Subject: [tcpinc] Selective protection of services provided by TCP headers

From: Tero Kivinen <[email-masked]>
Date: Sat, 15 Nov 2014 06:02:37 +0200

As we discussed in the meeting, everybody who cares about the
integrity protection of the services provided by the TCP headers
should send email to the list and explain why TCP header bit X should
be integrity protected, and what kind of attacks are possible if we do
not protect it.

Also explain what the receiving end should do if it detects attack
against the protected piece (i.e. either real active attacker attack,
or middlebox messing that thing up).

Note, that we do assume that the actual data stream is always
integrity protected and encrypted, and that integrity protection of
the data stream do include protection against replay, reordering and
data dropping attacks. That protection is integral part of the data
stream protection and how it is technically done is different thing,
but you can assume it is there.

I.e. sequence number offers two services, one is ordering of the
frames and to detect if there is gaps, replays, or reorderings, the
second one is the other end sending acks based on the sequence numbers
to acknowledge the data. The first service will be protected always,
and it can be done either by protecting the data stream offset
calculated from the sequence number, or with some other means (for
example tls record layer has its own internal sequence number
protecting against same things). The second service will not be
protected by default, so if that one also needs protection explain
what attacks is possible if we do not protect acks.

Send your commends to the list before the end of this month, i.e.
before 2014-11-30.

Also AD suggested that we should have interm meeting to talk about
those things before Christmas, but I do not have any proposals for
dates for it yet, but just wanted to give you heads up.
-- 
[email-masked]

---

From: ianG <[email-masked]>
Date: Sat, 15 Nov 2014 17:01:56 +0000

(thanks for the minutes, to Paul and Tero)

> be integrity protected, and what kind of attacks are possible if we do

I'm wondering why there is consideration of these active attacks.  In my 
mind it is fairly clear that the proposal for tcpinc is for passive 
protection only.

We know it is trivial for an active attacker to knock out the tcpinc 
additional exchange and force a fallback to cleartext.  Why do we care 
beyond that?

iang

---

From: David Mazieres <[email-masked]>
Date: Sat, 15 Nov 2014 10:08:54 -0800

> be integrity protected, and what kind of attacks are possible if we do

I know I was pushing for discussion of particular header fields, but
then someone (I thought it was you) made the excellent point that really
what we care about are *functions*, and that the header bits may be one
way to provide functions that could alternatively be provided some other
way.  So perhaps we should split the discussion into A) what functions
do we want to secure, and B) which of those functions are best secured
by protecting header bits.

Here is my list of functions.  At a high level, my goal is to ensure
that *if you establish a tcpinc connection to the right host*, then the
connection will behave as much as possible like vanilla TCP on a secure
network, even in the presence of active attackers (provided they don't
drop/mangle all of your packets).  Of course, since it's opportunistic,
you might connect to the wrong host, or you might use session IDs to
ensure you are talking to the right host.

* Payload integrity.  This is a given, but I list it because it means
  either disabling urgent data or protecting the urgent data pointer.
  There do exist legacy applications that use urgent data.  My linux
  system ships with GNU inetutils 1.9.2 (released 2013), which uses
  urgent data in rlogin/rlogind, telnet/telnetd, and ftp/ftpd.  We can
  maybe bid good riddance to rlogin and telnet, but not ftp.
  Fortunately, ftp uses SO_OOBINLINE, so strictly speaking byte-by-byte
  payload integrity will be preserved against an attacker toggling the
  URG bit (though read boundaries will change, which is not great).
  However, it looks like ftpd counts on receiving a SIGURG for the ABOR,
  STAT, and QUIT commands.  Injecting or suppressing such signals is an
  integrity attack (and, I'm now remembering, has even been abused for
  privilege escalation in the past).

* End of file.  EOF is part of the semantics of a TCP connection, used
  by applications for signaling (e.g., in old versions of HTTP).  An
  attacker who injects a premature EOF can cause data such as file
  transfers to be truncated without software noticing the error.  This
  is a data integrity attack, but like many integrity attacks poses a
  confidentiality threat as well.  Contrived example:  a bunch of people
  upload identical-length files to an anonymization server.  All
  participants say their uploads completed.  The server reveals the
  files but not their sources.  If the attacker truncated exactly one
  sender's file, he knows which user supplied the truncated file.

* Connection abort (the function played by TCP's RST bit).  Documented
  cases exist of attackers causing problems by injecting RST packets.
  There may be unilateral and/or non-cryptographic ways of protecting
  against RST forgery, but tcpinc should at least optionally provide
  connection integrity comparable to TCP-AO.  The attack here is the
  same as for TCP-AO.

* Keepalive.  It must not be possible for an attacker to cause a tcpinc
  endpoint to keep the connection (and associated crypto keys) around
  indefinitely if the other endpoint no longer exists.  This is a
  confidentiality attack that undermines forward secrecy.  For example,
  if an attacker keeps a defunct connection alive for many months, he
  then has plenty of time to compromise and extract session keys from
  the remaining endpoint.  (Note: TCP's keepalive feature is
  insufficient here, which is why tcpcrypt has the replay-protected
  SYNC_REQ.)

* Reliable data delivery over established connections.  Clearly an
  on-path attacker who tampers with packets can cause both downgrade
  attacks and denial of service attacks, most simply by dropping all
  packets.  However, an attacker who can eavesdrop on and forge packets
  but not suppress legitimate ones should not be able to disrupt
  established tcpinc connections.  This is a DoS attack that could
  happen if, for example, the attacker succeeds in desynchronizing the
  TCP connection.

* Deadlock freedom and buffer space control.  TCP implementations
  provide control over buffer space via socket options such as SO_SNDBUF
  and SO_RCVBUF.  Applications should be able to send data
  simultaneously in both directions without deadlocking if the data fits
  in the specified buffers.  An attacker who reduces the effective
  buffer space can mount a DoS attack.  Note this primarily applies to
  proposals that consume extra sequence numbers for non-payload data.  A
  straw man would be an attacker who manipulates the window to induce a
  bunch of one-byte segments with eight-byte MACs.  These must each
  consume one byte of buffer space, not nine.

* Flow control.  People assume they cannot continuously write large
  amounts of data to a TCP connection unless most of it is getting
  through to the other side.  Hence, applications use flow control as a
  means of reporting progress (based on bytes written), estimating
  transfer completion times, and adapting to network conditions.
  Unprotected flow control enables attacks that lie somewhere between
  mere DoS and integrity failure.  An example is someone trying to
  upload a video to either of two servers for safe keeping.  The user
  starts two file transfers then kills the one that seems slower, when
  in fact the fast-looking server is hardly getting any data at all
  because the attacker has disabled flow control and is faking the ACK
  and Window fields.

> or middlebox messing that thing up).

With the exception of RST, which should be discussed separately, an
integrity failure (detected attack) should be treated the same as a
checksum failure.  The segment should be discarded and should not affect
the TCP connection state.  It doesn't matter whether the corruption
happened in the payload or in header bits that are necessary to tcpinc's
security.

That said, there are really three cases to consider:  valid integrity,
invalid integrity, and as-yet unknown integrity.  The first is easy:
accept.  The second is easy:  discard.  In the third case, if, for
instance, an endpoint is waiting for a cumulative MAC in a different
segment, it should buffer the ambiguous segment until such point as it
determines whether or not the segment has been corrupted or receives an
unambiguously correct segment for the same sequence numbers.

David

---

From: David Mazieres <[email-masked]>
Date: Sat, 15 Nov 2014 10:21:47 -0800

> mind it is fairly clear that the proposal for tcpinc is for passive 

One reason is that if we go through all the trouble of getting tcpinc
out there and deployed, it would be a great shame if we had done 99% of
the work to protect against active attackers and then left a bunch of
security on the table.

In part because of this, the tcpinc charter specifically calls for
integrity protection and external authentication hooks.  External
authentication is kind of meaningless if you don't have data integrity.
Moreover, most integrity attacks undermine confidentiality as well.
(That's the reason cryptographers tell us to MAC ciphertexts rather than
encrypt authenticated plaintexts).

> additional exchange and force a fallback to cleartext.  Why do we care 

Well, if our effort is successful, then one possible outcome is that
most honest middleboxes allow tcpinc through, and which point
applications can use the authentication hooks to ensure tcpinc.

David

---

From: Joe Touch <[email-masked]>
Date: Mon, 17 Nov 2014 10:49:33 -0800

> or middlebox messing that thing up).

IMO a TCP-layer solution is needed only if TCP-level protection is
provided. That's why I think a TCP-level solution should provide some
TCP-level protection.

First, I'm assuming we still want to support TCPINC via NAT traversal,
which excludes:
	- TCP/IP SYN source IP (IP in that direction)
	- TCP/IP SYN source port

All other fields except those that depend on these *can* be protected
(i.e., the only one that can't is the checksum).

IP options MUST NOT be protected (even if they're IPv6 destination
options, they're not part of the TCP connection). Only the IP
pseudoheader should be protected, and only the portion that can't change
via a NAT (given NAT traversal).

So, IMO, the fields we CAN protect are
	IP SYN dest IP
	TCP dest port
	TCP flags and pointers
	TCP seq nos
	TCP options

Of these, the only values that are related to privacy would be the IP
and port, but we haven't considered making those private (i.e., we're
focused on data privacy, not connection privacy).

There are NO BENEFITS to the privacy of the data stream in protecting
ANY of these; the benefit is to protecting the TCP connection itself
against attack.

As a result, and because this WG is focused on a TCP solution, I'm in
favor of protecting ALL of these. If we try to get into the business of
deciding how to support middleboxes that play with these fields, then
we'll easily end up with TCP-in-TCP, at which point we ought to just use
BTNS IPsec and go home.

NOTE: regarding RSTs, they can and should be treated like any other
segment. TCPINC connections MUST use keepalives, which means that
they'll clean up after themselves if one side dies. There's NO need for
context-free RSTs.

Joe

---

From: "Scharf, Michael (Michael)" <[email-masked]>
Date: Tue, 18 Nov 2014 12:17:43 +0000

>   consume one byte of buffer space, not nine.

I am not sure if I understand this issue. Can you elaborate a bit? If the attacker can use SO_SNDBUF or SO_RCVBUF calls, it already has control over the application, right? If the attacker can manipulate RWND, it seems to be an on-path attacker that can do more harm by other means. The overhead of a MAC of 8 bytes for protecting one byte is independent of the encoding scheme (payload/option). Specifically, a TCP stack also requires memory for option storage and processing. Today's TCP stacks typically run dynamic buffer management and should not run into deadlock issues as long they have any available memory, as far as I understand it. Or do I miss something?
 
>   because the attacker has disabled flow control and is faking the ACK

This would be an on-path attacker, right? Note that manipulating RWND may not be an entirely uncommon Performance Enhancement Proxy (PEP) behavior; at least in the past people suggested this to "tune" TCP performance (or deal with legacy TCP stacks with broken window scaling). Manipulating RWND somehow violates e2e semantics and modern TCP stacks usually don't need this due to RWND autotuning. But I think at least in some environments this was in the past (and perhaps still is) a desired traffic engineering behavior by the operator of the PEP. As a result, I am not sure if such traffic engineering is an "attack".

Thanks

Michael

---

From: [email-masked]
Date: Tue, 18 Nov 2014 17:56:49 -0800

"Scharf, Michael (Michael)" <[email-masked]> writes:

> I am not sure if I understand this issue. Can you elaborate a bit? If
> the attacker can use SO_SNDBUF or SO_RCVBUF calls, it already has
> control over the application, right? If the attacker can manipulate
> RWND, it seems to be an on-path attacker that can do more harm by
> other means. The overhead of a MAC of 8 bytes for protecting one byte
> is independent of the encoding scheme (payload/option). Specifically,
> a TCP stack also requires memory for option storage and
> processing. Today's TCP stacks typically run dynamic buffer management
> and should not run into deadlock issues as long they have any
> available memory, as far as I understand it. Or do I miss something?

Currently things like TCP options, while they consume memory at
endpoints, do not consume window space.  What I am saying is that if you
change TCP so that things other than payload bytes written to a socket
consume sequence numbers, then the window logic MUST be adjusted to
account for that.  It's not impossible to do, but it's something that
has to be taken into account.

Note that the attack doesn't require the attacker making calls to
SO_SNDBUF or SO_RCVBUF, or even to be on path.  It just requires an
application that is deadlock free under the assumption that
SO_SNDBUF/SO_RCVBUF is a reasonable value.  Then the attacker simply
induces behavior that chews up way more sequence numbers than payload
bytes.  This could be Ajax code in the browser issuing a bunch of tiny
writes (after the browser disables Nagle).  Or it could be a server
sending a bunch of one-byte writes to a (Nagle-disabled) web proxy that
does cut-through routing in an effort to deadlock a TCP connection
between the browser and proxy.

> This would be an on-path attacker, right? Note that manipulating RWND
> may not be an entirely uncommon Performance Enhancement Proxy (PEP)
> behavior; at least in the past people suggested this to "tune" TCP
> performance (or deal with legacy TCP stacks with broken window
> scaling). Manipulating RWND somehow violates e2e semantics and modern
> TCP stacks usually don't need this due to RWND autotuning. But I think
> at least in some environments this was in the past (and perhaps still
> is) a desired traffic engineering behavior by the operator of the
> PEP. As a result, I am not sure if such traffic engineering is an
> "attack".

This is an attacker who can eavesdrop on traffic (or guess ISNs) and
forge packets, but not necessarily tamper with or block legitimate
packets.  For this particular attack, RWND isn't the issue so much as
forged ACK numbers.  By continuously forging ACKs for all sent traffic,
the attacker can de-synchronize the TCP connection and then continue on
to make the sender think all his data is getting through.  E.g., the
attacker makes curl -d report 500MiB uploaded at 10MiB/sec, 90%
complete, when in fact nothing of the sort has happened.

David

---

From: "Scharf, Michael (Michael)" <[email-masked]>
Date: Wed, 19 Nov 2014 09:44:25 +0000

> has to be taken into account.

Sure. This is very well-known. This was excessively discussed in the MPTCP WG during the encoding discussion some years ago.

I agree that drafts running into such issues have to address this - but this seems possible and is just a design detail that has to be solved.

(A general question could be if any long option scheme needs a solution to deal with endsystems that completely run out of memory. This has never been a real issue for 40B options. If we allow for 1KB+ options, low end devices could be a concern. For long options, this is not at all specific to tcpinc and encoding issues IMHO should be discussed in TCPM.)

> between the browser and proxy.

I still don't understand how this can be an issue if the actual buffer and possibly RWND considers the encoding overhead.

I think most TCP stacks don't run a static, fixed TCP buffer of size SO_SNDBUF or SO_RCVBUF anyway. In particular packet-based stacks seems to work differently internally.

> complete, when in fact nothing of the sort has happened.

There may be (legacy) middleboxes (PEPs) that send such Pre-ACKs if they know/assume that the path between the middlebox and the receiver is entirely reliable. This is a violation of TCP end-to-end semantics and probably not a PEP behavior that the IETF recommends. But, in particular in satellite environments, this could be a mechanism used for traffic engineering or performance optimization (I've read about it one decade ago, maybe those PEPs went away in the mean time). If used for performance tuning, these ACKs would not be a malicious attack.

If an attacker that can inject ACKs into the connection, that attacker could also send RST or other segments with more impact than a faked ACK, right? So, this is basically a variant of the question whether to protect against RST (e.g., with TCP-AO)?
 
Michael

---

From: David Mazieres <[email-masked]>
Date: Wed, 19 Nov 2014 15:04:48 -0800

"Scharf, Michael (Michael)" <[email-masked]> writes:

> I agree that drafts running into such issues have to address this -
> but this seems possible and is just a design detail that has to be
> solved.

I think we are basically in agreement.  This has to be added to the list
of things to check in any tcpinc draft, but we should anticipate that it
is a tractable problem.

> I still don't understand how this can be an issue if the actual buffer
> and possibly RWND considers the encoding overhead.

It doesn't have to be an issue, but it can be an issue if a protocol is
not properly designed.  And naively attempting to address the problem
could lead to other issues (such as exceeding the receive buffer size).

For example, one naive way to solve the problem is to implement payload
integrity entirely on top of TCP, almost at a different level of
abstraction.  Something like TLS already works over TCP, so intuitively
one might expect this to work.  Well obviously not because now the MACs
are going to consume options space and cause deadlock in situations
where TCP would have been fine.  Okay, so the next thing one might try
to do is increase the effective send buffer size to be SO_SNDBUF plus
the size of all buffered MACs.  But this still doesn't solve the
problem, because now what if the MAC is computed over more data than the
receive window size?  Now the receiver will have a bunch of data whose
integrity is unknown, and will not have buffer space to receive the MAC.
Okay so now allow sender to exceed the receive window and have the
receiver buffer data until the next MAC.  Okay, but now an ill-behaved
sender could create a giant message that massively exceeds the receive
buffer size.

These are all obvious straw-men.  But the point is that the real
solution is a bit more complicated.  Since as far as I know none of the
sequence-number-stretching tcpinc proposals has even been implemented,
it's an open question how exactly flow control interacts with such
proposals.

> I think most TCP stacks don't run a static, fixed TCP buffer of size
> SO_SNDBUF or SO_RCVBUF anyway. In particular packet-based stacks seems
> to work differently internally.

I haven't tested recently, but in my experience SO_RCVBUF definitely
impacted the receive window.  More importantly, though, even
applications that don't set SO_RCVBUF can reasonably assume they get
some sane default value, meaning at least 4 MSS and less than infinity.
If tcpinc changes this so the effective receive buffer is tiny, it could
cause deadlock.  If tcpinc changes it so the effective receive buffer is
infinity, this breaks flow control.  And obviously, while greater than
the buffer size, we want some kind of limit on how much buffer space the
kernel needs on the sender or receiver.

> There may be (legacy) middleboxes (PEPs) that send such Pre-ACKs if
> they know/assume that the path between the middlebox and the receiver
> is entirely reliable. This is a violation of TCP end-to-end semantics
> and probably not a PEP behavior that the IETF recommends. But, in
> particular in satellite environments, this could be a mechanism used
> for traffic engineering or performance optimization (I've read about
> it one decade ago, maybe those PEPs went away in the mean time). If
> used for performance tuning, these ACKs would not be a malicious
> attack.

Probably better just to terminate the TCP connection, no?  Particularly
given the prevalence of transparent proxies.  If there's a choice
between breaking flow control and slightly diminishing the throughput of
legacy devices that are new enough to use tcpcrypt but not to increase
their receive buffer size, I would prefer the latter.

> If an attacker that can inject ACKs into the connection, that attacker
> could also send RST or other segments with more impact than a faked
> ACK, right? So, this is basically a variant of the question whether to
> protect against RST (e.g., with TCP-AO)?

Well, you can say it's a variant of RST, but the consequences of the
attack are different and the defenses are different, so it doesn't seem
that useful to lump them together.  RST protection can be implemented
locally and unilaterally for a variety of TCP modifications.  Checking
the integrity of ACKs requires specific support from the wire format.

Also, RST kills the connection, so it's basically a DoS attack.  ACK
forgery causes the application to think things are okay, but return
incorrect values for things like percent complete and bytes/second.

David

---

From: Tero Kivinen <[email-masked]>
Date: Thu, 20 Nov 2014 16:50:40 +0200

> 	- TCP/IP SYN source IP (IP in that direction)

You do assume that the server is never behind NAT. This is not true.
It is actually very common to have servers also behind static NATs,
i.e. mapping external address + port to some internal address + port
in the DMZ.

> So, IMO, the fields we CAN protect are

So if we want to work through NATs I think we cannot protect the IP
dest IP nor the TCP dest port. 

> 	TCP options
...
> we'll easily end up with TCP-in-TCP, at which point we ought to just use

So can you answer the questions I asked. For each service provided by
each of the fields listed above, why it should be integrity protected,
and what kind of attacks are possible if we do not protect it, and
what should we do if we detect that someone has messed up with it.

It does not really help saying I want to do everything. If you want to
protect everything then you should really be just using IPsec... As we
discussed in the meeting and on the mailing list protecting those
fields will limit as when we try to go through certain kind of
middleboxes, so if we protect everything, we will not work through
certain middleboxes, which means tcpinc will be disabled quite often,
which means we do not get the maximal protection against the pervasive
monitoring.

So the trade off is provide as wide protection against pervasive
monitoring as possible, or provide protection against certain active
attacks, while not protecting against pervasive monitoring that well.

For almost every single field in the TCP header I think we can find
some middlebox that will mess up with it, thus cause tcpinc to be
disabled if we protect against it. Or even worse just cause connection
to hang when the feature is first time used. I.e. if we decide to
protect urgent field, and decide we drop the segment if someone messes
up with the field, and if there is middlebox in the middle that will
zero out the urgent field every single time, then first time we try to
use urgent data, the connection will hang. The connection will not
fall back to cleartext, and will not be disconnected. So protecting
urgent field would most likely require us to use policy where we
ignore the result of failures in the it, thus there is no point of
protecting it... 
-- 
[email-masked]

---

From: Joe Touch <[email-masked]>
Date: Thu, 20 Nov 2014 08:43:01 -0800

> i.e. mapping external address + port to some internal address + port

I realize that. It's a trade-off, and it prevents one box from spoofing
a server to another - in this case, it prevents the host behind the NAT
from spoofing as if it were the NAT itself.

However, this might be something that can be relaxed, either
per-connection or in general.

> dest IP nor the TCP dest port.

If you want to allow servers behind NATs, yes. But, IMO, that's a
particular kind of identity attack that I don't think we necessarily
need to support - or at least all the time.

> each of the fields listed above, why it should be integrity protected,

TCP. Those bits and fields are there for a reason. If you aren't
protecting them, you aren't protecting TCP and we'd all be better off
using existing TLS in the conventional manner.

> and what kind of attacks are possible if we do not protect it, and

All the currently known ones and all the future ones we don't yet know
about. Gaming what's protected and what isn't just opens TCP up to attacks.

> what should we do if we detect that someone has messed up with it.

Silently drop the packet, and perhaps log at most one such event per
connection for use by other attack analysis.

> protect everything then you should really be just using IPsec...

IPsec does not establish per-connection protection, as noted in RFC5925.
IMO, a TCP-level solution ought to protect TCP. As you note, protections
at other levels are already handled by other protocols.

> As we
> middleboxes,

Yes. And they should. You can't differentiate between cooperative
middleboxes and malicious ones merely by the fields they modify.

> so if we protect everything, we will not work through

"maximal" protection means minimal use.

Protection, by its nature, means restricted use.

Security means having to say NO.

Joe

---

From: "Scharf, Michael (Michael)" <[email-masked]>
Date: Thu, 20 Nov 2014 20:29:50 +0000

> could lead to other issues (such as exceeding the receive buffer size).

Sorry, I still don't understand what buffer problem you refer to, assuming the protocol design is bug-free. Could you please expand a bit?

> one might expect this to work.

What deadlock is caused when we would use something like TLS for confidentiality and something like TCP-AO integrity (or TCP-AO for both)?

> Well obviously not because now the MACs

Maybe it would help to me to better understand the deadlock if I had a concrete example.

> the size of all buffered MACs.  But this still doesn't solve the

As far as I know, SO_SNDBUF/SO_RCVBUF is the *socket* buffer size, which should not be affected if tcpinc offers a socket interface to the app, as far as I understand it.

But the socket buffer may not be the buffer TCP uses - if they were the same, this would be a specific implementation choice.

The TCP buffer can be larger. For instance, I think Linux uses for TCP buffering a buffer significantly larger than the socket buffer:

      SO_SNDBUF
              Sets or gets the maximum socket send buffer in bytes.  The
              kernel doubles this value (to allow space for bookkeeping
              overhead) when it is set using setsockopt(2), and this doubled
              value is returned by getsockopt(2).  The default value is set
              by the /proc/sys/net/core/wmem_default file and the maximum
              allowed value is set by the /proc/sys/net/core/wmem_max file.
              The minimum (doubled) value for this option is 2048.

       SO_RCVBUF
              Sets or gets the maximum socket receive buffer in bytes.  The
              kernel doubles this value (to allow space for bookkeeping
              overhead) when it is set using setsockopt(2), and this doubled
              value is returned by getsockopt(2).  The default value is set
              by the /proc/sys/net/core/rmem_default file, and the maximum
              allowed value is set by the /proc/sys/net/core/rmem_max file.
              The minimum (doubled) value for this option is 256.

Thus, as far as I understand it, the value of SO_RCVBUF is by and large decoupled from the TCP receiver advertised window. An application cannot expect that the receiver advertised window is equal to SO_RCVBUF etc.

> integrity is unknown, and will not have buffer space to receive the

If the MAC for header fields is added in a TCP option (e.g., TCP-AO), I don't understand why buffer space would be a problem.

I understand that securing certain TCP header fields from within the TCP byte stream may be challenging. But buffer space is by far not the only challenge for this, as far as I can see.

> sender could create a giant message that massively exceeds the receive

I don't understand why one would have to allow a sender to exceed the receive window. Could you please explain why this would be required? It violates the TCP specs.
 
> it's an open question how exactly flow control interacts with such

tcpcrypt seems to have to violate TCP flow control for INIT1/INIT2 (well, using EDO could possibly solve this problem).

Again, please help me better understanding why flow control would be an issue in any other solution. We don't need an implementation to understand whether protocol semantics are violated or not. But you have to explicitly state your assumptions and reason why there is no alternative to them.

> some sane default value, meaning at least 4 MSS and less than infinity.

As far as I can tell, the IETF does not mandate how a TCP stack allocates buffer internally, nor how it sets the TCP receive window.

> kernel needs on the sender or receiver.

Sorry, I cannot follow. It is up to the TCP stack to decide how much buffer to allocate, and what receive window to announce.  A TCP stack does not have to use SO_RCVBUF for calculating the receive window, as far as I know.

The tcpinc WG may have to specify that a stack has to ensure that the TCP buffer is larger than the socket buffer, or the tcpinc WG may specify that a TCP endpoint using tcpinc must allocate X additional bytes e.g. to store out-of-band key/MAC data, and fall back to vanilla TCP if no RAM us available. Why is this a problem?

> their receive buffer size, I would prefer the latter.

Pre-ACKs have been suggested in the past because of the huge RTT in satellite networks. This large RTT also affects congestion control (e.g., during Slow-Start). The performance difference could be significant.

I agree that a transparent TCP proxy may end up in less issues. Also, I have no data whether there are indeed middleboxes in the Internet not terminating the connection but faking ACKs. And none of these middleboxes are desirable regarding end-to-end TCP semantics. But we have toaccept that people may deploy middleboxes e.g. in satellite networks without having attacks in mind.

> incorrect values for things like percent complete and bytes/second.

TCP does not guarantee that data indeed arrives at the receiving application. TCP only guarantees delivery to the receiving TCP stack. Thus, an application that cares about payload integrity (in your example: complete upload) has to verify the byte stream at application level for standard TCP already. In that case, one can detect that attack already at application level, right? So, I really wonder faking a complete upload at TCP level is a real problem.

In addition, one could very easily detect this attack in many scenarios. If an application reports an upload rate of 42 Gbit/s on a 4 Mbit/s DSL uplink, that would certainly be strange, right? In other words: ACK forgery would have to detect the real path capacity in order to keep the displayed throughput without reasonable limits of the true value. Getting this right would be quite some effort; it seems basically equivalent to an active on-path attacker. Is this really what a tcpinc solution should focus on?

Michael

---

From: [email-masked]
Date: Fri, 21 Nov 2014 16:31:02 -0800

"Scharf, Michael (Michael)" <[email-masked]> writes:

> Sorry, I still don't understand what buffer problem you refer to,
> assuming the protocol design is bug-free. Could you please expand a
> bit?

Great, as long as we are in agreement that a bug-free tcpinc protocol
should not deadlock anyplace a vanilla tcp protocol does, then I think
you understand me.  In particular, since the specs don't go into this
level of detail, we should have an implementation of each proposed
protocol to kick the tires and have a sanity check that all of this
stuff works.

> What deadlock is caused when we would use something like TLS for
> confidentiality and something like TCP-AO integrity (or TCP-AO for
> both)?

TCP-AP does not have this problem, because TCP-AO uses TCP option space.
The concern I was raising arises from the complexity of sequence number
stretching, as happens when protocols consume sequence numbers for MACs.

> Maybe it would help to me to better understand the deadlock if I had a

Okay, here's a simple example.  Both sides set send and receive buffers
of 8192 bytes using SO_SNDBUF/SO_RCVBUF.  Then both sides write 16384
bytes.  Then both sides read 16384 bytes.  Under vanilla TCP, this is
deadlock-free because the sending kernel and receiving kernel each
buffer 8192 bytes not read by the application.

All I'm saying is that I'd like such scenarios to continue to be free
from deadlock under tcpinc.  They might already be under all existing
proposals.  Or if they aren't, you might already agree with me that this
would have to be fixed in any proposal, in which case your confusion
might be why I'm mentioning such an obvious point.  Only because I can't
tell if all the proposals have this.

> As far as I know, SO_SNDBUF/SO_RCVBUF is the *socket* buffer size,
> which should not be affected if tcpinc offers a socket interface to
> the app, as far as I understand it.
>
> But the socket buffer may not be the buffer TCP uses - if they were
> the same, this would be a specific implementation choice.
>
> The TCP buffer can be larger. For instance, I think Linux uses for TCP
> buffering a buffer significantly larger than the socket buffer:

Well, then this requires some investigation.  For instance, on at least
some variants of BSD, SO_RCVBUF directly affects the advertised window
size.  So perhaps this discussion should be detached a bit from
SO_SNDBUF/SO_RCVBUF.  The higher-level point is that scenarios that
don't deadlock on particular implementations of vanilla TCP should
continue not to deadlock when tcpinc is enabled.

> If the MAC for header fields is added in a TCP option (e.g., TCP-AO),
> I don't understand why buffer space would be a problem.

Correct.  I'm mostly concerned about sequence-number-stretching
proposals, in which tcpinc consumes more sequence numbers than vanilla
TCP.  TCP-AO does not have this problem.

> I don't understand why one would have to allow a sender to exceed the
> receive window. Could you please explain why this would be required?
> It violates the TCP specs.

I was just throwing out bad solutions.  All I'd really like to convince
you is that once established, a tcpinc connection should not experience
deadlock (even with a passive adversary who can forge packets) where
vanilla TCP over a secure network wouldn't experience deadlock.

The proposals I was putting forth for dealing with sequence-number
stretching were bad.  But I'm not necessarily advocating sequence-number
stretching.

> tcpcrypt seems to have to violate TCP flow control for INIT1/INIT2
> (well, using EDO could possibly solve this problem).

We would ideally specify a reasonable minimum window size (like 500
bytes).  If it turns out that the window is independent of SO_RCVBUF,
then this is easy just to mandate it.  Possibly EDO could help, too,
though ideally tcpinc would benefit from but not be blocked by other
proposals.

> Again, please help me better understanding why flow control would be
> an issue in any other solution. We don't need an implementation to
> understand whether protocol semantics are violated or not. But you
> have to explicitly state your assumptions and reason why there is no
> alternative to them.

Okay, so instead of debating whether it's an issue (some of the other
draft authors could just weigh in here), can we just agree that even
weird/contrived/controlled-by-malicious-javascript/etc. applications
that don't deadlock on vanilla TCP shouldn't deadlock on tcpinc?

> The tcpinc WG may have to specify that a stack has to ensure that the
> TCP buffer is larger than the socket buffer, or the tcpinc WG may
> specify that a TCP endpoint using tcpinc must allocate X additional
> bytes e.g. to store out-of-band key/MAC data, and fall back to vanilla
> TCP if no RAM us available. Why is this a problem?

Exactly.  We are agreed.  So the only problems are, A) calculating X,
and B) making sure we actually specify it.  If we agree on these points,
then I think you understand my main concern and were just thrown off by
my pessimism over whether this would happen with
sequence-number-stretching proposals.

> Pre-ACKs have been suggested in the past because of the huge RTT in
> satellite networks. This large RTT also affects congestion control
> (e.g., during Slow-Start). The performance difference could be
> significant.
>
> I agree that a transparent TCP proxy may end up in less issues. Also,
> I have no data whether there are indeed middleboxes in the Internet
> not terminating the connection but faking ACKs. And none of these
> middleboxes are desirable regarding end-to-end TCP semantics. But we
> have toaccept that people may deploy middleboxes e.g. in satellite
> networks without having attacks in mind.

Given how fringe these are, I would settle for such scenarios working
more slowly with tcpinc enabled, at least until the ACK-forging boxes
are upgraded to do something else (like downgrade connections or
terminate them and buffer data).

> TCP does not guarantee that data indeed arrives at the receiving
> application. TCP only guarantees delivery to the receiving TCP
> stack. Thus, an application that cares about payload integrity (in
> your example: complete upload) has to verify the byte stream at
> application level for standard TCP already. In that case, one can
> detect that attack already at application level, right? So, I really
> wonder faking a complete upload at TCP level is a real problem.

TCP does not guarantee that data arrives at the receiving application,
or even receiving host (given send buffering).  However, actual real
applications in widespread use rely on the fact that TCP's buffers are
not infinite so as to estimate upload bandwidth.

> In addition, one could very easily detect this attack in many
> scenarios. If an application reports an upload rate of 42 Gbit/s on a
> 4 Mbit/s DSL uplink, that would certainly be strange, right?

They could but they don't.  So again, if you believe my principle that
an established, authenticated tcpinc connection over an insecure network
should behave roughly like vanilla TCP over a secure network, flow
control should be secured.  Similarly, an established unauthenticated
tcpinc connection should behave the same if there was no
man-in-the-middle.

David

---

From: "Scharf, Michael (Michael)" <[email-masked]>
Date: Sun, 23 Nov 2014 15:32:48 +0000

Some few follow-up comments; I think my points are clear for the other ones.

> > Maybe it would help to me to better understand the deadlock if I had
> a
> > concrete example.
> 

OK, I think we agree that there are no *known* deadlock issue right now assuming reasonable implementations.

And we also agree that all specifications have to address how to deal with small send/receive buffers.

> continue not to deadlock when tcpinc is enabled.

Indeed. This could imply that those BSD variants may have to change their 1:1 mapping of SO_RCVBUF to the advertised receive window. I don't think any IETF specification mandates this - the IETF does not standardize the sockets interface. Many other stacks use much more sophisticated algorithms to set the advertised receive window.

> though ideally tcpinc would benefit from but not be blocked by other

I am not sure what you mean by "blocked".

Do you mean that TCPM may not complete the EDO spec fast enough? If so, please come to TCPM and help us moving forward at least the non-SYN case. We really look for feedback from implementers of EDO right now. 

Or is your concern the risk that EDO could be blocked on a number of Internet paths by middleboxes, thereby reducing the number of path over which tcpinc would work? If so, please speak up in TCPM as well. In particular, I think TCPM would be highly interested in insights about middleboxes that block the current EDO design but that would let through the current tcpcrypt design.

My own concern is that the INIT1/INIT2 mechanism seems to define an own, special long option encoding format that appears to be incompatible with EDO and the other encoding variants currently discussed in TCPM. Not only ideally the IETF should avoid having several incompatible TCP long option formats designed independently of each other. Interoperability could become a real mess, in particular if we would try to further extend any of the mechanisms in the upcoming years.

Thanks

Michael