Subject: Re: [tsvwg] Checksums in UDP options

From: "C. M. Heard" <[email-masked]>
Date: Thu, 6 Sep 2018 09:32:00 -0700

On Thu, 6 Sep 2018 08:11:50 -0700 Joe Touch wrote:
>> On Sep 6, 2018, at 8:04 AM, Gorry Fairhurst <gorry at erg.abdn.ac.uk> wrote:
>>> On 06/09/2018, 15:47, Tom Herbert wrote:
>>>> On Thu, Sep 6, 2018 at 2:48 AM, Gorry Fairhurst<gorry at erg.abdn.ac.uk>  wrote:
>>>> So, I see it normally this way:
>>>> - UDP checksum fails                            drop packet
>>>> - UDP checksum passes, OCS fails                deliver udp to upper layer, ignore all options
>>>> - UDP checksum passes, OCS passes               deliver packet, parse options
>>>>
>>>> ACS enabled*
>>>> - UDP checksum passes, OCS passes, ACS passes   deliver udp to upper layer,
>>>> process options
>>>> - UDP checksum passes, OCS passes, ACS fails    drop packet
>>>>
>>>> To achieve the coverage expected by ACS, I'd also require a receiver with
>>>> ACS enabled to also drop:
>>>> UDP packet with no option or no ACS option      drop packet
>>>> UDP checksum passes, OCS fails                  drop packet
>>>>
>>>> Does that make sense?
>>>>
>>> No, it doesn't. Consider that OCS and FRAG option can be used
>>> together. Per the above rules, if the OCS fails then all options,
>>> including the FRAG option, are ignored. This results in a fragment
>>> being sent so the user. What the user sees is gibberish and this
>>> directly violates the requirement: "Individual UDP fragments MUST NOT
>>> be forwarded to the user.". The receiver full knew there was a problem
>>> with the packet, and therefore it could not eliminate the very real
>>> possibility that it would be sending incorrect data to the user. The
>>> receiver needs to drop the packet on OCS failure.
>>>
>>> Tom
>> I wanted to first agree the processing of the checksums!
>>
>> I agree what you say is a severe problem. I suspect for this reason the
>> current design of the FRAG option is broken:
>> To me, the fragment data really needs to be sent in the options part of
>> the UDP datagram, because placing the data in the UDP payload can (and
>> will) result in case where fragments are unexpectedly returned to userland
>> when a stack does not understand options (e.g.. a legacy receiver or
>> legacy middlebox on path).
>
> Agreed.  I think we’ve been heading towards lite+frag as the only safe
> way to use frag for a while. It may be preferable to force that in the
> spec.

Indeed. I agree with Gorry's analysis in general, but I also agree with
Tom's (and Gorry's) concern in the case of FRAG, which egregiously
violates the desideratum that an option always be safe to ignore. The
obvious fix for that is to use FRAG in conjunction with LITE except
when it is used for negotiation. If that is done, then a failure of OCS
will not cause a fragment to be passed to the higher layer as if it were
a normal UDP payload. It also eliminates any possibility that a legacy
receiver could misinterpret a UDP fragment.

Another desirable feature of always using FRAG in conjunction with
LITE is that redundant checksum coverages go away. The checksums
in the UDP headers (if non-zero) cover the UDP headers only. The
checksum in the terminal FRAG option (if non-zero) covers the
reassembled payload only (ACS, if present, does the same). And
OCS of course covers only the options.

If we go this route, I believe that it is necessary to make an exception
to the following rule for FRAG and LITE:

   Receivers cannot treat unexpected option lengths as invalid, as this
   would unnecessarily limit future revision of options (e.g., defining
   a new ACS that is defined by having a different length).

Otherwise, it will be impossible to enforce desirable integrity checks.

Mike Heard

---

From: Tom Herbert <[email-masked]>
Date: Thu, 6 Sep 2018 09:49:54 -0700

> Otherwise, it will be impossible to enforce desirable integrity checks.
Mike,

Note the problems aren't just with FRAG. The second some implements
someone implements payload encryption in a UDP option (AE option?)
they'll hit the same problem. Generally, any option that performs a
transform on the data that needs to be undone before delivery can't be
ignored. So either each option needs to carefully considered to see it
requires LITE since it can't safely be ignored, or just require use of
LITE all the time to solve the problem for everyone in one shot.

Tom

---

From: "C. M. Heard" <[email-masked]>
Date: Thu, 6 Sep 2018 10:23:23 -0700

> LITE all the time to solve the problem for everyone in one shot.

That is certainly correct, and for that reason I would agree that AE
and any other option that requires reversal of a transform on the
data must be used in conjunction with LITE, in order to prevent
misinterpretation of payload data in case of an OCS failures.

I don't want to see this required for all UDP options, though. That
would exclude some desirable use cases. Consider the case of
DNSSEC, which is known to have issues with IPv6 fragmentation.
A client can safely include a FRAG option with offset zero,
length 12, and checksum zero in a UDP datagram sent to any server.
That FRAG option is a no-op that indicates that it is safe for the
server to return a reply using UDP fragmentation. If the server
doesn't understand UDP options, it will ignore the option, interpret
the UDP payload as a normal UDP payload (which is correct), and
reply in the same manner as is done today. An server that supports
UDP options, on the other hand, can take advantage of UDP
fragmentation if it needs to return an outsized reply.

Mike Heard

---

From: Tom Herbert <[email-masked]>
Date: Thu, 6 Sep 2018 10:45:19 -0700

On Thu, Sep 6, 2018 at 10:23 AM, C. M. Heard <[email-masked]> wrote:
> server to return a reply using UDP fragmentation.

Where's the security in that? If an attacker spoof my address and
sends a FRAG option to a DNS server, then AFAICT the DNS server will
think that the host with my address supports FRAG.So next time I do a
DNS look up, the the server may respond with UDP fragments. If my host
doesn't support UDP options, then the application gets DNS response
that are gibberish. This is a perfect Denial of Service Attack.

Tom

> If the server

---

From: "C. M. Heard" <[email-masked]>
Date: Thu, 6 Sep 2018 11:03:30 -0700

On Thu, Sep 6, 2018 at 10:45 AM Tom Herbert wrote:
> On Thu, Sep 6, 2018 at 10:23 AM, C. M. Heard wrote:

Your legacy host won't see gibberish if FRAG is combined with LITE as
we've been discussing. Rather, it will see a UDP length of zero in each of
the fragments, and its UDP layer will deliver no data to its application layer.
That's actually more innocuous that the threat that already exists today.

But you do bring up yet another reason why any UDP option that transforms
the UDP data needs to be "firewalled" behind a LITE option.

Mike Heard

---

From: Tom Herbert <[email-masked]>
Date: Thu, 6 Sep 2018 11:30:39 -0700

> That's actually more innocuous that the threat that already exists today.
It's still a problem. The server may infer that receiving the FRAG
option means that the sending host supports options. So then when the
server starts sending options with LITE, the packets are now dropped
at the client. The effect of DOS attack is still the same. The client
isn't seeing responses from the server, the server thinks everything
is okay since there's no error reports coming back to it,

You might get away with this negotiation method in a request/response
protocol like DNS if the server only responds using UDP options if it
the client indicated support in the request.  But note, only the
server side can use mandatory options like FRAG, not the client in
this case.

Tom

---

From: Joe Touch <[email-masked]>
Date: Thu, 6 Sep 2018 11:54:34 -0700

> On Sep 6, 2018, at 11:30 AM, Tom Herbert <[email-masked]> wrote:
> 
> . The client

That’s a poorly implemented server. UDP is best effort. Packets get lost. A positive confirmation is the only way to think anything is ok. 

Joe

---

From: Joe Touch <[email-masked]>
Date: Thu, 6 Sep 2018 11:55:58 -0700

> On Sep 6, 2018, at 11:03 AM, C. M. Heard <[email-masked]> wrote:
> 

As noted, this applies only where other info isn’t available, eg handshakes at other layers or or other ways to know context. 

Joe

---

From: "C. M. Heard" <[email-masked]>
Date: Thu, 6 Sep 2018 12:00:52 -0700

On Thu, Sep 6, 2018 at 11:30 AM Tom Herbert wrote:
> On Thu, Sep 6, 2018 at 11:03 AM, C. M. Heard wrote:
> the client indicated support in the request.

Indeed, in the DNS case, the negotiation I describe is safe only if
the server does not retain memory of the client's ability to use FRAG
across transactions. The negotiation has to apply to exactly one
request -- which is what I was implicitly assuming. But subject to
that proviso, I fail to see any security vulnerabilities (either from a
spoofing attack or a MITM attack) that do not already exist today.

I grant that this is a special case, but I believe that it is a potentially
important special case (though the DNSOP folks may not agree --
see draft-song-atr-large-resp for an alternative approach).

Again, though, you bring up a good point: negotiating the use of
options that transform the data can introduce security vulnerabilities
unless there is some means to authenticate the remote peer.

> But note, only the

I think what you meant to say here was not "mandatory options"
but rather something like "options that transform the data" -- is
that correct?

Mike Heard

---

From: Tom Herbert <[email-masked]>
Date: Thu, 6 Sep 2018 12:01:17 -0700

> That’s a poorly implemented server. UDP is best effort. Packets get lost. A positive confirmation is the only way to think anything is ok.
Okay, then please explain what the server should have done differently
to avoid this problem.

Tom

---

From: "C. M. Heard" <[email-masked]>
Date: Thu, 6 Sep 2018 12:09:03 -0700

On Thu, Sep 6, 2018 at 11:56 AM Joe Touch <[email-masked]> wrote:
> > On Sep 6, 2018, at 11:03 AM, C. M. Heard wrote:
> As noted, this applies only where other info isn’t available, eg handshakes at
> other layers or or other ways to know context.

Let me rephrase that, then: if we agree on the following behavior (or,
alternatively,
in situations where the following behavior is agreed between the two endpoints)

- UDP checksum passes, OCS fails: deliver UDP payload to upper layer,
ignore all options

then we need to firewall options such as FRAG that transform data behind LITE.

--cmh

---

From: Tom Herbert <[email-masked]>
Date: Thu, 6 Sep 2018 12:10:37 -0700

> see draft-song-atr-large-resp for an alternative approach).
I'm not sure I would call DNS a special case. Isn't that one of the
biggest users of UDP on the Internet?

> unless there is some means to authenticate the remote peer.
Look at it this way, we've spent about forty years trying to get TCP
handshake to be secure and efficient and it's still not perfect. These
are not easy problems to solve! :-)

> but rather something like "options that transform the data" -- is
"Mandatory" in the sense that they must be processed, not that they
must be in the packet. I consider options with strong CRCs set by the
sender to be in that class, since the effects of ignoring a CRC really
isn't innocuous.

Tom

---

From: "C. M. Heard" <[email-masked]>
Date: Thu, 6 Sep 2018 12:17:50 -0700

On Thu, Sep 6, 2018 at 12:10 PM Tom Herbert wrote:
> On Thu, Sep 6, 2018 at 12:00 PM, C. M. Heard wrote:

That is my understanding. I call it a special case because of the
strict request-response pairing.

> are not easy problems to solve! :-)

Indeed.

Mike Heard

---

From: "C. M. Heard" <[email-masked]>
Date: Thu, 6 Sep 2018 12:35:23 -0700

> sender to be in that class, since the effects of ignoring a CRC really

OK. I confused this with the notion of "required" option in the draft,
and there are two senses of "required" there, too: "required to
implement" (Section 5) and "required to be present" (Section 6).

Can we please use the phrase "mandatory to process" or something
similar to disambiguate this third sense? We can further debate just
which options those should be.

Thanks,

Mike Heard

---

From: Tom Herbert <[email-masked]>
Date: Thu, 6 Sep 2018 12:47:48 -0700

> which options those should be.

How about:

"Required to implement" = MUST implement
"Mandatory to process" = MUST be processed if received (or drop packet)
"Enforced to be present" = MUST be received and processed (or drop
packet). (Although, I'm not sure if something like that would
technically still be an option).

Tom

---

From: "C. M. Heard" <[email-masked]>
Date: Thu, 6 Sep 2018 12:57:27 -0700

On Thu, Sep 6, 2018 at 12:47 PM Tom Herbert wrote:
> On Thu, Sep 6, 2018 at 12:35 PM, C. M. Heard wrote:

Works for me.

--cmh

---

From: Joe Touch <[email-masked]>
Date: Fri, 7 Sep 2018 20:32:13 -0700

FWIW, I’m not sure all three of these are actually needed.

There are options that MUST be implemented, i.e., if you find out that an endpoint knows NOP, then you can (or should) safely assume it knows EOL, etc.

I don’t know what “mandatory to process” means. OCS, LITE, FRAG, NOP, EOL are are always processed in full if valid. TIME and MSS too, but that basically means just passing the info up with the packet. AE and ACS should be computed (IMO) only when the receiver enables those features, otherwise their presence is passed up to the app only (as are all otherwise unknown options). New options would have to decide for themselves what happens.

Nothing is enforced to be present per se, except as indicated by receiver configuration. I.e., all options are optional as a default.

Joe

> On Sep 6, 2018, at 12:57 PM, C. M. Heard <[email-masked]> wrote:
>

---

From: Tom Herbert <[email-masked]>
Date: Sat, 8 Sep 2018 10:32:04 -0700

On Fri, Sep 7, 2018 at 8:32 PM, Joe Touch <[email-masked]> wrote:
> I don’t know what “mandatory to process” means.

Joe,

It means that if the option is present on receive it must either be
processed or the packet must be dropped. Ignoring the option and
delivering the packet to the application is incorrect. An obvious
example of this is the FRAG option. Knowingly delivering an individual
fragment to an application is incorrect. If the FRAG option is present
and the receiver is unable to process it, for instance because the
option length or it's data aren't what's expected, the packet must be
dropped because the receiver still knows it as a fragment based on the
option type (option length and data don't change that fact).

A corollary is that if a receiver cannot reasonably eliminate the
possibility that a "mandatory to process" option was sent then the
packet must be dropped. This can happen if the options are corrupted
so that a mandatory option that was sent, such as FRAG, is no longer
discernible by the receiver. A receiver detects the corruption when it
observes a failed integrity check. For this reason it's incorrect to
ignore OCS, and furthermore checksum should always be present in a fix
header to ensure protocol correctness.

> OCS, LITE, FRAG, NOP, EOL are are always processed in full if valid. TIME and MSS too, but that basically means just passing the info up with the packet. AE and ACS should be computed (IMO) only when the receiver enables those features, otherwise their presence is passed up to the app only (as are all otherwise unknown options). New options would have to decide for themselves what happens.
>
That would raise the question as to how unknown options in fragments
are passed up if only reassembled packets are delivered. On the other
hand, if the app is expected to handle some options, it's just as easy
for them to handle all options. So instead of the OS dealing with the
complexities of UDP options, it can pass them up to the application.
The only thing an OS needs to do is allow applications to set the bits
for sending in the surplus area and provide received bits in ancillary
data. The application can implement options as it pleases which can be
done in a portable userspace library. This model gives a lot of
flexibility and allows for fast iteration to make changes (the QUIC
developers can attest to the benefits of this model).

Tom

---

From: "C. M. Heard" <[email-masked]>
Date: Sat, 8 Sep 2018 11:05:38 -0700

On Sat, Sep 8, 2018 at 10:32 AM Tom Herbert wrote:
> are passed up if only reassembled packets are delivered.

That's already covered in the spec:

   >> UDP options MUST NOT follow the FRAG header in non-terminal
   fragments. Any data following the FRAG header in non-terminal
   fragments MUST be silently dropped. All other options that apply to
   a reassembled packet MUST follow the FRAG header in the terminal
   fragment.

> On the other

That's actually an implementation issue, not a specification issue, isn't
it?

Mike Heard

---

From: Joe Touch <[email-masked]>
Date: Sat, 8 Sep 2018 11:25:00 -0700

> On Sep 8, 2018, at 10:32 AM, Tom Herbert <[email-masked]> wrote:
> 
> processed or the packet must be dropped.

That would make sense only for receivers that decide to configure the socket for that behavior.

Because a legacy receiver would silently ignore all options, it would not be useful to assume that any option would default to “drop” upon receive as default (note: I’m presuming that the options being sent leave a sensible payload when used; FRAG without LITE and encryption would not have that property, and should not be used when legacy behavior is possible).

> Ignoring the option and

We already discussed this; if a sender doesn’t know a receiver is legacy or not, it should not be using FRAG without LITE or encryption.

So there are no “mandatory to process” options with the exception of those two, but they should never be used unless the sender knows that the receiver has agreed. This is then consistent with what I summarized:

	- there are no “mandatory to process” *options* by default
	- “mandatory to process” is a setting that *each* receiver would need to set for a given option, when that behavior is desired

> header to ensure protocol correctness.

Again, we need behavior where legacy receivers and a default for option-aware receivers is the same UNTIL configured otherwise.

This means that corrupt but recoverable options need to be ignored and options that cannot be recovered should not be used unless the sender has some confidence that the receiver will behave otherwise.

> are passed up if only reassembled packets are delivered.

We do need to decide how best to handle that.

> for them to handle all options.

We aren’t passing the options in total up; we’re passing the list of options. This is still a transport protocol; we handle all the options we know about. 

If you want the app to handle all the options, that app would need to be able to send raw IP packets (i.e., the app would implement UDP).

> So instead of the OS dealing with the

You’re describing raw IP and user space UDP, which is an implementation choice.

Joe

---

From: "C. M. Heard" <[email-masked]>
Date: Sat, 8 Sep 2018 11:58:52 -0700

On Fri, Sep 7, 2018 at 8:32 PM Joe Touch wrote:
> There are options that MUST be implemented, i.e., if you find out that
> an endpoint knows NOP, then you can (or should) safely assume it knows
> EOL, etc.

Yes, those are the ones that that are called "required options" in
Section 5 and Section 14 of the draft. In Section 6 that very same
term is used in a different sense. This caused some confusion for me,
not while reading the draft (where the context was clear), but during
the subsequent discussion. For that reason, I'd like to suggest some
editorial changes along the following lines.

Section 5 - OLD:

   >> Required options MUST come before other options. Each required
   option MUST NOT occur more than once (if they are repeated in a
   received segment, all except the first MUST be silently ignored).

   The requirement that required options come before others is intended
   to allow for endpoints to implement DOS protection, as discussed
   further in Section 14
<https://tools.ietf.org/html/draft-ietf-tsvwg-udp-options-05#section-14>.

Section 5 - NEW:

   >> Options that are mandatory to implement (those marked with a

   "*" above) MUST come before other options.  Except for NOP, those

   options MUST NOT occur more than once (if they are repeated in a

received segment, all except the first MUST be silently ignored). The
requirement that mandatory-to-implement options come before others in
intended to allow for endpoints to implement DOS protection, as discussed
further in Section 14
<https://tools.ietf.org/html/draft-ietf-tsvwg-udp-options-05#section-14>.

Section 14 - s/required options/options that are mandatory to implement/

Section 6 - OLD:

   o  Extend the method to create receive ports to include receive
      options that are required. Datagrams not containing these
      required options MUST be silently dropped and MAY be logged.

NEW:

   o  Extend the method to create receive ports to include receive
      options that are required to be present. Datagrams not containing
      these options MUST be silently dropped and MAY be logged.

It would then be helpful if we would use the terms "mandatory to
implement" and "required to be present" (or whatever actually ends
up in the draft) in our discussions.

> I don’t know what “mandatory to process” means. OCS, LITE, FRAG, NOP,
> EOL are are always processed in full if valid. TIME and MSS too, but
> that basically means just passing the info up with the packet. AE and
> ACS should be computed (IMO) only when the receiver enables those
> features, otherwise their presence is passed up to the app only (as
> are all otherwise unknown options). New options would have to decide
> for themselves what happens.

Upon reflection, I see that I have slighted a vital distinction between
what Tom means by "mandatory to process" and what I mean by "unsafe to
ignore." I'll define the latter term below and discuss the relationship.

My understanding is that when Tom says that an option is "mandatory to
process" he means that it MUST be processed when present in a received
datagram, whether or not the application has explicitly marked it as
"required to be present." By that criterion, OCS, LITE, FRAG, NOP, EOL,
and MSS are all “mandatory to process.” IMO it would be incorrect to
characterize TIME as “mandatory to process” because it is not "mandatory
to implement"; a UDP layer that does not implement it would pass it to
the application but would otherwise ignore it (and thus would not use
the information e.g. to compute RTT).

The draft is silent on whether ACS and AE are “mandatory to process.”
Your proposal is that they are not. I agree with that for ACS and for
AE when it is used for authentication only but I would not agree for
AE when it is used for encryption. I understand that Tom advocates
making both of these “mandatory to process.”

What I mean when I say that a UDP option is "unsafe to ignore" is that
the semantics of the data preceded by the option area may depend upon
whether the option is processed or not. By that criterion, any option
that results in some transformation of that data is "unsafe to ignore."
FRAG (other than the special no-op case used for negotiation) is clearly
one such option. So is AE when it is used for encryption. If ACS were
"mandatory to process," as Tom suggests, it would also fall into this
category, since the data preceding the option area could be passed to
the application, or not, depending on whether the ACS check succeeded.
So also for AE even when used just for authentication.

I contend that any UDP option that is "unsafe to ignore" should be
firewalled behind a LITE option in the manner described for FRAG in
section 5.7.1
<https://tools.ietf.org/html/draft-ietf-tsvwg-udp-options-05#section-5.7.1> of
the draft. That protects both an option-aware receiver
in the case of an OCS failure and a legacy receiver, which does not
process UDP options at all.

I further contend that it must be possible for an implementation to
recognize any UDP option that is "unsafe to ignore," irrespective of
whether or not that option is implemented. We should consider changing
the encoding of the option "kind" field to have a bit that indicates
whether or not the datagram needs to be discarded if the option is not
recognized.

> header to ensure protocol correctness.

No, that is true only for "unsafe to ignore" options (which are NOT the
same as "mandatory to process" options). I do agree that if the receiver
cannot reasonably eliminate the possibility that an "unsafe to ignore"
option was sent then the datagram needs to be dropped. BTW, my reading
of the spec is that OCS cannot be ignored, if present.

On Sat, Sep 8, 2018 at 11:25 AM Joe Touch wrote:
> We already discussed this; if a sender doesn’t know a receiver is
> legacy or not, it should not be using FRAG without LITE or encryption.

Yes, but an OCS failure causes even an option-aware receiver
to behave as a legacy receiver, resulting in a gibberish being
delivered to the upper layer if FRAG without LITE or
encryption is used and OCS fails. And as Tom has pointed out,
an attacker sending a spoofed DNS request indicating that FRAG
is supported can cause an option-aware server to send a reply
using FRAG to an option-unaware victim. It's far safer to
always use LITE in connection with such options -- the ones
I call "unsafe to ignore" -- and to provide a means for an
implementation that does not implement them to easily discern
that they are indeed unsafe to ignore.

> Nothing is enforced to be present per se, except as indicated by
> receiver configuration. I.e., all options are optional as a default.

I would support a change that requires OCS to always be present.

>
> Joe
>
>> On Sep 6, 2018, at 12:57 PM, C. M. Heard wrote:

---

From: Tom Herbert <[email-masked]>
Date: Mon, 10 Sep 2018 07:59:25 -0700

On Sat, Sep 8, 2018 at 11:58 AM, C. M. Heard <[email-masked]> wrote:
>    further in Section 14.
>    The requirement that mandatory-to-implement options come before
>    others in intended to allow for endpoints to implement DOS
>    protection, as discussed further in Section 14.
>
>
> section 5.7.1 of the draft. That protects both an option-aware receiver
LITE option isn't the right mechanism for this. LITE's purpose is to
emulate UDP-LITE and we lose the UDP checksum (UDP-LITE has pretty
much be obsoleted by checksum offload anyway). The part about zeroing
the length and swapped payload bytes can be used. Here is an
alternative method:

1. Move first four bytes of payload to length offset (increase packet
length by four bytes)
2. Set UDP header length and UDP header checksum to zero
3. At payload offset zero set the UDP length, at offset two set the checksum

So packet looks something like this:

+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+\\
|        Source port            |      Destination port         | |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ UDP
|             0                 |              0                | |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+/
|           Length              |          Checksum             |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                                                               |
~                Payload after first four bytes                 ~
|                                                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                  First four bytes of payload                  |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

The encoding is detected by a receiver if UDP length and checksum are
both zero so no need for option type/length. The checksum is
calculated in the same manner as normal UDP checksum and it needs to
cover the pseudo header and everything from the start of UDP header
through the length of payload plus four bytes-- this should be
amendable to checksum offload.

> whether or not the datagram needs to be discarded if the option is not
The same option format used in IPv6 HBH and Destination options could
be used (RFC8200). A new ICMP type could be created as "Transport
Option Problem" or something like that.

> I would support a change that requires OCS to always be present.
Which would mean it's no longer an option. I proposed a four byte
header for the surplus area as:

1  type (i.e. UDP options, something else)
1 byte length
2 byte checksum

Tom

---

From: Joe Touch <[email-masked]>
Date: Mon, 10 Sep 2018 08:28:28 -0700

> On Sep 10, 2018, at 7:59 AM, Tom Herbert <[email-masked]> wrote:
> 
> (UDP-LITE has pretty
> much be obsoleted by checksum offload anyway).

Relieving processing load isn’t the purpose of LITE. It is to pass info that would have failed a checksum up to the app to interpret.

Joe

---

From: Joe Touch <[email-masked]>
Date: Mon, 10 Sep 2018 08:31:02 -0700

> The part about zeroing

That fails for all legacy receivers, which is the opposite of the goal of this effort. 

Joe

---

From: Tom Herbert <[email-masked]>
Date: Mon, 10 Sep 2018 08:40:21 -0700

> That fails for all legacy receivers, which is the opposite of the goal of this effort.
That's the point. If the sender includes a "mandatory to process"
option in the list and uses this encoding, then the packet is properly
dropped by a legacy receiver since it can't process manadory options
which is the correct behavior. If there are no manadory to process
options being used, then the sender, at its discretion, can send
options in the surplus space without the encoding. A legacy receiver
will ignore the suplus area which is equivalent to ignoring all the
options. That is correct since none of the options are mandatory to
process.

Tom

---

From: Joe Touch <[email-masked]>
Date: Mon, 10 Sep 2018 12:37:45 -0700

In that case you’re adding nothing to what we can already do. 

Joe

> On Sep 10, 2018, at 8:40 AM, Tom Herbert <[email-masked]> wrote:
>

---

From: Tom Herbert <[email-masked]>
Date: Mon, 10 Sep 2018 12:55:41 -0700

> In that case you’re adding nothing to what we can already do.
This proposal allows a format that allows the UDP length and checksum
field to be zero, but retains both of those in the packet by
effectively moving the the fields. I don't see how this can be
accomplished in the current UDP options draft, if it is possible
please give the specifics on how it can be done.

Tom

---

From: Joe Touch <[email-masked]>
Date: Mon, 10 Sep 2018 14:35:16 -0700

Lite + frag with one fragment does the same thing. It doesn’t force the UDP checksum to be zero though - but it can be.  

Joe

> On Sep 10, 2018, at 12:55 PM, Tom Herbert <[email-masked]> wrote:
>

---

From: Tom Herbert <[email-masked]>
Date: Mon, 10 Sep 2018 15:17:00 -0700

Joe,

I don't think it's the same. The checksum in the last fragment option
covers the whole reassembled packet, there is no checksum for
individual packets that contain fragments. The UDP checksum is defined
to be per each UDP packet. In the case of IPv6, a non-zero checksum is
required in each UDP packet so that it can't be put into an option
anyway.

>From a practical standpoint, Lite+Frag is fourteen bytes of overhead
whereas this encoding here is four bytes. Also, this encoding keeps
the UDP checksum in the front of the packet instead of putting it in
the trailer, this is a benefit since some devices doing checksum
offload limit the start and offset value to 255 (i.e. they are stored
in one byte).

If the UDP length is zero, the checksum should also be zero. Per
RFC768, a UDP packet with length zero is invalid, and so the meaning
of the checksum field is undefined in that case. If a device were to
attempt to verify a non-zero checksum when length is zero the results
are likely to be nondeterministic. It might reasonably interpret the
checksum in one of two ways: 1) checksum is only over the pseudo
header 2) checksum is over the pseudo header and UDP header. Safest
way is to just require UDP checksum field to be zero when length is
zero and have UDP checksum located elsewhere in packet.

Tom

---

From: Joe Touch <[email-masked]>
Date: Mon, 10 Sep 2018 18:03:09 -0700

> On Sep 10, 2018, at 3:17 PM, Tom Herbert <[email-masked]> wrote:
> 
> individual packets that contain fragments.

There is - for the UDP ports and UDP length fields in the individual fragments.

> The UDP checksum is defined

IPv6 makes that requirement, but it has been relaxed in many environments when other checksums are sufficient (e.g., tunnels). I.e., it’s fine to have zero checksum fragments as long as the reassembled whole has a checksum.

> RFC768, a UDP packet with length zero is invalid,

OK, let’s stop there.

You’re trying to optimize 10 bytes, but the net result is a packet that won’t traverse NATs. That’s a non-starter.

We’re always worked under the assumption that:
	- where possible, information should be available to either legacy or option-enabled receivers
	- packets can traverse NATs, firewalls, etc., as they currently do (or should)

Other alternatives aren’t on the table, AFAICT. If you want to create a protocol that won’t traverse a NAT, there are plenty of other transport code points available.

Joe

---

From: Tom Herbert <[email-masked]>
Date: Mon, 10 Sep 2018 18:52:13 -0700

On Mon, Sep 10, 2018 at 6:03 PM, Joe Touch <[email-masked]> wrote:
> Lite + frag with one fragment does the same thing. It doesn’t force the UDP
> checksum to be zero though - but it can be.
>
> Joe,
>
> IPv6 makes that requirement, but it has been relaxed in many environments
> when other checksums are sufficient (e.g., tunnels). I.e., it’s fine to have
> zero checksum fragments as long as the reassembled whole has a checksum.
>

IPv6 checksum has only been relaxed for tunnels under specific
conditions as described in RFC6935 and RFC6936. In no way are these
intended to generally relax the requirement of UDPv6 checksum. The
reason that the checksum is important in IPv6 is to prevent
misdelivery because of address corruption. IPv4 has a header checksum,
where IPv6 does not, so the only validation of addresses is via the
pseudo header of a transport protocol checksum. The fragmentation
checksum is not sufficient since it doesn' t include the pseudo
header. The part where the pseudo header is applied to the checksum
only after reassmbly is basically lying to the application that the
pseudo header was receieved as sent.

> You’re trying to optimize 10 bytes, but the net result is a packet that
> won’t traverse NATs. That’s a non-starter.
>
I'm trying to retain the UDP checksum as part of a UDP packet as well
as trying to be compatible with how devices have implemented it. The
saving of 10 bytes is a side effect.

> - packets can traverse NATs, firewalls, etc., as they currently do (or
If you want to pass through a NAT, then simply set the UDP checksum
field to a fixed non-zero value (maybe 0xffff for convenience). Then
when packet goes through a NAT the checksum will be update accordingly
to offset the change in address.

> Other alternatives aren’t on the table, AFAICT. If you want to create a
> protocol that won’t traverse a NAT, there are plenty of other transport code
> points available.
>
Maybe so, but I don't think arbitrarily changing core requirements
(like UDPv6 really isn't required) should be on the table either.

Tom

---

From: "C. M. Heard" <[email-masked]>
Date: Mon, 10 Sep 2018 19:02:37 -0700

On Mon, Sep 10, 2018 at 6:52 PM Tom Herbert <[email-masked]> wrote:
> IPv6 checksum

I'm  sorry, I'm confused. There is no IPv6 checksum. There was a header checksum
in IPv4 but it was eliminated in IPv6. Please clarify what you mean.

--cmh

---

From: Tom Herbert <[email-masked]>
Date: Mon, 10 Sep 2018 19:09:46 -0700

> in IPv4 but it was eliminated in IPv6. Please clarify what you mean.
Sorry, I meant the UDP checksum for IPv6. Here are the requirements
from RFC8200:

"Unlike IPv4, the default behavior when UDP packets are originated by
an IPv6 node is that the UDP checksum is not optional.  That is,
whenever originating a UDP packet, an IPv6 node must compute a UDP
checksum over the packet and the pseudo-header, and, if that
computation yields a result of zero, it must be changed to hex FFFF
for placement in the UDP header. IPv6 receivers must discard UDP
packets containing a zero checksum and should log the error."

RFC6935 and RFC6936 update this requirement for tunnels using UDP/IPv6
encapsulation that meet certain conditions.

Tom

---

From: Tom Herbert <[email-masked]>
Date: Mon, 10 Sep 2018 19:29:13 -0700

> to offset the change in address.
Actually, it's even simpler than that. Since the meaning of the
checksum field is undefined when length is zero, we can define to be
the normal UDP checksum where the payload length in the calculation
comes from elsewhere in the packet. As long as a NAT isn't looking for
a valid length or trying to verify the checksum, it will just update
the field like it always has. This is more amenable to checksum
offload (some implementations demand checksum to be at offset for
transmit to either UDP ot TCP) and saves two more bytes of overhead.

Tom

---

From: Joe Touch <[email-masked]>
Date: Mon, 10 Sep 2018 19:34:08 -0700

> On Sep 10, 2018, at 7:09 PM, Tom Herbert <[email-masked]> wrote:
> ...
> 

Our FRAG option meets the basic conditions of 6936 Sec 2.2.1 by checksumming the reassembled payload.

The fixed UDP header is useless to worry about checksumming as the checksum is recomputed in many places inside the network anyway.

Joe

---

From: Joe Touch <[email-masked]>
Date: Mon, 10 Sep 2018 19:43:45 -0700

> On Sep 10, 2018, at 6:52 PM, Tom Herbert <[email-masked]> wrote:
> 

You can’t claim compatibility when you start by using a “defined as invalid” packet format.

UDP options is designed to live within the *existing* wiggle room that UDP doesn’t explicitly prohibit. 

The LITE+FRAG mechanism already retains a UDP checksum over the payload after reassembly. There’s no way to usefully “protect” the UDP header over the reassembled packet because of NATs; those headers would already be protected within each fragment if using a regular UDP checksum there. 

Joe

---

From: Joe Touch <[email-masked]>
Date: Mon, 10 Sep 2018 19:49:14 -0700

> On Sep 6, 2018, at 12:01 PM, Tom Herbert <[email-masked]> wrote:
> 
> On Thu, Sep 6, 2018 at 11:54 AM, Joe Touch <[email-masked] <mailto:[email-masked]>> wrote:
>> 
>> 
>>> On Sep 6, 2018, at 11:30 AM, Tom Herbert <[email-masked] <mailto:[email-masked]>> wrote:
>>> 
>>> . The client

To repeat:

>> A positive confirmation is the only way to think anything is ok.

Joe

---

From: Tom Herbert <[email-masked]>
Date: Mon, 10 Sep 2018 20:43:06 -0700

> You can’t claim compatibility when you start by using a “defined as invalid”
The draft already does that when it allows UDP packets to have length
of zero in LITE+FRAG. RFC768 explicitly states "This means the minimum
value of the length is eight.". If an intermediate device decides to
drop packets because UDP length is zero there's little we can do about
it.

Tom

> The LITE+FRAG mechanism already retains a UDP checksum over the payload
> after reassembly. There’s no way to usefully “protect” the UDP header over
> the reassembled packet because of NATs; those headers would already be
> protected within each fragment if using a regular UDP checksum there.
>
> Joe
>

---

From: "C. M. Heard" <[email-masked]>
Date: Mon, 10 Sep 2018 20:50:25 -0700

On Mon, Sep 10, 2018 at 7:59 AM Tom Herbert wrote:

> On Sat, Sep 8, 2018 at 11:58 AM, C. M. Heard wrote:
>
> |        Source Port            |      Destination Port         | |
>
> ~               Payload after first four bytes                  |
>
> |                                                               |
>

I'm sorry, I fail to see how that is better than the proposal in the draft,
i.e.,

|        Source Port            |      Destination Port         | |

|       UDP Length = 8          |        UDP Checksum           | |

|       4       |       4       |        Option Offset          | LITE Option

|                                                               |

~               Payload after first four bytes                  |

|                                                               |

|                                                               |

~                 UDP Options other than LITE                   |

|                                                               |

Differences:

   - Tom's proposal has an invalid UCP length; the proposal in the draft
   does not. That may or may not matter, depends on what middleboxes do (we
   definitely need a real-world study to see what passes unmolested through
   common middleboxes and what does not).
   - Tom's proposal does not protect the UDP header with a checksum; the
   proposal in the draft does.

IMO, it's superior to Tom's proposal. I don't see why he says "LITE option
isn't the right mechanism for this." Maybe LITE is not quite the right name
-- it does have other uses -- but the option as defined will work just fine.

> Option Problem" or something like that.

Seems like ICMP is the wrong layer for such signaling.

> 1  type (i.e. UDP options, something else)

I am not necessarily opposed to such a proposal, but what I see in the
diagram above does not pass muster.

Mike Heard

---

From: Tom Herbert <[email-masked]>
Date: Mon, 10 Sep 2018 20:51:27 -0700

> transmit to either UDP ot TCP) and saves two more bytes of overhead.
Taking this to it's logical conclusion, the checksum can cover the
surplus area. So the the rule would be "If the UDP length is zero, UDP
checksum covers the pseudo header, UDP header, UDP payload, and all of
the surplus area". This meets all the requirements:

- Packets are dropped by lagacy receivers since they will see UDP
length is invalid.
- The canonical UDP is easily derived so the requirements that UDP
checksum is supported and the requirement that UDP checksum is enabled
for IPv6 is maintained.
- Works properly through NAT translation.
- Eliminates the need for a separate checksum in options area (no OCS
or checksum in an option header needed).
- Is compatible with checksum offload, so checksum over the whole
packet is essentially free on supporting devices.
- Incurs a mere two byte overhead.

Tom

---

From: Tom Herbert <[email-masked]>
Date: Mon, 10 Sep 2018 20:59:15 -0700

On Mon, Sep 10, 2018 at 8:50 PM, C. M. Heard <[email-masked]> wrote:
> Tom's proposal has an invalid UCP length; the proposal in the draft does
> not.

Unless I'm misreading it, section 5.7.1 is defining a packet format
where UDP length field is set to zero (an invalid length for UDP per
RFC768)  i.e.:

+---------+--------------+---------+
| UDP Hdr |   LiteFrag   |LITE|FRAG|
+---------+--------------+----+----+
<-------> ^^^^           ^^^^
Zero UDP Length  |              |
                             +--------------+

This technique actually makes sense because lagacy receivers will drop
such packets which is what we want to happen when madatory to process
options are present.

> That may or may not matter, depends on what middleboxes do (we
> Tom's proposal does not protect the UDP header with a checksum; the proposal
> in the draft does.

See the updated version of my proposal.

Tom

---

From: "C. M. Heard" <[email-masked]>
Date: Mon, 10 Sep 2018 21:02:30 -0700

On Mon, Sep 10, 2018 at 8:59 PM Tom Herbert <[email-masked]> wrote:
> where UDP length field is set to zero

You are misreading it. if LITE appears immediately after the UDP
header, the UDP length is set to 8.

--cmh

---

From: Joe Touch <[email-masked]>
Date: Mon, 10 Sep 2018 21:08:58 -0700

They have a payload length of zero. Not a total length of zero.

Joe

> On Sep 10, 2018, at 8:43 PM, Tom Herbert <[email-masked]> wrote:
>

---

From: Tom Herbert <[email-masked]>
Date: Mon, 10 Sep 2018 21:10:09 -0700

> header, the UDP length is set to 8.
Possibly, it's really not clear in the draft (we really need a
canonical diagram here).

If that is true, then those packets are accepted an sent up to the
application. That would presume that there are no side effects in
doing that.

Tom

---

From: Joe Touch <[email-masked]>
Date: Mon, 10 Sep 2018 21:11:33 -0700

> On Sep 10, 2018, at 8:51 PM, Tom Herbert <[email-masked]> wrote:
> 

Nope. They will see an invalid header and drop. So might any fireball that parses udp ports.

Joe

---

From: Joe Touch <[email-masked]>
Date: Mon, 10 Sep 2018 21:12:22 -0700

Agreed. It should say the payload is zero length.

> On Sep 10, 2018, at 9:02 PM, C. M. Heard <[email-masked]> wrote:
>

---

From: Joe Touch <[email-masked]>
Date: Mon, 10 Sep 2018 21:13:06 -0700

> On Sep 10, 2018, at 9:10 PM, Tom Herbert <[email-masked]> wrote:
> 

That is correct.

---

From: Tom Herbert <[email-masked]>
Date: Mon, 10 Sep 2018 21:18:04 -0700

> Agreed. It should say the payload is zero length.

"Zero UDP Length" implies UDP Length field is zero. I think you mean
to say that the UDP length field must be 8.

---

From: Tom Herbert <[email-masked]>
Date: Mon, 10 Sep 2018 21:23:00 -0700

> Nope. They will see an invalid header and drop. So might any fireball that parses udp ports.
How do you know they do that? Linux netfilter does not for instance.

Tom

---

From: Joe Touch <[email-masked]>
Date: Mon, 10 Sep 2018 21:30:18 -0700

> On Sep 10, 2018, at 9:18 PM, Tom Herbert <[email-masked]> wrote:
> 

The udp payload is zero, which means it’s total length is 8. Yes.

Joe

---

From: Joe Touch <[email-masked]>
Date: Mon, 10 Sep 2018 21:31:11 -0700

> On Sep 10, 2018, at 9:23 PM, Tom Herbert <[email-masked]> wrote:
> 

I don’t - but we’re not designing for changes that are already prohibited by 768.

Joe

---

From: Tom Herbert <[email-masked]>
Date: Mon, 10 Sep 2018 21:38:38 -0700

> The udp payload is zero, which means it’s total length is 8. Yes.
Very well. I believe my proposal is still valid in this case and can
be reworded as "When the UDP length is set to eight and UDP options
are present, then the UDP checksum covers the pseudo header, UDP
header, UDP payload, and surplus area". This has the additional
benefit in that the checksum will be seen as wrong by a legacy
receiver and so the packet would be dropped instead of being accepted.

Tom

---

From: Joe Touch <[email-masked]>
Date: Tue, 11 Sep 2018 00:00:10 -0700

You haven’t described a problem that is:
A) not solved already by the existing proposal
B) will still work in the presence of NATs

Further, you can’t know there are udp options are sent until you parse the packet, so are you intending to act on packet data before checking whether it is corrupt? That doesn’t seem safe or correct.

Remember that out primary case is not dual-stack because there will be no way to prevent an upgraded host from acting on the legacy format. So either a packet needs to support both legacy and options as a single message or the endpoints have coordinated and it is safe to send only the new format.
 
I see no reason to further waste time on this.

Joe

> On Sep 10, 2018, at 9:38 PM, Tom Herbert <[email-masked]> wrote:
>

---

From: Tom Herbert <[email-masked]>
Date: Tue, 11 Sep 2018 07:57:57 -0700

> I see no reason to further waste time on this.
The draft makes the implicit assumption that receiving packets with a
zero length that were not sent as such is innocuous. You need to prove
that assumption is always valid it you want to call the protocol
robust. If the assumption turns out to be incorrect, and the protocol
gets deployed and this causes problems, then it's people like me who
work with real systems that have to debug this. Instances where
packets change erroneously inflight, but otherwise look correct at the
receiver are among the hardest problems to address.

Tom

---

From: Derek Fawcus <[email-masked]>
Date: Tue, 11 Sep 2018 16:23:21 +0100

> B) will still work in the presence of NATs

Given that all NATs I'm aware of perform incremental checksum update,
with this version (UDP length field is 8) it should be able to pass all
NATs.  That might need to be verified, but failure strikes me as unlikely.

I interpreted his proposal as:

Old host, receiving such a (udp-length-field == 8) packet from an old host, without
surplus data would act properly as the checksum is still valid (when not zero).

Old host, receiving such a (udp-length-field == 8) packet from an old host, but with
surplus data, then in current undefined area.

Old host, receiving his scheme (udp-length-field == 8) packet from such a new host,
with options in the surplus area, would be dropped by an old host unless the checksum
over all of pseudo-header, udp-header, surplus data, just happened to equal the
checksum in the first case above.  The sender could check for this situation, and
alter the data in the surplus to avoid it.

> Further, you can’t know there are udp options are sent until you parse the packet, so are you intending to act on packet data before checking whether it is corrupt? That doesn’t seem safe or correc…

I suspect he intends to allow the receiver to perform the checksum over the packet
(including the surplus) before then trying to parse the surplus as udp options.
So he avoids the issue to refer to.

I'm not sure what he has in mind for the case of UDP being sent with legacy visible
data, and UDP options present.  Maybe he views those as being restricted to a subset
of options which are not mandatory to process and act upon.

DF

---

From: Tom Herbert <[email-masked]>
Date: Tue, 11 Sep 2018 08:38:02 -0700

> surplus data, then in current undefined area.
Old host to old host behavior should need to change.

> alter the data in the surplus to avoid it.

Right. That's a 1/65K possibility that checksums would happen to match
and zero length packet would be accepted.
> So he avoids the issue to refer to.
Yes. Checksum also covers UDP options, so if those are corrupted
packet is dropped.

> of options which are not mandatory to process and act upon.

Right. In that case the options don't contain mandatory to process
ones. The option area still needs its own checksum in that case.

Tom

---

From: "C. M. Heard" <[email-masked]>
Date: Tue, 11 Sep 2018 09:40:43 -0700

> NATs.  That might need to be verified, but failure strikes me as unlikely.

I sent this comment off-list to Joe and Tom, but I see that was a mistake.

The issue with Tom's revised proposal is not that it won't work in the
presence of NATs (the incremental checksum update will work just fine),
but rather the fact that it has a good chance of failing in the presence
of firewalls. Based on a survey of on-line product literature, it seems
that it is common for existing commercial firewalls to drop UDP
datagrams with invalid checksums by default. Being legacy devices,
these firewalls will consider the checksum to be invalid if it covers
the pseudo header, UDP header, UDP payload, and surplus area.

On the other hand, none of the firewalls that I looked at drop packets
with UDP length < IP payload length as a default. Most check only for
packets with UDP length > IP payload length.

So I have to agree with Joe that Tom's proposal does not solve
any problems not already addressed by what is currently in
draft-ietf-tsvwg-udp-options-05, and it does seem to have at least
one additional downside. It also adds another code path.

Mike Heard

---

From: Tom Herbert <[email-masked]>
Date: Tue, 11 Sep 2018 10:09:47 -0700

On Tue, Sep 11, 2018 at 9:40 AM, C. M. Heard <[email-masked]> wrote:
>> NATs.  That might need to be verified, but failure strikes me as unlikely..
>
Mike,

The problem with the proposal in the draft is that it is not robust.
The expectation for the past forty years of UDP is that if the sending
application sends a message of 100 bytes, then the receiving
application receives a message with those same one hundred bytes. The
UDP checksum is used to validate the packet is correct. This draft
allows scenarios in UDP where the application sender might send 100
bytes, but the application receiver sees zero bytes. That is data
corruption. If the checksum doesn't catch it, then it is silent data
corruption. I claim a protocol that allows silent data corruption is
not robust. As I said, the draft implicitly assumes all this is
innocuous.

In practice, it will create problems. Consider that UDP options are
enabled in a datacenter and some unsuspecting legacy production
application instance starts receiving massive numbers of zero length
messages. This is something that has not previously been seen. THE
APPLICATION OWNER WILL NOTICE THIS, AND THEY WILL DEBUG THIS! They
will assume there are three possible causes: 1) They are under a DOS
attack 2) The sending application is broken 3) The OS or something in
the network is broken. Notice that a change to the UDP protocol is not
on the list. This will take a long time for them to debug, and once
they figure out the problem they will not be at all happy that we
changed the protocol underneath them.

Protocol robustness trumps compatibility with firewalls.

Tom

---

From: Derek Fawcus <[email-masked]>
Date: Tue, 11 Sep 2018 19:08:08 +0100

On Tue, Sep 11, 2018 at 09:40:43AM -0700, C. M. Heard wrote:
> datagrams with invalid checksums by default.
[snip]

Interesting, I wasn't aware of such.

I'm a little surprised, but if they're  newish devices (and hence able to
use hw checksum offload), not too surprised.  I'd not have expected pure
s/w devices to take this approach, for the same reason NATs use incremental
update - it avoids referencing the rest of the packet, which can kill
forwarding performance.

> with UDP length < IP payload length as a default.

That was the case I was expecting might have been dropped by firewalls,
maybe under a configurable option, as it represents a covert channel.

DF

---

From: Joe Touch <[email-masked]>
Date: Tue, 11 Sep 2018 11:25:32 -0700

> On Sep 11, 2018, at 10:09 AM, Tom Herbert <[email-masked]> wrote:
> 

The only thing that might happen is receipt of an empty packet. If that is a concern, then don’t use hidden payloads. However, IMO it should be robust because to legacy receivers at most it signals that the endpoint is there - which is correct. 

The only errors would be if an apps own doing if they break the Postel Principle (unexpected but valid is not an indication if error).

I agree this could break some things, but IMO it is less likely to break something with a valid but perhaps unexpected packet than one that is already defined as invalid. 

Joe

---

From: Tom Herbert <[email-masked]>
Date: Tue, 11 Sep 2018 12:13:35 -0700

> The only thing that might happen is receipt of an empty packet. If that is a concern, then don’t use hidden payloads. However, IMO it should be robust because to legacy receivers at most it signals…
It would not surprise me at all if some applications interpret recvmsg
returning zero bytes to mean the UDP flow is disconnected which would
result is sockets being closed.

> The only errors would be if an apps own doing if they break the Postel Principle (unexpected but valid is not an indication if error).
That's not how it will be perceived in the real world. If an existing
production application has been running perfectly fine for ten years
and someone starts sending UDP options breaking the application in
some way, then the problem _is_ UDP options. Application owners are
far more interested in keeping their apps running then getting
availability of new protocol features like UDP options. So, if there's
ever any question whether a new feature might be hurting applications,
the default action is to disable the new feature. Once feature is
turned off, it's hard to get it turned back on. I've seen this story
run time and time again.

> I agree this could break some things, but IMO it is less likely to break something with a valid but perhaps unexpected packet than one that is already defined as invalid.
Maybe, maybe not. But, if you want to see UDP options ever get
significant deployment make sure it doesn't ever break existing
applications.

Tom

---

From: "C. M. Heard" <[email-masked]>
Date: Tue, 11 Sep 2018 12:20:01 -0700

> The problem with the proposal in the draft is that it is not robust.

On the contrary, it is quite robust.

> application receives a message with those same one hundred bytes.
> The UDP checksum is used to validate the packet is correct.

Indeed, and that will continue to be the case if, as required by Section 10
of the draft, an option-aware endpoint refrains from using any option that
cannot be safely ignored until and unless it has been determined, either by
negotiation of prior arrangement, that the the other end of the connection
supports that option.

> This draft allows scenarios in UDP where the application sender might
> send 100 bytes, but the application receiver sees zero bytes. This draft
> bytes, but the application receiver sees zero bytes.

That is not something that will routinely happen if the use of LITE with
a data area of greater than zero bytes is negotiated or arranged in advance
with the remote peer.

It is true, as you have pointed out, that a third-party attacker may, by
using forged packets, cause an option-aware host to send a sequence of
apparently empty datagrams to a legacy victim host. As far as I can see,
that's no worse than what can happen today.

> That is data corruption. If the checksum doesn't catch it, then it is
> silent data corruption. I claim a protocol that allows silent data
> corruption is not robust. As I said, the draft implicitly assumes all
> this is innocuous.

Like Joe, I do not share your level of concern about empty datagrams
being presented to a legacy receiver. But I do also would not mind
seeing Section 10 strengthened to be more explicit about requiring
negotiation or prearrangement before using any UDP options that
affect the data that will be delivered to an application. It would
be good to explicitly list which options are safe to send without
negotiation or prearrangement and which ones are not.

> changed the protocol underneath them.

I fail to see  where "massive numbers of zero length messages" would
come from if rule "don't send it unless you know that the receiver can
take it" is strictly followed.

> Protocol robustness trumps compatibility with firewalls.

No, actually, it doesn't. They are both vital. If we can't design a
robust protocol than stands a decent chance of deployment, we
have lost the game.

Mike Heard

---

From: Tom Herbert <[email-masked]>
Date: Tue, 11 Sep 2018 13:00:11 -0700

Mike,

There's no protocol to enforce that rule. In a giant data centers of
upwards of hundreds of thousands of hosts there is no simply way to
esure such a rule is followed. There are no flags days in the DC and
misconfiguation isn't exactly non-existent. At scale we've seen
problems pop up that are even less likely to occur. If there was a
real negotiation mechanism in the protocol (not the weak, insecure
attempt to negotiate FRAG) then that might be a different story. But
short of adding a TCP like handshake to UDP I don't know how to do
that. Would be nice though, all this business about legacy receivers
would just go away.

Tom

---

From: Joe Touch <[email-masked]>
Date: Tue, 11 Sep 2018 13:03:24 -0700

> On Sep 11, 2018, at 12:20 PM, C. M. Heard <[email-masked]> wrote:
> 

Agreed. 

Joe

---

From: Joe Touch <[email-masked]>
Date: Tue, 11 Sep 2018 13:05:11 -0700

> On Sep 11, 2018, at 1:00 PM, Tom Herbert <[email-masked]> wrote:
> 
> But
> that.

That is what is required to use some of these options safely, but it need not be hard state (soft is ok) and it need not be in-band.  

Joe

---

From: Joe Touch <[email-masked]>
Date: Tue, 11 Sep 2018 13:10:32 -0700

> On Sep 11, 2018, at 12:13 PM, Tom Herbert <[email-masked]> wrote:
> 

Well that would be wrong for two reasons:

1) UDP doesn’t have connectors. 

2) getting a packet on a connection should never imply a connection is already disconnected (it can be explicit as per a FIN or RST in TCP.)

But yes, all approaches come with some risk. The goal is to meet minimize that risk in sensible ways. To me that starts with staying within what the protocol already allows. 

Joe

---

From: "C. M. Heard" <[email-masked]>
Date: Tue, 11 Sep 2018 13:22:43 -0700

On Tue, Sep 11, 2018 at 1:00 PM Tom Herbert wrote:
> On Tue, Sep 11, 2018 at 12:20 PM, C. M. Heard  wrote:

The use cases do need to be looked at. For the DNS scenario I described
a few days ago the FRAG negotiation -- which is client to server only and
on a transaction-by-transaction basis -- is not weak or insecure. I think
that I adequately addressed your objections in that particular case.

Mike Heard

---

From: Tom Herbert <[email-masked]>
Date: Tue, 11 Sep 2018 13:29:59 -0700

> Well that would be wrong for two reasons:
There's absoutely nothing wrong with this at all. UDP makes no
requirements as to what content is in the payload and how that payload
is interpreted. If an application wants to interpret a zero length
message as end of communication it can do that. If an application
wants interpret it as a signal to have the server self-destruct server
when receiving a zero length message that's its prerogative. Again, if
applications break because someone starts using UDP options, the fault
is in UDP options, not in the application.

> But yes, all approaches come with some risk. The goal is to meet minimize that risk in sensible ways. To me that starts with staying within what the protocol already allows.

I obviously disagree with what constitutes "sensible". Dropping the
packet instead of deliver something incorrect seems more sensible to
me.

Tom

---

From: Joe Touch <[email-masked]>
Date: Tue, 11 Sep 2018 13:43:33 -0700

On Sep 11, 2018, at 1:29 PM, Tom Herbert <[email-masked]> wrote:

> message as end of communication it can do that.

It can. You said disconnected - I took that to mean unreachable, not a signal *to* disconnect.

Joe

---

From: Lloyd Wood <[email-masked]>
Date: Mon, 17 Sep 2018 04:20:31 +0000 (UTC)

"I claim a protocol that allows silent data corruption is
not robust."

With its zero-checksum option, UDP has never been robust. 
Lloyd Wood [email-masked] http://about.me/lloydwood 

________________________________
From: Tom Herbert <[email-masked]>
To: C. M. Heard <[email-masked]> 
Cc: tsvwg <[email-masked]>
Sent: Wednesday, 12 September 2018, 3:10
Subject: Re: [tsvwg] Terminology for UDP options (was: Checksums in UDP options)

On Tue, Sep 11, 2018 at 9:40 AM, C. M. Heard <[email-masked]> wrote:
> I sent this comment off-list to Joe and Tom, but I see that was a mistake..
>
Mike,

Tom

---

From: Tom Herbert <[email-masked]>
Date: Mon, 17 Sep 2018 09:23:11 -0700

> Lloyd Wood [email-masked] http://about.me/lloydwood

Lloyd,

UDP checksum is optional by the sender (in IPv4 at least), it's never
optional at the receiver. If a UDP checksum is non-zero then a
receiver MUST validate the checksum before accepting the packet.

My statement was more about protocols that knowingly or encourage
allow silent data corruption even when a checksum is present.

Tom

---

From: Lloyd Wood <[email-masked]>
Date: Tue, 18 Sep 2018 03:14:06 +0000 (UTC)

RFC6935 and 6936 make the UDP checksum optional in v6 as well.

security directorate might have a view on how corruption

is undesirable, once it's rephrased to them as an attack...
robustness has different interpretations.

L.

Ethernet CRCs are insecure, and therefore bad. Don't use CRCs! 
Lloyd Wood [email-masked] http://about.me/lloydwood 

________________________________
From: Tom Herbert <[email-masked]>
To: Lloyd Wood <[email-masked]> 
Cc: C. M. Heard <[email-masked]>; tsvwg <[email-masked]>
Sent: Tuesday, 18 September 2018, 2:23
Subject: Re: [tsvwg] Terminology for UDP options (was: Checksums in UDP options)

Lloyd,

Tom

---

From: Joe Touch <[email-masked]>
Date: Tue, 18 Sep 2018 23:46:20 -0700

> On Sep 17, 2018, at 8:14 PM, Lloyd Wood <lloyd.wood=[email-masked]> wrote:
> 
> is undesirable, once it's rephrased to them as an attack…

Checksums never check for attacks. They’re trivial for an attacker to generate correctly.

Joe

---

From: Lloyd Wood <[email-masked]>
Date: Thu, 20 Sep 2018 04:26:00 +0000 (UTC)

Well, CRCs aren't checksums...

My take is that a checksum is what shown-to-be-insecure
shown-to-be-subject-to-duplicates security hash function
can end its life as.

The hash has to have a unique input and match, the
checksum (or CRC, whose input matches increase with size of the
input data it computes across) is happy with a very
small set of matches.

(MD5 starting life as a hash function and ending life
as a very useful cheap-to-compute checksum for very large
files is a very good example of this. 
https://en.wikipedia.org/wiki/MD5)

Security code gets repurposed for checksumming,
eventually.

But convincing security types that checksums
still have any useful purpose despite being insecure
is a lost cause -- because they're insecure.

L.

________________________________
From: Joe Touch <[email-masked]>
To: Lloyd Wood <[email-masked]> 
Cc: Tom Herbert <[email-masked]>; C. M. Heard <[email-masked]>; tsvwg <[email-masked]>
Sent: Wednesday, 19 September 2018, 16:46
Subject: Re: [tsvwg] Terminology for UDP options (was: Checksums in UDP options)

Joe

---

From: Joe Touch <[email-masked]>
Date: Wed, 19 Sep 2018 21:45:41 -0700

> On Sep 19, 2018, at 9:26 PM, Lloyd Wood <[email-masked]> wrote:
> 

That reflects only the ignorance of those doing such. Protocol design cannot fix those types of flaws. 

Joe

---

From: [email-masked]
Date: Thu, 20 Sep 2018 13:17:57 +0000 (UTC)

the street finds its own uses for things

udp wasn't designed for gaming, video, or multicast tv, yet here we are.

Lloyd [email-masked]

Joe

---

From: Tom Herbert <[email-masked]>
Date: Thu, 20 Sep 2018 07:28:24 -0700

> is a lost cause -- because they're insecure.
LLoyd,

Who is trying to convince them of that? It's common knowledge that 1's
complement checksum is a weak form of integrity check and was _never_
intended to a be a security mechanism. But it's been fully deployed in
TCP, UDP, and IP for the past forty years and is well known. If you're
suggesting that UDP (or TCP or IP) checksum be replaced by MD-5, which
by the way has been shown to be insecure, then I think that proposal
will fall on deaf ears.

If a stronger data integregity check or security is needed in UDP then
there are several options. DTLS is defined for use with UDP, ULPs
running over UDP (like QUIC) can implement their own security. UDP
options is a nice addition that includes a CRC and header
athentication. The issue concerning the latter is rather CRCs,
checksums, authentication options that are set as UDP options MUST be
validated by a receiver before accepting a packet. As I've stated, I
believe they should be "mandatory to process" as described earlier.
CRCs and checksum can only be optional at a sender, not a receiver.

Tom