Subject: [tsvwg] Checksums in UDP options

From: Tom Herbert <[email-masked]>
Date: Tue, 4 Sep 2018 12:35:52 -0700

A few comments on checksums in UDP options:

>From the draft "Note that OCS is direct, i.e., it is not negated or
adjusted if zero (unlike the Internet checksum as used in IPv4, TCP,
and UDP headers)."

Since this use is inconsistent with other typical uses of checksum, it
would be nice to demonstrate the process to verify the checksum. Note
that when the checksum is negated, all one has to do is a one's
complement sum over the data and check is result is zero.

>From the draft: "When present, the option checksum SHOULD occur as
early as possible, preferably preceded by only NOP options for
alignment and the LITE option if present."

- and -

>From the draft: "If the option checksum fails, all options MUST be
ignored and any trailing surplus data (and Lite data, if used)
silently discarded."

That latter statement implies that the option checksum MUST be
processed before other options regardless of where it is in the option
list which is not guaranteed to be first one as the position
requirement is only a SHOULD. So in an implementation this means the
options must be scanned for OCS before processing any of them. As I've
mention previously, I believe that a simple four byte header that
includes checksum and length avoids a lot of issues and complexity
like this.

>From the draft about ACS: "Its purpose is to detect errors that the
UDP checksum might not detect.". This should read "Its purpose is to
detect errors in the UDP payload that the UDP checksum might not
detect." since it doesn't include pseudo header or UDP header.

What is the behavior if ACS verification fails? (that needs to be
specified). Assuming the intended behavior is to silently drop the
packets, wouldn't that be inconsistent with behavior when OCS fails
which is to accept the packet?

Tom

---

From: Joe Touch <[email-masked]>
Date: Tue, 04 Sep 2018 13:05:35 -0700

Hi, Tom, 

> complement sum over the data and check is result is zero.

That's actually part of the reason why I didn't want to use the the
typical one. It's far too easy to confuse the result with "oh, there's
nothing there" if you're not careful. 

> requirement is only a SHOULD.

It only means that any effect of processing the other options has to be
"undone". That's why it's a SHOULD not a MUST. At least one possible
reason for OCS not being first is the LITE option, which is easy to
"undo". 

> So in an implementation this means the

It creates complexity and overhead in other cases, too. I've already
addressed that. 

> specified).

Agreed. It means the entire packet is treated as a UDP checksum error
would have. 

> Assuming the intended behavior is to silently drop the

No - OCS is a check on the option space, which is checked separately.
There are several reasons why the CS/ACS would succeed but OCS might
fail, and in those cases we want the behavior to revert to "UDP without
options". 

Joe

---

From: Tom Herbert <[email-masked]>
Date: Tue, 4 Sep 2018 14:19:22 -0700

On Tue, Sep 4, 2018 at 1:05 PM, Joe Touch <[email-masked]> wrote:
> That's actually part of the reason why I didn't want to use the the typical
> one. It's far too easy to confuse the result with "oh, there's nothing
> there" if you're not careful.

Sorry, I don't understand what you're saying here. I don't know what
"oh, there's nothing there" means. It's not like the UDP checksum
field is necessarily set zero. Also, we have almost forty years of
deployment experience with IP checksums-- if there there is a problem
with them can you provide more details on exactly what the problem is
and whom it's hurting.

> "undone". That's why it's a SHOULD not a MUST. At least one possible reason
> for OCS not being first is the LITE option, which is easy to "undo".
>
>
>

I don't think so. The possibility a corrupted OCS type field still
need to be discussed in the draft.

> Agreed. It means the entire packet is treated as a UDP checksum error would
> have.
>
That's means it's silently drop packets. Please be explicit on this
for next version of the draft.

> No - OCS is a check on the option space, which is checked separately. There
> are several reasons why the CS/ACS would succeed but OCS might fail, and in
> those cases we want the behavior to revert to "UDP without options".
>

That leads to some really bizzaire behaviors.

Consider that someone enables both the OCS and ACS because they know
they are communicating over a path that can introduce errors and
corrupt packets. Now a packet is received that is corrupted in both
the options area and UDP payload. The check on OCS fails, so according
to the draft the other options, including ACS, are ignored and the
packet is accepted even though CRC check would have dopped it.
Essentially, that means that a corrupted packet was accepted _because_
a checksum verification failed! That fundamentally contradicts the
purpose of checksums and CRCs.

Even better, if you combine this with the requirement above that
options must be undone when OCS fails, it's possible that a node would
process the ACS first and detect that the packet should be dropped.
But then later on the OCS is processed and if that fails then the
processing of ACS and other options needs to be undone. But, undoing
the ACS processing would mean that the packet that is known to be
corrupt now has to be accepted. As I said, this is bizzaire behavior!

Tom

---

From: Joe Touch <[email-masked]>
Date: Tue, 04 Sep 2018 14:50:11 -0700

Hi, Tom,

On 2018-09-04 14:19, Tom Herbert wrote:

> "oh, there's nothing there" means.

I didn't want a solution where "all zeroes" was a valid checksum/body
combination. 

E.g., if the length field of UDP is incorrectly updated to match the IP
length, I don't want the UDP checksum to remain correct (which it would
be in if specified as you suggest).  

> It's not like the UDP checksum

Those 40 years include issues such as the following: 
Partridge, Craig, Jim Hughes, and Jonathan Stone. "Performance of
checksums and CRCs over real data." _ACM SIGCOMM Computer Communication
Review_. Vol. 25. No. 4. ACM, 1995. 

That potential for incorrect reassembly also exists with IPv4 when IDs
are cycled too quickly, in addition to the issue above. 

> need to be discussed in the draft.

I can add that text; AFAICT: 
- if OCS is expected and does not appear, then the packet body should
still be accepted if possible 
- if OCS is interpreted in the wrong place, presumably the checksum
would fail 

>> those cases we want the behavior to revert to "UDP without options".

FWIW, this is already the current text. 

> purpose of checksums and CRCs.

We already talk about soft state. I can add other expectations, e.g., a
receiver can be configured or told to do anything specific, and then can
require any of these options and drop when that requirement is not met. 

The above is only the default behavior, which is required as stated. You
can't force a legacy endpoint to drop based on ACS failing when it might
not understand options at all. 

> corrupt now has to be accepted. As I said, this is bizzaire behavior!

If OCS fails, you're telling me that the option area was corrupt. So it
doesn't matter what ACS said - ACS is part of the option space and has
to be ignored. 

Again, in that case, for legacy receivers, you want the packet to get
through. For a preconfigured receiver that demands ACS, the packet would
be dropped - because OCS failing means that there was, in effect, no
ACS. 

Joe

---

From: Tom Herbert <[email-masked]>
Date: Tue, 4 Sep 2018 15:08:16 -0700

On Tue, Sep 4, 2018 at 2:50 PM, Joe Touch <[email-masked]> wrote:
> length, I don't want the UDP checksum to remain correct (which it would be
> in if specified as you suggest).
>
>
>
> Partridge, Craig, Jim Hughes, and Jonathan Stone. "Performance of checksums
> and CRCs over real data." ACM SIGCOMM Computer Communication Review. Vol.
> 25. No. 4. ACM, 1995.
>
Joe,

Can you be more specific? Maybe a reference to problems and data that
are more recent?

> That potential for incorrect reassembly also exists with IPv4 when IDs are
> cycled too quickly, in addition to the issue above.
>
Sorry, I don't see it. IP IDs could only possibly affect IPv4 checksum
and I really don't understand how they can creates a problem for
checksum computation. It's just another field that is summed.

> - if OCS is expected and does not appear, then the packet body should still
> be accepted if possible

What does "OCS is expected" mean?

> - if OCS is interpreted in the wrong place, presumably the checksum would
> fail

What does "interpreted in the wrong place" mean?

> can't force a legacy endpoint to drop based on ACS failing when it might not
> understand options at all.
>
>
> doesn't matter what ACS said - ACS is part of the option space and has to be
> ignored.
>
> through. For a preconfigured receiver that demands ACS, the packet would be
> dropped - because OCS failing means that there was, in effect, no ACS.
>
Which again, I think is very odd and counterintuitive behavior. My
recommendation is to use the header with checksum, require the UDP
options header checksum to be set and validated, and drop the packet
if any checksum or CRC fails. Legacy receivers that don't support UDP
options will accept such packets, but those that do support them will
properly handle checksums in a consistent manner with other protocols.
This will be a far more robust solution.

Tom

---

From: Joe Touch <[email-masked]>
Date: Tue, 04 Sep 2018 16:08:39 -0700

On 2018-09-04 15:08, Tom Herbert wrote:

>> ...

The issue is reassembly of fragments, regardless of the source (ATM, IP
tunneling, etc.). It has been known for 23 years so it's not like any
venue is going to publish a more recent discovery of it. 

> Sorry, I don't see it.

IP ID cycling can cause fragment reassembly errors. If the UDP data
portion is the same length and if the OCS sums to zero, it's possible
that OCS won't catch reassembly of different options with different
payloads. 

>> ...

An endpoint configured to require OCS on a given UDP association. 

> What does "interpreted in the wrong place" mean?

Corruption could do three things to OCS: 
1) wipe out the OCS KIND value (so there would be no OCS to fail) 
2) change either the checksum or the rest of the header (OCS would
presumably fail) 
3) change a KIND somewhere else to "OCS" (at which point that "false"
OCS would presumably fail) 

I was speaking above of #3. 

> ... for legacy receivers, you want the packet to get
> Which again, I think is very odd and counterintuitive behavior.

A receiver has one of the following behaviors: 

- doesn't know about options 
    at which point we require that the packet body be received and
options ignored, for legacy support 

- knows about options, but doesn't require ACS 
    at which point ACS'd packets that are correct will get through, 
    ACS'd packets with errors that ACS catches will not, 
    and non-ACS'd packets with errors that are not caught by the UDP CS
will get through (including those whose ACS kind is corrupted) 

    which is correct behavior, because the receiver didn't care 

- knows about options and requires ACS 
    at which point only ACS'd packets that are correct will get through 
    i.e., if OCS fails, then the packet doesn't have ACS and will be
dropped 

This is predictable and robust. 

Joe

---

From: Tom Herbert <[email-masked]>
Date: Tue, 4 Sep 2018 19:31:47 -0700

On Tue, Sep 4, 2018 at 4:08 PM, Joe Touch <[email-masked]> wrote:
> tunneling, etc.). It has been known for 23 years so it's not like any venue
> is going to publish a more recent discovery of it.
>
>
>
> IP ID cycling can cause fragment reassembly errors. If the UDP data portion
> is the same length and if the OCS sums to zero, it's possible that OCS won't
> catch reassembly of different options with different payloads.
>

Joe,

Suppose SUM = sum(bytes with checksum field zeroed)

Mehtod #1 (as in your draft):
    Sender write SUM into checksum field
    Receiver does CHECK1 = sum(received bytes with checksum zeroed)
    If CHECK1 != SUM, then drop packet

Method #2 (as in IP checksum):
    Sender write negation of SUM into checksum field
    Receiver does CHECK2 = sum(received bytes)
    If CHECK2 != 0 then drop packet

Note that,

CHECK2 == CHECK1 - SUM, CHECK1 == CHECK2 + SUM

If checksum is correct, then in one's complement arithmetic:

CHECK1 == SUM <=>
CHECK2 + SUM == SUM <=>     (substituting CHECK1)
CHECK2 == 0

So the methods are equivalent. I think the paper you referred two
describes problems at ATM reassmbly where blocks contain all zeores,
so different blocks sum to the same value. But that doesn't affect the
outcome of validating a checksum. IMO, it's much better to be
consistent with the rest of IP including the use of 2 byte one's
complement checksum. This also simplifies the implementation so we can
reuse common functions.

Tom

> 3) change a KIND somewhere else to "OCS" (at which point that "false" OCS
> would presumably fail)
>
>     at which point we require that the packet body be received and options
> ignored, for legacy support
>
>     and non-ACS'd packets with errors that are not caught by the UDP CS will
> get through (including those whose ACS kind is corrupted)
>
>     i.e., if OCS fails, then the packet doesn't have ACS and will be dropped
>

---

From: Gorry Fairhurst <[email-masked]>
Date: Wed, 5 Sep 2018 13:12:46 +0100

This is slightly off-topic...

W're planning tests with various options for the OCS across various 
networks with middleboxes and expect to have results by the next IETF 
that shed some light on what OCS works in the wild and what doesn't.  
We're hoping to be able to present the results to map-rg if there is 
time at the next meeting.

Gorry

---

From: Tom Herbert <[email-masked]>
Date: Wed, 5 Sep 2018 08:56:31 -0700

On Tue, Sep 4, 2018 at 8:08 PM, Joe Touch <[email-masked]> wrote:
>
>
>
>
> On 2018-09-04 18:17, Tom Herbert wrote:
>
> ...
>
>
> Yes, except that the _sender_ does care.
>
>
> This is UDP, not TCP. The sender doesn't control what is required because
> there's no handshake. Only the receiver can force changes in behavior beyond
> what a legacy host would do - which is ignore options completely.
>
Yes, UDP does not have a handshake but neither does IPv6. In IPv6
Destination and Hop-by-hop options it is the sender that decides the
option is mandatory via a bit in the option type. Similarly, the high
order bit could be set by a sender in a UDP option to indicate of the
option is mandatory. Also, if a sender wants to enforce that mandatory
UDP options are processed it could use LITE and setting UDP length to
zero. A legacy receiver that doesn't implement UDP options will drop
the packet which is the correct behavior per the senders intent.

> ...
>
>
> In any case, the draft is not clear on the receiver requirements for
> ACS. There should be a requirement like: "If a receiver receives a
> packet with the ACS options, it {MAY/SHOULD/MUST} validate the CRC
> before accepting the packet. If the receiver is unable to verify a CRC
> it MUST silently discard the packet"
>
>
> Agreed, though I had assumed that a substitute for the UDP checksum would
> have the same consequences as the UDP checksum failing.
>
Assumptions lead to ambiguity which is not befitting of a protocol
specification. Please be specific on the actions to be taken for each
error condition and the processing steps for each algorithm. The more
detail about the protocol provided to the implementor, the better
chance we have to get a deployable and interoperable protocol.

Another example of ambiguity is how to deal with option length
mismatch. The draft says:

"Receivers cannot treat unexpected option lengths as invalid, as this
would unnecessarily limit future revision of options (e.g., defining a
new ACS that is defined by having a different length)."

Okay, then how do they treat options with a length that is different
from the standard definition? Do they ignore the option? Try to make
use of the available data somehow? What if an ACS is received with a
length of 3, do they need accept the packet?

> I don't see how accepting a packet on a failure to verify a checksum
> can be considered robust.
>
>
> A receiver that doesn't require ACS is exactly as robust as UDP to a legacy
> receiver would be. If that's not what you want (i.e., you want a receiver
> that requires ACS), then you have to confirm that via some other mechanism
> (a stateful protocol exchange). Once you have that confirmation, you should
> get the behavior you desire - but it would not be predictable or robust to
> do otherwise.
>
I'm not referring to ACS, this is about OCS. From the draft:

"UDP data that is validated by a correct UDP checksum MUST be
delivered to the application layer, even if the UDP option checksum
fails"

This implies that a packet with a checksum failure can be accepted. A
checksum failure in OCS is _proof_ that the packet is somehow
corrupted or malformed. The packet should be dropped. Accepting such a
packet is incorrect. Also, notice that the behavior is inconsistent
with other error cases like when an option length exceeds the extent
of the packet, in that case the packet must is considered malformed
and must be dropped as per section 5.

Tom

---

From: Tom Herbert <[email-masked]>
Date: Wed, 5 Sep 2018 09:53:53 -0700

> and must be dropped as per section 5.
Also noticed last sentence in security considerations:

"Note that when a packet's options cannot be processed, it MUST be
discarded; the packet and its options should always share the same
fate.".

That requirement is at odds with skipping option processing when the
OCS checksum fails, as well as allowing a legacy node to ignore UDP
options altogether.

Tom

---

From: "Gorry (erg)" <[email-masked]>
Date: Wed, 5 Sep 2018 18:10:28 +0100

> On 5 Sep 2018, at 17:53, Tom Herbert <[email-masked]> wrote:
> 

I like that.
> OCS checksum fails,

Which is a rather different property, and I am less sure that this is the simplest way to work.

> as well as allowing a legacy node to ignore UDP
> options altogether.
> 
To me - that is quite ok. An app knows it is on a stack that can send options. An app knows it is on a stack that should be able to receive options ... and it certainly can easy exchange an option to know the path allows options. If an app ignores all the options that is not a huge. I think the failed acs should mean all options fail.

If an app (transport in the app) sees packets with some options do not achieve the expected effect - then it needs to desist from using them. 

Gorry

---

From: Tom Herbert <[email-masked]>
Date: Wed, 5 Sep 2018 10:40:00 -0700

> To me - that is quite ok. An app knows it is on a stack that can send options. An app knows it is on a stack that should be able to receive options ... and it certainly can easy exchange an option…
Hi Gorry,

I was commenting more on getting the internal consistency or
requirements in the draft right. However, I disagree that allowing an
application to ignore options is not a problem.

Consider the fragmentation option. If a receiver completely ignores
options (or doesn't know what UDP options) it may forward fragments to
an application. The application does not know what fragments are so it
tries to process them as normal UDP packets. This is systematic silent
data corruption! This will occur not just for fragmentation, but also
any sort of option that does a transformation on the data that must be
undone before delivery to the application (e.g. encryption,
compression, etc.). Ignoring CRCs is almost as bad because it
increases the chances of a receiver accepted corrupted packets. The
use of LITE with UDP length zero should be a MUST if fragmentation or
other transformation is present, and IMO maybe should be used all the
time to ensure that options are either processed or packet is dropped
as per the security section requirement.

> If an app (transport in the app) sees packets with some options do not achieve the expected effect - then it needs to desist from using them.e e

How does a sender know whether the options its sending are achieving
the desired effect? How does a sender using a CRC actually known that
corrupted packets are being properly identified and dropped by the
receiver?

Tom

---

From: Joe Touch <[email-masked]>
Date: Wed, 05 Sep 2018 12:30:28 -0700

On 2018-09-05 08:56, Tom Herbert wrote:

> option is mandatory via a bit in the option type.

IPv6 defined that behavior at the outset for all options. 

For UDP, those flags would be meaningless for legacy endpoints, which
silently ignore all options. 

> Similarly, the high

Except that LITE isn't covered by the UDP checksum, so if you send a
packet without ACS, the packet body isn't covered anymore. 

But yes, at that point the options would definitely be processed. 

> specification.

I already agreed to add the text; I was explaining the logic behind it.
What's the issue? 

   >> Receivers MUST silently ignore unknown options. That includes
   options whose length does not indicate the specified value.

> packet is incorrect.

You're implying that the behavior of legacy UDP and UDP with options
would be different; I am requiring them to be the same. 

If the (UDP option capable) receiver says "oh, packets without options
should be dropped" they can do so at their app layer, where they would
know that this happened. 

> Also, notice that the behavior is inconsistent

Good catch; in that case, the option exceeding the length should be
silently ignored instead. It would be up to a receiver expecting that
option to decide that the packet should be dropped because it is
missing, not UDP. 

Joe

---

From: Joe Touch <[email-masked]>
Date: Wed, 05 Sep 2018 12:34:25 -0700

Hi, Gorry, 

So it looks like we agree, but I want to check: 

- UDP options should not drop a packet because of an option area error;
it should pass the body up with only whatever options it deems valid and
let the app decide what to do 

(this would change the security considerations statement to the
contrary, i.e., that implied fate-sharing) 

I see this as the best way forward because otherwise the sender thinks
it has control over the receiver when it does not. 

Joe 

On 2018-09-05 10:10, Gorry (erg) wrote:

> On Wed, Sep 5, 2018 at 8:56 AM, Tom Herbert <[email-masked]> wrote: On Tue, Sep 4, 2018 at 8:08 PM, Joe Touch <[email-masked]> wrote:
> 

> That requirement is at odds with skipping option processing when the

Which is a rather different property, and I am less sure that this is
the simplest way to work.

> as well as allowing a legacy node to ignore UDP
 To me - that is quite ok. An app knows it is on a stack that can send
options. An app knows it is on a stack that should be able to receive
options ... and it certainly can easy exchange an option to know the
path allows options. If an app ignores all the options that is not a
huge. I think the failed acs should mean all options fail.

If an app (transport in the app) sees packets with some options do not
achieve the expected effect - then it needs to desist from using them. 

Gorry

---

From: Tom Herbert <[email-masked]>
Date: Wed, 5 Sep 2018 13:46:55 -0700

On Wed, Sep 5, 2018 at 12:30 PM, Joe Touch <[email-masked]> wrote:
> Except that LITE isn't covered by the UDP checksum, so if you send a packet
> without ACS, the packet body isn't covered anymore.
>
Doesn't LITE supplant the UDP checksum in this case?

> You're implying that the behavior of legacy UDP and UDP with options would
> be different; I am requiring them to be the same.
>
> should be dropped" they can do so at their app layer, where they would know
> that this happened.
>
>
> Good catch; in that case, the option exceeding the length should be silently
> ignored instead. It would be up to a receiver expecting that option to
> decide that the packet should be dropped because it is missing, not UDP.
>
Joe,

Sorry, but that's going the wrong direction. Ignoring the symptoms
that a packet is sick doesn't magically make its illness go away.

In any case, I'm pretty sure when we implement UDP options in Linux,
and probably other OSes, the rules will be very straightforward (like
they are with implementation of IP options, TCP options, etc.). If
there's any problem with options that bring the integrity of the
packet into question, a receiver will drop the packet. Bad OCS,
unknown ACS length, option going past the end of the packet-- should
all result in dropped packets.

Tom

---

From: Gorry Fairhurst <[email-masked]>
Date: Thu, 06 Sep 2018 10:48:28 +0100

On 05/09/2018, 20:34, Joe Touch wrote:
> - UDP options should not drop a packet because of an option area 
> error; it should pass the body up with only whatever options it deems 
> valid and let the app decide what to do
>

UDP checksum covers only the UDP Payload.
OCS covers only the options

So, I see it normally this way:
- UDP checksum fails				drop packet
- UDP checksum passes, OCS fails		deliver udp to upper layer, ignore all options
- UDP checksum passes, OCS passes		deliver packet, parse options

ACS enabled*
- UDP checksum passes, OCS passes, ACS passes	deliver udp to upper layer, process options
- UDP checksum passes, OCS passes, ACS fails	drop packet

To achieve the coverage expected by ACS, I'd also require a reciever with ACS enabled to also drop:
UDP packet with no option or no ACS option 	drop packet
UDP checksum passes, OCS fails			drop packet

Does that make sense?

Gorry

>>> On 5 Sep 2018, at 17:53, Tom Herbert <[email-masked] 
>>> <mailto:[email-masked]>> wrote:
>>>
>>>> On Wed, Sep 5, 2018 at 8:56 AM, Tom Herbert <[email-masked] 
>>>> <mailto:[email-masked]>> wrote:
>>>>> On Tue, Sep 4, 2018 at 8:08 PM, Joe Touch <[email-masked] 
>>>>> <mailto:[email-masked]>> wrote:
>>>>>
>>>>>
>>>>>
>>>>>
>> If an app (transport in the app) sees packets with some options do 
>> not achieve the expected effect - then it needs to desist from using 
>> them.
>>
>> Gorry
>>
>>> Tom

---

From: Joe Touch <[email-masked]>
Date: Thu, 6 Sep 2018 06:59:15 -0700

Yes, but you didn’t discuss non-ACS-enabled receivers.  They should work like the OCS case.

Joe

> On Sep 6, 2018, at 2:48 AM, Gorry Fairhurst <[email-masked]> wrote:
> 
>> - UDP options should not drop a packet because of an option area error; it should pass the body up with only whatever options it deems valid and let the app decide what to do
>> 
>> (this would change the security considerations statement to the contrary, i.e., that implied fate-sharing)
>> 
>> I see this as the best way forward because otherwise the sender thinks it has control over the receiver when it does not.
>> 
>> Joe
>> 
> 
>>>>> On 5 Sep 2018, at 17:53, Tom Herbert <[email-masked] <mailto:[email-masked]>> wrote:
>>>>> 
>>>>>> On Wed, Sep 5, 2018 at 8:56 AM, Tom Herbert <[email-masked] <mailto:[email-masked]>> wrote:
>>>>>> On Tue, Sep 4, 2018 at 8:08 PM, Joe Touch <[email-masked] <mailto:[email-masked]>> wrote:
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>>

---

From: Gorry Fairhurst <[email-masked]>
Date: Thu, 06 Sep 2018 15:09:57 +0100

> Yes, but you didn’t discuss non-ACS-enabled receivers.  They should work like the OCS case.
OK. I can comment on my thoughts here also...

An app designed to rely on ACS is made by an app developer that knows 
that he needs the ACS to be checked:
- At the sender, the app tells the UDP-Options stack to add the ACS option.
- At the receiver, the app tells the UDP-Options stack to check the ACS 
option. If the stack doesn't support the ACS option, it shouldn't 
implement an API that lets you set this mode. If the path/sender stack 
happens to later strip the option, then the receiver is made unhappy 
because delivery fails.

To me, that implies a UDP-Options receiver will now behave in the way I 
expect to handle ACS.  An app could also (if it so likes) choose to 
negotiate this in some way, or an unhappy app could fall back to a 
socket that doesn't set this mode. That's up to the app to do the right 
thing.

Does that help?

Gorry
>> On Sep 6, 2018, at 2:48 AM, Gorry Fairhurst<[email-masked]>  wrote:
>>
>>>>>> On 5 Sep 2018, at 17:53, Tom Herbert<[email-masked]<mailto:[email-masked]>>  wrote:
>>>>>>
>>>>>>> On Wed, Sep 5, 2018 at 8:56 AM, Tom Herbert<[email-masked]<mailto:[email-masked]>>  wrote:
>>>>>>> On Tue, Sep 4, 2018 at 8:08 PM, Joe Touch<[email-masked]<mailto:[email-masked]>>  wrote:
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>

---

From: Joe Touch <[email-masked]>
Date: Thu, 6 Sep 2018 07:27:30 -0700

Hi, Gorry,

Yes - i.e., “require ACS” is a setting on the receiver (i.e., per-socket, per-machine, etc.). It is not “enabled by default across a machine” simply by supporting UDP options. The default is “not-require ACS”.

But that means if ACS isn’t required it works like OCS, i.e.:
	ACS-not-required, UDP CS passes, OCS passes, ACS fails		deliver the packet and options to the app, indicating ACS failed

At that point, the app can decide what to do on a per-packet basis.

I.e., IMO, when UDP CS passes and OCS passes, the options (perhaps both that work and those that fail) to the app

In this regard ACS is no different from any other UDP option:
	- the sender can add them as desired
	- the receiver can REQUIRE them or not, as desired (per socket)
		when REQUIRED, packets that don’t have that option are silently ignored
	- when an option isn’t REQUIRED, it is still checked and passed up to the app

The “REQUIRED” part is an optional feature of a UDP implementation that helps a UDP-option receiver support desired option behavior for legacy apps.

Joe

> On Sep 6, 2018, at 7:09 AM, Gorry Fairhurst <[email-masked]> wrote:
> 
> An app designed to rely on ACS is made by an app developer that knows that he needs the ACS to be checked:
> - At the receiver, the app tells the UDP-Options stack to check the ACS option. If the stack doesn't support the ACS option, it shouldn't implement an API that lets you set this mode. If the path/sender stack happens to later strip the option, then the receiver is made unhappy because delivery fails.
> 
> To me, that implies a UDP-Options receiver will now behave in the way I expect to handle ACS.  An app could also (if it so likes) choose to negotiate this in some way, or an unhappy app could fall back to a socket that doesn't set this mode. That's up to the app to do the right thing.
> 
> Does that help?
> 
> Gorry

---

From: Joe Touch <[email-masked]>
Date: Thu, 6 Sep 2018 07:32:52 -0700

> On Sep 5, 2018, at 1:46 PM, Tom Herbert <[email-masked]> wrote:
> 

LITE doesn’t have a checksum; it’s intended to pass payloads without UDP CS coverage.

> that a packet is sick doesn't magically make its illness go away.

The issue is default behavior. We have a problem that TCP (which negotiates options) and IPv6 (which required the ‘required’ flag from the start) didn’t have - legacy receivers that ignore all options, valid or not. A UDP-option receiver needs to act like a legacy receiver *until instructed otherwise*. The only party that has control over that is the receiver, not the sender.

We need to remember that UDP options are optional - even at the receiver - until specifically configured otherwise.

See the summary Gorry posted.

Joe

---

From: Tom Herbert <[email-masked]>
Date: Thu, 6 Sep 2018 07:47:36 -0700

On Thu, Sep 6, 2018 at 2:48 AM, Gorry Fairhurst <[email-masked]> wrote:
>> - UDP options should not drop a packet because of an option area error; it
>> should pass the body up with only whatever options it deems valid and let
>> the app decide what to do
>>
>> I see this as the best way forward because otherwise the sender thinks it
>> has control over the receiver when it does not.
>>
>> Joe
>>
>
No, it doesn't. Consider that OCS and FRAG option can be used
together. Per the above rules, if the OCS fails then all options,
including the FRAG option, are ignored. This results in a fragment
being sent so the user. What the user sees is gibberish and this
directly violates the requirement: "Individual UDP fragments MUST NOT
be forwarded to the user.". The receiver full knew there was a problem
with the packet, and therefore it could not eliminate the very real
possibility that it would be sending incorrect data to the user. The
receiver needs to drop the packet on OCS failure.

Tom

>>>>>> will
>>> options ... and it certainly can easy exchange an option to know the path
>>> allows options. If an app ignores all the options that is not a huge. I
>>> think the failed acs should mean all options fail.
>>>

---

From: Gorry Fairhurst <[email-masked]>
Date: Thu, 06 Sep 2018 15:55:56 +0100

I don't quite see it this way...

> Yes - i.e., “require ACS” is a setting on the receiver (i.e., per-socket, per-machine, etc.). It is not “enabled by default across a machine” simply by supporting UDP options. The default is “not-r…
That makes sense.
> 	ACS-not-required, UDP CS passes, OCS passes, ACS fails		deliver the packet and options to the app, indicating ACS failed
I'm not sure I'd evaluate the ACS when it was not enabled by the 
receiver. That seems like triggering code that shouldn't be executed, 
... and if it executes then I'd also expect the packet to be dropped 
when the ACS fails.

(1) One idea could be to ignore the option, and not tell the app. I 
don't like this.
(2) Another idea could be to pass the ACS option value to the app 
without evaluating - perhaps the app would then decide to enable the ACS 
because it sees it being sent end-to-end? I like this less.
(3) A receiver could  drop any datagram that has an unexpected ACS 
option present.

My problem with (1,2) are that this means that a receiver can start 
accepting packets that were meant to check a more powerful integrity 
check, because the receiver forgot to enable it. I expect that 
communication path to fail as in (3).
> 	- when an option isn’t REQUIRED, it is still checked and passed up to the app
I think it seems weird to "force" this extra work on an endpoint  that 
doesn't request it. Evaluating a checksum consumes cycles. If the 
receiver does the work and the integrity check fails, then the packet 
needs to be thrown. I'm concerned that work is done irrespective of 
whether the ACS is valid or not. If it is invalid, I'm reluctant to for 
the stack to continue processing because a weaker checksum says so.
Gorry
>
>> On Sep 6, 2018, at 7:09 AM, Gorry Fairhurst<[email-masked]>  wrote:
>>

---

From: Gorry Fairhurst <[email-masked]>
Date: Thu, 06 Sep 2018 16:04:38 +0100

On 06/09/2018, 15:47, Tom Herbert wrote:
> On Thu, Sep 6, 2018 at 2:48 AM, Gorry Fairhurst<[email-masked]>  wrote:
I wanted to first agree the processing of the checksums!

I agree what you say is a severe problem. I suspect for this reason the 
current design of the FRAG option is broken:
To me, the fragment data really needs to be sent in the options part of 
the UDP datagram, because placing the data in the UDP payload can (and 
will) result in case where fragments are unexpectedly returned to 
userland when a stack does not understand options (e.g.. a legacy 
receiver or legacy middlebox on path).

Gorry

---

From: Joe Touch <[email-masked]>
Date: Thu, 6 Sep 2018 08:11:50 -0700

> On Sep 6, 2018, at 8:04 AM, Gorry Fairhurst <[email-masked]> wrote:
> 
> I agree what you say is a severe problem. I suspect for this reason the current design of the FRAG option is broken:
> To me, the fragment data really needs to be sent in the options part of the UDP datagram, because placing the data in the UDP payload can (and will) result in case where fragments are unexpectedly returned to userland when a stack does not understand options (e.g.. a legacy receiver or legacy middlebox on path).

Agreed.  I think we’ve been heading towards lite+frag as the only safe way to use frag for a while. It may be preferable to force that in the spec.

Joe

---

From: Tom Herbert <[email-masked]>
Date: Thu, 6 Sep 2018 08:25:52 -0700

On Thu, Sep 6, 2018 at 8:04 AM, Gorry Fairhurst <[email-masked]> wrote:
>>>> it
>>>> it
>>> - UDP checksum passes, OCS fails                deliver udp to upper
>>> layer,
>>> ignore all options
>>> - UDP checksum passes, OCS passes, ACS passes   deliver udp to upper
>>> layer,
>>> process options
> To me, the fragment data really needs to be sent in the options part of the
> UDP datagram, because placing the data in the UDP payload can (and will)
> result in case where fragments are unexpectedly returned to userland when a
> stack does not understand options (e.g.. a legacy receiver or legacy
> middlebox on path).
>
Gorry,

Middleboxes are not a relevant consideration. If they are inspecting
UDP payload data based on port number that is fundamentally incorrect
and being done at their own peril. If they are dropping packets
because of that, then the sender can backoff from using UDP options
because they are not viable in the path. That would be the same as
dealing with a firewall that drops packet when UDP slack space is
non-zero.

Legacy receivers are a bigger concern. The assumption in the draft is
UDP options should be ignored by a legacy receiver. I am not sure that
can ever be correct. Unless all options were completely innocuous to
ignore, or there is a well defined handshake and with some sort of
session context, the possibility of the protocol allowing incorrect
data being delivered to an application is present (like the FRAG
option problem). Protocol incorrectness is not a good tradeoff to
allow legacy receivers.

The alternative is to not allow legacy receivers to ignore options.
This can be done by setting UDP length to zero and using LITE options
(a UDP checksum option would also be needed). This eliminates the
ambiguity: either a receiver processes the UDP options or it drops the
packet. Processing of options is deterministic, mandatory options
(those that must be processed) are facilitates, and the requirement of
UDP length being zero helps disambiguate UDP options from other case
where slack space may be non-zero.

Tom

>>>>>>>> there's no handshake. Only the receiver can force changes in
>>>>>>>> behavior
>>>>>>>> beyond
>>>>> the
>>>>> simplest way to work.
>>>>> path

---

From: Gorry Fairhurst <[email-masked]>
Date: Thu, 06 Sep 2018 16:35:45 +0100

On 06/09/2018, 16:25, Tom Herbert wrote:
> On Thu, Sep 6, 2018 at 8:04 AM, Gorry Fairhurst<[email-masked]>  wrote:
Welcome to my world: Some middleboxes will strip the UDP options field. 
I don't like that. However, it will happen. Fortunately,iIt's easy 
enough to spot this if your app sends an option!
> where slack space may be non-zero.
I agree - you could place the Frag data in the options payload, like (or 
using Lite), we talked about that some months ago, and I'd suggest this.
Gorry

---

From: Tom Herbert <[email-masked]>
Date: Thu, 6 Sep 2018 08:50:56 -0700

On Thu, Sep 6, 2018 at 8:35 AM, Gorry Fairhurst <[email-masked]> wrote:
>>>>>> error;
>>>>> To achieve the coverage expected by ACS, I'd also require a reciever
>>>>> with
>>>>> ACS enabled to also drop:
> Welcome to my world: Some middleboxes will strip the UDP options field. I
> don't like that. However, it will happen. Fortunately,iIt's easy enough to
> spot this if your app sends an option!

Gorry,

Don't forget, they're not stripping UDP options field, they're
stripping UDP slack space. Honestly, I'm surprised that firewalls
don't commonly do this. You would think they'd see this as an obvious
covert channel.

Tom

>>>>>>>>>>      and non-ACS'd packets with errors that are not caught by the
>>>>>>>>>> UDP
>>>>>>>>>> CS
>>>>>>>>>> will
>>>>>>>>>> Agreed, though I had assumed that a substitute for the UDP
>>>>>>>>>> checksum
>>>>>>>>>> would
>>>>>>>>> would unnecessarily limit future revision of options (e.g.,
>>>>>>>>> defining
>>>>>>>>> a
>>>>>>>>>> A receiver that doesn't require ACS is exactly as robust as UDP to
>>>>>>>>>> a
>>>>>>>>>> legacy
>>>>>>>>>> (a stateful protocol exchange). Once you have that confirmation,
>>>>>>>>>> you
>>>>>>>>>> should
>>>>>>>>> corrupted or malformed. The packet should be dropped. Accepting
>>>>>>>>> such
>>>>>>>>> a
>>>>>>> not

---

From: Gorry Fairhurst <[email-masked]>
Date: Thu, 06 Sep 2018 16:54:01 +0100

On 06/09/2018, 16:50, Tom Herbert wrote:
> On Thu, Sep 6, 2018 at 8:35 AM, Gorry Fairhurst<[email-masked]>  wrote:
Just to be sure is "slack space" the same field we now call UDP options?

Gorry

---

From: Joe Touch <[email-masked]>
Date: Thu, 6 Sep 2018 09:05:47 -0700

On Sep 6, 2018, at 8:35 AM, Gorry Fairhurst <[email-masked]> wrote:

> I agree - you could place the Frag data in the options payload, like (or using Lite), we talked about that some months ago, and I'd suggest this.
>> Tom
> Gorry

Ip options just make packers more likely to be dropped, and transport info should never be inside ip space. 

Yes, if you want legacy receivers to ignore packers with UDP option, you might try lite. But that also changes the semantics of the delivered data as not summed. You really want lite+frag with one fragment.  

But that’s a choice. It shouldn’t be a requirement. 

Joe

---

From: Tom Herbert <[email-masked]>
Date: Thu, 6 Sep 2018 09:19:50 -0700

On Thu, Sep 6, 2018 at 8:54 AM, Gorry Fairhurst <[email-masked]> wrote:
>>>>>>>> should pass the body up with only whatever options it deems valid
>>>>>>>> and
>>>>>>>> let
>>>>> result in case where fragments are unexpectedly returned to userland
>>>>> when
>>>>> a
No, not quite. Slack space was created in RFC768 with the introduction
of UDP length. It's the difference between IP length and UDP length
and has existed for 38 years. There were never any requirements, nor
any prohibition, on using this space. UDP options proposes use of
slack space, but that doesn't preclude that people aren't already
using slack space for their own purposes (a reason to include data to
disambiguate UDP options from other use cases). Also, UDP options as
defined, don't necessarily take up the whole slack space, the protocol
allows data beyond the end of the options.

Tom

>>>>>>>>>>>> This is UDP, not TCP. The sender doesn't control what is
>>>>>>>>>>>> required
>>>>>>>>>>>> because
>>>>>>>>>>> UDP options are processed it could use LITE and setting UDP
>>>>>>>>>>> length
>>>>>>>>>>> to
>>>>>>>>>>>> before accepting the packet. If the receiver is unable to verify
>>>>>>>>>>>> a
>>>>>>>>>>>> CRC
>>>>>>>>>>> use of the available data somehow? What if an ACS is received
>>>>>>>>>>> with
>>>>>>>>>>> a
>>>>>>>>>>>>       at which point only ACS'd packets that are correct will
>>>>>>>>>>>> get
>>>>>>>>>>>> through
>>>>>>>>>>>>       i.e., if OCS fails, then the packet doesn't have ACS and
>>>>>>>>>>>> will
>>>>>>>>>>>> be
>>>>>>>>>>>> dropped
>>>>>>>>>>>>
>>>>>>>>>>> This implies that a packet with a checksum failure can be
>>>>>>>>>>> accepted.
>>>>>>>>>>> A
>>>>>>>>> allows options. If an app ignores all the options that is not a
>>>>>>>>> huge.
>>>>>>>>> I

---

From: Gorry Fairhurst <[email-masked]>
Date: Thu, 06 Sep 2018 17:23:42 +0100

On 06/09/2018, 17:05, Joe Touch wrote:
>
> On Sep 6, 2018, at 8:35 AM, Gorry Fairhurst<[email-masked]>  wrote:
>
I menat UDP-Options part of the IP Payload.
> Yes, if you want legacy receivers to ignore packers with UDP option, you might try lite. But that also changes the semantics of the delivered data as not summed. You really want lite+frag with one…
Yes - or something similar that isn't Lite, depending on how you 
integrity check the reassembled datagram.
Gorry

---

From: Gorry Fairhurst <[email-masked]>
Date: Thu, 06 Sep 2018 17:31:42 +0100

On 06/09/2018, 17:19, Tom Herbert wrote:
> On Thu, Sep 6, 2018 at 8:54 AM, Gorry Fairhurst<[email-masked]>  wrote:
> of UDP length.
> It's the difference between IP length and UDP length
OK - Now I don't disagree with what you say (some decades ago this was 
explored in detail when we were looking at using the UDP port number for 
UDP-Lite). I was curious about the usage of the term and couldn't find 
it anywhere in the RFC series.

Gorry