Subject: [tsvwg] A counterproposal to Section 5.5 of draft-ietf-tsvwg-udp-options-12

From: "C. M. Heard" <[email-masked]>
Date: Thu, 3 Jun 2021 19:22:22 -0700

TSVWG,

In a previous message, I made some rather unkind comments about he FRAG
option as proposed in Section 5.5 of draft-ietf-tsvwg-udp-options-12. In
this missive I wish to present proposed relacement text for the entire
section. If some of the packet diagrams end up a bit garbled, please accept
my apologies; I am working within the limits of the gmail client, and I
can't always tell what will look good to list recipents or those who browse
the TSWG archive. This proposal first appeared in
https://mailarchive.ietf.org/arch/msg/tsvwg/Wv--BLVMPAX6g5umok9BQsXAEGg/.
The central idea is to make the FRAG option the very last option in the
packet and to have it consume all following octets. I want to acknowledge
freeley borrowing/stealing the best of some of Joe Touch's excellent text
in draft-ietf-tsvwg-udp-options-12.

--cut here--

5.5. Fragmentation (FRAG)

   The Fragmentation (FRAG) option supports UDP fragmentation and
   reassembly, which can be used to transfer UDP messages larger than
   limited by the IP receive MTU (EMTU_R [RFC1122]). It is typically
   used with the UDP MSS option to enable more efficient use of large
   messages, both at the UDP and IP layers.  FRAG is designed similar
   to the IPv6 Fragmentation Header [RFC8200], except that the UDP
   variant uses a 16-bit Offset measured in bytes, rather than IPv6's
   13-bit Fragment Offset measured in 8-byte units.  FRAG

   provides transport-layer fragmentation and reassembly in which each
   fragment includes a copy of the same UDP transport ports, enabling
   the fragments to traverse Network Address (and port) Translation
   (NAT) devices, in contrast to the behavior of IP fragments.

   >> When FRAG is present, the UDP payload MUST be empty. If the
   payload is not empty, all UDP options MUST be silently ignored
   and the UDP payload received MUST be sent to the user.

   Legacy receivers interpret FRAG messages as zero-length payload
   packets (i.e., UDP Length field is 8, the length of just the UDP
   header), which would not affect the receiver unless the presence
   of the packet itself were a signal.

   >> When FRAG is present, it MUST come LAST in the UDP options list.

   All remaining octets in the packet are interpreted as fragment data.

   The FRAG option has two formats; non-terminal fragments set the
   More Fragments (MF) field to 1 and terminal fragments set the MF
   field to 0.  The latter includes stand-alone fragments, i.e., when
   data is contained in the FRAG option but reassembly is not required.

   In both cases the option length is implicitly eight octets.

                   +--------+--------+--------+--------+
                   | Kind=5 |  MF=1  |  Frag. Offset   |
                   +--------+--------+--------+--------+
                   |          Identification           |
                   +--------+--------+--------+--------+
                   |       ... Fragment Data ...       |
                   +--------+--------+--------+--------+

                    UDP non-terminal FRAG option format

                   +--------+--------+--------+--------+
                   | Kind=5 |  MF=0  |  Frag. Offset   |
                   +--------+--------+--------+--------+
                   |          Identification           |
                   +--------+--------+--------+--------+
                   |       ... Fragment Data ...       |
                   +--------+--------+--------+--------+

                      UDP terminal FRAG option format

   >> The Fragment Offset is 16 bits and indicates the location of the
   UDP payload fragment in bytes from the beginning of the original
   unfragmented payload.  The MF field indicates whether there are more
   fragments (MF=1) or no more fragments (MF=0).

   >> The Identification field is a 32-bit value that MUST be unique
   over the expected fragment reassembly timeout.

   >> The Identification field SHOULD be generated in a manner similar
   to that of the IPv6 Fragment ID [RFC8200].

   >> UDP fragments MUST NOT overlap.  Note that exact duplicates may

   occur when packets are duplicated in the network.

   UDP fragmentation relies on a fragment expiration timer, which can
   be preset or could use a value computed using the UDP Timestamp
   option.

   >> The default UDP reassembly SHOULD be no more than 2 minutes.

   Implementers are advised to limit the space available for UDP
   reassembly.

   >> UDP reassembly space SHOULD be limited to reduce the impact of
   DOS attacks on resource use.

   >> UDP reassembly space limits SHOULD NOT be implemented as an
   aggregate, to avoid cross-socketpair DOS attacks.

   >> Individual UDP fragments MUST NOT be forwarded to the user.  The
   reassembled datagram is received only after complete reassembly,
   and continued processing of the remaining UDP options.

   >> Only per-fragment UDP options may accompany the FRAG header in
   non-terminal fragments.  These include NOP, OCS, and AE.  All options
   that apply to a reassembled UDP datagram MUST accompany the FRAG
   header in the terminal fragment.  These include ACS, MSS, MRSS,
   TIME, REQ, and RES.  UNSAFE may appear in either place, depending.
   on the nature of the UNSAFE Ukind.

   In general, UDP packets are fragmented as follows:

   1. Create a datagram with data and any UDP options that apply to the
      complete UDP datagram, which we will call "D".  Note that these
      options must accompany the terminal fragment.  They are prepared
      before the rest of the fragmentation steps below.

   2. Identify the desired fragment size, which we will call "S".  This
      value should take into account the path MTU (if known) and allow
      space for per-fragment options (e.g., OCS).

   3. Fragment "D" into chunks of size no larger than "S"-8 each.  Note
      that all of the per-datagram options in step #1 MUST appear in
      the terminal fragment.

   4. For each chunk of "D" in step #3, create a zero-data UDP packet
      followed by the per-fragment options, with the final option being
      the FRAG option followed by the FRAG data chunk.

      The last chunk includes the non-FRAG options noted in step #1.
      These UDP options apply to the reassembled data as a whole.

   5. Process the UDP options of each fragment, e.g., computing its
      OCS.

   Receivers reverse the above sequence.  They process all received
   options in each fragment.  When the FRAG option is encountered, the
   FRAG data is used in reassembly.  After all fragments are received,
   the entire packet is processed with any per-datagram UDP options
   accompanying the terminal fragment applying to the reassembled data.

--cut here--

This is what typical UDP datagrams containing UDP fragments would look like:

                   +--------+--------+--------+--------+
                   |  Source Port    |   Dest. Port    |

                   +--------+--------+--------+--------+
                   |   UDP Len=8     |  UDP Checksum   |

                   +--------+--------+--------+--------+
                   | NOP=1  | OCS=2  | Option Checksum |
                   +--------+--------+--------+--------+
                   | FRAG=5 |  MF=1  |  Frag. Offset   |
                   +--------+--------+--------+--------+
                   |          Identification           |
                   +--------+--------+--------+--------+
                   |       ... Fragment Data ...       |
                   +--------+--------+--------+--------+

                     UDP non-terminal FRAG packet

                   +--------+--------+--------+--------+
                   |  Source Port    |   Dest. Port    |

                   +--------+--------+--------+--------+
                   |   UDP Len=8     |  UDP Checksum   |

                   +--------+--------+--------+--------+
                   | NOP=1  | OCS=2  | Option Checksum |
                   +--------+--------+--------+--------+
                   |       ... Other Options ...       |
                   +--------+--------+--------+--------+
                   | FRAG=5 |  MF=0  |  Frag. Offset   |
                   +--------+--------+--------+--------+
                   |          Identification           |
                   +--------+--------+--------+--------+
                   |       ... Fragment Data ...       |
                   +--------+--------+--------+--------+

                     UDP terminal FRAG packet

Kindly review and send send constructive comments to the WG list

Thanks and regards,

Mike Heard

---

From: Joseph Touch <[email-masked]>
Date: Thu, 10 Jun 2021 21:42:51 -0700

Hi, Mike, 

> On Jun 3, 2021, at 7:22 PM, C. M. Heard <[email-masked]> wrote:
> 
> TSVWG,
> 
> In a previous message, I made some rather unkind comments about he FRAG option as proposed in Section 5.5 of draft-ietf-tsvwg-udp-options-12. In this missive I wish to present proposed relacement text for the entire section. 

Your proposal focuses on making the  FRAG option last in non-terminal fragments.

FIRST, note that “offset” in the FRAG option should really be named “”frag length”; it’s there so there can be options after the frags and (at least originally) helped the FRAG/LITE combination support RDMA.

I appreciate that this was garbled/lost in the conversion from FRAG/LITE to just FRAG; I’ll be fixing that.

Overall, I think this proposal does raise two issues indirectly:
a. Do we need to support RDMA for fragments
b. Do we need a post-reassembly checksum

See below for details. Until we answer those, we don’t quite know how to proceed.

Joe

—

So regarding a new format, I think the only real question is whether we keep RDMA support. Here’s why:

Regarding non-terminal FRAG be last:
- we could require options before the non-terminal frags
- however, that also means that we need to consider whether not having FRAG as early as possible would impact RDMA
	right now, we recommend moving it early to make RDMA move fewer bytes to ‘adjust’

Regarding the formats: 
- I prefer uniform TLV format even for options whose length doesn’t change (yes, even changing OCS to do this; only NOP and EOL should be exceptions)
- we still need options after the terminal fragment, e.g., if someone wants to perform AE on the reassembled packet but not the individual fragments (which could improve performance and reduce overhead), or if there are other behaviors (time, echo, etc.) intended on the reassembled packet but not any individual fragment (such as echo only when the whole packet gets there, not just the terminal one).

The question of whether we need a MF bit (or byte) depends on whether we want to retain the post-reassembly required checksum or not. Yes, it’s possible this might not be needed, but it was added by what I understood to be consensus. If we don’t need that, we can drop it.

So at that point, we really have only a few decisions:

- do we want/need RDMA support?
	if not, we ditch the frag length in non-terminal frags

- do we want to ditch the required post-reassembly checksum?
	if so, we ditch that field in the terminal frag

THEN we compare the answers to decide how to handle the MF issue:
	- if we still end up with different length options, length indicates MF
	- if not, we might be better off burning two codepoints on FRAG/TERMFRAG than a byte
	(a single byte throws off our fields as odd-length AND needs to be checked for values not 0 or 1)

Joe

---

From: "C. M. Heard" <[email-masked]>
Date: Sun, 13 Jun 2021 15:09:26 -0700

> checked for values not 0 or 1)

I'm OK with this as a path forward. I would like to add that I have no
objection to using different KIND codepoints in place of a MF bit or byte.

I'll address the two points above separately.

Mike

---

From: "C. M. Heard" <[email-masked]>
Date: Sun, 13 Jun 2021 16:37:30 -0700

>         if not, we ditch the frag length in non-terminal frags

I am going to reveal my ignorance by asking what is meant by RDMA (Remote
Direct Memory Access). But I'm fine with that as long as we end up with an
unambiguous problem statement.

The specification that I found with a Google search is "InfiniBand™
Architecture Specification Release 1.2.1 Annex A17: RoCEv2," available at
https://cw.infinibandta.org/document/dl/7781/.

This is a datacenter protocol and requires very specific conditions in
order to work, not the least of which is that it assumes that the IP/UDP
layer is nearly lossless and free from reordering.

This protocol handles its own data placement and integrity checks -- indeed
it requires UDP CS=0 to avoid redundant work -- and AFAICT would not in any
way benefit from UDP options.

If you have something else in mind, please let us know what it is.
Otherwise, I'd say that UDP options does not need to make any effort to
support RDMA.

Mike Heard

---

From: Joseph Touch <[email-masked]>
Date: Sun, 13 Jun 2021 17:01:24 -0700

> On Jun 13, 2021, at 4:37 PM, C. M. Heard <[email-masked]> wrote:
> 
> I am going to reveal my ignorance by asking what is meant by RDMA (Remote Direct Memory Access). But I'm fine with that as long as we end up with an unambiguous problem statement.

I may have conflated two issues.

1. Whether UDP supports zero-copy in the presence of reassembly.
2. Whether FRAG needs a frag length per se

I think #2 might strictly depend only on whether non-terminal fragments can have post-fragment options. There doesn’t appear to be reason for doing this; options that apply to the whole reassembled payload would occur after the last fragment and any options that apply to individual fragments appear before FRAG in each non-terminal fragment.

#1 determines whether the data occurs in normal order or whether data in each fragment (including the final one) are moved around. Here’s what that looks like:

- in normal order, each fragment’s data just occurs in-order in each fragment, e.g.,:

	abcde	fghijkl	mnopqrst

- to support zero-copy, we previously had a technique that swapped bytes. Showing this for a full option is a bit complex; a simple version here would assume that each fragment had 3 bytes of options from the beginning of the option space to the start of the FRAG data (it’s always longer, but let’s make it 3 just to show it more simply). In that case we do the following:

	123deabc	123ijklfgh	123parstmno

The idea is that the data can be placed directly in memory, with the last 3 bytes overwriting the first three. By moving only the space covered by the options, direct data placement can be used:

	123de	- write up to the last 3 bytes
	a23de	- overwrite the last 3 bytes starting at the beginning (next 2 lines too)
	ab3de
	abcde

So this doesn’t affect the option format, but does affect whether FRAG data is in-place or wraps-around to support zero-copy.

Joe

---

From: Tom Herbert <[email-masked]>
Date: Wed, 16 Jun 2021 10:16:43 -0700

> a23de - overwrite the last 3 bytes starting at the beginning (next 2 lines too)

That presumes that the memory holding the payload is writeable which
is not always true. For instance, that would not be the case if the
device DMAs the data into GPU memory.

Tom

---

From: Joseph Touch <[email-masked]>
Date: Wed, 16 Jun 2021 17:16:44 -0700

> On Jun 16, 2021, at 10:16 AM, Tom Herbert <[email-masked]> wrote:
> 
>> ...

This approach is already OBE and thus not relevant.

However, the approach above remains an expectation of any DMA transfer - there are always edge aspects that need to be adjusted, whether by a separate DMA or some other mechanism. It doesn’t have to be one byte at a time, as shown above.

Joe

---

From: Tom Herbert <[email-masked]>
Date: Wed, 16 Jun 2021 17:48:21 -0700

On Wed, Jun 16, 2021, 5:16 PM Joseph Touch <[email-masked]> wrote:

> >> The idea is that the data can be placed directly in memory, with the
> last 3 bytes overwriting the first three. By moving only the space covered
> by the options, direct data placement can be used:
> >>

What does OBE mean?

However, the approach above remains an expectation of any DMA transfer -
> there are always edge aspects that need to be adjusted, whether by a
> separate DMA or some other mechanism. It doesn’t have to be one byte at a
> time, as shown above.
>
> Joe
>
>

---

From: "C. M. Heard" <[email-masked]>
Date: Wed, 16 Jun 2021 17:52:53 -0700

> What does OBE mean?

Overtaken By Events

---

From: Gorry Fairhurst <[email-masked]>
Date: Thu, 17 Jun 2021 15:41:06 +0100

I'd personally perfer the simpler formats - unless someone tells me that 
coping with message buffers is made much easier by any more fiddlesome 
(re)ordering of bytes.

When teh dust has settled, I expect we should see an updated 
fragmentation text.  I suspect we should consider the final format for 
suitability for offload (perhaps Tom Herbert would look?) and for ease 
of processing in message nbuffere (Tom Jones offered to look).

Gorry

---

From: Tom Herbert <[email-masked]>
Date: Thu, 17 Jun 2021 10:30:08 -0700

> of processing in message nbuffere (Tom Jones offered to look).

Gorry,

Please look at draft-herbert-udp-space-hdr-01 for my proposal as to
how the UDP surplus space should be formatted. If there is interest, I
could update draft to include considerations for UDP options
fragmentation and reassembly offload as well as header/data split
which is needed for zero-copy receive (i.e. packet headers are DMA'ed
into one set of buffers to be processed by the stack, payload is
DMA'ed to another set of buffers to be processed by the application).

Tom

---

From: Gorry Fairhurst <[email-masked]>
Date: Fri, 18 Jun 2021 10:36:06 +0100

> DMA'ed to another set of buffers to be processed by the application).

Thanks, I saw that a while ago, thanks for reminding people.

I think it would be good for the WG to focus on how to finish 
draft-ietf-tsvwg-udp-options, but I do seem some opportunities to use 
some of these ideas for making the fragment header - because that also 
places all data in the "option".

Gorry

---

From: Joseph Touch <[email-masked]>
Date: Fri, 18 Jun 2021 08:11:45 -0700

> On Jun 18, 2021, at 2:36 AM, Gorry Fairhurst <[email-masked]> wrote:
> 
> I think it would be good for the WG to focus on how to finish draft-ietf-tsvwg-udp-options, but I do seem some opportunities to use some of these ideas for making the fragment header - because that also places all data in the "option”.

The difference between this draft and where we are (and have been before that draft):
	1. it adds an option area length field 
	2. it requires length on all packets with options
	3. It requires OCS on all packets with options

(1) repeats TCP's mistake of limiting the options space (see draft-ietf-tcpm-tcp-edo)
(1) undermines the use of FRAGs where fragments are larger than 1K (not to mention being smaller for per-frag options)
(2) adds a field that is not needed in packets that are not FRAGs
(3) requires checks that are redundant on packets already protected, undermining the flexibility that continues to be available via UDP CS=0

I don’t understand “places all the data in the ‘option’”. It doesn’t appear to do that unless used with FRAG, which we already do in the WG doc.

Joe

---

From: Tom Herbert <[email-masked]>
Date: Fri, 18 Jun 2021 09:44:13 -0700

> This approach is already OBE and thus not relevant.
If you don't like the receive problems, consider those on transmit.
The most common use case of zero copy is sendfile where an application
sends data directly from files, specifically this is done by DMA'ing
buffers in the page cache directly to the NIC without data going
through userspace application. This technique is extremely common in
video servers so there is no question that it's relevant. The
algorithm requires reading and writing three bytes of the application
payload.

If the algorithm is performed in the userspace application then we
need to read the first three bytes of the zero copy data into
userspace, perform a sendmsg consisting of at least three parts: the
three byte FRAG option, the zero copy data offset by three bytes,
followed by the trailer containing the originally value. So any time
we want to invoke sendfile we'd need to read part of the data that we
wanted to do zero copy on (expensive syscall). But there is another
subtle problem, between the time we read the first three bytes of data
and invoke the sendmsg, the page cache data may be changed by a writer
of the file. For instance, the first 128 bytes of the page cache may
have been modified such that the data actually sent is now corrupted.

If the algorithm is performed in the kernel, one might think that we
can just modify the page cache directly, however page cache cache is
shared so modifying it, even temporarily, risks data corruption for
another reader-- so this is a non-starter. We would need to perform
the same steps that I described above in the userspace case where
bytes are copied and we need to offset the zero copy data. The problem
of the page cache being modified still exists, although since we're in
the kernel we may be able to lock the pages during the operation.

In any case, to make zero copy work correctly for both TX and RX with
an algorithm requires manipulating application payload is going to be
much harder to implement and get right than zero copy in plain UDP or
TCP.

Tom

---

From: Joe Touch <[email-masked]>
Date: Fri, 18 Jun 2021 10:04:26 -0700

Tom, 

This thread is OBE because we are no longer proposing individual byte moves at all. I posted the new frag option earlier this week.

Joe

> On Jun 18, 2021, at 9:44 AM, Tom Herbert <[email-masked]> wrote:
>

---

From: Tom Herbert <[email-masked]>
Date: Fri, 18 Jun 2021 10:21:20 -0700

> (1) repeats TCP's mistake of limiting the options space (see draft-ietf-tcpm-tcp-edo)

Allowing an unlimited size and number of options repeats the same
mistake in RFC8200, the only use case for unlimited options is Denial
of Service attack.

> (1) undermines the use of FRAGs where fragments are larger than 1K (not to mention being smaller for per-frag options)

I don't know what that means.

> (2) adds a field that is not needed in packets that are not FRAGs

Not sure to which fields you are referring.

> (3) requires checks that are redundant on packets already protected, undermining the flexibility that continues to be available via UDP CS=0
UDP checksum is required to be non-zero for IPv6, which means this
point is being rendered moot as IPv6 deployment continues. Besides
that, as I've mentioned several times now, if the checksum is in a TLV
then the checksum cannot protect against it's type field being
corrupted and also the checksum serves to validate that the surplus
space is indeed UDP options and not some random bits some hosts
decided to put in there.

---

From: "C. M. Heard" <[email-masked]>
Date: Fri, 18 Jun 2021 13:18:00 -0700

On Fri, Jun 18, 2021 at 2:36 AM Gorry Fairhurst wrote:

> > On Thu, Jun 17, 2021 at 7:41 AM Gorry Fairhurst wrote:
> >> When [the] dust has settled, I expect we should see an updated

Indeed. I wanted to undertake a detailed analysis of the effects of the
primary competing formats on checksum offload, and I was looking around for
the treatment in Appendix A of draft-herbert-udp-space-hdr-01,  especially
the discussion of how common offload hardware works, when the reminder came
in.

Since draft-herbert-udp-space-hdr-01 does not take into account the OCS
pseudo-header (which is one 16-bit word consisting of the surplus area
length), I'm going to do the analysis from scratch in order to account for
this point. My main conclusions are that for the purposes of checksum
offload, IT DOESN'T MATTER whether OCS is at a fixed position in an option
header or in a TLV. The differences are very minor.

What follows is a fairly detailed comparison of how the offload logic
differs between the format proposed in draft-ietf-tsvwg-udp-options-12 and
that in draft-herbert-udp-space-hdr-01. Those who do not care to pick
through the entire analysis should skip down to the CONCLUSIONS section
below.

Tom's draft describes typical hardware support for transmit checksums as
follows:

   In generic checksum offload, for each packet the host indicates to
   the device the starting offset where the checksum calculation begins
   and the offset of the field to write the resultant checksum. The
   extent of the checksum coverage is assumed to be the end of the
   packet. In particular, this means that even if the UDP checksum is
   being offloaded, the UDP surplus space is included in the device's
   computation.

As the draft-herbert-udp-space-hdr-01 notes, there are (in general) two
transmit checksums to be computed: the UDP checksum, and the OCS. The idea
is to use the hardware support to compute whichever checksum covers more
data and relegate the other to the host CPU.

When the packet is a standard UDP datagram with an options trailer in the
surplus area, OCS is typically the one that covers fewer bytes, so the host
CPU prepares and formats the option trailer either as described in
draft-ietf-tsvwg-udp-options-12 or as in draft-herbert-udp-space-hdr-01 and
computes the OCS as specified in draft-ietf-tsvwg-udp-options-12 (i.e.,
including the 1-word pseudo-header). Then the host CPU calculates a
modified UDP pseudo-header where the length is the IP payload length (not
the UDP length) and puts this in the UDP checksum field. Finally, the host
instructs the offload engine to calculate the Internet checksum over the
entire IP payload and store the complement in the UDP checksum field.
Clearly, nothing in the offload logic depends on the details of how the
trailer is formatted, only that the 1s complement sum of all 16-bit words
therein adds up to the 1s complement of the trailer length.

When the packet is a UDP fragment (including a degenerate one that is both
an initial and terminal fragment), the host CPU calculates the UDP header
checksum in the standard way (including using the UDP length of 8 in the
pseudo-header). It then formats the trailer area either as described in
draft-ietf-tsvwg-udp-options-12 or as in draft-herbert-udp-space-hdr-01 and
populates the OCS field with the surplus area length (i.e., IP payload
length - 8). Note that in the case when OCS is a TLV
(draft-ietf-tsvwg-udp-options-12) the issue of alignment may arise: the OCS
pseudo-header needs to be byte-swapped if the OCS checksum field is on an
odd byte boundary. This can be avoided by judicious use of NOPs. The host
CPU then instructs the offload engine to calculate the Internet checksum
over the surplus area and store the complement in the OCS checksum field.
Some offload hardware may not be able to store at an arbitrary byte
boundary, but any hardware alignment requirements can again be accommodated
by judicious use of NOPs. In this case some details of the offload logic
are dependent on the surplus area format, but it is clear, I think, that
either of the competing formats draft-ietf-tsvwg-udp-options-12 or
draft-herbert-udp-space-hdr-01) can be accommodated.

It's easy to accommodate optional UDP checksum (UDP CS=0) and
(conditionally) optional OCS: just omit the unneeded checksum
computation(s). Conditionally optional OCS could be accommodated in the
format proposed in draft-herbert-udp-space-hdr-01 by allowing OCS (therein
called USH checksum) equal to zero to indicate absence.

 The draft describes typical hardware support for receive checksums as
follows:

   In the most generic form of receive checksum offload, a device
   performs a running checksum calculation across a packet as it is
   received. That is, it performs a running ones complement addition
   over two byte words as they are received. The device then provides
   the computed value, referred to as the "checksum complete" value, to
   the host in the meta data (receive descriptor) for the packet. The
   host can use this value to verify one or more packet checksums
   contained in the packet.

In order to use the raw checksum provided in this way. the host CPU needs
to do the following:

1.) Compensate for any lower layer headers that may have been included in
the raw checksum. This could
the IPv4/IPv6 header and any IPv6 extension headers that the NIC does not
strip off. I'm not going to dwell further on this point because the effects
are the same under the different approaches to UDP options.

2.) Determine whether the received packet is a standard UDP packet with an
options trailer and also whether the UCP checksum is present or absent, and
if the UDP checksum is absent, whether OCS is also absent. Determining
whether the a standard UDP packet with an options trailer or a UDP fragment
(possibly a degenerate one that is both an initial and terminal fragment)
is easy: just check to see if UDP Length = 8. Determining whether the UDP
checksum is present is easy: just check to see if UDP checksum = 0.
Determining whether OCS is present is a bit more difficult with the TLV
format in draft-ietf-tsvwg-udp-options-12, but not much so, PROVIDED THAT
OCS IS ALWAYS THE FIRST OPTION, apart from any NOPs that may be present for
alignment.

Let's consider first the case where both checksums are present. The first
step for the host CPU is to compute the ones complement sum of a
pseudo-header that is similar to the standard UDP pseudo-header but uses
the IP payload length in place of the UDP length. When that is added to the
raw checksum coming from the offload engine (following compensation for
lower layer headers) the result will be zero if both the UDP checksum and
the OCS are correct or both have errors that happen to cancel
(see draft-fairhurst-udp-options-cco-00). So if this check passes, the host
CPU needs to separately verify either the UDP checksum or the OCS in order
to ensure that both are valid.

When the packet is a standard UDP datagram with an options trailer in the
surplus area, OCS is usually the cheaper one to verify, and the calculation
required to do so is very little different whether the trailer uses the TLV
format in draft-ietf-tsvwg-udp-options-12 or the format in
draft-herbert-udp-space-hdr-01. Assertions to the contrary notwithstanding,
IT IS NOT NECESSARY TO PARSE the OCS TLV in order to perform the OCS
validation. One need only compute the Internet checksum over the trailer
area using the trailer length as the pseudo-header, same as for the trailer
format described in draft-herbert-udp-space-hdr-01.

When the packet is a UDP fragment (including a degenerate one that is both
an initial and terminal fragment), the host CPU validates the UDP header
checksum in the standard way (including using the UDP length of 8 in the
pseudo-header). There is no difference in the two formats for this.

If UDP CS=0 and OCS is present, then the host CPU needs to validate OCS
only. When the packet is a standard UDP datagram with an options trailer in
the surplus area, the cheapest thing to do is likely just to calculate OCS
directly. When the packet is a UDP fragment, then the host CPU can
profitably use the output of the offload engine by adding the 1's
complement of the 1's complement checksum of the UDP header and then adding
the OCS pseudo-header. In both cases the amount of work is identical (or
nearly so) for both formats.

CONCLUSIONS:

1.) For checksum offload, the differences in the packet formats in
draft-ietf-tsvwg-udp-options-12 and format in
draft-herbert-udp-space-hdr-01 vary only at the edges. For the former it is
necessary to impose a constraint on the placement of the OCS TLV in order
for a receiver to safely determine whether it is present in the case where
UDP CS=0. For the latter this placement constraint is automatic, but on the
other hand there is now an alignment requirement when the options are in a
trailer, which is something that the receiver needs to check. To me, that's
six of one or half a dozen of the other, a wash. I do concede that there is
a question of what happens if an OCS TLV shows up later in the packet when
UDP CS = 0. My sense is that the receiver should just discard the option in
that case and not check OCS.

2.) In both cases the receive logic would be cleaner if we reduced this to
two cases: UDP CS<>0 and OCS is required (as in
draft-ietf-tsvwg-udp-options-12) or UCP CS=0 and OCS is not required (and
ignored if present). Then there would be no need to constrain the placement
of the OCS TLV (since, as I point out above, it's always possible to check
it without parsing the option TLVs). The alignment annoyance with the
herbert-udp-space-hdr-01 trailer format remains. I personally have trouble
seeing a lot of value in UDP CS=0 with OCS present, no matter how hard I
squint; but if the WG really wants it, I'm OK to leave it in.

3.) If both checksums are always present whenever either is, a substantive
simplification suggests itself, namely, not to worry about offsetting
errors in UDP CS and OCS. That weakens the check, but not by a lot, and it
streamlines both implementations that use offload and those that do not.
Again, though, this is just a suggestion; it is not a point on which I feel
strongly.

> places all data in the "option".

I agree that we need to focus on getting this deliverable finished. I'm
rapidly coming to the conclusion that to some extent at least -- and
certainly for checksum offload -- the disagreements that Joe, Tom, and I
have about the packet formats amount largely to bike-shedding. I'm going to
shut up for a while, review the 50+ messages in the "A review of
draft-ietf-tsvwg-udp-options-12," and come back with some substantive
suggestions for convergence.

Mike Heard

---

From: Joe Touch <[email-masked]>
Date: Fri, 18 Jun 2021 13:22:05 -0700

> On Jun 18, 2021, at 10:22 AM, Tom Herbert <[email-masked]> wrote:
> 

Receivers can always decide to drop packets with whatever they deem as too large an option area.  We don’t need to bake a fixed number in the option definition. 

> I don't know what that means.

Please read the draft. 

> Not sure to which fields you are referring.

The length field. 

> UDP checksum is required to be non-zero for IPv6,

Please recheck 8200 and 6936. Zero is still allowed for tunneled packets. 

> which means this

---

From: Tom Herbert <[email-masked]>
Date: Sat, 19 Jun 2021 10:11:29 -0700

On Fri, Jun 18, 2021 at 1:18 PM C. M. Heard <[email-masked]> wrote:
> Indeed. I wanted to undertake a detailed analysis of the effects of the primary competing formats on checksum offload, and I was looking around for the treatment in Appendix A of draft-herbert-udp-space-hdr-01,  especially the discussion of how common offload hardware works, when the reminder came in.
>
> Since draft-herbert-udp-space-hdr-01 does not take into account the OCS pseudo-header (which is one 16-bit word consisting of the surplus area length), I'm going to do the analysis from scratch in order to account for this point. My main conclusions are that for the purposes of checksum offload, IT DOESN'T MATTER whether OCS is at a fixed position in an option header or in a TLV. The differences are very minor.
>
> What follows is a fairly detailed comparison of how the offload logic differs between the format proposed in draft-ietf-tsvwg-udp-options-12 and that in draft-herbert-udp-space-hdr-01. Those who do not care to pick through the entire analysis should skip down to the CONCLUSIONS section below.
>
> Tom's draft describes typical hardware support for transmit checksums as follows:
>
> As the draft-herbert-udp-space-hdr-01 notes, there are (in general) two transmit checksums to be computed: the UDP checksum, and the OCS. The idea is to use the hardware support to compute whichever checksum covers more data and relegate the other to the host CPU.
>
> When the packet is a standard UDP datagram with an options trailer in the surplus area, OCS is typically the one that covers fewer bytes, so the host CPU prepares and formats the option trailer either as described in draft-ietf-tsvwg-udp-options-12 or as in draft-herbert-udp-space-hdr-01 and computes the OCS as specified in draft-ietf-tsvwg-udp-options-12 (i.e., including the 1-word pseudo-header). Then the host CPU calculates a modified UDP pseudo-header where the length is the IP payload length (not the UDP length) and puts this in the UDP checksum field. Finally, the host instructs the offload engine to calculate the Internet checksum over the entire IP payload and store the complement in the UDP checksum field. Clearly, nothing in the offload logic depends on the details of how the trailer is formatted, only that the 1s complement sum of all 16-bit words therein adds up to the 1s complement of the trailer length.
>
Mike,

There is another serious problem with transport checksums and use of
the UDP surplus area. Most of this discussion has presumed that the
UDP checksum is the one in the packet being offloaded, but that may
not be the case. Consider the case where a sender sends a TCP packet
that is encapsulated GRE/UDP or VXLAN (a very common use case in
virtual networks where VMs send packets on their virtual networks).
The stack will attempt to offload the innermost checksum which is the
TCP checksum. The TCP checksum is the one offloaded regardless of
whether or not the outer UDP checksum is zero (if it's non-zero then
the stack would set it using local checksum offload (LCO)). The
offloaded TCP checksum computation would start the computation at the
first byte of the TCP header through the end of the whole packet. So
unless the surplus area is properly checksummed then the computed TCP
checksum will be invalid and the packet will be dropped at the
receiver. This is not just a problem for offload for offload, I
believe that this wouldn't work properly in existing software stacks
without some major changes.

So to make all uses of transport checksum computation and offload
reasonably robust, when the UDP surplus area is being used both the
UDP checksum and checksum over the surplus area MUST always be set.

FYI, here is some nice background checksum offload is
https://www.kernel.org/doc/html/latest/networking/checksum-offloads.html

Tom

> When the packet is a UDP fragment (including a degenerate one that is both an initial and terminal fragment), the host CPU calculates the UDP header checksum in the standard way (including using the UDP length of 8 in the pseudo-header). It then formats the trailer area either as described in draft-ietf-tsvwg-udp-options-12 or as in draft-herbert-udp-space-hdr-01 and populates the OCS field with the surplus area length (i.e., IP payload length - 8). Note that in the case when OCS is a TLV (draft-ietf-tsvwg-udp-options-12) the issue of alignment may arise: the OCS pseudo-header needs to be byte-swapped if the OCS checksum field is on an odd byte boundary. This can be avoided by judicious use of NOPs. The host CPU then instructs the offload engine to calculate the Internet checksum over the surplus area and store the complement in the OCS checksum field. Some offload hardware may not be able to store at an arbitrary byte boundary, but any hardware alignment requirements can again be accommodated by judicious use of NOPs. In this case some details of the offload logic are dependent on the surplus area format, but it is clear, I think, that either of the competing formats draft-ietf-tsvwg-udp-options-12 or draft-herbert-udp-space-hdr-01) can be accommodated.
>
> It's easy to accommodate optional UDP checksum (UDP CS=0) and (conditionally) optional OCS: just omit the unneeded checksum computation(s). Conditionally optional OCS could be accommodated in the format proposed in draft-herbert-udp-space-hdr-01 by allowing OCS (therein called USH checksum) equal to zero to indicate absence.
>
>  The draft describes typical hardware support for receive checksums as follows:
>
> In order to use the raw checksum provided in this way. the host CPU needs to do the following:
>
> 1.) Compensate for any lower layer headers that may have been included in the raw checksum. This could
> the IPv4/IPv6 header and any IPv6 extension headers that the NIC does not strip off. I'm not going to dwell further on this point because the effects are the same under the different approaches to UDP options.
>
> 2.) Determine whether the received packet is a standard UDP packet with an options trailer and also whether the UCP checksum is present or absent, and if the UDP checksum is absent, whether OCS is also absent. Determining whether the a standard UDP packet with an options trailer or a UDP fragment (possibly a degenerate one that is both an initial and terminal fragment) is easy: just check to see if UDP Length = 8. Determining whether the UDP checksum is present is easy: just check to see if UDP checksum = 0. Determining whether OCS is present is a bit more difficult with the TLV format in draft-ietf-tsvwg-udp-options-12, but not much so, PROVIDED THAT OCS IS ALWAYS THE FIRST OPTION, apart from any NOPs that may be present for alignment.
>
> Let's consider first the case where both checksums are present. The first step for the host CPU is to compute the ones complement sum of a pseudo-header that is similar to the standard UDP pseudo-header but uses the IP payload length in place of the UDP length. When that is added to the raw checksum coming from the offload engine (following compensation for lower layer headers) the result will be zero if both the UDP checksum and the OCS are correct or both have errors that happen to cancel (see draft-fairhurst-udp-options-cco-00). So if this check passes, the host CPU needs to separately verify either the UDP checksum or the OCS in order to ensure that both are valid.
>
> When the packet is a standard UDP datagram with an options trailer in the surplus area, OCS is usually the cheaper one to verify, and the calculation required to do so is very little different whether the trailer uses the TLV format in draft-ietf-tsvwg-udp-options-12 or the format in draft-herbert-udp-space-hdr-01. Assertions to the contrary notwithstanding, IT IS NOT NECESSARY TO PARSE the OCS TLV in order to perform the OCS validation. One need only compute the Internet checksum over the trailer area using the trailer length as the pseudo-header, same as for the trailer format described in draft-herbert-udp-space-hdr-01.
>
> When the packet is a UDP fragment (including a degenerate one that is both an initial and terminal fragment), the host CPU validates the UDP header checksum in the standard way (including using the UDP length of 8 in the pseudo-header). There is no difference in the two formats for this.
>
> If UDP CS=0 and OCS is present, then the host CPU needs to validate OCS only. When the packet is a standard UDP datagram with an options trailer in the surplus area, the cheapest thing to do is likely just to calculate OCS directly. When the packet is a UDP fragment, then the host CPU can profitably use the output of the offload engine by adding the 1's complement of the 1's complement checksum of the UDP header and then adding the OCS pseudo-header. In both cases the amount of work is identical (or nearly so) for both formats.
>
> CONCLUSIONS:
>
> 1.) For checksum offload, the differences in the packet formats in draft-ietf-tsvwg-udp-options-12 and format in draft-herbert-udp-space-hdr-01 vary only at the edges. For the former it is necessary to impose a constraint on the placement of the OCS TLV in order for a receiver to safely determine whether it is present in the case where UDP CS=0. For the latter this placement constraint is automatic, but on the other hand there is now an alignment requirement when the options are in a trailer, which is something that the receiver needs to check. To me, that's six of one or half a dozen of the other, a wash. I do concede that there is a question of what happens if an OCS TLV shows up later in the packet when UDP CS = 0. My sense is that the receiver should just discard the option in that case and not check OCS.
>
> 2.) In both cases the receive logic would be cleaner if we reduced this to two cases: UDP CS<>0 and OCS is required (as in draft-ietf-tsvwg-udp-options-12) or UCP CS=0 and OCS is not required (and ignored if present). Then there would be no need to constrain the placement of the OCS TLV (since, as I point out above, it's always possible to check it without parsing the option TLVs). The alignment annoyance with the herbert-udp-space-hdr-01 trailer format remains. I personally have trouble seeing a lot of value in UDP CS=0 with OCS present, no matter how hard I squint; but if the WG really wants it, I'm OK to leave it in.
>
> 3.) If both checksums are always present whenever either is, a substantive simplification suggests itself, namely, not to worry about offsetting errors in UDP CS and OCS. That weakens the check, but not by a lot, and it streamlines both implementations that use offload and those that do not. Again, though, this is just a suggestion; it is not a point on which I feel strongly.
>
> Gorry also wrote:
>>
> I agree that we need to focus on getting this deliverable finished. I'm rapidly coming to the conclusion that to some extent at least -- and certainly for checksum offload -- the disagreements that Joe, Tom, and I have about the packet formats amount largely to bike-shedding. I'm going to shut up for a while, review the 50+ messages in the "A review of draft-ietf-tsvwg-udp-options-12," and come back with some substantive suggestions for convergence.
>
> Mike Heard

---

From: "C. M. Heard" <[email-masked]>
Date: Sat, 19 Jun 2021 11:00:48 -0700

> virtual networks where VMs send packets on their virtual networks).

Is there a use case for employing UDP options with GRE/UDP? Or GUE?

Mike Heard

---

From: Tom Herbert <[email-masked]>
Date: Sat, 19 Jun 2021 11:09:45 -0700

Mike,

We can divide UDP encapsulations into two flavors. Those that are
extensible and contain their own TLVs or equivalent method to encode
ancillary data, and those that don't have that. GUE and Geneve are
examples of UDP encapsulation protocols that have their own
extensibility mechanisms and hence wouldn't be used with UDP options,
GRE/UDP and VXLAN don't have extensibility so UDP options could be
useful with those protocols.

Tom

---

From: Joseph Touch <[email-masked]>
Date: Sat, 19 Jun 2021 12:56:39 -0700

Tom,

> On Jun 19, 2021, at 10:11 AM, Tom Herbert <[email-masked]> wrote:
> 
> ...
> TCP checksum.

I’m assuming this means TCP in GRE/UDP (it seems ambiguous).

> The TCP checksum is the one offloaded regardless of
> the stack would set it using local checksum offload (LCO)).

Can you explain this? Packets are both defined and processed from the outside in; to do anything else might not yield a meaningful result.

> The
> first byte of the TCP header through the end of the whole packet.

If TCP interprets “end” to mean anything beyond what the UDP header indicates, it is that TCP stack that is broken.

(Presumably we’re talking about non-UDP fragmented packets; with UDP fragmentation, the TCP operation should never happen until the UDP fragments are reassembled).

> So
> receiver.

See above.

> This is not just a problem for offload for offload, I

Same problem, same answer. 

> UDP checksum and checksum over the surplus area MUST always be set.

If protocol stacks that try to peek ahead in layers don’t follow the rules, there’s not much we can do, ever. 

> https://www.kernel.org/doc/html/latest/networking/checksum-offloads.html

There is an error in draft-herbert-remotecsumoffload in Section 2.1; it states that the UDP checksum is over the upper layer packet length (it is not). RFC768 defines the UDP checksum over the pseudoheader, UDP header, and data (not referring to the IP packet except for pseudo header info). In fact, it also refers to the pseudoheader as using the UDP Length, not the IP length (adjusted or not). 

The same error appears in the draft-herbert-vxlan-rco in the definition of packet_csum.

If either doc reflects how offloading is implemented, then those are bugs that should be fixed.

In TCPM we’ve identified a number of additional issues with TCP offload, notably regarding how they incorrectly coalesce packets with different TCP headers.  

If we’re not calling out these behaviors as the bugs they are, there’s little point in doing much of anything in the IETF.

Joe

---

From: Tom Herbert <[email-masked]>
Date: Sat, 19 Jun 2021 13:40:19 -0700

> Can you explain this? Packets are both defined and processed from the outside in; to do anything else might not yield a meaningful result.
It is described in
https://www.kernel.org/doc/html/latest/networking/checksum-offloads.html.

> If TCP interprets “end” to mean anything beyond what the UDP header indicates, it is that TCP stack that is broken.
The assumption is that there are no bits in the packet beyond the
transport layer on transmit. This is a valid assumption since there
are currently no use cases where this is true; UDP options would be
the first instance of supporting trailers. On the receive side, stacks
will properly handle checksum offload with data in the surplu space.

> If we’re not calling out these behaviors as the bugs they are, there’s little point in doing much of anything in the IETF.
You're welcome to call these behavior bugs if you want, but the fact
is they are correct behavior and robust behavior for all currently
defined IETF protocols as evidenced by the fact that the Internet runs
on billions of devices with these behaviors. The problem you are
hitting is that we have over thirty years of deployment and
implementation experience with Internet protocols that follow some
basic principles and conventions like use protocol headers and not
trailers. So while a protocol that diverges from those principles and
conventions might be academically correct on paper, in deployment it
may be replete with a myriad of issues which is what we see when
delving into the details of how UDP options interact with real stacks
and devices. If the goal is to produce a deployable and performant
protocol, which I believe is the purpose of IETF, then we need to take
realities of deployment and implementation into account.

Tom

---

From: Joseph Touch <[email-masked]>
Date: Sat, 19 Jun 2021 14:58:13 -0700

> On Jun 19, 2021, at 1:40 PM, Tom Herbert <[email-masked]> wrote:
> 
> are currently no use cases where this is true;

That is the fundamental error; this is not a valid assumption.

> UDP options would be
> the first instance of supporting trailers.

TCP has had trailers too, at one point.

> On the receive side, stacks
> on billions of devices with these behaviors.

“Currently works” is not the same as “correct”.

Correct follows the spec; the above notes do not. They were never in WG docs or that would have been presumably corrected.

> The problem you are
> trailers.

It’s not about headers vs. trailers. It’s about whether you follow the specs or not.

> So while a protocol that diverges from those principles and

We need to start bu not continuing to refer to false claims in unpublished drafts.

When we find errors, we should fix them, not propagate them.

Joe

---

From: Tom Herbert <[email-masked]>
Date: Sat, 19 Jun 2021 15:07:30 -0700

On Sat, Jun 19, 2021, 2:58 PM Joseph Touch <[email-masked]> wrote:

> >> Can you explain this? Packets are both defined and processed from the
> outside in; to do anything else might not yield a meaningful result.
> >>
> > It is described in
> .
> >> If TCP interprets “end” to mean anything beyond what the UDP header
> indicates, it is that TCP stack that is broken.
> >>
> >> (Presumably we’re talking about non-UDP fragmented packets; with UDP
> fragmentation, the TCP operation should never happen until the UDP
> fragments are reassembled).
> >>
> >>> So
> >> There is an error in draft-herbert-remotecsumoffload in Section 2.1; it
> states that the UDP checksum is over the upper layer packet length (it is
> not). RFC768 defines the UDP checksum over the pseudoheader, UDP header,
> and data (not referring to the IP packet except for pseudo header info). In
> fact, it also refers to the pseudoheader as using the UDP Length, not the
> IP length (adjusted or not).
> >>
> >> In TCPM we’ve identified a number of additional issues with TCP
> offload, notably regarding how they incorrectly coalesce packets with
> different TCP headers.
> >>
> >> If we’re not calling out these behaviors as the bugs they are, there’s
> little point in doing much of anything in the IETF.
> >>
> Correct follows the spec; the above notes do not. They were never in WG
> docs or that would have been presumably corrected.
>
> > The problem you are

Joe,

A nice thing about an open source project like Linux is that anyone who
thinks there's a bug they can submit a patch and it will be accepted *if*
you can justify the patch to the maintainers. Good luck, if you want to
take that on!

Tom

---

From: "C. M. Heard" <[email-masked]>
Date: Sat, 19 Jun 2021 16:39:52 -0700

On Sat, Jun 19, 2021 at 11:09 AM Tom Herbert wrote:

> On Sat, Jun 19, 2021 at 11:01 AM C. M. Heard wrote:

> https://www.kernel.org/doc/html/latest/networking/checksum-offloads.html
This was also helpful:
https://www.ietf.org/proceedings/95/slides/slides-95-nvo3-3.pdf

Even after looking at the referenced material, however, I must confess that
I still need clarification on the following points:

1.) I do not understand why the UDP checksum (or anything prior to
csum_start) would matter for the inner packet checksum offload.

2.) Assuming that the trailer checksum (OCS) is present, it is not clear
how offload for the inner packet checksum could be expected to work
properly without change. Remember, OCS does not cause the data in the
trailer to sum to zero. It causes it to sum to the ones-complement of the
trailer length.

The modifications needed to handle point #2 are small -- specifically,
adding the trailer length to whatever would normally be preloaded into the
inner packet. The FRAG proposal
in draft-ietf-tsvwg-udp-options-13#section-5.5, which proposes to sandwich
the payload in between option headers, would make it a whole lot harder.

Looking forward to your reply.

Mike

---

From: Tom Herbert <[email-masked]>
Date: Sat, 19 Jun 2021 18:01:06 -0700

On Sat, Jun 19, 2021 at 4:40 PM C. M. Heard <[email-masked]> wrote:
> This was also helpful: https://www.ietf.org/proceedings/95/slides/slides-95-nvo3-3.pdf
>
> Even after looking at the referenced material, however, I must confess that I still need clarification on the following points:
>
> 1.) I do not understand why the UDP checksum (or anything prior to csum_start) would matter for the inner packet checksum offload.
>
It's relevant to checksum offload where devices can only offload one
checksum in the packet hence the best one to offload is the innermost
one since we can take advantage of techniques like LRO.

> 2.) Assuming that the trailer checksum (OCS) is present, it is not clear how offload for the inner packet checksum could be expected to work properly without change. Remember, OCS does not cause the data in the trailer to sum to zero. It causes it to sum to the ones-complement of the trailer length.
>
Regardless, of anything else, the hard requirement is that all
checksums in the packet must be correctly computed when transmitting.
If we are offloading a checksum in the packet, be it the outer UDP
checksum or an inner TCP checksum, the result must be correct. So if
there's bits in the surplus area and that the offload checksum covers
then the surplus area needs to sum to zero or whatever value is
necessary to ensure the offloaded transport checksum is correctly
computed.

> The modifications needed to handle point #2 are small -- specifically, adding the trailer length to whatever would normally be preloaded into the inner packet. The FRAG proposal in draft-ietf-tsvwg-udp-options-13#section-5.5, which proposes to sandwich the payload in between option headers, would make it a whole lot harder.
>
Protocol trailers are a fundamental problem. At this point it's not
clear to me that there is a way to make them work correctly in all use
cases with deployed HW.  Protocol headers, like those used per FRAG,
shouldn't be a problem,

Tom

---

From: Joseph Touch <[email-masked]>
Date: Sat, 19 Jun 2021 21:57:48 -0700

> On Jun 19, 2021, at 3:07 PM, Tom Herbert <[email-masked]> wrote:
> 
> Joe,
> 
> A nice thing about an open source project like Linux is that anyone who thinks there's a bug they can submit a patch and it will be accepted *if* you can justify the patch to the maintainers. 

And the bad thing is that they take lots of erroneous stuff too.

The point is not whether those bugs exist or any of us need to care, it’s that they do not dicate our protocol designs.

Bugs are bugs.

Joe

---

From: "Rodney W. Grimes" <[email-masked]>
Date: Sun, 20 Jun 2021 05:52:20 -0700 (PDT)

Joe,
	I caught this comment and wish for some eloboration so I
can better understand what may of already been identified, where it
is called out, and what can be done to make corrections.

...
> 
> In TCPM we?ve identified a number of additional issues with TCP offload, notably regarding how they incorrectly coalesce packets with different TCP headers.  
> 
> If we?re not calling out these behaviors as the bugs they are, there?s little point in doing much of anything in the IETF.
> 
> Joe

I have a larger scope on the "different TCP headers" in that IMHO
it should be clearly spelled out that even presently reserved bits
should be considered when coalesce of packets occur, as if it is
not considered potential future uses of these reserved bits may cause
issues.  An example is that in our experimenting with SCE we found
that some ACK thinning code out there combines ACK's that have
different values in the former NS bit we used for feeding back
ESCE (Echo Some Congestion Experienced).  Investigation of the
code infact yielded that it ignored all reserved bits in its
decision to combine packets.

Has there been any published RFC text that addresses these problems?

Regards,
-- 
Rod Grimes                                                 [email-masked]

---

From: "Rodney W. Grimes" <[email-masked]>
Date: Sun, 20 Jun 2021 06:06:13 -0700 (PDT)

> And the bad thing is that they take lots of erroneous stuff too.

And a further worse thing, especially with mind sets that made the
above statement, is that Linux defines the internet protocols,
not the IETF if one allows the above argument to be the norm.

> The point is not whether those bugs exist or any of us need to care, it?s that they do not dicate our protocol designs.

Sadly that has become a reality, I often run into the mentality
of "well how does Linux do it" as the golden standard, and not
"how do the specs say to do it."

> Bugs are bugs.
Yes, and left unaddressed and uncorrected are leading to some
very real world problems in protocol developement.

---

From: Tom Herbert <[email-masked]>
Date: Sun, 20 Jun 2021 09:41:59 -0700

Joe,

All of this grandstanding and Linux bashing aside, technical issues
concerning UDP options are being raised that may be detrimental to the
running implementation or deployability of the protocol. I believe
these issues warrant a better answer than just calling them
implementation bugs and otherwise ignoring them.

Tom

---

From: Joseph Touch <[email-masked]>
Date: Sun, 20 Jun 2021 10:01:18 -0700

> On Jun 20, 2021, at 9:41 AM, Tom Herbert <[email-masked]> wrote:
> 

The only technical issue shown so far is in the *assumptions* about UDP, not in UDP itself.

If there is a technical issue *in UDP*, then prove it.

Joe

---

From: Tom Herbert <[email-masked]>
Date: Sun, 20 Jun 2021 10:51:48 -0700

> If there is a technical issue *in UDP*, then prove it.

Nope, that's not how it works. As an author of the draft the impetus
is on you to demonstrate the robustness of the protocol and
feasibility of implementation and deployment. As I don't believe
further discussion on this would be productive, I'll defer to the WG
as to whether the authors have sufficiently addressed comments made on
the list.

Tom

---

From: Joseph Touch <[email-masked]>
Date: Sun, 20 Jun 2021 11:47:07 -0700

> On Jun 20, 2021, at 10:51 AM, Tom Herbert <[email-masked]> wrote:
> 
> On Sun, Jun 20, 2021 at 10:01 AM Joseph Touch <[email-masked] <mailto:[email-masked]>> wrote:
>> ...
> feasibility of implementation and deployment.

We have so far; your non-WG drafts with demonstrated errors are the only counterexample.

But I agree that we can leave the rest to the WG.

Joe

---

From: "C. M. Heard" <[email-masked]>
Date: Sun, 20 Jun 2021 16:36:18 -0700

On Sat, Jun 19, 2021 at 6:01 PM Tom Herbert wrote:

> On Sat, Jun 19, 2021 at 4:39 PM C. M. Heard wrote:
>

I believe that fear is unjustified. In every case analyzed in detail so far
-- including this one -- trailers involve only minor rework at most.  I
believe that I proved my case conclusively for the usual case (UDP used as
a straight transport, not as an encapsulation) in
https://mailarchive.ietf.org/arch/msg/tsvwg/RZULHOKRgrSYIvsI-5Hg7sNtSS8/.
Additionally, while an encapsulator that uses UDP options may need to be
modified in order to use hardware checksum offload, I think it's clear that
the modifications needed are bounded and small. Actually for the case of a
TCP inner packet the pseudo-header will be OK to use with a trailer that
has OCS included if instead of the actual RCP length the pseudo-header
includes TCP + trailer length. The driver still has to remember to copy
only UDP length - 8 bytes. Arguably, if it were written correctly in the
first place, it would be ready now.

The proposal in the -13 draft does have problems in this regard. I'll reply
to that thread in the next day or so with an explanation.

Thanks,

Mike Heard

---

From: Joseph Touch <[email-masked]>
Date: Sun, 20 Jun 2021 16:37:11 -0700

> On Jun 20, 2021, at 5:52 AM, Rodney W. Grimes <[email-masked]> wrote:
> 
> Joe,

None that I am aware. We mention it in draft-ietf-tcpm-edo as a concern. In TCPM, there were proposals to require an EDO integrity check to detect such erroneous coalescing, but we decided there not to require it for similar reasons - i.e., that it’s a losing game to design protocols against deliberate incorrect implementation.

It would be useful to explore current offloading behavior regarding unknown options, though.

Joe

---

From: Tom Herbert <[email-masked]>
Date: Sun, 20 Jun 2021 18:24:31 -0700

On Sun, Jun 20, 2021 at 4:36 PM C. M. Heard <[email-masked]> wrote:
> I believe that fear is unjustified. In every case analyzed in detail so far -- including this one -- trailers involve only minor rework at most.  I believe that I proved my case conclusively for the usual case (UDP used as a straight transport, not as an encapsulation) in https://mailarchive.ietf.org/arch/msg/tsvwg/RZULHOKRgrSYIvsI-5Hg7sNtSS8/.

Mike,

Please consider the simple case where the UDP checksum is set. There
are two common methods to offload the checksum: protocol agnostic and
protocol specific. In the protocol agnostic case the stack provides
the starting offset of the checksum area (offset of the UDP header)
and the offset of the checksum field to write the result (UDP
checksum)-- these values are placed in the TX descriptor for a packet.
The checksum field is primed with the pseudo header. The end of the
checksum area is the end of the packet (trying to add a length of the
area in the hardware interface is a non-starter). If there is surplus
and that sums to zero then the algorithm works unchanged. If the
surplus area sums to non-zero then that requires an offsetting sum by
the host adding in the negative sum value of the surplus area into the
checksum field. The good news is that this works with any case of
checksum offload in a UDP packet including encapsulated checksums. In
any case, if the surplus area always sums to zero that would not
require any rework, a non-zero sum surplus area is going to require
work in the OS (I have not looked at what would be required, but it
might be viewed as significant in especially if we need to add a new
field to mbuf, skbuff, etc.)

Protocol specific transmit checksum offload is when the device parses
the protocol and does all the work to send a checksum. It is not the
preferred method, however it is still quite prevalent. We simply have
no idea how devices will do checksum offload done in the presence of
surplus area, I suspect implementations are all over the place. Some
might only checksum over the UDP length, some might checksum over the
surplus area also, some might use the wrong length in the pseudo
header like described in draft. If I had to take a guess on what gives
the best chances for this to work correctly for the most devices, it
would be to always ensure the surplus space sums to zero.

It's a similar story for receive checksum offload, there is a protocol
agnostic and protocol specific method. The protocol generic method
works in all cases, but does require that the host computes the
checksum over the surplus area regardless of whether a checksum in the
area was explicitly set. For protocol specific checksum offload, again
we really have no idea how various devices will deal with surplus
area. Like the send case, I believe that the greatest chance for
compatibility would be to ensure the surplus area sums to zero.

Tom

> Additionally, while an encapsulator that uses UDP options may need to be modified in order to use hardware checksum offload, I think it's clear that the modifications needed are bounded and small. Actually for the case of a TCP inner packet the pseudo-header will be OK to use with a trailer that has OCS included if instead of the actual RCP length the pseudo-header includes TCP + trailer length. The driver still has to remember to copy only UDP length - 8 bytes. Arguably, if it were written correctly in the first place, it would be ready now.
>
> The proposal in the -13 draft does have problems in this regard. I'll reply to that thread in the next day or so with an explanation.
>
> Thanks,
>
> Mike Heard
>

---

From: Joseph Touch <[email-masked]>
Date: Sun, 20 Jun 2021 20:04:43 -0700

> On Jun 20, 2021, at 6:24 PM, Tom Herbert <[email-masked]> wrote:
…
> checksum area is the end of the packet

No, it is not and has never been. Again, see the text I quoted from RFCs 768, repeated below:

Checksum is the 16-bit one's complement of the one's complement sum of a
pseudo header of information from the IP header, the UDP header, and the
data,  padded  with zero octets  at the end (if  necessary)  to  make  a
multiple of two octets.

Data, which is the UDP field. Not entire IP packet payload, as noted just before the above paragraph:

Length  is the length  in octets  of this user datagram  including  this
header  and the data.   (This  means  the minimum value of the length is
eight.)

Please stop repeating any claim to the contrary, except to note an error in an existing implementation (which should be corrected, not propagated).

> (trying to add a length of the
> checksum offload in a UDP packet including encapsulated checksums.

Checksums of packets that encapsulate IP/UDP as a payload would cover the entire IP packet, not just the UDP portion.

Checksums of IP/UDP that encapsulate other packets as a payload would place that packet in the UDP data area, preceding the options if not fragmented.

If errors are corrected, we can stop trying to do cartwheels to enable them.

Joe

---

From: Tom Herbert <[email-masked]>
Date: Sun, 20 Jun 2021 20:43:36 -0700

On Sun, Jun 20, 2021, 8:04 PM Joseph Touch <[email-masked]> wrote:

> Please stop repeating any claim to the contrary, except to note an error
> in an existing implementation (which should be corrected, not propagated).
>

I am describing how real implementation actually *works*. There is no error
because the correct checksum per RFC768 can be produced.

> Checksums of packets that encapsulate IP/UDP as a payload would cover the
> entire IP packet, not just the UDP portion.
>
> Checksums of IP/UDP that encapsulate other packets as a payload would
> place that packet in the UDP data area, preceding the options if not
> fragmented.
>

---

From: Joseph Touch <[email-masked]>
Date: Sun, 20 Jun 2021 20:58:22 -0700

> On Jun 20, 2021, at 8:43 PM, Tom Herbert <[email-masked]> wrote:
> 
> 
> 
> On Sun, Jun 20, 2021, 8:04 PM Joseph Touch <[email-masked] <mailto:[email-masked]>> wrote:
> 
> 
>> On Jun 20, 2021, at 6:24 PM, Tom Herbert <[email-masked] <mailto:[email-masked]>> wrote:
> …
> 
> I am describing how real implementation actually *works*.

I do not doubt that, but you *are* propagating an error.

> There is no error because the correct checksum per RFC768 can be produced.

That logic is equivalent to “we do math in base 6” because you only ever tried to add 2+2=4. 

It’s both faulty as logic and incorrect.

Joe

---

From: Tom Herbert <[email-masked]>
Date: Sun, 20 Jun 2021 21:11:49 -0700

> I do not doubt that, but you *are* propagating an error.

That is your opinion.

> It’s both faulty as logic and incorrect.

No, it's not. The algorithms I described are provably correct. One's
complement addition is commutative and associative so:

Sum (whole_packet_from_UDP_header) = Sum(UDP_packet) + Sum(Surplus_area)

So if Sum(Surplus_area == 0) then Sum (whole_packet_from_UDP_header) =
Sum(UDP_packet), and therefore rather an offload performs the computation
over the just the UDP packet or includes the surplus area they get the same
correct result.

Tom

---

From: Joseph Touch <[email-masked]>
Date: Sun, 20 Jun 2021 21:34:10 -0700

> On Jun 20, 2021, at 9:11 PM, Tom Herbert <[email-masked]> wrote:
> 
> 
> 
> On Sun, Jun 20, 2021, 8:58 PM Joseph Touch <[email-masked] <mailto:[email-masked]>> wrote:
> 
> 
>> On Jun 20, 2021, at 8:43 PM, Tom Herbert <[email-masked] <mailto:[email-masked]>> wrote:
>> 
>> 
>> 

It is explicit in the text cited above. 

> No, it's not. The algorithms I described are provably correct. One's complement addition is commutative and associative so:
> 

Agreed, where the first one is “whole ***IP** packet from UDP header”. The UDP packet has a length and that’s not it.

> So if Sum(Surplus_area == 0)

That is not a requirement anywhere in UDP, but note that there are two zeroes in ones complement arithmetic.

> then Sum (whole_packet_from_UDP_header) = Sum(UDP_packet),

Sure. And if Sum(Surplus_area == 1, then Sum(IP payload) == Sum(UDP packet) + 1.

> and therefore rather an offload performs the computation over the just the UDP packet or includes the surplus area they get the same correct result.

Said more specifically, an offload that is implemented INCORRECTLY will still work.

Yes.

Again, we already support that when OCS is included for many versions of the draft now. We don’t support it when it isn’t, but that’s only allowed when UDP CS=0. These two are coupled for reasons other than making erroneous checksum work.

What’s your point?

Joe

---

From: Tom Herbert <[email-masked]>
Date: Mon, 21 Jun 2021 07:34:02 -0700

> Said more specifically, an offload that is implemented INCORRECTLY will still work.
IETF standards specify behavior of protocols, they do not mandate how
protocols are implemented. An implementation that produces the
specified behavior is correct and conformant with the standard.

> Again, we already support that when OCS is included for many versions of the draft now. We don’t support it when it isn’t, but that’s only allowed when UDP CS=0. These two are coupled for reasons o…

In my opinion, the best chance for UDP options to be deployable and
successful is to require that the surplus area always sums to zero.

---

From: Joseph Touch <[email-masked]>
Date: Mon, 21 Jun 2021 08:47:29 -0700

> On Jun 21, 2021, at 7:34 AM, Tom Herbert <[email-masked]> wrote:
...
> IETF standards specify behavior of protocols,

They DEFINE protocols, including their behavior.

> they do not mandate how
> protocols are implemented.

Agreed, except that the implementation is faulty unless if follows the definition.

> An implementation that produces the

If the ones you cite always produced correct answers, yes.

But your claim that “I’ve never seen an error” is like saying “but my [broken] watch is fine because I only look at it twice a day - at 8am each time”.

Just because you never experienced a corner case doesn’t mean your code is correct. It means the code hasn’t been sufficiently tested.

…
> successful is to require that the surplus area always sums to zero.

When UDP CS==0, OCS provides that capability. When OCS isn’t zero, it doesn’t (and cannot) matter to the UDP layer. It should never matter to any other layer, because that layer is either entirely inside the UDP data or completely encloses the IP packet. There are no protocols that encapsulate UDP but not its outer IP; UDP (like all IETF transport protocols) is not even defined in the absence of an IP wrapper.

If you have a case you believe falls outside what I’ve just indicated, please provide it. I believe it will demonstrate an error in how protocols are encapsulated, not an error where OCS will be required when UDP CS == 0.

Joe

---

From: Tom Herbert <[email-masked]>
Date: Mon, 21 Jun 2021 09:22:10 -0700

On Mon, Jun 21, 2021, 8:47 AM Joseph Touch <[email-masked]> wrote:

> But your claim that “I’ve never seen an error” is like saying “but my
> [broken] watch is fine because I only look at it twice a day - at 8am each
> time”.
>

I have NEVER said that, in fact have seen and FIXED many bugs in this area.
You are completely misconstruing my words.

> Just because you never experienced a corner case doesn’t mean your code is
> correct. It means the code hasn’t been sufficiently tested.

> When UDP CS==0, OCS provides that capability. When OCS isn’t zero, it
> doesn’t (and cannot) matter to the UDP layer. It should never matter to any
> other layer, because that layer is either entirely inside the UDP data or
> completely encloses the IP packet. There are no protocols that encapsulate
> UDP but not its outer IP; UDP (like all IETF transport protocols) is not
> even defined in the absence of an IP wrapper.
>
> If you have a case you believe falls outside what I’ve just indicated,
> please provide it. I believe it will demonstrate an error in how protocols
> are encapsulated, not an error where OCS will be required when UDP CS == 0.
>

I have provided that, like offloading a transport checksum in UDP
encapsulation.

---

From: "C. M. Heard" <[email-masked]>
Date: Mon, 21 Jun 2021 09:31:40 -0700

Tom,

Had that statement been "the best chance for UDP options to be deployable
and successful is to require that the surplus area always sums to the
ones-complement of the surplus area length." But the statement above, taken
literally, is NOT TRUE.

You may remember the lengthy discussion that centered around the discovery
by the Aberdeen group (Fairhurst, Jones, and Zullo) that a significant
proportion of paths across the Internet BLOCK packets with a surplus area,
even if the UDP checksum is correct, unless it has been arranged that the
checksum ALSO comes to (ones complement) zero when computed over the IP
payload length and with the IP payload length substituted for the UDP
length in the pseudo-header.

See
https://datatracker.ietf.org/meeting/103/materials/slides-103-maprg-a-tale-of-two-checksums-tom-jones

I'm pretty sure that you knew of this, but forgot, because I saw messages
in the archive wherein you discussed this matter with Raffaele Zullo.

The root cause appears to be a BUG in middlebox implementations that
substitute IP payload for UDP payload length in the UDP checksum
computation. See pages 5 and 6 of the presentation. Note that the BUG was
corrected in Freebsd.

Making UDP options viable across a large proportion of real-world paths in
the Internet is a vital step in making it successful, and that is why the
WG (seems to have) converged on computing OCS so that it has the same
effect as the checksum compensation option (CCO, see
draft-ietf-fairhurst-udp-options-cco).

If the Linux stack pervasively takes the shortcut of substituting IP
payload length for UDP length, or making a similar substitution in
encapsulations, that is a BUG of the very same kind that CCO was invented
to work around. I have to agree with Joe on this point.

Having said that, I do still agree that if it is possible without undue
burden, it is desirable for the UDP options spec to play well with the
generic hardware checksum offload that you described in
draft-herbert-udp-space-hdr-01 that sum to the end of the packet. It is NOT
a bug for the device to offer the offload capability in that form (though
it would be a bug if the host software or the NIC firmware did not use that
capability correctly).

Happily, generic checksum offload works just fine for the usual case
(non-encapsulated data), as I think I proved  conclusively in
https://mailarchive.ietf.org/arch/msg/tsvwg/RZULHOKRgrSYIvsI-5Hg7sNtSS8/. I
think that the version of FRAG in -13 of the current draft may have some
problems with encapsulations, as I stated in
https://mailarchive.ietf.org/arch/msg/tsvwg/thc0mIjU6CcyppULA1L0aV3Mk7M/,
and I will offer specific comments on that. What I am not going to do is go
down any more ratholes where I pose a question to you about what is wrong
and then get in return a non-response that simply asserts that what we are
doing won't work with Linux. You have not made your case, and I will not
respond further until you do. Life is too short.

I have, of course, not made the case about protocol-specific checksum
offload hardware, but I do not think that it's reasonable to expect a
priori that a modification to UDP can be expected to work with them. If you
will provide pointers to specs for (or clear descriptions of) some common
ones I'll look when I have time.

Mike

---

From: "C. M. Heard" <[email-masked]>
Date: Mon, 21 Jun 2021 09:37:17 -0700

On Mon, Jun 21, 2021 at 8:48 AM Joseph Touch wrote:

> When UDP CS==0, OCS provides that capability.

It would, if the requirement for a receiver to check OCS when it is present
were dropped when UCP CS==0.

If that's not in -13, I'll make a comment requesting it.

Thanks for bringing this up.

Mike

---

From: Tom Herbert <[email-masked]>
Date: Mon, 21 Jun 2021 09:46:20 -0700

> in the archive wherein you discussed this matter with Raffaele Zullo.

Yes, I am aware of that. That is in fact a bug (middle boxes also shouldn't
be looking at UDP checksum in the first). The fix in the draft will
workaround that particular bug, but that would conflict with devices that
include the surplus area in their computation but use the correct length in
the pseudo header. Unfortunately these are conflicting requirements, so
100% correctness across all currently deployed devices may be impossible.
On the host side we at least have some control, for instance we could
disable TX checksum offload if we don't if the device can do it correctly.
RX offload would simply fail if the device doesn't do it right and host
would do checksum in CPU. The software stack can reworked to deal with any
use case and protocol agnostic offloads can work, but as I said the
simplest solution to that end would be surplus area summing to zero.

Tom

> I think that the version of FRAG in -13 of the current draft may have some

---

From: "C. M. Heard" <[email-masked]>
Date: Mon, 21 Jun 2021 11:05:48 -0700

On Mon, Jun 21, 2021 at 9:46 AM Tom Herbert  wrote:

> On Mon, Jun 21, 2021, 9:31 AM C. M. Heard wrote:
>
>> You may remember the lengthy discussion that centered around the
>> discovery by the Aberdeen group (Fairhurst, Jones, and Zullo) that a
>> significant proportion of paths across the Internet BLOCK packets with a
>> surplus area, even if the UDP checksum is correct, unless it has been
>> arranged that the checksum ALSO comes to (ones complement) zero when
>> computed over the IP payload length and with the IP payload length
>> substituted for the UDP length in the pseudo-header.
>>
>> See
> Yes, I am aware of that. That is in fact a bug (middle boxes also
> shouldn't be looking at UDP checksum in the first). The fix in the draft
> will workaround that particular bug, but that would conflict with devices
> that include the surplus area in their computation but use the correct
> length in the pseudo header. Unfortunately these are conflicting
> requirements, so 100% correctness across all currently deployed devices may
> be impossible.
>

That's true, it's not possible to compensate for all of the middlebox bugs.
But thanks to the work of the Aberdeen group, we have some actual data.
They found that found that paths that passed only the following oddball
combinations

"3rd CS" == "Full Payload Checksum, UDP length Pseudoheader"
"4th CS" == "UDP Length Checksum, IP length Pseudoheader"

were MUCH smaller in number than those that worked with CCO (" Full Payload
Checksum, IP length Pseudoheader").

The chosen solution works for the majority of cases and so is a logical
choice.

> disable TX checksum offload if [ ... ] the device can[not] do it correctly.

Making the surplus area sum to zero may well make SOME host software easier
to update to be compatible with UDP options, and it would work around SOME
host bugs.

But it would cause UDP options to fail over 25% - 25% of Internet paths.
Note the word PATH, which includes middleboxes AND hosts.

That IMO is not a good design tradeoff.

Mike

---

From: Joe Touch <[email-masked]>
Date: Mon, 21 Jun 2021 11:17:41 -0700

> On Jun 21, 2021, at 9:22 AM, Tom Herbert <[email-masked]> wrote:
> 
>> 
> I have NEVER said that, in fact have seen and FIXED many bugs in this area. You are completely misconstruing my words.

I was assuming this discussion did not need compilation-level precision. 

I should have said “never seen *this* error” (of erroneously using the IP length to infer the UDP data length rather than using the UDP length field). 

The point remains. Not seeing a bug does not mean the bug does not exist or that the code implements the spec. 

Joe

---

From: Joe Touch <[email-masked]>
Date: Mon, 21 Jun 2021 11:19:39 -0700

> On Jun 21, 2021, at 9:37 AM, C. M. Heard <[email-masked]> wrote:
> 
> It would, if the requirement for a receiver to check OCS when it is present were dropped when UCP CS==0.

Why / how does this help? 

Joe

---

From: Tom Herbert <[email-masked]>
Date: Mon, 21 Jun 2021 11:52:54 -0700

> I should have said “never seen *this* error” (of erroneously using the IP length to infer the UDP data length rather than using the UDP length field).
That would still be wrong with respect to what I actually said. I said
the algorithm I described was provably correct and gave the proof.

> The point remains. Not seeing a bug does not mean the bug does not exist or that the code implements the spec.
Neither does not seeing a bug mean that there is a bug and the code
does not implement spec correctly. If you want to claim implementation
has a bug, then you'll need to show us the specific reproducible use
case and configuration for the specific implementation that produces
the bug.

Tom

---

From: "C. M. Heard" <[email-masked]>
Date: Mon, 21 Jun 2021 12:48:00 -0700

> Why / how does this help?

You said (or at least I thought you said) that when UDP CS==0, OCS provides
the capability to force the surplus area to sum to zero.

If I understand the current spec correctly, then that is not really true.

It's true that OCS is optional when UDP CS==0, but my understanding is that
per the current spec, if it is present, the receiver is required to check
it and drop the options if it does not follow the rule specified for OCS.
And that rule is not that the surplus area sums to zero, but that the
surplus area plus the pseudo-header consisting of the surplus are length
sums to zero.

Mike

---

From: Joe Touch <[email-masked]>
Date: Mon, 21 Jun 2021 13:24:24 -0700

I’m not going to engage in a discussion that requires quotation of a “court record”,

Any code that uses IIP length to determine the UDP data area, eg gir the UDP checksum, it would be incorrect. The correct UDP data length Is indicated in the UDP header, per RFC 768. 

Joe

> On Jun 21, 2021, at 11:53 AM, Tom Herbert <[email-masked]> wrote:
>

---

From: Joseph Touch <[email-masked]>
Date: Mon, 21 Jun 2021 15:20:20 -0700

Hi, Mike,

> On Jun 21, 2021, at 12:48 PM, C. M. Heard <[email-masked]> wrote:
> 
> On Mon, Jun 21, 2021 at 11:19 AM Joe Touch <[email-masked] <mailto:[email-masked]>> wrote:
>> On Jun 21, 2021, at 9:37 AM, C. M. Heard <[email-masked] <mailto:[email-masked]>> wrote:
> You said (or at least I thought you said) that when UDP CS==0, OCS provides the capability to force the surplus area to sum to zero.

OCS works like CCO did mostly, at least computationally - or at lest that’s what it’s supposed to do.

> It's true that OCS is optional when UDP CS==0, but my understanding is that per the current spec, if it is present, the receiver is required to check it and drop the options if it does not follow the rule specified for OCS.

Yes, and that makes sense - if you’ve added OCS, regardless of whether the UDP CS==0 or not, then you want the options checked. All we currently have been saying is that “OCS can be omitted when CS==0”, not that it MUST be.

> And that rule is not that the surplus area sums to zero, but that the surplus area plus the pseudo-header consisting of the surplus are length sums to zero.

It was originally (through draft-ietf-tsvwg-udp-options-05)  to cover the surplus area only; it was in the additional CCO-like behavior that it changed to add the pseudo header  (the surplus length). So are we going back to the pre-CCO definition? 

Joe

---

From: "C. M. Heard" <[email-masked]>
Date: Mon, 21 Jun 2021 15:29:44 -0700

On Mon, Jun 21, 2021 at 3:20 PM Joseph Touch wrote:

> It's true that OCS is optional when UDP CS==0, but my understanding is
> that per the current spec, if it is present, the receiver is required to
> check it and drop the options if it does not follow the rule specified for
> OCS.
>
>
> Yes, and that makes sense - if you’ve added OCS, regardless of whether the
> UDP CS==0 or not, then you want the options checked. All we currently have
> been saying is that “OCS can be omitted when CS==0”, not that it MUST be.
>
> It was originally (through draft-ietf-tsvwg-udp-options-05)  to cover the
> surplus area only; it was in the additional CCO-like behavior that it
> changed to add the pseudo header  (the surplus length). So are we going
> back to the pre-CCO definition?
>

I'm certainly not advocating for that! Tom Hernert seems to want that, but
I sure don't, and as I indicated in my reply to him, I think that going
back to the pre-CCO definition would undermine the chances of successful
deployment.

Mike

---

From: Joseph Touch <[email-masked]>
Date: Mon, 21 Jun 2021 15:43:27 -0700

> On Jun 21, 2021, at 3:29 PM, C. M. Heard <[email-masked]> wrote:
> 
> I'm certainly not advocating for that! Tom Hernert seems to want that, but I sure don't, and as I indicated in my reply to him, I think that going back to the pre-CCO definition would undermine the chances of successful deployment.

OK. I thought I saw something in your response that indicated either that we needed to adjust the calculation or that we need to force it being ignored when UDP CS==0.

Joe