Subject: [tsvwg] Fwd: New Version Notification for draft-herbert-udp-space-hdr-01.txt

From: Tom Herbert <[email-masked]>
Date: Mon, 8 Jul 2019 16:52:23 -0700

Hello,

I've posted a new draft version for the UDP surplus space header. The
version provides detail on the use cases of the UDP Surplus Header as
a protocol trailer and protocol header. Also, I added a motivations
section and added appendices for implementation considerations.

Thanks,
Tom

---------- Forwarded message ---------
From: <[email-masked]>
Date: Mon, Jul 8, 2019 at 4:48 PM
Subject: New Version Notification for draft-herbert-udp-space-hdr-01.txt
To: Tom Herbert <[email-masked]>

A new version of I-D, draft-herbert-udp-space-hdr-01.txt
has been successfully submitted by Tom Herbert and posted to the
IETF repository.

Name:           draft-herbert-udp-space-hdr
Revision:       01
Title:          UDP Surplus Header
Document date:  2019-07-08
Group:          Individual Submission
Pages:          16
URL:
https://www.ietf.org/internet-drafts/draft-herbert-udp-space-hdr-01.txt
Status:         https://datatracker.ietf.org/doc/draft-herbert-udp-space-hdr/
Htmlized:       https://tools.ietf.org/html/draft-herbert-udp-space-hdr-01
Htmlized:
https://datatracker.ietf.org/doc/html/draft-herbert-udp-space-hdr
Diff:           https://www.ietf.org/rfcdiff?url2=draft-herbert-udp-space-hdr-01

Abstract:
   This specification defines the UDP Surplus Header that is an
   extensible and generic format applied to the UDP surplus space. The
   UDP surplus space comprises the bytes between the end of the UDP
   datagram, as indicated by the UDP Length field, and the end of the IP
   packet, as indicated by IP packet or payload length. The UDP Surplus
   Header can be either a protocol trailer of the UDP datagram, or a
   protocol header which effectively serves as an extended UDP header.

Please note that it may take a couple of minutes from the time of submission
until the htmlized version and diff are available at tools.ietf.org.

The IETF Secretariat

---

From: Joe Touch <[email-masked]>
Date: Wed, 10 Jul 2019 07:51:21 -0700

This approach serves absolutely no useful purpose. 

The UDP options already provide a vehicle for others to use the surplus space and it already consumes the entirety of that space.

I hope those in the WG interested in improving UDP options will comment on the other posted issues instead.

Joe

> On Jul 8, 2019, at 4:52 PM, Tom Herbert <[email-masked]> wrote:
> 
> Hello,
>

---

From: Tom Herbert <[email-masked]>
Date: Wed, 10 Jul 2019 09:18:01 -0700

> This approach serves absolutely no useful purpose.

Obviously, I disagree with that assessment. See the motivations section in
the draft.

> The UDP options already provide a vehicle for others to use the surplus
> space and it already consumes the entirety of that space.
>

Well, I've already posted many comments on that UDP options proposal, I
would hope my proposal gets equally constructive feedback from the WG (more
than just an arbitrary statement that it has no useful purpose).

Tom

> > Status:
> https://datatracker.ietf.org/doc/draft-herbert-udp-space-hdr/
> > Htmlized:
> https://tools.ietf.org/html/draft-herbert-udp-space-hdr-01
> > Htmlized:
> > Diff:
> https://www.ietf.org/rfcdiff?url2=draft-herbert-udp-space-hdr-01
> >
> > Abstract:

---

From: Joe Touch <[email-masked]>
Date: Wed, 10 Jul 2019 09:49:05 -0700

a) I gave a very specific pair of reasons. To repeat, in detail: 

- UDP options already provides the same vehicle (type flags) for others
to use the space 

- UDP options are not limited in length and (now) already consume the
entirety of that space when used 

b) it is a waste of time to even provide further reasons. 

It would be much more useful IMO for others to comment on the other
pending suggestions. 

(so others can feel free to skip the rest of this post) 

But since you asked: 

- the length field unnecessarily limits use of the space to chunks of
1020 bytes 

- the front padding isn't helpful (the checksum can be adjusted with a
byte-swap if not half-word aligned) 

- the length field assumes (incorrectly) that all use of the space is in
chunks that end at word boundaries 

- the excessive space for experiments is not needed, given RFC6994
techniques 

- it wastes space and provides no additional capability for the current
UDP option approach 

But most importantly: 

- there are no currently known uses of the surplus space 

- but even if there were, they would need to be adapted to this scheme
(and could instead be adapted to the existing UDP options) 

Again, this serves no useful purpose and is actually wasteful. 

Joe 

On 2019-07-10 09:18, Tom Herbert wrote:

> Obviously, I disagree with that assessment. See the motivations section in the draft. 
> 
> Well, I've already posted many comments on that UDP options proposal, I would hope my proposal gets equally constructive feedback from the WG (more than just an arbitrary statement that it has no useful purpose). 
> 
> Tom 
> 
>> Joe
>> 
>>> https://datatracker..ietf.org/doc/html/draft-herbert-udp-space-hdr [1]
>>> until the htmlized version and diff are available at tools.ietf..org [2].
>>> 
>>> The IETF Secretariat
>>> 

Links:
------
[1] https://datatracker.ietf.org/doc/html/draft-herbert-udp-space-hdr
[2] http://tools.ietf.org

---

From: Tom Herbert <[email-masked]>
Date: Wed, 10 Jul 2019 11:36:54 -0700

On Wed, Jul 10, 2019, 9:49 AM Joe Touch <[email-masked]> wrote:

> - UDP options already provides the same vehicle (type flags) for others to
> use the space
>
> - the length field unnecessarily limits use of the space to chunks of 1020
> bytes
>

The UDP surplus header provides alignment, integrity check, disambiguation
for other uses of surplus space, a means to extend the UDP header,
friendliness with HW and SW implentations, extensibility, uniformity with
the other notable transport protocol that contains options, addresses the
requirements for UDPv6 checksum without having to invoke RFC6939. Overhead
is four bytes (seven with worse case padding) which is insignificant
compared to the benefits.

Joe
>> > https://datatracker..ietf.org/doc/html/draft-herbert-udp-space-hdr
>> <https://datatracker.ietf.org/doc/html/draft-herbert-udp-space-hdr>
>> > Diff:
>> > until the htmlized version and diff are available at tools.ietf..org
>> <http://tools.ietf.org>.
>> >
>> > The IETF Secretariat
>> >
>>
>>

---

From: Joe Touch <[email-masked]>
Date: Wed, 10 Jul 2019 12:23:56 -0700

On 2019-07-10 11:36, Tom Herbert wrote:

> ... 
> The UDP surplus header provides alignment,

UDP options already has NOPs for that. 

> integrity check,

UDP options already has a OCS for that. 

> disambiguation for other uses of surplus space,

This isn't needed. It won't support existing uses (not that any are
known) because they won't follow the format.  UDP options already allows
for experimental uses and other standards-track extensions through
codepoint assignment. 

> a means to extend the UDP header,

Actually, it hinders UDP options by limiting them to 1020 bytes, as with
all "other uses". 

> friendliness with HW and SW implentations,

It has no benefit vs. the existing approach in this regard. 

> extensibility,

It's actually less extensible than doing things within the current UDP
options, due to the end alignment requirement and length limit per type.

> uniformity with the other notable transport protocol that contains options,

UDP options tracks TCP options much more closely. 

> addresses the requirements for UDPv6 checksum without having to invoke RFC6939.

There are many ways to address this as have already been proposed - but
we're not talking about them. We're wasting time debating this instead. 

> Overhead is four bytes (seven with worse case padding) which is insignificant compared to the benefits.

There are no benefits (as noted above), only detriments IN ADDITION to
the overhead. 

Can we please discuss more useful topics? 

Joe

---

From: Tom Herbert <[email-masked]>
Date: Wed, 10 Jul 2019 15:17:03 -0700

> UDP options already has NOPs for that.
The NOPs align the options to the start of surplus. If the surplus
space is unaligned to begin with then there's no point to using NOPs
to align individual options.

> UDP options already has a OCS for that.

A one byte optional checksum is next to useless as an integrity check.
This has already been brought up in reference to OCS.

> This isn't needed. It won't support existing uses (not that any are known) because they won't follow the format.  UDP options already allows for experimental uses and other standards-track extensions through codepoint assignment.

If the checksum fails then surplus space isn't processed. So,
probability of misinterpretation of some random use of surplus space
is < 0.0015%.

> Actually, it hinders UDP options by limiting them to 1020 bytes, as with all "other uses".
>
That is intentional. Allowing unlimited options is a path towards DOS attack.

> It has no benefit vs. the existing approach in this regard.
The checksum ensures that the surplus space sums to zero so that the
UDP checksum computation yields the same result rather it covers the
UDP datagram only or the UDP datagram and surplus space. If we don't
have this property, we will lose checksum offload which is important
for UDP performance. Along similar lines, a per packet checksum is
much better than the fragmentation checksum since we can offload the
former and not the latter.

Tom

---

From: Joe Touch <[email-masked]>
Date: Wed, 10 Jul 2019 15:41:33 -0700

> to align individual options.

That's already supported in the UDP options. They can start with NOPs
too. 

> This has already been brought up in reference to OCS.

It was changed to 2 bytes before the last IETF, between v5 and v6 (we're
on v7; the doc table does need to be updated, but the text is accurate
and no longer talks about folding it over).

---

From: Joe Touch <[email-masked]>
Date: Wed, 10 Jul 2019 15:49:48 -0700

Re-sending the full response this time: 

> probability of misinterpretation of some random use of surplus space

See above. 

> That is intentional. Allowing unlimited options is a path towards DOS attack.

There are only 256 codepoints; options you don't support you skip over.
And only NOPs can be repeated and only up to 3x in a row. 

And if you see enough of these with options you don't know or care
about, maybe you should throttle. 

One thing we do know is that any fixed size limit WILL be insufficient.

> The checksum ensures that the surplus space sums to zero...

We've already been over this and the updated text already does EXACTLY
the same thing, except that it doesn't need alignment to work. 

> ... Along similar lines, a per packet checksum is

That remains one of the items that we're waiting for others to comment
on, ones we're NOT discussing because we're still wasting time on this. 

--- 

In summary, please look at the current draft and note that OCS is now 3
bytes. That is incorrect in the table (we already noted that on the
list; it's in the next round of opdates) but not in the text of the OCS
section. 

Joe

---

From: Tom Herbert <[email-masked]>
Date: Wed, 10 Jul 2019 16:24:14 -0700

On Wed, Jul 10, 2019 at 3:49 PM Joe Touch <[email-masked]> wrote:
> That's already supported in the UDP options. They can start with NOPs too.
>
>
>
>
>
> integrity check,
>
>
> It was changed to 2 bytes before the last IETF, between v5 and v6 (we're on v7; the doc table does need to be updated, but the text is accurate and no longer talks about folding it over).
>
>
>
>
>
> There are only 256 codepoints; options you don't support you skip over. And only NOPs can be repeated and only up to 3x in a row.
>
> And if you see enough of these with options you don't know or care about, maybe you should throttle.
>
> We've already been over this and the updated text already does EXACTLY the same thing, except that it doesn't need alignment to work.
>
>
>
> That remains one of the items that we're waiting for others to comment on, ones we're NOT discussing because we're still wasting time on this.
>
> ---
>
> In summary, please look at the current draft and note that OCS is now 3 bytes. That is incorrect in the table (we already noted that on the list; it's in the next round of opdates) but not in the text of the OCS section.
>
Actually, there are two places that say it is a one byte checksum:

+--------+--------+
| Kind=2 |checksum|
 +--------+--------+

2*      2         Option checksum (OCS)

Assuming that this is supposed to be a two byte checksum:

1) An optional checksum cannot protect against corruption of the type
field containing the option. I've have mentioned this several times
and there has never be a reasonable response as to why this isn't a
problem
2) The checksum option is at least three bytes, or four bytes if
alignment is required. A fixed checksum only consumes two bytes.
3) A fixed checksum disambiguates uses standard uses from legacy uses
of the surplus space. An optional checksum doesn't help with that.

---

From: Joe Touch <[email-masked]>
Date: Wed, 10 Jul 2019 16:45:15 -0700

> Actually, there are two places that say it is a one byte checksum:

The first one doesn't "say" anything per se, but yes, it could be
extended to make it more clear. As I noted, this was mentioned already
on the list and is on the pending items. 

> and there has never be a reasonable response as to why this isn't a

It's a choice, just as whether to use checksums in IPv4 UDP. It's the
user's choice and it saves bytes when not desired. The default is to use
it, though. 

> alignment is required. A fixed checksum only consumes two bytes.

Yes, and we've been around that block before too. A fixed checksum could
take 2-4 bytes too - due to the same alignment issues. So we're talking
about 1 byte and the design is based on the principle that ALL options
are optional in UDP - there is no "default" header. 

> of the surplus space. An optional checksum doesn't help with that.

Yes, it does. Optionally. However, note that we still have exactly ZERO
legacy uses discovered. And if - or when - we get comfortable with that,
it can be optionally omitted. 

Joe

---

From: Tom Herbert <[email-masked]>
Date: Wed, 10 Jul 2019 17:07:23 -0700

On Wed, Jul 10, 2019 at 4:45 PM Joe Touch <[email-masked]> wrote:
> The first one doesn't "say" anything per se, but yes, it could be extended to make it more clear. As I noted, this was mentioned already on the list and is on the pending items.
>
>
> It's a choice, just as whether to use checksums in IPv4 UDP. It's the user's choice and it saves bytes when not desired. The default is to use it, though.

The checksum field in UDP is *not* optional, it is in every UDP
header. Use of the checksum is optional only by the sender setting the
field to zero in IPv4, it is not optional in IPv6. The risk of
undetected corruption, other than the unfortunate situation where
mutliple bit corruptions occur that cancel out each other in the one's
complement sum, is if a non-zero checksum gets flipped to be zero then
then will be accepted by an IPv4 stack (not IPv6). Contrast this to
the situation where the type field of checksum option gets corrupted.
A receiver will no longer see the checksum. Barring some other
mechanism to detect a corrupted checksum, a corrupted packet is
accepted.

> Yes, and we've been around that block before too. A fixed checksum could take 2-4 bytes too - due to the same alignment issues. So we're talking about 1 byte and the design is based on the principle that ALL options are optional in UDP - there is no "default" header.
>
>
>
> Yes, it does. Optionally. However, note that we still have exactly ZERO legacy uses discovered. And if - or when - we get comfortable with that, it can be optionally omitted.

No it doesn't. I suggest you send your implementation of UDP options a
bunch of random bytes in surplus space and see what happens. As far as
the fact that zero legacy uses have been discovered, that does not
mean that in the forty year history of UDP that no one else though out
using the surplus. The space was NEVER reserved, so if anyone is using
the space then we can't break them. The checksum is a reasonable
defense against that.

---

From: Joe Touch <[email-masked]>
Date: Wed, 10 Jul 2019 17:20:57 -0700

> On Jul 10, 2019, at 5:07 PM, Tom Herbert <[email-masked]> wrote:
> 
> header.

This isn’t a new header. We’ve already covered that. 

> Use of the checksum is optional only by the sender setting the

Receivers who care can require the checksum.

> the space then we can't break them. The checksum is a reasonable

And you can require it if this keeps you up at night. The rest of us don’t need to be stuck with it forever. 

We’re not designing a Byzantine robust protocol here. 

Joe

---

From: Tom Herbert <[email-masked]>
Date: Wed, 10 Jul 2019 18:29:54 -0700

On Wed, Jul 10, 2019, 5:21 PM Joe Touch <[email-masked]> wrote:

> bytes. That is incorrect in the table (we already noted that on the list;
> it's in the next round of opdates) but not in the text of the OCS section.
> >>
> extended to make it more clear. As I noted, this was mentioned already on
> the list and is on the pending items.
> >>
> >>
> >> Yes, and we've been around that block before too. A fixed checksum
> could take 2-4 bytes too - due to the same alignment issues. So we're
> talking about 1 byte and the design is based on the principle that ALL
> options are optional in UDP - there is no "default" header.
> >>
> >>
> >>
> legacy uses discovered. And if - or when - we get comfortable with that, it
> can be optionally omitted.
> >

Yes, that's why checksum is in the fixed header. The benefits far outweigh
the cost.

---

From: Joe Touch <[email-masked]>
Date: Wed, 10 Jul 2019 20:20:06 -0700

> On Jul 10, 2019, at 6:29 PM, Tom Herbert <[email-masked]> wrote:
> 
> nd you can require it if this keeps you up at night. The rest of us don’t need to be stuck with it forever. 
> 
> Yes, that's why checksum is in the fixed header. The benefits far outweigh the cost.

The cost is a lack of flexibility for 1 byte.

So it took the better part of a day and quite a few emails to conclude that the only remaining issue in this proposal is one that we’ve already discussed at length.

Joe

---

From: Tom Herbert <[email-masked]>
Date: Wed, 10 Jul 2019 20:56:01 -0700

On Wed, Jul 10, 2019, 8:20 PM Joe Touch <[email-masked]> wrote:

> So it took the better part of a day and quite a few emails to conclude
> that the only remaining issue in this proposal is one that we’ve already
> discussed at length.
>

Hardly the only remaining issue if you're referring to UDP options. Please
look at minutes from last IETF. Both the subject of disambiguation of
surplus space as well as negotiation of UDP options was raised (i.e. if the
receiver requires checksum how does sender know that). I haven't seen these
addressed by UDP options draft at least...

---

From: Joe Touch <[email-masked]>
Date: Wed, 10 Jul 2019 22:16:04 -0700

Updating the subject line...

> On Jul 10, 2019, at 8:56 PM, Tom Herbert <[email-masked]> wrote:
> ...
> Hardly the only remaining issue if you're referring to UDP options.

> Please look at minutes from last IETF. Both the subject of disambiguation of surplus space
> as well as negotiation of UDP options was raised (i.e. if the receiver requires checksum how
> does sender know that). I haven't seen these addressed by UDP options draft at least…

If you review the minutes from IETF 104, you will note that the next steps were yours and QUICs, not mine.

But let me jump in:

- regarding other uses of the option space:
	- as I’ve already noted in this thread, this was discussed on this list long before IETF 104
	- past uses (if there are any) would be protected (if necessary) by use of OCS (when OCS is used)
	- future uses need not be differentiated independently; they should be using the UDP option code points instead

Regarding soft-state coordination:
	- Sec 13 already addresses the general notion of soft state in a general sense.
	- on 3/20/19, I noted on the list that soft state mechanisms are not specified because each option might vary in how this is achieved
	- if this isn’t sufficient, can you please clarify?

Regarding use of options with encrypted transports:
	- I don’t know what “encrypted transports” means
	- UDP options includes AE, which is encryption and compatible with UDP options
	- transport *payload* encryption is independent of transport encryption and should have no impact on UDP AE
	- nobody from QUIC has contacted me on this issue

Joe

---

From: "[email-masked]" <[email-masked]>
Date: Thu, 11 Jul 2019 07:09:47 +0000 (UTC)

At least the UDP Options draft acknowledges that the prior UDP-Lite exists...

Lloyd Wood [email-masked] http://about.me/lloydwood

On Thursday, 11 July 2019, 00:51:51 GMT+10, Joe Touch <[email-masked]> wrote: 

Joe

---

From: Tom Herbert <[email-masked]>
Date: Thu, 11 Jul 2019 08:11:34 -0700

On Wed, Jul 10, 2019, 10:16 PM Joe Touch <[email-masked]> wrote:

> - on 3/20/19, I noted on the list that soft state mechanisms are not
> specified because each option might vary in how this is achieved

It's not sufficient. The point of a protocol specification is to describe
how the protocol works, without normative requirements and procedures there
is no interoperability and no robustness.

Consider the proposal that receivers could require optional checksums. How
would the sender know that it must use the checksum option for some
particular peer? This isn't magic, somehow the sender needs to acquire
information about the receiver. That's a protocol. It needs some sort of
negotiation, or probing, or other. And since UDP is stateless "negotiation"
may become obsolete so that fallbacks and periodic renegotiation are needed
which I assume is what you mean by soft state. Furthermore, as reported,
network devices in the path may require the checksum. So the sender needs
to consider how to deal with that, and paths can similarly change so sender
needs to be prepared for that also.

I suggest that you spell out the protocol for receivers requiring
checksums. I don't think the protocol for this is at all trivial, and doing
the exercise to specify the protocol may very well lead you to the
conclusion that it's far simpler to just put checksum in fixed header as
the designers of IP, TCP, and UDP already figured out.

Tom

---

From: Joe Touch <[email-masked]>
Date: Thu, 11 Jul 2019 08:55:45 -0700

> On Jul 11, 2019, at 8:11 AM, Tom Herbert <[email-masked]> wrote:
> 
> 
> 
> On Wed, Jul 10, 2019, 10:16 PM Joe Touch <[email-masked] <mailto:[email-masked]>> wrote:
>> On Jul 10, 2019, at 8:56 PM, Tom Herbert <[email-masked] <mailto:[email-masked]>> wrote:
>> ...
> It's not sufficient. The point of a protocol specification is to describe how the protocol works, without normative requirements and procedures there is no interoperability and no robustness.

It’s a USER protocol; the user decides. 

Other notes:
	- UDP remains unreliable, so there’s no hard state
	- UDP options are optional - UNLIKE TCP headers and UDP headers, so everything can and should be optional as the user desires
	(i.e., the UDP options are like the TCP option *field*, not the TCP header + option field) 

Here’s a simple version that I can include as an example:

- user sends a zero-length UDP with the options the user wants to use and the ECHO-REQ (including either using OCS or not using OCS)

- if the user receives an ECHO-RESP with the same options, set a timer and use those options until the timer expires

Users can do this with more than one variation of options set and use any one of the sets that is successfully echoed.

Why is this difficult?

Joe

---

From: Wesley Eddy <[email-masked]>
Date: Thu, 11 Jul 2019 12:30:47 -0400

On 7/11/2019 11:11 AM, Tom Herbert wrote:
>     - on 3/20/19, I noted on the list that soft state mechanisms are
>     not specified because each option might vary in how this is achieved
> It's not sufficient. The point of a protocol specification is to 
> describe how the protocol works, without normative requirements and 
> procedures there is no interoperability and no robustness.
>
> Consider the proposal that receivers could require optional checksums. 
> How would the sender know that it must use the checksum option for 
> some particular peer? This isn't magic, somehow the sender needs to 
> acquire information about the receiver. That's a protocol. It needs 
> some sort of negotiation, or probing, or other. And since UDP is 
> stateless "negotiation" may become obsolete so that fallbacks and 
> periodic renegotiation are needed which I assume is what you mean by 
> soft state. Furthermore, as reported, network devices in the path may 
> require the checksum. So the sender needs to consider how to deal with 
> that, and paths can similarly change so sender needs to be prepared 
> for that also.
>
> checksums. I don't think the protocol for this is at all trivial, and 
> doing the exercise to specify the protocol may very well lead you to 
> the conclusion that it's far simpler to just put checksum in fixed 
> header as the designers of IP, TCP, and UDP already figured out.
>

The way I understand it, this is supposed to be handled by the 
application (or other protocol) on top of UDP.

Section 9 talks about how the API needs to be extended for the 
application to indicate receive options that are required, and which 
ones should be included on a sent datagram.

Maybe that section should also just explicitly say that the application 
protocol (either by its definition or in its signalling) is supposed to 
work out compatible sets of sent and received options, based on what the 
application needs.  I think that would solve the question you raise.

---

From: Tom Herbert <[email-masked]>
Date: Thu, 11 Jul 2019 09:46:44 -0700

> Users can do this with more than one variation of options set and use any one of the sets that is successfully echoed.

If it's so easy then there shouldn't be any issue with specifying the
protocol in the draft in normative language and including
consideration for various edge conditions and recommendations for
default value of protocol parameters such as the timeout in your
algorithm.

Tom

---

From: Joe Touch <[email-masked]>
Date: Thu, 11 Jul 2019 11:32:17 -0700

> On Jul 11, 2019, at 9:46 AM, Tom Herbert <[email-masked]> wrote:
> 

We can give examples, but this is user level not UDP. We don’t have or want an integrated, normative approach. 

Joe

---

From: "Black, David" <[email-masked]>
Date: Thu, 11 Jul 2019 20:01:06 +0000

Commenting as an individual, **not** as a WG chair.

-- Option Checksum (OCS)

The IETF 104 tsvwg minutes match my impression that the topic of whether the offsetting option checksum (OCS) should be optional vs. mandatory remains an open issue for UDP options.

For my part, I've seen strong "running code" evidence of what breaks when the OCS is omitted, in contrast to almost no evidence of things broken by the presence of OCS (computed to offset the UDP checksum calculation when that is done over IP length instead of UDP length).  For that reason, my current view is that the OCS ought to be mandatory.  Beyond that, I think Tom has a strong argument for putting the OCS in a fixed position of some sort, although my initial sense is that the OCS checksum should be at the end of the IP length, as that works better for pipelined implementations.

One exception to OCS being mandatory that makes sense to me is that OCS doesn't seem to make a lot of sense with LITE, as the OCS would cover all the LITE payload data, thereby defeating the LITE goal of not having to checksum data whose reliability is not of interest.   One consequence is that UDP Options become unreliable when LITE is used, which may have implications for which UDP options are acceptable to use with LITE.  An important tradeoff is that LITE won't work on network paths that pass  UDP Options only when OCS is present.

-- Other uses of surplus space

Any hypothetical existing use of surplus space is incompatible with both UDP options and a new surplus header.  While I haven't seen evidence of any such existing use "running code," making the OCS mandatory helps defend against that possibility, as well as against bad endpoint implementations that put whatever junk bytes happen to be lying around in memory into that extra space, improving UDP Options robustness.

Both UDP options and the surplus header are extensible, which is the "80%" conclusion from my perspective.  Beyond that, I don't see a lot of difference in what the two extensibility approaches enable.

-- Additional motivations

Tom asked " See the motivations section in the draft." ... so I did ...

Assuming that the option checksum (OCS) issues are addressed (see above), the remaining motivations for discarding UDP options and doing something different appear to be:
- Alignment: the NOP option in UDP options covers this
- Protocol headers vs. protocol trailers: Something is wrong with that motivation as it leads to putting OCS immediately after the UDP length, i.e., before the data that it covers, whereas pipelined implementations would prefer to put it at the end of the IP length, i.e., after the data that it covers.

-- Bottom Line --

I see clear opportunity for improvement in the option checksum (OCS) support in UDP options.  **If** suitable OCS improvements are made, then I don't see a strong case for an alternative approach.

Thanks, --David

> -----Original Message-----
> From: tsvwg <[email-masked]> On Behalf Of Joe Touch
> Sent: Thursday, July 11, 2019 2:32 PM
> To: Tom Herbert
> Cc: tsvwg
> Subject: Re: [tsvwg] draft-ietf-udp-options issues from IETF 104
> 
> 
> [EXTERNAL EMAIL]
> 
> 
> 
> >> It's not sufficient. The point of a protocol specification is to describe how
> the protocol works, without normative requirements and procedures there
> >> - UDP options are optional - UNLIKE TCP headers and UDP headers, so
> everything can and should be optional as the user desires
> >> - user sends a zero-length UDP with the options the user wants to use
> and the ECHO-REQ (including either using OCS or not using OCS)
> >>
> >> - if the user receives an ECHO-RESP with the same options, set a timer and
> use those options until the timer expires
> >>
> >> Users can do this with more than one variation of options set and use any
> one of the sets that is successfully echoed.
> >>
> >> Why is this difficult?
> >

---

From: Joe Touch <[email-masked]>
Date: Thu, 11 Jul 2019 13:45:07 -0700

Notes embedded below...

So far, I'm beginning to see that this argument boils down to 1 byte. If
that's all it is, then fine - let's add OCS as a fixed field at the
head, possibly after NOPs. 

NOTE: OCS would still probably need to have a way to be disabled, ala
UDPv4 checksums, by setting to zero. 

So no matter what you do, they're still easily optional unless we decide
to FORCE the user to do otherwise. Why exactly would that be the case?

Joe 

> The IETF 104 tsvwg minutes match my impression that the topic of whether the offsetting option checksum (OCS) should be optional vs. mandatory remains an open issue for UDP options.

Tom has kept this issue open for over a year; there hasn't exactly been
a groundswell behind the issue. I also note that you seem to argue
yourself out of your own support below, FWIW. 

> For my part, I've seen strong "running code" evidence of what breaks when the OCS is omitted,

In SOME Internet paths. The issue is whether we need to require this
even when we might not need it in the future. 

But note that the same thing is largely true for UDP checksums - they
find real errors, yet some people choose to turn them off (esp. for
IPv4). So even if we kept OCS as a required field, we would still need a
way to turn them off in a corresponding way (e.g., set to zero). 

> in contrast to almost no evidence of things broken by the presence of OCS (computed to offset the UDP checksum calculation when that is done over IP length instead of UDP length).

> One exception to OCS being mandatory that makes sense to me is that OCS doesn't seem to make a lot of sense with LITE, as the OCS would cover all the LITE payload data, thereby defeating the LITE g…

There are some other possibilities, such as including an OCS on transmit
but not checking it on each fragment received. That would trick
middleboxes as needed but avoid duplicate work at the receiver (which is
where load tends to be bigger anyway). 

This is both why the current doc indicates OCS as both optional and
default enabled. In the future, if/when it isn't needed, we can remove
it that way - as well as disabling it if needed/possible for LITE. 

> Any hypothetical existing use of surplus space is incompatible with both UDP options and a new surplus header.  While I haven't seen evidence of any such existing use "running code," making the OCS…

*Using* OCS accomplishes this - and remember, it's default to being
used. 

Making OCS mandatory is a decision - why do you think that's critical
for us to make for all future users? 

And regarding the last point, if you really care, users are always
welcome to just leave OCS on as defaulted anyway. 

--

---

From: Tom Herbert <[email-masked]>
Date: Thu, 11 Jul 2019 14:46:43 -0700

On Thu, Jul 11, 2019, 1:01 PM Black, David <[email-masked]> wrote:

> The IETF 104 tsvwg minutes match my impression that the topic of whether
> the offsetting option checksum (OCS) should be optional vs. mandatory
> remains an open issue for UDP options.
>
> For my part, I've seen strong "running code" evidence of what breaks when
> the OCS is omitted, in contrast to almost no evidence of things broken by
> the presence of OCS (computed to offset the UDP checksum calculation when
> that is done over IP length instead of UDP length).  For that reason, my
> current view is that the OCS ought to be mandatory.  Beyond that, I think
> Tom has a strong argument for putting the OCS in a fixed position of some
> sort, although my initial sense is that the OCS checksum should be at the
> end of the IP length, as that works better for pipelined implementations.
>
> One exception to OCS being mandatory that makes sense to me is that OCS
> doesn't seem to make a lot of sense with LITE, as the OCS would cover all
> the LITE payload data, thereby defeating the LITE goal of not having to
> checksum data whose reliability is not of interest.   One consequence is
> that UDP Options become unreliable when LITE is used, which may have
> implications for which UDP options are acceptable to use with LITE.  An
> important tradeoff is that LITE won't work on network paths that pass  UDP
> Options only when OCS is present.
>
> Any hypothetical existing use of surplus space is incompatible with both
> UDP options and a new surplus header.  While I haven't seen evidence of any
> such existing use "running code," making the OCS mandatory helps defend
> against that possibility, as well as against bad endpoint implementations
> that put whatever junk bytes happen to be lying around in memory into that
> extra space, improving UDP Options robustness.
>
> Both UDP options and the surplus header are extensible, which is the "80%"
> conclusion from my perspective.  Beyond that, I don't see a lot of
> difference in what the two extensibility approaches enable.
>
> Assuming that the option checksum (OCS) issues are addressed (see above),
> the remaining motivations for discarding UDP options and doing something
> different appear to be:
> - Protocol headers vs. protocol trailers: Something is wrong with that
> motivation as it leads to putting OCS immediately after the UDP length,
> i.e., before the data that it covers, whereas pipelined implementations
> would prefer to put it at the end of the IP length, i.e., after the data
> that it covers.
>

David,

I would agree with that if we were developing the IP checksum from scratch,
but reality is we have thirty years experience optimizing checksum for TCP
and UDP. For instance, in TX checksum offload the device is given the
offset to write the checksum. Since the checksum is in header, some
implentations express the offset in a byte to save a byte in TX descriptor.
This is one example of implementation optimizing processing for headers, I
doubt it's the only one.

Tom

> -- Bottom Line --
>
> I see clear opportunity for improvement in the option checksum (OCS)
> support in UDP options.  **If** suitable OCS improvements are made, then I
> don't see a strong case for an alternative approach.
>
> Thanks, --David
>
> > >>> Please look at minutes from last IETF. Both the subject of
> disambiguation
> > of surplus space
> > >>>
> > >>> as well as negotiation of UDP options was raised (i.e. if the
> receiver
> > requires checksum how
> > >>>
> > >>> - as I’ve already noted in this thread, this was discussed on this
> list long
> > before IETF 104
> > >>> - past uses (if there are any) would be protected (if necessary) by
> use of
> > OCS (when OCS is used)
> > >>> - future uses need not be differentiated independently; they should
> be
> > using the UDP option code points instead
> > >>>
> > >>> - Sec 13 already addresses the general notion of soft state in a
> general
> > sense.
> describe how
> > >> (i.e., the UDP options are like the TCP option *field*, not the TCP
> header
> > + option field)
> > >>
> > We can give examples, but this is user level not UDP. We don’t have or
> want
> > an integrated, normative approach.
> >
> > Joe
> >
> >
> > > Tom
> > >
> > >>
> > >> Joe
>
>

---

From: Joe Touch <[email-masked]>
Date: Thu, 11 Jul 2019 19:07:39 -0700

> On Jul 11, 2019, at 2:46 PM, Tom Herbert <[email-masked]> wrote:
> 
> I would agree with that if we were developing the IP checksum from scratch, but reality is we have thirty years experience optimizing checksum for TCP and UDP. For instance, in TX checksum offload the device is given the offset to write the checksum. Since the checksum is in header, some implentations express the offset in a byte to save a byte in TX descriptor. This is one example of implementation optimizing processing for headers, I doubt it's the only one.
> 
> Tom

Not sure I follow here - so there are only a few variants that seem viable:

1.- at the front of the surplus space
2.- at the front of the surplus space after alignment NOPs
3.- at the end of the surplus space
4.- at the end of the surplus space with alignment

AFAICT, there’s no real help in requiring OCS be aligned (it can be designed to tolerate any alignment), which means we don’t need #2 or #4.

So what’s preferable here? #1 or #3?

Joe

---

From: Tom Herbert <[email-masked]>
Date: Thu, 11 Jul 2019 19:14:47 -0700

On Thu, Jul 11, 2019, 7:07 PM Joe Touch <[email-masked]> wrote:

> I would agree with that if we were developing the IP checksum from
> scratch, but reality is we have thirty years experience optimizing checksum
> for TCP and UDP. For instance, in TX checksum offload the device is given
> the offset to write the checksum. Since the checksum is in header, some
> AFAICT, there’s no real help in requiring OCS be aligned (it can be
> designed to tolerate any alignment), which means we don’t need #2 or #4.
>

#2 gives greatest probability of working with existing implementation of
checksum offload.

---

From: Joe Touch <[email-masked]>
Date: Thu, 11 Jul 2019 20:48:40 -0700

> On Jul 11, 2019, at 7:14 PM, Tom Herbert <[email-masked]> wrote:
> 
> #2 gives greatest probability of working with existing implementation of checksum offload.

A few questions on that:
- Can we deal with a small amount of “cleanup” operations after the offload?
- if so can that allow arbitrary byte start? 
- or at least16-bit instead of 32-bit alignment? 

Keep in mind that regardless of where we start, we shouldn’t need aligned ends, so there’s always some “cleanup” with such support anyway.

I.e., why can’t we start at the 32-bit alignment and just add the rest and adjust later, rather than pad?

Joe

---

From: Tom Herbert <[email-masked]>
Date: Thu, 11 Jul 2019 21:51:56 -0700

> I.e., why can’t we start at the 32-bit alignment and just add the rest and adjust later, rather than pad?
Joe,

Yes, we can do all these things. But, it's not a question of what can
be done, it's a question of what should be done. If you want to
maximize the chances of a deployable and successful protocol then the
best course of action is to follow the conventions and design patterns
previously set by successfully deployed protocols. Implementations,
both end hosts and middleboxes, have long baked in and optimized
around these conventions. For instance, we know TCP is a header not a
trailer, the multi-byte fields in the TCP header are properly aligned
including checksum, the start of the TCP options is aligned, the TCP
header length is multiple of four bytes, the checksum is in a fixed
location and not optional, etc. Implementations assume these
conventions, not just for TCP, but most IP protocols adhere to some
set of common conventions. Obviously we can't follow all conventions,
using the UDP surplus space is certainly unconventional, but the less
deviation from convention the higher probability of success in the
real world. Note that optimizing out a few bytes here and there is
immaterial as to whether the protocol will be ultimately prove
deployable-- it really isn't the highest priority issue we need to
worry about.

Tom

---

From: Joe Touch <[email-masked]>
Date: Thu, 11 Jul 2019 22:25:02 -0700

> deployable-- it really isn't the highest priority issue we need to

Sure, but here’s the issue:

Neither TCP nor UDP require their payloads to end on a word boundary. So transport checksums already need to deal with up to 3 bytes of “cleanup”.

Given that, why force the front alignment? At most it only doubles that cleanup, which ought to be nominal anyway.

Joe

---

From: "Black, David" <[email-masked]>
Date: Fri, 12 Jul 2019 14:09:57 +0000

Joe,

> So no matter what you do, they're still easily optional unless we decide to FORCE the user to do otherwise. Why exactly would that be the case?

I think I’m over on the “FORCE” side of that dichotomy (i.e., I don’t agree with “easily optional”), although we can split hairs on what degree of “FORCE” is appropriate.   My underlying reasons are these two:
                1) strong "running code" evidence of what breaks when the OCS is omitted,
                2) helps defend against … possibility [of] … other uses of surplus space

IMHO, OCS being the “default” is a good starting point, but … my current view is that on its own, item 1) justifies “MUST implement/SHOULD use” language for OCS (and “SHOULD use” is complementary to and stronger than “default”) and setting the space aside for OCS, at least when LITE is not present.  That view is reinforced by item 2), and I note neither of those two items are within the control of the endpoint implementer or application that uses the endpoint’s UDP option support.

> One exception to OCS being mandatory that makes sense to me is that OCS doesn't seem to make a lot of sense with LITE,

Point taken – my response is “not exactly” … but I clearly need to explain my thinking in more detail :-).  IMHO, LITE is “special” …

My rationale for OCS not making a lot of sense with LITE is that use of LITE is a clear indication from the sending application that correctness of the LITE data on receipt is optional.  Beyond that, I believe you’ve pointed out that zeroing the UDP checksum is a related scenario where correctness of all of the UDP data may be optional – that scenario is more subtle, because one of the rationales for zeroing the UDP checksum is presence of another integrity mechanism elsewhere in the protocol stack that covers the data (although protocol designers need to watch out for uncovered IPv6 headers – see RFC 6935 and RFC 6936).  In both cases, if the scenario is one in which “correctness of .. data is optional,” then IMHO (as an individual) correctness of UDP options also has to be optional, and we (WG) need to carefully consider which options are ok vs. ought not to be used when correctness of them on receipt doesn’t matter.

Thanks, --David

From: Joe Touch <[email-masked]>
Sent: Thursday, July 11, 2019 4:45 PM
To: Black, David
Cc: Tom Herbert; tsvwg
Subject: Re: [tsvwg] draft-ietf-udp-options issues from IETF 104

[EXTERNAL EMAIL]

Notes embedded below...

So far, I'm beginning to see that this argument boils down to 1 byte. If that's all it is, then fine - let's add OCS as a fixed field at the head, possibly after NOPs.

NOTE: OCS would still probably need to have a way to be disabled, ala UDPv4 checksums, by setting to zero.

So no matter what you do, they're still easily optional unless we decide to FORCE the user to do otherwise. Why exactly would that be the case?

Joe

-- Option Checksum (OCS)

Tom has kept this issue open for over a year; there hasn't exactly been a groundswell behind the issue. I also note that you seem to argue yourself out of your own support below, FWIW.

In SOME Internet paths. The issue is whether we need to require this even when we might not need it in the future.

But note that the same thing is largely true for UDP checksums - they find real errors, yet some people choose to turn them off (esp. for IPv4). So even if we kept OCS as a required field, we would still need a way to turn them off in a corresponding way (e.g., set to zero).

...

There are some other possibilities, such as including an OCS on transmit but not checking it on each fragment received. That would trick middleboxes as needed but avoid duplicate work at the receiver (which is where load tends to be bigger anyway).

This is both why the current doc indicates OCS as both optional and default enabled. In the future, if/when it isn't needed, we can remove it that way - as well as disabling it if needed/possible for LITE.

*Using* OCS accomplishes this - and remember, it's default to being used.

Making OCS mandatory is a decision - why do you think that's critical for us to make for all future users?

And regarding the last point, if you really care, users are always welcome to just leave OCS on as defaulted anyway.

--

---

From: Joe Touch <[email-masked]>
Date: Fri, 12 Jul 2019 07:38:36 -0700

> On Jul 12, 2019, at 7:09 AM, Black, David <[email-masked]> wrote:
> 
> My rationale for OCS not making a lot of sense with LITE is that use of LITE is a clear indication from the sending application that correctness of the LITE data on receipt is optional.  Beyond that, I believe you’ve pointed out that zeroing the UDP checksum is a related scenario where correctness of all of the UDP data may be optional – that scenario is more subtle, because one of the rationales for zeroing the UDP checksum is presence of another integrity mechanism elsewhere in the protocol stack that covers the data (although protocol designers need to watch out for uncovered IPv6 headers – see RFC 6935 and RFC 6936). 

Agreed, but keep in mind that NAT boxes that modify IP addresses and transport ports - and other fields - simply recalculate checksums to cover their tracks. So there’s little point to worrying about whether they’re ‘covered’ by transport protections.

Users who really care about this sort of behavior really need to use some sort of encryption or authentication over the whole packet. Mere checksums aren’t enough.

Joe

---

From: Joe Touch <[email-masked]>
Date: Fri, 12 Jul 2019 07:42:31 -0700

> IMHO, OCS being the “default” is a good starting point, but … my current view is that on its own, item 1) justifies “MUST implement/SHOULD use” language for OCS (and “SHOULD use” is complementary t…

Also note that the “running code” evidence is based on an error. That’s why I worry about designing overhead we live with forever only to overcome a bug that may be ephemeral.

That said, again, is there some reason you think we CANNOT allow the user to turn off OCS (in this case, set to zero)? Again, this is UDP. The risk - and the decision - is up to the user, IMO.

Joe

---

From: Tom Herbert <[email-masked]>
Date: Fri, 12 Jul 2019 08:25:45 -0700

David,

The integrity provided by the checksum is only part of the story. An
important observation is that many devices will include the surplus
space in their calculation of the UDP checksum. Unless the surplus
space sums to zero, an incorrect UDP checksum may be derived and the
result is a high probability that packets will be dropped somewhere in
the network. The other important use is disambiguation with legacy
uses of the surplus space.

So the first requirement is simple:

"If the UDP checksum is used (non-zero) then a checksum MUST be used
in the surplus space such that the surplus space sums to zero"

Given that IPv6 requires the UDP checksum, the surplus space checksum
needs to be present in all IPv6 packets. For cases in IPv4 where the
UDP checksum is zero, I think use of the surplus space checksum is
still a MUST. A common misnomer is that optional UDP checksums is
somehow a benefit to hosts since it allows them to omit the checksum
calculation. It's not. We've already accounted for presence required
checksums in TCP and UDPv6. For instance, RFC6936 only benefited
routers not hosts; the effect of supporting RFC6936 in hosts has been
more complexity, more test cases, and more bugs to fix. Making surplus
space checksum optional is more work for little gain.

Tom

---

From: "Black, David" <[email-masked]>
Date: Fri, 12 Jul 2019 15:30:20 +0000

Inline …

Thanks, --David

From: Joe Touch <[email-masked]>
Sent: Friday, July 12, 2019 10:43 AM
To: Black, David
Cc: Tom Herbert; tsvwg
Subject: Re: [tsvwg] draft-ietf-udp-options issues from IETF 104

[EXTERNAL EMAIL]

On Jul 12, 2019, at 7:09 AM, Black, David <[email-masked]<mailto:[email-masked]>> wrote:

Also note that the “running code” evidence is based on an error. That’s why I worry about designing overhead we live with forever only to overcome a bug that may be ephemeral.
[David>] Albeit a widely deployed error.  I acknowledge the worry, and suggest that it deserves further discussion on what can reasonably be expected in deployed Internet infrastructure.

That said, again, is there some reason you think we CANNOT allow the user to turn off OCS (in this case, set to zero)? Again, this is UDP. The risk - and the decision - is up to the user, IMO.
[David>] Asked in that form, I think my “SHOULD use” is in rough agreement with your views.  I suspect that we may not agree on the appropriate tone/emphasis of the text, in that I believe that “SHOULD use” provides an important warning to those who are inclined not to use OCS, and ought to be backed by more specific discussion of the potential things that can go wrong.   I’m also concerned that this discussion may be oversimplifying a subtle/complex area – e.g., RFC 8085 devotes about 3 pages of text to this topic, including UDP-Lite and IPv6 considerations for UDP checksum usage.

Joe

---

From: "Black, David" <[email-masked]>
Date: Fri, 12 Jul 2019 15:49:11 +0000

Tom,

Inline ...

Reminder: All of my comments in this thread are as an individual, not as a WG chair.

Thanks, --David

> From: Tom Herbert <[email-masked]>
> Sent: Friday, July 12, 2019 11:26 AM
> To: Black, David
> Cc: Joe Touch; Tom Herbert; tsvwg
> > > So no matter what you do, they're still easily optional unless we decide to
> FORCE the user to do otherwise. Why exactly would that be the case?
> >
> > I think I’m over on the “FORCE” side of that dichotomy (i.e., I don’t agree
> with “easily optional”), although we can split hairs on what degree of
> “FORCE” is appropriate.   My underlying reasons are these two:
> >
> > IMHO, OCS being the “default” is a good starting point, but … my current
> view is that on its own, item 1) justifies “MUST implement/SHOULD use”
> language for OCS (and “SHOULD use” is complementary to and stronger than
> “default”) and setting the space aside for OCS, at least when LITE is not
> present.  That view is reinforced by item 2), and I note neither of those two
> items are within the control of the endpoint implementer or application that
> uses the endpoint’s UDP option support.
> >
> >
> >
> > > One exception to OCS being mandatory that makes sense to me is that
> OCS doesn't seem to make a lot of sense with LITE,
> >
> >
> >
> > Point taken – my response is “not exactly” … but I clearly need to explain
> my thinking in more detail :-).  IMHO, LITE is “special” …
> >
> >
> >
> > My rationale for OCS not making a lot of sense with LITE is that use of LITE is
> a clear indication from the sending application that correctness of the LITE
> data on receipt is optional.  Beyond that, I believe you’ve pointed out that
> zeroing the UDP checksum is a related scenario where correctness of all of
> the UDP data may be optional – that scenario is more subtle, because one of
> the rationales for zeroing the UDP checksum is presence of another integrity
> mechanism elsewhere in the protocol stack that covers the data (although
> protocol designers need to watch out for uncovered IPv6 headers – see RFC
> 6935 and RFC 6936).  In both cases, if the scenario is one in which
> “correctness of .. data is optional,” then IMHO (as an individual) correctness
> of UDP options also has to be optional, and we (WG) need to carefully
> consider which options are ok vs. ought not to be used when correctness of
> them on receipt doesn’t matter.
> 
> David,
> 

[David>] With the exception of LITE, I agree.

[David>]I think LITE deserves more thought/discussion, as a user of LITE has already agreed to tolerate things that the network is not supposed to do to non-LITE traffic, and that tolerance may extend further.
> needs to be present in all IPv6 packets.

[David>] That IPv6 requirement statement is incorrect, even though it's "digging in the right place" (IMHO) - see RFC 6935 and RFC 6936.

> For cases in IPv4 where the

[David>] I acknowledge that view, and for now I will agree to disagree, as I currently see a clear case for "SHOULD use OCS" but not for "MUST use OCS" ... and reserve the right to change my mind as more information is added to the discussion.
> that's all it is, then fine - let's add OCS as a fixed field at the head, possibly
> after NOPs.
> >
> > So no matter what you do, they're still easily optional unless we decide to
> FORCE the user to do otherwise. Why exactly would that be the case?
> >
> > Joe
> >
> > Tom has kept this issue open for over a year; there hasn't exactly been a
> groundswell behind the issue. I also note that you seem to argue yourself
> out of your own support below, FWIW.
> >
> >
> >
> > But note that the same thing is largely true for UDP checksums - they find
> real errors, yet some people choose to turn them off (esp. for IPv4). So even
> if we kept OCS as a required field, we would still need a way to turn them off
> in a corresponding way (e.g., set to zero).
> >
> >
> >
> > in contrast to almost no evidence of things broken by the presence of OCS
> (computed to offset the UDP checksum calculation when that is done over IP
> length instead of UDP length).
> >
> >
> >
> doesn't seem to make a lot of sense with LITE, as the OCS would cover all the
> LITE payload data, thereby defeating the LITE goal of not having to checksum
> data whose reliability is not of interest.   One consequence is that UDP
> Options become unreliable when LITE is used, which may have implications
> for which UDP options are acceptable to use with LITE.  An important
> tradeoff is that LITE won't work on network paths that pass  UDP Options
> only when OCS is present.
> >
> >
> >
> > There are some other possibilities, such as including an OCS on transmit but
> not checking it on each fragment received. That would trick middleboxes as
> needed but avoid duplicate work at the receiver (which is where load tends
> to be bigger anyway).
> >
> >
> >
> > This is both why the current doc indicates OCS as both optional and default
> enabled. In the future, if/when it isn't needed, we can remove it that way -
> as well as disabling it if needed/possible for LITE.
> >
> >
> >
> against that possibility, as well as against bad endpoint implementations that
> put whatever junk bytes happen to be lying around in memory into that

---

From: Tom Herbert <[email-masked]>
Date: Fri, 12 Jul 2019 09:16:00 -0700

> Neither TCP nor UDP require their payloads to end on a word boundary. So transport checksums already need to deal with up to 3 bytes of “cleanup”.
Per RFC768, a zero byte is appended if the UDP data length is odd.

> Given that, why force the front alignment? At most it only doubles that cleanup, which ought to be nominal anyway.

It's a straightforward application of sending clause the robustness
principle. Other IP protocols ensure checksum and headers aligned so
this protocol should too in order to maximize robustness.

Tom

---

From: Tom Herbert <[email-masked]>
Date: Fri, 12 Jul 2019 09:40:08 -0700

> [David>]I think LITE deserves more thought/discussion, as a user of LITE has already agreed to tolerate things that the network is not supposed to do to non-LITE traffic, and that tolerance may ext…

The user may see "random" packet loss which could be a bear to debug.
I'm not sure we're doing them a favor by making checksums optional.

> [David>] That IPv6 requirement statement is incorrect, even though it's "digging in the right place" (IMHO) - see RFC 6935 and RFC 6936.

Yes, that's correct. However, I think you'll agree that the
requirements of RFC6935 and RFC6936 are very narrow and only apply to
UDP tunneling protocols that explicitly meet the requirements (which
AFAIK are only MPLS/UDP, GRE/UDP, an GUE). UDP options proposal
doesn't generally meet the requirements of RFC6936 for using UDPv6
zero checksum.

---

From: Joe Touch <[email-masked]>
Date: Fri, 12 Jul 2019 10:19:16 -0700

On 2019-07-12 09:40, Tom Herbert wrote:et the requirements (which

> ... UDP options proposal

FRAG+LITE does, though - because it provides its own post-reassembly
checksum. 

Joe

---

From: Joe Touch <[email-masked]>
Date: Fri, 12 Jul 2019 10:22:48 -0700

On 2019-07-12 09:16, Tom Herbert wrote: 

...

>> Neither TCP nor UDP require their payloads to end on a word boundary. So transport checksums already need to deal with up to 3 bytes of "cleanup".

That's in the description of how the checksum calculation works; there's
no requirement to make the payload of actual even length. 

Padding with a single zero byte is equivalent to dealing with the
"cleanup" (which for IP checksum, is only ever 1 byte anyway). 

> this protocol should too in order to maximize robustness.

That's not the robustness principle. That's ossification. 

Joe

---

From: Tom Herbert <[email-masked]>
Date: Fri, 12 Jul 2019 10:44:48 -0700

On Fri, Jul 12, 2019 at 10:19 AM Joe Touch <[email-masked]> wrote:
> FRAG+LITE does, though - because it provides its own post-reassembly checksum.
>
Unless the checksum covers the IP addresses, it doesn't meet the
requirements (not to mention as hard as I squint, I cannot fathom that
UDP options is tunneling protocol). In any case, the requirments that
UDPv6 checksum must be non-zero are baked in-- it's a done deal in the
Internet. For instance, there's deployed middleboxes that will drop
packets with UDP v6 zero checksum.

Tom

---

From: Joe Touch <[email-masked]>
Date: Fri, 12 Jul 2019 11:00:38 -0700

On 2019-07-12 10:44, Tom Herbert wrote:

> requirements

Given NATs rewriting those addresses, what's the point? 

> (not to mention as hard as I squint, I cannot fathom that
> packets with UDP v6 zero checksum

"baked in" or not, it a bug according to the requirements as of RFC6936
- and it needs to be addressed. 

Joe

---

From: Tom Herbert <[email-masked]>
Date: Fri, 12 Jul 2019 11:07:53 -0700

On Fri, Jul 12, 2019 at 11:00 AM Joe Touch <[email-masked]> wrote:
> "baked in" or not, it a bug according to the requirements as of RFC6936 - and it needs to be addressed.
>

What is the bug?

---

From: Joe Touch <[email-masked]>
Date: Fri, 12 Jul 2019 11:12:07 -0700

> On Jul 12, 2019, at 11:07 AM, Tom Herbert <[email-masked]> wrote:
> 

Intermediate devices Dropping UDP packets with zero checksums. 

The have no idea what’s inside or whether is is a valid tunneling protocol. 

Ps - UDP is used for tunnels because it traverses NATs (or should) and doesn’t delay, unnecessarily reorder, or layer congestion control the way a TCP tunnel would. 

Joe

---

From: Tom Herbert <[email-masked]>
Date: Fri, 12 Jul 2019 12:10:48 -0700

> Intermediate devices Dropping UDP packets with zero checksums.
They have been doing this for years and it is consistent with the
Internet standard, i.e. RFC2460 and now RFC8200 are very clear that
checksums are required for UDPv6. RFC6935 is also clear that it's the
responsibility of the tunnel protocols to deal with any
incompatibility that the use of UDPv6 zero checksums creates:

"Tunnel protocols intending to use a zero UDP checksum need to ensure
that they have defined a method for handling cases when a middlebox
prevents the path between the tunnel ingress and egress from
supporting transmission of datagrams with a zero UDP checksum.  This
is especially important as middleboxes that conform to RFC 2460 are
likely to discard datagrams with a zero UDP checksum."

---

From: Joe Touch <[email-masked]>
Date: Fri, 12 Jul 2019 12:40:36 -0700

On 2019-07-12 12:10, Tom Herbert wrote: 

...

 They have been doing this for years 

It's been wrong for years... 

> and it is consistent with the
> Internet standard, i.e. RFC2460

Yes, but not since RFC6936 (granted, that should have updated 2460)... 

> and now RFC8200 are very clear that
> checksums are required for UDPv6.

It's also very clear that they are NOT always required, as per Sec 8.1. 

> RFC6935 is also clear that it's the

That recognizes existing deployment *at that time*. It doesn't provide
for continued ignorance of the change.

Further, middleboxes that conform to RFC 2460 shouldn't be modifying the
headers anyway... 

Joe

---

From: Tom Herbert <[email-masked]>
Date: Fri, 12 Jul 2019 14:26:57 -0700

On Fri, Jul 12, 2019 at 12:40 PM Joe Touch <[email-masked]> wrote:
> That recognizes existing deployment *at that time*. It doesn't provide for continued ignorance of the change.
>
> Further, middleboxes that conform to RFC 2460 shouldn't be modifying the headers anyway...
>
And yet they do... have you ever heard of NAT?

In any case, I don't get the point of this coversation. We already
_have_ a working solution for the Internet. Non-zero UDP checksums
have long been accounted for and optimized. We have no users clamoring
to use zero UDP checksums. Turning up UDPv6 zero checksums is not a
leap forward any more than trying to make TCP checksums optional.

Tom

---

From: Joe Touch <[email-masked]>
Date: Fri, 12 Jul 2019 16:45:17 -0700

> On Jul 12, 2019, at 2:26 PM, Tom Herbert <[email-masked]> wrote:
> 
> We have no users clamoring
> to use zero UDP checksums.

I doubt that given the other RFC. But that I ant the point. 

> Turning up UDPv6 zero checksums is not a
> leap forward

Not turning it up, but definitely supporting that STANDARDS-based case. 

Joe

---

From: Derek Fawcus <[email-masked]>
Date: Sat, 13 Jul 2019 19:02:47 +0100

On Thu, Jul 11, 2019 at 01:45:07PM -0700, Joe Touch wrote:
> a groundswell behind the issue.

Well strictly it is not just Tom, I also have expressed concerns about OCS,
and preferring to have a checksum field at some fixed offset.  I don't recall
what position Mike Heard took.

So I'd prefer the checksum as a fixed field, at either the start or end of
the surplus area, and probably optionally set to zero indicating not used.
The latter could then be of use in the LITE case.

Apart from the reasons already rehearsed in this thread, part of my concern
is simply the implementation robustness.

Having coding up the logic to efficiently parse and check an earlier version
of the OCS option scheme, the code becomes complex and fragile.

The alternate scheme of a fixed field was simpler, and also has the advantage
of allowing for a naive approach where the surplus is first validated,
then subsequently parsed (thus avoiding TLV parsing and verification until
after checksum validation).

DF

---

From: Derek Fawcus <[email-masked]>
Date: Sat, 13 Jul 2019 19:25:54 +0100

> AFAICT, there’s no real help in requiring OCS be aligned (it can be designed to tolerate any alignment), which means we don’t need #2 or #4.

Agree.  I see no need for alignment.

> So what’s preferable here? #1 or #3?

Not sure.

I'd be inclined to pick 1, but could be convinced that 3 is better.

If we're talking of h/w offload engines [*], I imagine that they could
handle 'write to 2 bytes before end of packet' more easily than
'write to start of surplus'.

DF

[*] How many do this with random logic?  My current impression is that
    most such devices are programmable cores, e.g. ARM or MIPS cores.

---

From: Tom Herbert <[email-masked]>
Date: Sat, 13 Jul 2019 11:49:10 -0700

> 'write to start of surplus'.

Derek,

It's the other way around because offload engines are primarily built
to deal TCP and UDP checksums which have always been in headers
towards beginning of the packet. Similarly, HW can optimize for
checksum being aligned since UDP and TCP checksums have always been
aligned.

>     most such devices are programmable cores, e.g. ARM or MIPS cores.

Actually no, checksum offload is in ASICs and maybe some FPGA. NICs
are getting cores now in so called Smart-NICs, but those are more
likely to be applied towards more complex protocol parsing and
processing. CPUs in the NIC won't fundamentally improve how checksum
offload, the solutions for that are well established and understood.

If you're interested, we will be presenting a technlogy deep dive
@IETF105 that will cover a lot of topics around NIC technology:
https://datatracker.ietf.org/meeting/105/materials/agenda-105-tdd-01

Tom

---

From: Joe Touch <[email-masked]>
Date: Sun, 14 Jul 2019 08:52:08 -0700

So, *given* that we *are* different from UDP and TCP, i.e., we do NOT have the luxury of a fixed-offset location for this checksum:

- why does the offset distance matter, i.e., front vs. end?
- how *MUCH* does the offset alignment matter, keeping in mind:
	- no offset might require 1 byte-swap at some point, but consumes the least space
	- half-word alignment wastes 4 bits on avg.
	- full-word alignment wastes 12 bits (1.5 bytes) on avg

In comms, the typical figure of merit is 1 opcode per bit. So, IMO:

- if we can save 4 bits with 4 or fewer opcodes, it’s a win
- if we can save 12 bits with 12 or fewer opcodes, it’s a win

My assumption is that offload engines or the core processor need to do “non-halfword length” cleanup regardless.

That cleanup should be at most 5 opcodes:
	load
	mask the addend
	add w/carry
	mask the sum (for the 1’s compl wrap)
	add w/carry (for the 1’s compl wrap)
	swap (if not half-word aligned)

That suggests we’re either very close to no real penalty for “no offset” or that we win by using at least half-word offset.

Note: the alignment overhead will also NEED to be zero - and checked that it’s zero, which are additional opcodes needed that are not too far from the cleanup ones above.

i.e., given the similarity of the zero-check and the cleanup, why is it useful to waste these bytes?

Joe

> On Jul 13, 2019, at 11:49 AM, Tom Herbert <[email-masked]> wrote:
>

---

From: Tom Herbert <[email-masked]>
Date: Mon, 15 Jul 2019 08:50:40 -0700

On Sun, Jul 14, 2019 at 8:52 AM Joe Touch <[email-masked]> wrote:
>
> So, *given* that we *are* different from UDP and TCP, i.e., we do NOT have the luxury of a fixed-offset location for this checksum

Joe,

There's no requirement that precludes defining a fixed header that
precedes the surplus space. The fact that UDP is different from TCP is
immaterial to the requirements, except that it's good practice to
adopt some of the design principles of TCP to leverage lessons learned
and implementation. The UDP Extended Header in
draft-herbert-udp-space-hdr-01 demonstrates that.

> - why does the offset distance matter, i.e., front vs. end?

Because some implementations assume that checksums are in protocol
headers towards the beginning of the packet. For instance, for
checksum offload the e1000e NIC stores offsets of checksum start and
checksum field in a byte. If either offset is greater than 255 then
the checksum can't be offloaded to the hardware.

> - if we can save 12 bits with 12 or fewer opcodes, it’s a win
The tradeoff between maintaining alignment and efficiency in overhead
is well known. For instance RFC8200 states:

"Individual options may have specific alignment requirements, to
ensure that multi-octet values within Option Data fields fall on
natural boundaries"

Note RFC8200 does not require option alignment; per robustness
principle receivers should be prepared to deal with unaligned options.
However, also by the robustness principle senders should send aligned
options especially if they want to minimize the chances that a
receiver drops packets or relegates them to slow path processing.
Personally, in my implemenation and protocols, I will always align
protocol fields to their natural boundaries if possible-- I have never
seen any complain about too much space being wasted for alignment, but
I have seen issues when protocol fields were unaligned.

Tom

---

From: Joe Touch <[email-masked]>
Date: Mon, 15 Jul 2019 09:14:59 -0700

> On Jul 15, 2019, at 8:50 AM, Tom Herbert <[email-masked]> wrote:
> 
> precedes the surplus space.

It clearly can’t be a fixed location. Thats all I was asserting. 

Or do you disagree?

> the checksum can't be offloaded to the hardware.

That’s a nonstarter. We cannot make that limit. Any other reason?
> I have seen issues when protocol fields were unaligned.

That would be good reasons for a SHOULD, but clearly not a MUST - regarding starting aligned. 

Joe

---

From: Tom Herbert <[email-masked]>
Date: Mon, 15 Jul 2019 09:39:45 -0700

> It clearly can’t be a fixed location. Thats all I was asserting.
In draft-herbert-udp-space-hdr-01 the checksum is a header that
precedes the surplus space. The surplus space header is the head of
the surplus space aligned to a four byte offset (in case of UDP
Extended Header no padding is required). The location and presence of
the checksum field is deterministic.

> That’s a nonstarter. We cannot make that limit. Any other reason?

Well, this "nonstarter" has been in widescale deployment for many
years with proven benefits. This is just one example of how
implementations have been tuned to work on protocol headers as opposed
to protocol trailers, and it's probably not even the one with the
worst implications. There are going to be many hardware
implementations that simply can't process protocol trailers. So if
they think they need to, either they'll either drop the packet or
relegate them to some intolerably slow software path. As we've seen,
many times before, if hardware doesn't properly support a datapath
protocol in fast path the protocol is doomed (e.g. see the fate of
IPv4 options).

Tom

---

From: Joe Touch <[email-masked]>
Date: Mon, 15 Jul 2019 10:02:30 -0700

> precedes the surplus space.

It clearly can't be a fixed location. Thats all I was asserting.

Or do you disagree?

I don't understand why this is confusing, so I'll be blunt: 

- OCS WILL NOT come at a *fixed offset* from the UDP header 

If you disagree, explain why you think that's true in your previous doc
or any other. 

It also can't come at a fixed offset from the start of the surplus space
UNLESS you assume that alignment doesn't matter. 

Is this clear enough? 

> The surplus space header is the head of

It's a deterministic EQUATION, but that equation is variable. 

I.e., the checksum is NOT at a fixed location from the *start of the
packet*, as it is in the UDP and TCP headers. 

> the checksum can't be offloaded to the hardware.

That's a nonstarter. We cannot make that limit. Any other reason? 
Well, this "nonstarter" has been in widescale deployment for many
years with proven benefits. 

That works fine for the fixed headers in TCP and UDP, which are less
than 255 bytes from the front of the packet in most cases (though not
all). They're certainly less than 255 bytes from the front of the TCP or
UDP packet start. 

However, are you also now asserting that we need to put OCS no more than
255 bytes from the front of the TCP or UDP packet? 

You do realize that widescale deployment of UDP and TCP use packet sizes
of 500+ byte payloads, which means WE CANNOT BE STUCK WITH THAT LIMIT. 

Yes, OCS comes less than 255 bytes *from the beginning of the surplus
space*, which itself comes some offset (>255) from the packet start. 

That's all I'm saying. 

> This is just one example of how
> to protocol trailers,

Sure, but if that's the limit AND it applies to the distance from the
front of the UDP packet, are you claiming it is now a limit for how far
OCS can be from the start? 

If that's the case, you're now claiming packets need to have payloads
smaller than 255 bytes, which IS A NONSTARTER. 

So again, if that's where you're getting the limits of using CS offload,
then use of that offload is already OFF THE TABLE. If not, please
explain. 

Joe

---

From: Tom Herbert <[email-masked]>
Date: Mon, 15 Jul 2019 10:27:36 -0700

Joe,

On that I agree. But, draft-herbert-udp-space-hdr-01 proposes to put a
checksum, which is not OCS, in a fixed header at the beginning of the
surplus space that covers the whole surplus. The motivations for this
are articulated in section 4 of that draft, have been brought several
times on the mailing list, as well as tsvwg meeting @IETF104.

Tom

> If you disagree, explain why you think that's true in your previous doc or any other.
>
> It also can't come at a fixed offset from the start of the surplus space UNLESS you assume that alignment doesn't matter.
>
> Is this clear enough?
>
>
>
> I.e., the checksum is NOT at a fixed location from the *start of the packet*, as it is in the UDP and TCP headers.
>
>
>
> That works fine for the fixed headers in TCP and UDP, which are less than 255 bytes from the front of the packet in most cases (though not all). They're certainly less than 255 bytes from the front of the TCP or UDP packet start.
>
> However, are you also now asserting that we need to put OCS no more than 255 bytes from the front of the TCP or UDP packet?
>
> You do realize that widescale deployment of UDP and TCP use packet sizes of 500+ byte payloads, which means WE CANNOT BE STUCK WITH THAT LIMIT.
>
> Yes, OCS comes less than 255 bytes *from the beginning of the surplus space*, which itself comes some offset (>255) from the packet start.
>
> That's all I'm saying.
>
>
> Sure, but if that's the limit AND it applies to the distance from the front of the UDP packet, are you claiming it is now a limit for how far OCS can be from the start?
>
> If that's the case, you're now claiming packets need to have payloads smaller than 255 bytes, which IS A NONSTARTER.
>
> So again, if that's where you're getting the limits of using CS offload, then use of that offload is already OFF THE TABLE. If not, please explain.
>
> Joe
>

---

From: Joe Touch <[email-masked]>
Date: Mon, 15 Jul 2019 10:40:49 -0700

On 2019-07-15 10:27, Tom Herbert wrote:

> surplus space that covers the whole surplus.

Sorry, but you're not following where were are with the UDP options. 

1) we already agreed (in the past week) to make the OCS field mandatory 

2) OCS always covered the entire option space 

3) we already agreed that there would NOT be any surplus space beyond
the options, i.e., that options would take up the whole surplus space 
(anyone who wants to use surplus space should coordinate with the UDP
options using that mechanism, as I noted when evaluating your proposal
as not necessary) 

NOTE: we still can use EOL to halt option processing, but the assumption
is that the remainder is *ignored*, not there for other uses. If that's
an issue, we can omit EOL. 

**Given that context**, all that remains regarding OCS now are the
following questions (which is why I started with them and hope to get
back to): 
- OCS at head or tail of the surplus space 
- OCS aligned, and if so, at what level (16-bit, 32-bit) 

NOTE1: your claims about offloading driving alignment seem to be
undermined by the need for using an untenable 255-byte offset from the
beginning of the UDP packet (not the surplus space - whose start offset
itself *varies from packet to packet* relative to the UDP header, unlike
the UDP checksum). 

NOTE2: *as repeatedly already noted*, again - GIVEN THIS CONTEXT - there
is no utility in the additional pre-option header proposed in
draft-herbert-udp-space-hdr-01. 

Joe

---

From: Tom Herbert <[email-masked]>
Date: Mon, 15 Jul 2019 12:46:32 -0700

On Mon, Jul 15, 2019 at 10:40 AM Joe Touch <[email-masked]> wrote:
> 3) we already agreed that there would NOT be any surplus space beyond the options, i.e., that options would take up the whole surplus space
> (anyone who wants to use surplus space should coordinate with the UDP options using that mechanism, as I noted when evaluating your proposal as not necessary)
>

Please explain how LITE mode works robustly if the UDP checksum is set.

> NOTE: we still can use EOL to halt option processing, but the assumption is that the remainder is *ignored*, not there for other uses. If that's an issue, we can omit EOL.
>
> **Given that context**, all that remains regarding OCS now are the following questions (which is why I started with them and hope to get back to):
> NOTE1: your claims about offloading driving alignment seem to be undermined by the need for using an untenable 255-byte offset from the beginning of the UDP packet (not the surplus space - whose start offset itself *varies from packet to packet* relative to the UDP header, unlike the UDP checksum).
>
> NOTE2: *as repeatedly already noted*, again - GIVEN THIS CONTEXT - there is no utility in the additional pre-option header proposed in draft-herbert-udp-space-hdr-01.
>
> Joe
>
>

---

From: Joe Touch <[email-masked]>
Date: Mon, 15 Jul 2019 13:18:57 -0700

> Please explain how LITE mode works robustly if the UDP checksum is set.

LITE isn't robust, by design, but... 

- we have another thread about that issue, but we're NOT DISCUSSING THAT
RIGHT NOW 

- IF WE WERE DISCUSSING THAT NOW, I'd note that OCS could do the
following: 
    1- SHOULD be used, but MAY be set to zero (at the user's own
discretion and potential peril) 
    2- LITE ***might*** use OCS set on transmission but ignored on
receipt (to *allow* errors, because LITE ISN'T ROBUST **by design**) 
    3- LITE+FRAG can do as LITE does (#2), with a reassembly checksum
instead (ESP. given the individual checksums null-out with the data,
that may be more appropriate) 

However, we're still not talking about that yet. 

Joe

---

From: Joe Touch <[email-masked]>
Date: Mon, 15 Jul 2019 13:34:37 -0700

Trying to get back on topic, with explicit clarification and context: 

Where we are on UDP options: 

- this thread is currently discussing only OCS 

- we need to pickup the previous thread on variants of LITE and FRAG,
but for now we're assuming the existing definitions of those 

- options will consume ALL the surplus space when present (which means
we might end up removing EOL) 

Where we currently are on OCS: 

- 16-bit, as per current UDP-options draft TEXT (figs and tables need to
be updated) 

- zeroes-out with the checksummed data, to enable transit of widespread
implementation bugs in middleboxes 

- the OCS field is now manditory (not signalled with a KIND field or
optional as a field) 

- OCS *SHOULD be computed, but MAY be set to zero (e.g., when UDP CS=0,
at user discretion and peril) 

- with LITE, OCS might be transmit-side computed and receive-side
ignored to allow for the intended NON-ROBUST capability (at least when
NOT transiting buggy middbleboxes; see below) of LITE 

(if LITE data doesn't change, that will transit middleboxes; if the LITE
data does change, there's no way to help it transit middleboxes anyway) 

------------ 

So we need to decide: 

- where OCS is (head or tail of the surplus space) 

- whether OCS is aligned, and if so is that 16-bit or 32-bit 

I propose that OCS can come first with no padding or alignment. 

Here's why and questions to guide the discussion (please address the
questions below or indicate why they're not sufficient):

*Given* that we *are* different from UDP and TCP in that we do NOT have
the luxury of a fixed-offset FROM THE BEGINNING OF THE TRANSPORT HEADER
for this checksum: 

- why does the offset distance matter, i.e., front vs. end? 
  NOTE that arguments based on existing implementations that limit 
  the "offset FROM TRANSPORT START" to 255 are not viable, 
  because we're already in a surplus area that's likely to be 
  500+ bytes from that location 

- how *MUCH* does the offset alignment matter, keeping in mind: 
    - ALL offloading requires dealing with the last byte not being
aligned (even when treated as zero-fill)
    - no offset might require 1 byte-swap at some point, but consumes
the least space
    - half-word alignment wastes 4 bits on avg.
    - full-word alignment wastes 12 bits (1.5 bytes) on avg

- if we can save 4 bits with 4 or fewer opcodes, it's a win
- if we can save 12 bits with 12 or fewer opcodes, it's a win

My assumption is that offload engines or the core processor need to do
"non-halfword length" cleanup regardless.

That cleanup should be at most 5 opcodes:
    load
    mask the addend
    add w/carry
    mask the sum (for the 1's compl wrap)
    add w/carry (for the 1's compl wrap)
    swap (if not half-word aligned)

That suggests we're either very close to no real penalty for "no offset"
or that we win by using at least half-word offset.

Note: the alignment overhead will also NEED to be zero-fill - and
checked that it's zero-fill, which are additional opcodes needed that
are not too far from the cleanup ones above.

i.e., given the similarity of the zero-check and the cleanup, why is it
useful to waste these bytes?

Joe

---

From: "C. M. Heard" <[email-masked]>
Date: Mon, 15 Jul 2019 14:43:14 -0700

On Mon, Jul 15, 2019 at 10:41 AM Joe Touch wrote:
> On Mon, Jul 15, 2019 at 10:28 AM Tom Herbert  wrote:
> > On Mon, Jul 15, 2019 at 10:02 AM Joe Touch wrote:
> undermined by the need for using an untenable 255-byte offset from
> the beginning of the UDP packet (not the surplus space - whose start
> offset itself *varies from packet to packet* relative to the UDP
> header, unlike the UDP checksum).
>
> NOTE2: *as repeatedly already noted*, again - GIVEN THIS CONTEXT -
> there is no utility in the additional pre-option header proposed in

Actually, there is something in the draft that IMO has great utility --
not the pre-option header per-se, but the **functionality** provided by
the draft's protocol header packet format. Specifically, that packet
format allows options to share fate with the payload data that they
accompany (i.e., either both are accepted and processed or neither is).
I think that functionality is essential for protocol correctness in the
presence of options that that affect the handling of payload data. This
point may have gotten lost because so much of the discussion has been
focused on matters efficiency and ease of implementation. While
important, those considerations are secondary to correctness.

Besides defining a UDP surplus header (USH, Section 2), the draft also
defines two variations of is usage: a protocol trailer variant (Section
2.1) and a protocol header variant (Section 2.2).

In the trailer variant the UDP user data (and padding bytes as needed
for alignment) precede the USH and trailing options appear at the the
offset indicated by the UDP Length. It applies only when UDP Length > 8.
It provides functionality similar to the packet format defined in
draft-ietf-tsvwg-udp-options-07.

In the header variant the UDP Length is set to 8. The USH immediately
follows the UDP header (with no padding), the options are next, and the
payload is last. This is functionally similar to what you get if you
start with draft-ietf-tsvwg-udp-options-07 and add a "user data" option
that consumes all trailing bytes, like the modified FRAG proposed in
https://mailarchive.ietf.org/arch/msg/tsvwg/Wv--BLVMPAX6g5umok9BQsXAEGg.

One can reasonably ask, why on earth would we want both formats? My
answer is that they provide different but complementary functions.

- The trailer variant can be sent to a legacy receiver with a reasonable
expectation that the user data will be processed normally and the
options will be ignored. This is useful in situations where one needs to
communicate with an endpoint whose capabilities are not known in advance
but where options that can be safely ignored could improve performance
in some way if that endpoint understands them. My canonical example is a
DNS client sending a request to a server with an option indicating that
it supports UDP fragmentation. The server can then send back a large
response using UDP fragmentation instead of IP fragmentation, if it
has that capability.

- The header variant can be used to safely send options that need to
share the fate of the payload data that they accompany (the options
and payload share the same checksum, so both will be discarded if
the checksum fails). Options that cause the payload to be handled
differently when they are present fall into that category. Examples
include FRAG (for obvious reasons), ACS (as data failing the alternate
checksum would normally be discarded instead of being passed to the
application), and AE (as data failing authentication or decryption would
normally be discarded instead of being passed to the application).

Note that draft-herbert-udp-space-hdr-01 stipulates that options which
cause the payload to be handled differently when they are present may
reside in the header variant only.

Why is this important?

Clearly, for legacy receivers, options in the trailer cannot be
guaranteed to share fate with the payload because they discard all
options. Similarly, if a new option option that affects data
handling is defined in the future, option-aware receivers that do
not recognize it will skip over it. These issues can in principle
be dealt with by insisting (as draft-ietf-tsvwg-udp-options-07 does)
that options that are not safe to ignore must not be sent without
prior knowledge (obtained by negotiation or configuration) that the
target receiver can properly handle them.

BUT, it's not just legacy receivers that feel the impact; option-aware
receivers treat the entire options trailer as being absent if the option
checksum fails. If the UDP checksum passes, and there are options that
change the handling of the preceding payload data, then that data will
be handled incorrectly. In other words, a correctly implemented receiver
that recognizes all options nonetheless has the potential to pass
corrupted data to the application. In my opinion that's not acceptable.

AFAICT, draft-ietf-tsvwg-udp-options-07 currently has no means for
providing the fate-sharing of options and payload. The protocol header
format in draft-herbert-udp-space-hdr-01 does. I'd like to see
draft-ietf-tsvwg-udp-options-07 revised to provide that functionality.

I have my own ideas on how best to do that, but I am going to hold them
in abeyance for a later time because I think it's much more important
for us to discuss whether I am correct to insist on the need for an
option and payload fate-sharing capability for protocol correctness.

Thanks

Mike Heard

---

From: Joe Touch <[email-masked]>
Date: Mon, 15 Jul 2019 15:01:06 -0700

> accompany (i.e., either both are accepted and processed or neither is).

How is that true? We can't redefine the UDP checksum, which covers the
payload. 

A legacy receiver doesn't care about the surplus area and can ignore it.
At which point, whether the surplus is corrupted or not, a legacy
receiver will accept it. 

A UDP-option-aware receiver would do the same thing with OCS (as
currently defined) as with the one in draft-herbert. Remember, OCS was
already redefined before the last IETF to have the same "zero out"
property as in draft-herbert-00. 

> answer is that they provide different but complementary functions.

Didn't you just prove they're equivalent to, in essence, LITE and
non-LITE? 

> options will be ignored.

that's how UDP options already work... 

> the checksum fails).

How is this not LITE? 

Note, however, that a packet still shows up (albeit zero length) with
either approach. So this isn't exactly the same as true fate-sharing. 

> Options that cause the payload to be handled 

That's not a feature of draft-herbert vs the exising UDP options; that
is a choice in Sec 8 of udp-options. 

We chose to have that case ignore options to emulate legacy receivers;
if that's not what we want, we change that rule. 

If that rule is changed, then you'd get the fate sharing you seek. 

But you can't have it both ways - with either solution. They're the same
here. Either failed options means "act like legacy" or it means "fail",
but that means that incorrect options would succeed in a legacy receiver
but fail in an options-aware one - EITHER WAY. 

> providing the fate-sharing of options and payload.

' 
It does, as shown above. 

> The protocol header 

I agree with that - let's decide what functionality we want and THEN
figure out how to deliver it. Or at least separate those two discussions
where possible. 

However, header/trailer is a misnomer. It's really about whether and how
much to use LITE. 

Joe

---

From: Tom Herbert <[email-masked]>
Date: Mon, 15 Jul 2019 16:14:43 -0700

On Mon, Jul 15, 2019 at 3:01 PM Joe Touch <[email-masked]> wrote:
> How is that true? We can't redefine the UDP checksum, which covers the payload.
>
> A legacy receiver doesn't care about the surplus area and can ignore it. At which point, whether the surplus is corrupted or not, a legacy receiver will accept it.
>
> A UDP-option-aware receiver would do the same thing with OCS (as currently defined) as with the one in draft-herbert. Remember, OCS was already redefined before the last IETF to have the same "zero out" property as in draft-herbert-00.
>
>
> ...
> Didn't you just prove they're equivalent to, in essence, LITE and non-LITE?
>
>
> Note, however, that a packet still shows up (albeit zero length) with either approach. So this isn't exactly the same as true fate-sharing.
>
>
> That's not a feature of draft-herbert vs the exising UDP options; that is a choice in Sec 8 of udp-options.
>
> We chose to have that case ignore options to emulate legacy receivers; if that's not what we want, we change that rule.
>
> But you can't have it both ways - with either solution. They're the same here. Either failed options means "act like legacy" or it means "fail", but that means that incorrect options would succeed in a legacy receiver but fail in an options-aware one - EITHER WAY.
>
>
>
> I agree with that - let's decide what functionality we want and THEN figure out how to deliver it. Or at least separate those two discussions where possible.
>
> However, header/trailer is a misnomer. It's really about whether and how much to use LITE.
>
I disagree. We have at least thirty years experience with protocol
_headers_ on the Internet. All implementations have been tuned for
them, and it would be naive to think that introducing protocol
trailers in data path protocols won't cause issues with established
deployment (some have already been highlighted). IMO, it is a very
relevant and pertinent question as to whether IETF wants to pursue
protocol trailers in data path protocols at this time.

Tom

---

From: Joe Touch <[email-masked]>
Date: Mon, 15 Jul 2019 16:44:36 -0700

On 2019-07-15 16:14, Tom Herbert wrote:

> _headers_ on the Internet.

That doesn't stop every solution for UDP from being a "trailer". 

They either trail conventional UDP data or zero-length UDP packets (no
data). Either way, they're still trailers, not headers. 

As I said, you can coin new words for these, but it's just misleading.
What you call a "header" is really what we already have as LITE, and
it's not a header. 

> All implementations have been tuned for
> deployment (some have already been highlighted).

LITE (or your "header") isn't compatible with established deployment. It
will show up as zero-length packets; it won't just "not show up". 

The real question here is how to interact with legacy receivers - Sec 8
already addresses that based on WG feedback. If we want to change that
algorithm, fine - but there are consequences. 

In the current approach: 

- IF receiver behavior is unknown, then the only safe thing to do is use
only truly optional options - ones that can be silently ignored. 

- If that's NOT the case, the ONLY TRULY SAFE behavior is to negotiate
use of those options that are NOT optional. It simply is NOT KNOWN TO BE
EQUIVALENTLY SAFE to simply send the options as part of LITE per se. 

LITE has its uses, but it is not a substitute for NOT SENDING A PACKET. 

Note that udp-opts already includes that advice. 

Joe

---

From: "C. M. Heard" <[email-masked]>
Date: Mon, 15 Jul 2019 22:43:18 -0700

payload.

The answer is that for this variant, the the UDP length is set to 8,
so the UDP checksum just covers the UDP header (as with FRAG+LITE in
draft-ietf-tsvwg-udp-options-07). Everything else -- the pre-option
header, options, and user data go in the surplus space, and is covered
by the checksum in the pre-option header. I think this this clip from
the draft will help to clarify:

2.2 <https://tools.ietf.org/html/draft-herbert-udp-space-hdr-01#section-2.2>
Protocol header format (Extended UDP header)

   The UDP Surplus Header can be used as a protocol header. Effectively,
   this creates an extended UDP header format. The logical protocol
   layering is:

                      +-+-+-+-+-+-+-+-+-+-+-+ \
                      |      UDP header     |  |
                    / +-+-+-+-+-+-+-+-+-+-+-+  | Extended
                   |  |   UDP space header  |  | UDP header
           Surplus |  +-+-+-+-+-+-+-+-+-+-+-+  |
           space   |  | Type specific data  |  |
                   |  +-+-+-+-+-+-+-+-+-+-+-+ /
                   |  |       UDP data      |
                    \ +-+-+-+-+-+-+-+-+-+-+-+

   The packet format containing an extended UDP header is:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+\
   |        Source port            |      Destination port         | |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ UDP
   |           Length              |         Checksum              | |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+/
   |     Type      |   TSLength    |         Checksum              |\
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
   |                                                               | |
   ~                      Type Specific Data                       ~ USH
   |                                                               | |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ /
   |                                                               |
   ~                           UDP data                            ~
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

   Notes:

      - Since the UDP header is aligned and a multiple of four bytes, no
        padding for USH is necessary.

      - The UDP length is fixed to be eight so that all bytes beyond the
        UDP header are contained in the surplus space.

      - The UDP checksum covers the eight bytes of UDP header and the
        checksum pseudo header. The USH checksum covers the entire
        surplus space which includes the UDP Surplus Header and UDP
        data.

The USH checksum does needs to include a compensation pseudo-header that
is equal to the length of the surplus area (the draft omits that through
oversight, but that is easily fixed and does not affect the discussion
in the draft in a fundamental way).

receiver will accept it.

Yes, but what it will see is a zero-length UDP datagram.

Agreed -- subject of course to adding the compensation pseudo-header.

> Didn't you just prove they're equivalent to, in essence, LITE and non-LITE?

No, not at all. By definition, LITE data is excluded from OCS. The
protocol header packet format (aka extended UDP packer format) includes
the payload data in its checksum, along with the options. One could
certainly do that with OCS, but it would be necessary to add an
"payload data" option of some sort.

> that's how UDP options already work...

Exactly so. To expand on this a bit, here is the packet format:

2.1 <https://tools.ietf.org/html/draft-herbert-udp-space-hdr-01#section-2.1>
Protocol trailer format

   When used as a protocol trailer, the UDP Surplus Header immediately
   follows the UDP data. The logical protocol layering is:

                      +-+-+-+-+-+-+-+-+-+-+-+
                      |      UDP header     |
                      +-+-+-+-+-+-+-+-+-+-+-+
                      |       UDP data      |
                    / +-+-+-+-+-+-+-+-+-+-+-+ \
           Surplus |  |   USH base header   |  |
           space   |  +-+-+-+-+-+-+-+-+-+-+-+  |  USH
                   |  | Type specific data  |  |
                    \ +-+-+-+-+-+-+-+-+-+-+-+ /

   The packet format of UDP Surplus Header as a protocol trailer is:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+\
   |        Source port            |      Destination port         | |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ UDP
   |           Length              |         Checksum              | |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+/
   |                                                               |
   ~                           UDP data                            ~
   |                                                               |
   +               +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |               |                      Padding                  |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
   |     Type      |   TSLength    |         Checksum              |\
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
   |                                                               | |
   ~                      Type Specific Data                       ~ USH
   |                                                               | |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ /

   Notes:

      - The offset of the UDP Surplus Header from the start of the UDP
        header, including possible padding for the USH, is equal the UDP
        Length.

      - The number of padding bytes is 3 - ((udp_length - 1) % 4), where
        udp_length is equal to the UDP Length field. The offset of the
        Type field of the USH is 4 * ((udp_length - 1) / 4 + 1).

      - If the size of the USH header (four plus four times TSLength) is
        less than the size of the UDP surplus space in a packet, then
        the USH is considered to be malformed (see section 3.2).

      - The UDP checksum covers the UDP header and UDP data. The USH
        checksum covers the entire UDP surplus space.

      - A legacy receiver, one that does not understand the UDP Surplus
        Header, will ignore the contents of the UDP surplus space and
        process the UDP data as normal. Protocol data that cannot
        correctly be ignored by a receiver, such as the fragmentation
        option in the [UDPOPT], MUST NOT be in a surplus space trailer.

hese are the main functional differences between the protocol trailer
packet format and what is currently in draft-ietf-tsvwg-udp-options-07:

- The checksum is not optional; it is required. It covers all data
from the padding preceding the fixed surplus header to the end of
the packet. As a consequence, full checksum compensation is provided.

- Option data (not including fixed surplus header) is limited to 1020
bytes, and it must be aligned on a four-byte boundary.

- Options that affect the handling of the data preceding the trailer
are not allowed there. In other words, an option would not be allowed
in the trailer if the interpretation of the data preceding the trailer
would be different depending on whether the option was processed or
ignored. Examples include FRAG (for obvious reasons), ACS (as data
failing the alternate checksum would normally be discarded), and AE
(as data failing authentication or decryption would normally be
discarded). Such options are allowed in the header variant only.

- LITE data is not accommodated, both because of the above rule and
also because of the mandatory nature of the surplus space checksum.

EVALUATION: I do understand from what you wrote elsewhere on this thread
that OCS will incorporate the essential checksum functionality described
above in the -08 rev, which I applaud. Indeed, I already proposed that
OCS, if present, should cover the entire UDP options trailer (excluding
LITE data), and that it should be present whenever the checksum in the
UDP header is non-zero (and I'd still to see "required when UDP CS<>0"
instead of "required"). The other functional requirement that I'd
like to see levied is to ban, or at least strongly discourage, options
from the trailer if they affect the handling of the payload. I include
FRAG, ACS, and AE in this category (see comments below about LITE).
Neither of those changes require the use of a fixed pre-option header.

<OPINIONATED COMMENTARY>
For the protocol trailer packet format I see only downsides to the
fixed pre-option header as compared to what is currently defined in
draft-ietf-tsvwg-udp-options-07, including OCS as a 3-byte option.
When sending I can align OCS with NOPs if I need to, and when receiving
I don't care if it's aligned or not, since I'll use a standard Internet
checksum calculation (with compensation pseudo-header) over the whole
trailer area, conceptually prepending or appending zero bytes if it
starts or ends on an odd byte boundary. If the checksum passes, I don't
have to do anything with OCS when I go looping through the option TLVs.
So, the protocol trailer packet format has no advantage when it comes to
checksum handling. Limiting the size of the trailer area limits the
ability of DPLPMTUD to pad user data packet for MTU probing (that is a
good use for EOL, please keep it). And it precludes LITE, though given
that an OCS failure causes LITE data to be discarded while the non-LITE
data is delivered, it may be better to tell folks to use UDP-Lite if
they need a partial checksum coverage capability.
</OPINIONATED COMMENTARY>

> and payload share the same checksum, so both will be discarded if

If you mean LITE with UDP Length = 8, the answer is because LITE data
is not checksummed. In the usual case, one wants the user data to be
protected by a checksum.

That's true; as I noted above, it is possible for a the zero-length UDP
datagram to affect a legacy receiver. However, it need not affect an
options-aware receiver; the transport module can certainly distinguish
between UDP Length = 8 and no additional data vs. UDP Length = 8
and a additional data that fails OCS or the USH checksum.

> But you can't have it both ways - with either solution. They're the same
here. Either failed options means "act like legacy" or it means "fail", but
that means that incorrect options would succeed in a legacy receiver but
fail in an options-aware one - EITHER WAY.

It seems to me that you are actually making the case for devoting a bit
in the Option Kind that specifies what to when the option is not
recognized (or recognized but not supported). This shows up in the IETF
104 slides under the rubric of a "require/ignore" but that misrepresents
what the advocates (myself included) had in mind. What we (or I, at
least) had in mind was something similar to what's in RFC 8200:

      0 - skip over this option and continue processing the packet

      1 - discard the packet.

With this capability available, we could assign an Option Kind with the
flag set if the option affects the handling of the accompanying payload
(FRAG, AE, and ACS, at a minimum) and an Option Kind with the flag clear
if the option can be safely ignored by a legacy receiver (EOL, NOP, OCS,
MSS, TIME, REQ, RSP). There should be an EXP option of each flavor to
be used as appropriate for the experiment. Then the rule is, only those
options whose flag is clear can be in an options trailer; options whose
flag is set have to be in the options header. Then the desired behavior
results: an option whose flag is set will be discarded, along with its
data, either by a legacy receiver that does not understand options at
all (thought it will see a zero-length datagram) or by an options-aware
receiver that does not understand the option.

I'm of two minds about LITE: I don't see a way to make it work with the
protocol header format or something similar, but I don't see a way to
make it safe with the trailer format of draft-ietf-tsvwg-udp-options-07.
I could accept seeing it remain with a "for mutually consenting parties
only" health warning, but unless it solves actual problems that
UDP-Lite does not, maybe it's best to say "just use UDP-Lite." It does
seem as if the fond hopes of easy traversal of middleboxes that block
protocol 136 are not to be, as LITE defeats checksum compensation.

> providing the fate-sharing of options and payload.

LITE data is delivered or dscarded depending on the fate if the LITE
option, but only at the cost of having no checksum protection for the
data. The protocol header packet format in draft-herbert does not
have that drawback.

where possible.

Excellent, thank you.

much to use LITE.

Sorry, but I disagree with that. LITE does not and cannot deliver the
functionality of the protocol header packet format in draft-herbert.
LITE  does not checksum the payload data; the protocol header packet
format in draft-herbert does. I do believe that improvements can be
made to the latter; details for those who care are below my .sig

Mike

--
The main properties of of the protocol header packet format in
draft-herbert are:

- The UDP Length is required to be 8. The fixed-size surplus header
begins immediately after the UDP header, without padding.

- The checksum is not optional; it is required. It covers all data
from the the fixed surplus header to the end of the packet (as above).
As a consequence, full checksum compensation is provided.

- Option data (not including the fixed surplus header) is limited to
1020 bytes, and it must be aligned on a four-byte boundary (as above).

- LITE data is not accommodated.

- All other options are allowed.

- Data following the end of the options area (as indicated by the length
field in the surplus header) starts on a four-byte boundary. It is
processed as indicated in the options and passed to the application
(if appropriate) following such processing. In many cases there is no
processing and it's just passed upward, but that would not be the case
for FRAG, ACS, and AE.

The limitation to 1020 bytes of options seems needless and arbitrary,
and it interferes with the capability of this format to support DPLPMTUD
probe packets that consist of padding only (for more information see
draft-ietf-tsvwg-datagram-plpmtud-08 Section 4.1). This limitation is
a result of reserving a byte in the pre-option header as a type for
extensibility; if we agree that the options themselves provide enough
flexibility, then this issue can be overcome by modifying the format
to be as follows:

    0                   1                   2                   3
    0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+\
   |        Source port            |      Destination port         | |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ -Base Hdr
   |        UDP Length = 8         |   UDP Header Checksum         | |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+/
   |        Payload Offset         |  Option+Payload Checksum      |\
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
   |                                                               | |
   ~                           UDP Options                           ~Ext Hdr
   |                                                               | |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ /
   |                                                               |
   ~                         Payload Data                          ~
   |                                                               |
   +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

If alignment of the payload data is seen as desirable (which is likely the
case), then Payload Offset can be restricted to be a multiple of 2, 4, or 8
as appropriate, with a minimum value of 4. Fragments should be
similarly aligned.

Several alternatives to this format were considered, notably including the
currently-defined 3-byte OCS (one byte Kind, two byte checksum) with a
to-be-defined one-byte "user data" option that would appear at the end and
consume all following data. That combination uses at least as much space
and will use more if NOPs are inserted for alignment. So in this case
(unlike the trailer case), a fixed header appears to be advantageous
compared to possible alternatives. FRAG would be as in
https://mailarchive.ietf.org/arch/msg/tsvwg/Wv--BLVMPAX6g5umok9BQsXAEGg but
with the interpretation changed to have it consume all payload data instead
of all data that follows.

---

From: Joe Touch <[email-masked]>
Date: Mon, 15 Jul 2019 23:47:38 -0700

Mike, 

Did you had read my earlier post today about using OCS only on transmit for LITE - as a user choice?

That costs on transmit *but not receive* and allows errors in the LITE data *if* the packet doesn’t cross a buggy middlebox afterwards (which kills the packet no matter what we do anyway).

And in that context, can you update your response below?

Joe

> On Jul 15, 2019, at 10:43 PM, C. M. Heard <[email-masked]> wrote:
> 
>>> On 2019-07-15 14:43, C. M.. Heard wrote:
>>> ...
>>> accompany (i.e., either both are accepted and processed or neither is)..
>>  
> 2.2 Protocol header format (Extended UDP header)
> 
> 2.1 Protocol trailer format
> 
> what the advocates (myself included) had in mind.. What we (or I, at
> If alignment of the payload data is seen as desirable (which is likely
> the case), then Payload Offset can be restricted to be a multiple of 2,
> 4, or 8 as appropriate, with a minimum value of 4. Fragments should be
> similarly aligned.
> 
> Several alternatives to this format were considered, notably including
> the currently-defined 3-byte OCS (one byte Kind, two byte checksum)
> with a to-be-defined one-byte "user data" option that would appear at
> the end and consume all following data. That combination uses at least
> as much space and will use more if NOPs are inserted for alignment. So
> in this case (unlike the trailer case), a fixed header appears to be
> advantageous compared to possible alternatives. FRAG would be as in
> https://mailarchive.ietf.org/arch/msg/tsvwg/Wv--BLVMPAX6g5umok9BQsXAEGg
> but with the interpretation changed to have it consume all payload
> data instead of all data that follows.
>

---

From: Tom Herbert <[email-masked]>
Date: Tue, 16 Jul 2019 07:12:52 -0700

On Mon, Jul 15, 2019 at 4:44 PM Joe Touch <[email-masked]> wrote:
> They either trail conventional UDP data or zero-length UDP packets (no data). Either way, they're still trailers, not headers.
>
> As I said, you can coin new words for these, but it's just misleading. What you call a "header" is really what we already have as LITE, and it's not a header.

Joe,

I am using the correct terminology. Protocol headers are protocol
bytes in a packet that precede the payload data and occur at the
beginning of a packet, protocol trailers are protocol bytes that
follow the payload data and are at the end of the packet. LITE defines
both a header and trailer, UDP Extended Header defines a header, and
UDP Surplus Trailer defines a protocol trailer. The distinction and
relevance becomes when considering how implementations process these.

Please take a look at slides "CONSIDERATIONS Things to keep in mind
when designing a new protocol" starting at slide #30 in
https://datatracker.ietf.org/meeting/104/materials/slides-104-wgtlgo-forwarding-plane-realities-00
(technology deep dive @IETF104), particularly the advice on designing
protocol headers to be compatible with routers that will inevitably
process them.

Tom

> LITE (or your "header") isn't compatible with established deployment. It will show up as zero-length packets; it won't just "not show up".
>
> The real question here is how to interact with legacy receivers - Sec 8 already addresses that based on WG feedback. If we want to change that algorithm, fine - but there are consequences.
>
> In the current approach:
>
> - IF receiver behavior is unknown, then the only safe thing to do is use only truly optional options - ones that can be silently ignored.
>
> - If that's NOT the case, the ONLY TRULY SAFE behavior is to negotiate use of those options that are NOT optional. It simply is NOT KNOWN TO BE EQUIVALENTLY SAFE to simply send the options as part of LITE per se.
>

---

From: Joe Touch <[email-masked]>
Date: Tue, 16 Jul 2019 07:26:23 -0700

> On Jul 16, 2019, at 7:12 AM, Tom Herbert <[email-masked]> wrote:
> 
> Protocol headers are protocol
> bytes in a packet that precede the payload data ...

Everything in the surplus area comes after the UDP payload.

You can try to call it the header of the surplus area, but that’s all it can be. It’s not a header any more than TCP options are their own header.

Joe

---

From: "C. M. Heard" <[email-masked]>
Date: Tue, 16 Jul 2019 07:37:29 -0700

Responding to some points below ...

> - 16-bit, as per current UDP-options draft TEXT (figs and tables need to
OK

> implementation bugs in middleboxes
Actually it needs to include the surplus length as a pseudo-header per
https://tools.ietf.org/html/draft-fairhurst-udp-options-cco

> - the OCS field is now manditory (not signalled with a KIND field or
As mentioned in
https://mailarchive.ietf.org/arch/msg/tsvwg/XZxL29UA-95ReA72mxv5-kEytK0 for
use in the trailer I'd prefer leaving it as an option so that NOPs can be
prepended for alignment, at the discretion of the sender

> - OCS *SHOULD be computed, but MAY be set to zero (e.g., when UDP CS=0, at
> user discretion and peril)
>
Partly agree. I do not want to see any receiver obligated to accept packets
with UDP trailers that don't have a computed OCS if UDP CS<>0 (it's fine to
allow a mode that does otherwise as long as it's an optional-to-implement
capability). Remember, in IPV6 the IP Payload Length is NOT protected by an
IP header checksum or by the pseudo-header in the UDP checksum, so the ONLY
protection against corruption of the computed trailer length is the OCS.

> - with LITE, OCS might be transmit-side computed and receive-side ignored
> to allow for the intended NON-ROBUST capability (at least when NOT
> transiting buggy middbleboxes; see below) of LITE
>
As a point of clarification: this means that OCS would include the LITE
data (which it does NOT do now) and the options; OCS would not be checked
on receive; is that correct?

Mike

---

From: Tom Herbert <[email-masked]>
Date: Tue, 16 Jul 2019 07:54:54 -0700

> Everything in the surplus area comes after the UDP payload.
You can call it user data if you want which is the terminology used in
the UDP options draft. Protocol headers preceded user data, trailers
follow it.

---

From: Joe Touch <[email-masked]>
Date: Tue, 16 Jul 2019 09:08:26 -0700

Hi, Mike,

On 2019-07-16 07:37, C. M. Heard wrote:

>> - 16-bit, as per current UDP-options draft TEXT (figs and tables need to be updated)
> 
> OK 
> 
>> - zeroes-out with the checksummed data, to enable transit of widespread implementation bugs in middleboxes
> 
> Actually it needs to include the surplus length as a pseudo-header per https://tools.ietf.org/html/draft-fairhurst-udp-options-cco

Yes - I think that was cached in my to-do list. 

>> - the OCS field is now manditory (not signalled with a KIND field or optional as a field)
> 
> As mentioned in https://mailarchive.ietf.org/arch/msg/tsvwg/XZxL29UA-95ReA72mxv5-kEytK0for use in the trailer I'd prefer leaving it as an option so that NOPs can be prepended for alignment, at the discretion of the sender

Yes, that's the trade-off. If it's mandatory, we need to decide its
alignment (none, 16-bit, 32-bit) ahead of time for all time. AND it has
to be there even if it's not used. 

If we use the 1-byte flag, the alignment can be at the user's discretion
and we save space when not in use. 

>> - OCS *SHOULD be computed, but MAY be set to zero (e.g., when UDP CS=0, at user discretion and peril)
> 
> Partly agree. I do not want to see any receiver obligated to accept packets with UDP trailers that don't have a computed OCS if UDP CS<>0 (it's fine to allow a mode that does otherwise as long as it's an optional-to-implement capability).

Optional to implement seems odd; it seems fine to say the receiver can
treat ignore the surplus area in that case (i.e., act like a legacy
receiver). 

> Remember, in IPV6 the IP Payload Length is NOT protected by an IP header checksum or by the pseudo-header in the UDP checksum,

Good point - that needs to be noted. 

> so the ONLY protection against corruption of the computed trailer length is the OCS.

Again, this is certainly a risk, but why isn't it just a choice? When
UDP picks CS=0, they're saying they are willing to take that risk. 

>> - with LITE, OCS might be transmit-side computed and receive-side ignored to allow for the intended NON-ROBUST capability (at least when NOT transiting buggy middbleboxes; see below) of LITE 
>> 
>> (if LITE data doesn't change, that will transit middleboxes; if the LITE data does change, there's no way to help it transit middleboxes anyway)
> 
> As a point of clarification: this means that OCS would include the LITE data (which it does NOT do now) and the options; OCS would not be checked on receive; is that correct?

That is what I'm thinking, but there are certainly variants of what
might or might not be covered. 

I.e., we could do things a few different ways: 
- in the current draft text OCS is "all or none" for the CS area, where
LITE is "not covered" because it's jumped over completely 
- if we want protection for PART of the surplus area, we would need two
different CS. That argues for going back to Gorry's CCO proposal. 

--- 

As a note, I think we're diverging from our intended goals. Those might
be useful to include in udp-opts. My recollection was: 

- support a version of what LITE does, if possible 
- support frag/reassy 
- support single-packet (each way) exchanges with legacy receivers
(notably for the DNSSEC case, e.g., send a request with a null option to
allow enabled receivers to respond directly either in legacy mode or
option-enabled-mode) 
- support transport-level security (akin to TCP-AO) 
- support PLPMTUD (via the echo exchange) 

Many of these simply won't work as automatically legacy-compatible with
draft-herbert; the legacy data can easily exceed its limits, at a
minimum. 

Joe

---

From: Joe Touch <[email-masked]>
Date: Tue, 16 Jul 2019 09:11:54 -0700

The option refers to user data as the UDP payload that precedes the
surplus area (except in one or two cases where additional adjectives are
used). In that draft, the surplus area always begins after the user
data, so it is always considered a trailer. 

Joe

---

From: Tom Herbert <[email-masked]>
Date: Tue, 16 Jul 2019 12:46:16 -0700

Hello,

I have implemented basic support in Linux to send and receive UDP
datagrams with UDP Extended Headers. The patch is attached. This is
only IPv4 right now and the USH trailers aren't supported yet. If
anyone is interested in hacking on this please let me know.

Thanks,
Tom

> until the htmlized version and diff are available at tools.ietf.org.
From 99c24574c202b07e3f12d8a2bcc871e96826540a Mon Sep 17 00:00:00 2001
From: Tom Herbert <[email-masked]>
Date: Sat, 13 Jul 2019 20:05:59 -0700
Subject: [PATCH] UDP Surplus Header and Extended Header

This implemented basic support for UDP Surplus Header and UDP
Extended Header as defined in draft-herbert-udp-space-hdr-01.

What is supported:
	- Socket options, UDP_UEH and UDP_USH_TRAILER, to set USH
	  on a socket for sending either as an UDP Extended Header
	  or trailer.
	- Socket option, UDP_USH_RECV, to enable receiving USH in
	  ancilary data.
	- Sending of UEH set on a socket.
	- Receiving of UEH in acilary data.

Not yet:
	- Sending USH trailer (may require significant changes in
	  datagram transmit functions).
	- Receive USH trailers. Some code is implemented, but not
	  tested.
	- Only tested on lookback which uses CHECKSUM_PARTIAL. There
	  is probably a bit of work to make checksums correct.
	- IPv4 only in this patch (IPv6 support should be straightforward.

Example application code:

* Set UEH on UDP socket with some type specific data:

	struct {
		struct ushhdr ush;
		char tsd[32];
	} ush;

	ush.ush.type = 1; /* UDP Options */
	ush.ush.tsd_len = sizeof(ush.tsd) >> 2;
	ush.ush.check = 0;
	for (i = 0; i < sizeof(ush.tsd); i++)
		ush.tsd[i] = i;

	if (setsockopt(fd, IPPROTO_UDP, UDP_UEH, &ush,
		       sizeof(ush)) < 0) {
		perror("setsockopt");
		exit(-1);
	}

Set UDP socket to enable receiving USH:

	if (setsockopt(fd, SOL_UDP, UDP_USH_RECV, &on, sizeof(on)) < 0) {
		perror("ush recv setsockopt");
		exit(-1);
	}

Receive USH in ancilary data:

	for (cmsg = CMSG_FIRSTHDR(msg); cmsg != NULL;
	     cmsg = CMSG_NXTHDR(msg, cmsg)) {
		if (cmsg->cmsg_level == SOL_UDP &&
		    cmsg->cmsg_type == UDP_UEH) {
			ush = (struct ushhdr *)CMSG_DATA(cmsg);
			process_ueh(ush);
		}
	}

Signed-off-by: Tom Herbert <[email-masked]>
---
 include/linux/udp.h      |  18 ++--
 include/net/udp.h        |  21 ++++-
 include/net/ush.h        |  68 ++++++++++++++
 include/uapi/linux/udp.h |   3 +
 net/ipv4/Makefile        |   2 +-
 net/ipv4/udp.c           | 239 ++++++++++++++++++++++++++++++++++++++++++++---
 net/ipv4/udp_ush.c       | 193 ++++++++++++++++++++++++++++++++++++++
 7 files changed, 516 insertions(+), 28 deletions(-)
 create mode 100644 include/net/ush.h
 create mode 100644 net/ipv4/udp_ush.c

diff --git a/include/linux/udp.h b/include/linux/udp.h
index aa84597bdc33..a2f7e0358f1a 100644
--- a/include/linux/udp.h
+++ b/include/linux/udp.h
@@ -35,6 +35,8 @@ static inline u32 udp_hashfn(const struct net *net, u32 num, u32 mask)
 	return (num + net_hash_mix(net)) & mask;
 }
 
+struct ush_sock;
+
 struct udp_sock {
 	/* inet_sock has to be the first member */
 	struct inet_sock inet;
@@ -51,7 +53,8 @@ struct udp_sock {
 					   * different encapsulation layer set
 					   * this
 					   */
-			 gro_enabled:1;	/* Can accept GRO packets */
+			 gro_enabled:1,	/* Can accept GRO packets */
+			 recv_ush:1;
 	/*
 	 * Following member retains the information to create a UDP header
 	 * when the socket is uncorked.
@@ -89,6 +92,8 @@ struct udp_sock {
 
 	/* This field is dirtied by udp_recvmsg() */
 	int		forward_deficit;
+
+	struct ush_sock __rcu *ush;	/* USP Surplus Header to send */
 };
 
 #define UDP_MAX_SEGMENTS	(1 << 6UL)
@@ -118,17 +123,6 @@ static inline bool udp_get_no_check6_rx(struct sock *sk)
 	return udp_sk(sk)->no_check6_rx;
 }
 
-static inline void udp_cmsg_recv(struct msghdr *msg, struct sock *sk,
-				 struct sk_buff *skb)
-{
-	int gso_size;
-
-	if (skb_shinfo(skb)->gso_type & SKB_GSO_UDP_L4) {
-		gso_size = skb_shinfo(skb)->gso_size;
-		put_cmsg(msg, SOL_UDP, UDP_GRO, sizeof(gso_size), &gso_size);
-	}
-}
-
 static inline bool udp_unexpected_gso(struct sock *sk, struct sk_buff *skb)
 {
 	return !udp_sk(sk)->gro_enabled && skb_is_gso(skb) &&
diff --git a/include/net/udp.h b/include/net/udp.h
index 79d141d2103b..3af6be8b9dae 100644
--- a/include/net/udp.h
+++ b/include/net/udp.h
@@ -44,6 +44,8 @@ struct udp_skb_cb {
 	} header;
 	__u16		cscov;
 	__u8		partial_cov;
+	__u16		ush_len;
+	__u16		ush_offset;
 };
 #define UDP_SKB_CB(__skb)	((struct udp_skb_cb *)((__skb)->cb))
 
@@ -156,11 +158,21 @@ void udp_set_csum(bool nocheck, struct sk_buff *skb,
 
 static inline void udp_csum_pull_header(struct sk_buff *skb)
 {
+	unsigned int len = sizeof(struct udphdr) + UDP_SKB_CB(skb)->ush_len;
+
+	if (UDP_SKB_CB(skb)->ush_len &&
+	    udp_hdr(skb)->len == sizeof(struct udphdr)) {
+		/* UDP Extended Header is present which immediately
+		 * follows the UDP header, need to include that in the
+		 * header pull.
+		 */
+		len += UDP_SKB_CB(skb)->ush_len;
+	}
+
 	if (!skb->csum_valid && skb->ip_summed == CHECKSUM_NONE)
-		skb->csum = csum_partial(skb->data, sizeof(struct udphdr),
-					 skb->csum);
-	skb_pull_rcsum(skb, sizeof(struct udphdr));
-	UDP_SKB_CB(skb)->cscov -= sizeof(struct udphdr);
+		skb->csum = csum_partial(skb->data, len, skb->csum);
+	skb_pull_rcsum(skb, len);
+	UDP_SKB_CB(skb)->cscov -= len;
 }
 
 typedef struct sock *(*udp_lookup_t)(struct sk_buff *skb, __be16 sport,
@@ -285,6 +297,7 @@ int udp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len);
 int udp_push_pending_frames(struct sock *sk);
 void udp_flush_pending_frames(struct sock *sk);
 int udp_cmsg_send(struct sock *sk, struct msghdr *msg, u16 *gso_size);
+void udp_cmsg_recv(struct msghdr *msg, struct sock *sk, struct sk_buff *skb);
 void udp4_hwcsum(struct sk_buff *skb, __be32 src, __be32 dst);
 int udp_rcv(struct sk_buff *skb);
 int udp_ioctl(struct sock *sk, int cmd, unsigned long arg);
diff --git a/include/net/ush.h b/include/net/ush.h
new file mode 100644
index 000000000000..e639e97f3233
--- /dev/null
+++ b/include/net/ush.h
@@ -0,0 +1,68 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef __NET_USH_H
+#define __NET_USH_H
+
+#include <net/udp.h>
+
+/* Definitions for the UDP Surplus Header.
+ *
+ * Diagram of USH header (with padding preceding the header):
+ *
+ *                 +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ *                 |          Padding to 4 byte alignment          |
+ * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
+ * |     Type      |   TSLength    |         Checksum              |\
+ * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ |
+ * |                                                               | |
+ * ~                      Type Specific Data                       ~ USH
+ * |                                                               | |
+ * +-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+ /
+ *
+ * Padding aligned offset of Type field to four bytes relative to the UDP
+ * header. Type indicates type of Type Specific Data . TSLength is the length of
+ * the Type Specific Data in units of four bytes. Checksum is standard Internet
+ * checksum that covers the whole surplus space (starting from Type field offset
+ * since padding bytes are zero). Checksum ensures that the surplus space sums
+ * to zero which is necessary for compatility with UDP checksum offload. Type
+ * Specific Data is optional data of some type that is part of the USH header
+ * and interpreted per the value of the Type field. An example fo Type Specific
+ * Data is UDP Options.
+ */
+
+struct ushhdr {
+	__u8 type;
+	__u8 tsd_len;
+	__sum16 check;
+	__u8 tsd[0];
+};
+
+struct ush_sock {
+	int tot_len;
+	struct rcu_head rcu;
+	bool do_ueh;
+	struct ushhdr ush;
+};
+
+/* Defined types */
+#define USH_UDP_OPTIONS		1
+
+static inline __wsum ush_ueh_csum(struct sk_buff *skb)
+{
+	struct ushhdr *ush;
+	__wsum csum;
+
+	ush = (struct ushhdr *)(skb_transport_header(skb) +
+				sizeof(struct udphdr));
+
+	csum = csum_partial(ush, sizeof(*ush) + (ush->tsd_len << 2), skb->csum);
+
+	for (skb = skb_shinfo(skb)->frag_list; skb; skb = skb->next)
+		csum = csum_add(csum, skb->csum);
+
+	return csum;
+}
+
+int ush_recv(struct sk_buff *skb, struct udphdr *uh);
+void udp4_ueh_hwcsum(struct sk_buff *skb);
+
+#endif
diff --git a/include/uapi/linux/udp.h b/include/uapi/linux/udp.h
index 30baccb6c9c4..d5317373db62 100644
--- a/include/uapi/linux/udp.h
+++ b/include/uapi/linux/udp.h
@@ -34,6 +34,9 @@ struct udphdr {
 #define UDP_NO_CHECK6_RX 102	/* Disable accpeting checksum for UDP6 */
 #define UDP_SEGMENT	103	/* Set GSO segmentation size */
 #define UDP_GRO		104	/* This socket can receive UDP GRO packets */
+#define UDP_UEH		105	/* Extended UDP Header */
+#define UDP_USH_TRAILER	106	/* UDP Surplus Header Trailer */
+#define UDP_USH_RECV	107	/* Enable receiving USH in ancillary data */
 
 /* UDP encapsulation types */
 #define UDP_ENCAP_ESPINUDP_NON_IKE	1 /* draft-ietf-ipsec-nat-t-ike-00/01 */
diff --git a/net/ipv4/Makefile b/net/ipv4/Makefile
index d57ecfaf89d4..79186aef2a19 100644
--- a/net/ipv4/Makefile
+++ b/net/ipv4/Makefile
@@ -14,7 +14,7 @@ obj-y     := route.o inetpeer.o protocol.o \
 	     udp_offload.o arp.o icmp.o devinet.o af_inet.o igmp.o \
 	     fib_frontend.o fib_semantics.o fib_trie.o fib_notifier.o \
 	     inet_fragment.o ping.o ip_tunnel_core.o gre_offload.o \
-	     metrics.o netlink.o nexthop.o
+	     metrics.o netlink.o nexthop.o udp_ush.o
 
 obj-$(CONFIG_BPFILTER) += bpfilter/
 
diff --git a/net/ipv4/udp.c b/net/ipv4/udp.c
index c21862ba9c02..b7ceb4ba587f 100644
--- a/net/ipv4/udp.c
+++ b/net/ipv4/udp.c
@@ -112,6 +112,7 @@
 #include <net/sock_reuseport.h>
 #include <net/addrconf.h>
 #include <net/udp_tunnel.h>
+#include <net/ush.h>
 
 struct udp_table udp_table __read_mostly;
 EXPORT_SYMBOL(udp_table);
@@ -897,6 +898,104 @@ static int udp_send_skb(struct sk_buff *skb, struct flowi4 *fl4,
 	return err;
 }
 
+static int udp_send_ueh_skb(struct sk_buff *skb, struct flowi4 *fl4,
+			    struct inet_cork *cork, struct ush_sock *ush_sock)
+{
+	size_t ush_len = sizeof(struct ushhdr) + (ush_sock->ush.tsd_len << 2);
+	int offset = skb_transport_offset(skb);
+	int len = skb->len - offset;
+	struct sock *sk = skb->sk;
+	struct inet_sock *inet = inet_sk(sk);
+	struct ushhdr *ush;
+	struct udphdr *uh;
+	__wsum csum = 0;
+	int err = 0;
+
+	/* Create Extended UDP Header */
+
+	uh = udp_hdr(skb);
+	uh->source = inet->inet_sport;
+	uh->dest = fl4->fl4_dport;
+	uh->len = htons(sizeof(*uh)); /* UDP length is 8 in UEH */
+
+	ush = (struct ushhdr *)&uh[1];
+	memcpy(ush, &ush_sock->ush, ush_len);
+	ush->check = 0;
+
+	/* Set up UDP checksum */
+
+	if (sk->sk_no_check_tx) {
+		uh->check = 0;
+	} else {
+		/* Set UDP checksum. Just covers UDP header and pseudo
+		 * header.
+		 */
+
+		__wsum csum = csum_partial(skb_transport_header(skb),
+					   sizeof(struct udphdr), 0);
+		uh->check = csum_tcpudp_magic(fl4->saddr, fl4->daddr, 8,
+					      sk->sk_protocol, csum);
+
+		if (uh->check == 0)
+			uh->check = CSUM_MANGLED_0;
+	}
+
+	if (cork->gso_size) {
+		const int hlen = skb_network_header_len(skb) +
+				 sizeof(struct udphdr) + ush_len;
+
+		if (hlen + cork->gso_size > cork->fragsize) {
+			kfree_skb(skb);
+			return -EINVAL;
+		}
+		if (skb->len > cork->gso_size * UDP_MAX_SEGMENTS) {
+			kfree_skb(skb);
+			return -EINVAL;
+		}
+		if (sk->sk_no_check_tx) {
+			kfree_skb(skb);
+			return -EINVAL;
+		}
+		if (skb->ip_summed != CHECKSUM_PARTIAL ||
+		    dst_xfrm(skb_dst(skb))) {
+			kfree_skb(skb);
+			return -EIO;
+		}
+
+		skb_shinfo(skb)->gso_size = cork->gso_size;
+		skb_shinfo(skb)->gso_type = SKB_GSO_UDP_L4;
+		skb_shinfo(skb)->gso_segs = DIV_ROUND_UP(len - sizeof(uh),
+							 cork->gso_size);
+		goto csum_partial;
+	}
+
+	if (skb->ip_summed == CHECKSUM_PARTIAL) { /* UDP hardware csum */
+csum_partial:
+		udp4_ueh_hwcsum(skb);
+		goto send;
+	}
+
+	csum = ush_ueh_csum(skb);
+
+	ush->check = csum_fold(csum);
+	if (ush->check == 0)
+		ush->check = CSUM_MANGLED_0;
+
+send:
+	err = ip_send_skb(sock_net(sk), skb);
+	if (err) {
+		if (err == -ENOBUFS && !inet->recverr) {
+			UDP_INC_STATS(sock_net(sk),
+				      UDP_MIB_SNDBUFERRORS, false);
+			err = 0;
+		}
+	} else {
+		UDP_INC_STATS(sock_net(sk),
+			      UDP_MIB_OUTDATAGRAMS, false);
+	}
+	return err;
+}
+
 /*
  * Push out all pending data as one UDP datagram. Socket is locked.
  */
@@ -978,6 +1077,8 @@ int udp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)
 	int (*getfrag)(void *, char *, int, int, int, struct sk_buff *);
 	struct sk_buff *skb;
 	struct ip_options_data opt_copy;
+	unsigned int header_len = sizeof(struct udphdr);
+	struct ush_sock *ush_sock;
 
 	if (len > 0xFFFF)
 		return -EMSGSIZE;
@@ -1007,7 +1108,17 @@ int udp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)
 		}
 		release_sock(sk);
 	}
-	ulen += sizeof(struct udphdr);
+
+	ush_sock = rcu_dereference(up->ush);
+	if (ush_sock) {
+		if (ush_sock->do_ueh) {
+			header_len += sizeof(struct ushhdr) +
+			    (ush_sock->ush.tsd_len << 2);
+		}
+		/* Don't support protocol trailers yet */
+	}
+
+	ulen += header_len;
 
 	/*
 	 *	Get and verify the address.
@@ -1167,11 +1278,20 @@ int udp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)
 		struct inet_cork cork;
 
 		skb = ip_make_skb(sk, fl4, getfrag, msg, ulen,
-				  sizeof(struct udphdr), &ipc, &rt,
+				  header_len, &ipc, &rt,
 				  &cork, msg->msg_flags);
+
+		/* Provide USH info to udp_send_skb. Note we don't need
+		 * to take reference on ush_sock in fast path.
+		 */
 		err = PTR_ERR(skb);
-		if (!IS_ERR_OR_NULL(skb))
-			err = udp_send_skb(skb, fl4, &cork);
+		if (!IS_ERR_OR_NULL(skb)) {
+			if (ush_sock)
+				err = udp_send_ueh_skb(skb, fl4, &cork,
+						       ush_sock);
+			else
+				err = udp_send_skb(skb, fl4, &cork);
+		}
 		goto out;
 	}
 
@@ -1198,7 +1318,7 @@ int udp_sendmsg(struct sock *sk, struct msghdr *msg, size_t len)
 do_append_data:
 	up->len += ulen;
 	err = ip_append_data(sk, fl4, getfrag, msg, ulen,
-			     sizeof(struct udphdr), &ipc, &rt,
+			     header_len, &ipc, &rt,
 			     corkreq ? msg->msg_flags|MSG_MORE : msg->msg_flags);
 	if (err)
 		udp_flush_pending_frames(sk);
@@ -1684,6 +1804,54 @@ struct sk_buff *__skb_recv_udp(struct sock *sk, unsigned int flags,
 }
 EXPORT_SYMBOL(__skb_recv_udp);
 
+void udp_cmsg_recv(struct msghdr *msg, struct sock *sk, struct sk_buff *skb)
+{
+	if (udp_sk(sk)->gro_enabled &&
+	    (skb_shinfo(skb)->gso_type & SKB_GSO_UDP_L4)) {
+		int gso_size;
+
+		gso_size = skb_shinfo(skb)->gso_size;
+		put_cmsg(msg, SOL_UDP, UDP_GRO, sizeof(gso_size), &gso_size);
+	}
+
+	if (udp_sk(sk)->recv_ush && UDP_SKB_CB(skb)->ush_len) {
+		unsigned int offset = UDP_SKB_CB(skb)->ush_offset;
+		size_t len = UDP_SKB_CB(skb)->ush_len;
+		__u8 *ptr;
+
+		if (!offset) {
+			/* UEH is present, we know the header immediately
+			 * follows the UDP header and has been pulled up.
+			 */
+
+			ptr = skb_transport_header(skb) + sizeof(struct udphdr);
+			put_cmsg(msg, SOL_UDP, UDP_UEH, len, ptr);
+		} else {
+			__u8 data[sizeof(struct ushhdr) + 32];
+			__u8 *ptr_malloc = NULL;
+
+			ptr = data;
+			if (len > sizeof(data)) {
+				ptr_malloc = kmalloc(len, GFP_KERNEL);
+
+				/* If kmalloc fails, use the stqck buffer
+				 * so that the USH header provided will be
+				 * truncate. Userspace can detect this
+				 * situation.
+				 */
+				if (ptr_malloc)
+					ptr = ptr_malloc;
+			}
+			ptr = skb_header_pointer(skb,
+						 offset + sizeof(struct udphdr),
+						 len, ptr);
+			put_cmsg(msg, SOL_UDP, UDP_USH_TRAILER, len, ptr);
+
+			kfree(ptr_malloc);
+		}
+	}
+}
+
 /*
  * 	This should be easy, if there is something there we
  * 	return it, otherwise we block.
@@ -1771,8 +1939,7 @@ int udp_recvmsg(struct sock *sk, struct msghdr *msg, size_t len, int noblock,
 							(struct sockaddr *)sin);
 	}
 
-	if (udp_sk(sk)->gro_enabled)
-		udp_cmsg_recv(msg, sk, skb);
+	udp_cmsg_recv(msg, sk, skb);
 
 	if (inet->cmsg_flags)
 		ip_cmsg_recv_offset(msg, sk, skb, sizeof(struct udphdr), off);
@@ -2263,17 +2430,21 @@ int __udp4_lib_rcv(struct sk_buff *skb, struct udp_table *udptable,
 
 	if (ulen > skb->len)
 		goto short_packet;
+	if (udp4_csum_init(skb, uh, proto))
+		goto csum_error;
 
 	if (proto == IPPROTO_UDP) {
 		/* UDP validates ulen. */
-		if (ulen < sizeof(*uh) || pskb_trim_rcsum(skb, ulen))
+		if (ulen < sizeof(*uh)) {
 			goto short_packet;
+		} else if (ulen < skb->len) {
+			/* UDP surplus space is present */
+			if (ush_recv(skb, uh) < 0)
+				goto drop;
+		}
 		uh = udp_hdr(skb);
 	}
 
-	if (udp4_csum_init(skb, uh, proto))
-		goto csum_error;
-
 	sk = skb_steal_sock(skb);
 	if (sk) {
 		struct dst_entry *dst = skb_dst(skb);
@@ -2587,6 +2758,52 @@ int udp_lib_setsockopt(struct sock *sk, int level, int optname,
 		up->pcflag |= UDPLITE_RECV_CC;
 		break;
 
+	case UDP_UEH:
+	case UDP_USH_TRAILER: {
+		struct ush_sock *ush_sock = NULL;
+		size_t alloc_len;
+
+		if (optlen) {
+			if (!optval || optlen < sizeof(struct ushhdr) ||
+			    (optlen & 0x3))
+				return -EINVAL;
+
+			alloc_len = sizeof(*ush_sock) - sizeof(struct ushhdr) +
+				     optlen;
+
+			ush_sock = sock_kmalloc(sk, alloc_len, GFP_KERNEL);
+			if (IS_ERR(ush_sock))
+				return PTR_ERR(ush_sock);
+
+			if (copy_from_user(&ush_sock->ush, optval, optlen)) {
+				kfree(ush_sock);
+				return -EFAULT;
+			}
+
+			if ((ush_sock->ush.tsd_len << 2) +
+			    sizeof(struct ushhdr) != optlen) {
+				kfree(ush_sock);
+				return -EINVAL;
+			}
+
+			ush_sock->tot_len = alloc_len;
+			ush_sock->do_ueh = (optname == UDP_UEH);
+		}
+
+		ush_sock = xchg((__force struct ush_sock **)&up->ush,
+				ush_sock);
+		if (ush_sock) {
+			atomic_sub(ush_sock->tot_len, &sk->sk_omem_alloc);
+			kfree_rcu(ush_sock, rcu);
+		}
+
+		break;
+	}
+
+	case UDP_USH_RECV:
+		up->recv_ush = !!val;
+		break;
+
 	default:
 		err = -ENOPROTOOPT;
 		break;
diff --git a/net/ipv4/udp_ush.c b/net/ipv4/udp_ush.c
new file mode 100644
index 000000000000..8aef08f731df
--- /dev/null
+++ b/net/ipv4/udp_ush.c
@@ -0,0 +1,193 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+#include <linux/skbuff.h>
+#include <net/udp.h>
+#include <net/ush.h>
+
+static int process_tld(struct sk_buff *skb, struct ushhdr *ush)
+{
+	switch (ush->type) {
+	case USH_UDP_OPTIONS:
+		return 0;
+	default:
+		return -1;
+	}
+}
+
+static int ush_verify_checksum(struct sk_buff *skb, unsigned int ush_off)
+{
+	size_t surplus_len = skb->len - ush_off;
+	__wsum csum;
+
+	/* If a valid USH header is present the the UDP surplus space
+	 * will sum to zero in one's complement arithmetic, so
+	 * validating the UDP checksum will also simultaneously
+	 * validate the surplus space.
+	 */
+
+	if (skb->ip_summed == CHECKSUM_COMPLETE && !skb->csum_valid)
+		return 0;
+
+	csum = skb_checksum(skb, ush_off, surplus_len, 0);
+	if (csum) {
+		/* Surplus space checksum is bad. Since we now
+		 * know the sum over the surplus space we can
+		 * adjust the checskum for the trim on return.
+		 */
+		skb->csum = csum_block_sub(skb->csum, csum, surplus_len);
+		return -1;
+	}
+
+	return 0;
+}
+
+static int ush_ueh_recv(struct sk_buff *skb, struct udphdr *uh)
+{
+	size_t tld_len, len = sizeof(*uh) + sizeof(struct ushhdr);
+	struct ushhdr *ush;
+	int err;
+
+	if (!pskb_may_pull(skb, len))
+		return -1;
+
+	/* Temporarily pull up the UDP header so we can use the standard
+	 * checksum validation functions.
+	 */
+	skb_pull(skb, sizeof(*uh));
+	err = skb_checksum_simple_validate(skb);
+	skb_push(skb, sizeof(*uh));
+
+	if (err)
+		return -1;
+
+	ush = (struct ushhdr *)&uh[1];
+
+	tld_len = ush->tsd_len << 2;
+	len += tld_len;
+
+	if (!pskb_may_pull(skb, len))
+		return -1;
+
+	/* Hooray, we have a validated USH */
+
+	if (process_tld(skb, ush) < 0)
+		return -1;
+
+	/* Set skb->data to UDP payload following USH */
+
+	UDP_SKB_CB(skb)->ush_len = sizeof(*ush) + tld_len;
+	UDP_SKB_CB(skb)->ush_offset = 0;
+
+	return 0;
+}
+
+static int ush_trailer_recv(struct sk_buff *skb, struct udphdr *uh)
+{
+	size_t ulen = ntohs(uh->len);
+	unsigned int padding = 4 * ((ulen - 1) / 4 + 1);
+	unsigned int surplus_len = skb->len - ulen;
+	struct ushhdr *ush, *ush_malloc = NULL;
+	unsigned int tsd_len;
+	__u8 *ptr;
+	int i;
+	struct {
+		__u8 padding[4];
+		struct ushhdr ush;
+		__u8 tsd[32];
+	} tsd_space;
+
+	ptr = skb_header_pointer(skb, ulen, padding + sizeof(*ush),
+				 (void *)&tsd_space.ush - padding);
+	if (!ptr)
+		return -1;
+
+	for (i = 0; i < padding; i++) {
+		if (ptr[i] != 0)
+			return -1;
+	}
+
+	ush_verify_checksum(skb, ulen + padding);
+
+	ush = (struct ushhdr *)(ptr + padding);
+	tsd_len = ush->tsd_len << 2;
+
+	if (tsd_len > sizeof(tsd_space.tsd)) {
+		ush_malloc = kmalloc(sizeof(*ush) + tsd_len, GFP_KERNEL);
+		ush = ush_malloc;
+	} else {
+		ush = (struct ushhdr *)&tsd_space.ush;
+	}
+
+	ush = skb_header_pointer(skb, ulen + padding,
+				 sizeof(*ush) + tsd_len, ush);
+	if (!ush)
+		return -1;
+
+	/* Hooray, we have a validated USH */
+
+	if (process_tld(skb, ush) < 0)
+		return -1;
+
+	pskb_trim_rcsum(skb, surplus_len);
+
+	kfree(ush_malloc);
+
+	return 0;
+}
+
+int ush_recv(struct sk_buff *skb, struct udphdr *uh)
+{
+	if (ntohs(uh->len) == sizeof(*uh)) {
+		/* UDP Extended Header */
+
+		return ush_ueh_recv(skb, uh);
+	}
+
+	/* Protocol trailer */
+
+	return ush_trailer_recv(skb, uh);
+}
+
+/**
+ *	udp_ueh_hwcsum  -  handle outgoing HW checksumming for
+ *			   the UDP Extended Header
+ *	@skb:	sk_buff containing the filled-in UDP header
+ *		(checksum field must be zeroed out)
+ */
+void udp4_ueh_hwcsum(struct sk_buff *skb)
+{
+	struct udphdr *uh = udp_hdr(skb);
+	struct ushhdr *ush = (struct ushhdr *)&uh[1];
+	int offset = skb_transport_offset(skb) + sizeof(*uh);
+	int len = skb->len - offset;
+	int hlen = len;
+	__wsum csum = 0;
+
+	csum = skb_checksum(skb, offset, skb->len - offset, 0);
+	skb->csum = 0;
+	ush->check = csum_fold(csum);
+
+	if (!skb_has_frag_list(skb)) {
+		/* Only one fragment on the socket. */
+
+		skb->csum_start = (__u8 *)ush - skb->head;
+		skb->csum_offset = offsetof(struct ushhdr, check);
+		ush->check = 0;
+	} else {
+		struct sk_buff *frags;
+
+		/* HW-checksum won't work as there are two or more
+		 * fragments on the socket so that all csums of sk_buffs
+		 * should be together
+		 */
+		skb_walk_frags(skb, frags) {
+			csum = csum_add(csum, frags->csum);
+			hlen -= frags->len;
+		}
+
+		ush->check = csum_fold(skb_checksum(skb, offset, hlen, csum));
+		if (ush->check == 0)
+			ush->check = CSUM_MANGLED_0;
+
+		skb->ip_summed = CHECKSUM_NONE;
+	}
+}
-- 
2.14.1

---

From: "C. M. Heard" <[email-masked]>
Date: Tue, 16 Jul 2019 13:42:01 -0700

In-line followups where where they seem to be warranted.

On Tue, Jul 16, 2019 at 9:08 AM Joe Touch wrote:
> > https://mailarchive.ietf.org/arch/msg/tsvwg/XZxL29UA-95ReA72mxv5-kEytK0,
> > for use in the trailer I'd prefer leaving it as an option so that NOPs
> > can be prepended for alignment, at the discretion of the sender
>

Good. We agree on the trade-offs, I've said where I stand, so for now I
move on.

> > Partly agree. I do not want to see any receiver obligated to accept
> > packets with UDP trailers that don't have a computed OCS if UDP CS<>0
> > (it's fine to allow a mode that does otherwise as long as it's an
> > optional-to-implement capability).
>

What I advocate to be "optional to implement" are the ability for a receiver
to turn off verification of OCS when UDP CS<>0 and the ability for a
receiver turn on verification of OCS when UDP CS=0. This allows a receiver
to determine whether and how to do an OCS verification by looking only at
the IP Payload Length and the fields in the UDP header -- there is no need
to inspect, or worse still, loop through the options trailer to make that
determination. And if OCS does need to be verified, it can be done by a
simple and well-understood Internet checksum computation (adjusted by the
compensation pseudo-header and conceptually prepending/appending zero bytes
when the trailer starts or ends on an odd byte boundary). As a bonus, CS
offload can be done as for a TCP segment (with appropriate adjustment for
the protocol number in the pseudo-header) since the combination of that
checksum and OCS being valid is mathematically equivalent to the combination
of the UDP checksum being valid and OCS being valid. And finally, the API
can be simpler because there are fewer combinations to support.

If the above receive capabilities are optional to implement, the
transmit capabilities to generate OCS when UCP CS<>0 and to omit OCS when
UDP CS=0 can also be optional to implement. The primary benefit in this
case is simplification of the API.

Granted, this is another tradeoff; we would reduce implementation
complexity at the cost of flexibility. My judgment call is that the
capabilities in question are not likely to be wanted and so are not
worth inducing implementation complexity.

> > Remember, in IPV6 the IP Payload Length is NOT protected by an IP
header
> > checksum or by the pseudo-header in the UDP checksum,
>
> Again, this is certainly a risk, but why isn't it just a choice? When UDP
> picks CS=0, they're saying they are willing to take that risk.

And to make it clear, I'm OK with saying that OCS can be omitted if
UDP CS=0 is selected. Under the appropriate circumstances (and in
conformance with RFC 6936) that selection is indeed a valid user choice.

> > > ignored to allow for the intended NON-ROBUST capability (at least
> > > when NOT transiting buggy middbleboxes; see below) of LITE
> > >
> > > (if LITE data doesn't change, that will transit middleboxes; if the
> > > LITE data does change, there's no way to help it transit middleboxes
> > > anyway)
> >
> > data (which it does NOT do now) and the options; OCS would not be
> > checked on receive; is that correct?
>
> That is what I'm thinking, but there are certainly variants of what might
> or might not be covered.
>

Thanks for the clarification. I'll respond presently with an update to
the other (long) message.

> - support PLPMTUD (via the echo exchange)

To that last add MTU probing, which draft-ietf-tsvwg-udp-options-07
actually handles just fine, but yes, it would be good to write down
the goals in -08.

> draft-herbert; the legacy data can easily exceed its limits, at a minimum.

My take (as stated in the long message) is that we probably want both the
trailer format pretty much as defined in draft-ietf-tsvwg-udp-options-07
for the backward compatible / safe to ignore  options (MSS, TIME, REQ, RES,
dummy FRAG used for negotiation) and and a variant of the header format as
defined in draft-herbert for the others. I do have to admit that that this
has an unwanted level of complexity, but I don't think it's unwarranted
given the goals. In the end "unwarranted" is a judgment call, and YMMV.

Mike

---

From: Derek Fawcus <[email-masked]>
Date: Tue, 16 Jul 2019 22:04:24 +0100

I need to draw some pictures to work things out, and also consider the
FRAG implications, but I'm sort of liking the general direction
of Mike's suggestions in this sub thread.

I also now need to ponder upon OCS staying an option, or being a fixed
field, since as I understand it, the current scheme suggested would
allow for a naïve scan over the surplus using the standard 16 bit
checksum (suitably initialised).

The implications for LITE data seem poor, but I guess if/when middleboxes
get fixed (or in controlled environments) LITE with no checksum coverage
could be more easily handled using the fixed position checksum (i.e. no
OCS option), if we used 0000 to represent no validation of the surplus,
and it only carried the LITE data - i.e. no other options.

DF

---

From: Joe Touch <[email-masked]>
Date: Tue, 16 Jul 2019 14:24:43 -0700

Focusing on the end part...

On 2019-07-16 13:42, C. M. Heard wrote:

> defined in draft-herbert for the others.

Safe-to-ignore needs the current structure, which isn't draft-herbert
compatible (because data can easily exceed the limit). 

udp-options in general aren't draft-herbert compatible because we do not
have (nor IMO do we want) a strict limit on using the option length for
udp-options. 
Unsafe-to-ignore can use LITE, at which point we can add - if needed - a
flag to force receivers to check OCS on receipt (to differentiate the
closest we can come to LITE vs. something close enough IMO to the
draft-herbert header approach).. 

Joe

---

From: "Black, David" <[email-masked]>
Date: Tue, 16 Jul 2019 21:31:26 +0000

Still commenting as an individual …

The following text exhibits a peculiar use of “optional to implement” wording, but I think the overall simplification is useful:

> the IP Payload Length and the fields in the UDP header.

In other words the receiver does one of two things:
- If UDP CS != 0, expect to find that OCS !=0.  If OCS == 0, something is wrong. (MUST check OCS).
- If UDP CS == 0, don’t bother doing anything with OCS, even if OCS is non-zero. (I think “don’t bother” winds up being: SHOULD ignore OCS).

> case is simplification of the API.

The use of “optional” in this second chunk of text is definitely misleading.   The easy requirement is:
- if UDP CS !=0 , OCS MUST be computed and sent.
I think there’s broad agreement on that requirement.

The UDP CS == 0 case is subtle, even if the receiver SHOULD ignore OCS as indicated above.  The reason is that OCS increases the likelihood of packet delivery, leading to Tom advocating that OCS MUST be used in this case.  As noted earlier, I’m ok with a “SHOULD use OCS” requirement here based on increased likelihood of packet delivery, but not a “MUST” even with a “receiver SHOULD ignore OCS” requirement.

> conformance with RFC 6936) that selection is indeed a valid user choice.

This is ok wrt corruption concerns, but “MAY” seems too weak (IMHO) given OCS’s visible increase in likelihood of packet delivery – the RFC 2119 admonition that “the full implications must be understood and carefully weighed before choosing a different course” seems to be a very good fit to this likelihood of packet delivery concern.

Thanks, --David

From: tsvwg <[email-masked]> On Behalf Of C. M. Heard
Sent: Tuesday, July 16, 2019 4:42 PM
To: Joe Touch
Cc: tsvwg
Subject: Re: [tsvwg] draft-ietf-udp-options issues from IETF 104

[EXTERNAL EMAIL]

> and we save space when not in use.

Good. We agree on the trade-offs, I've said where I stand, so for now I move on.

> > Remember, in IPV6 the IP Payload Length is NOT protected by an IP header

Mike

---

From: Tom Herbert <[email-masked]>
Date: Tue, 16 Jul 2019 14:37:00 -0700

On Tue, Jul 16, 2019 at 2:25 PM Joe Touch <[email-masked]> wrote:
> Safe-to-ignore needs the current structure, which isn't draft-herbert compatible (because data can easily exceed the limit).
>
> udp-options in general aren't draft-herbert compatible because we do not have (nor IMO do we want) a strict limit on using the option length for udp-options.

IPv6 extension headers began without a limit, and then we realized we
needed to limit them to fit into one MTU, and then we realized that
unbounded limits are a good DOS attack vector so we added more limits
to the receiver which are now default in Linux. Saying that we need
unlimited options is an academic statement and doesn't reflect
realities of deplyment.

If needed, it's easy enough to extend USH with an type that indicates
a next header. Given that in all the history of options being defined
in any protocol, I don't think this a high priority. I see no purpose
in defining a protocol that allow 65,500 bytes of options in a single
packet.

Tom

> Unsafe-to-ignore can use LITE, at which point we can add - if needed - a flag to force receivers to check OCS on receipt (to differentiate the closest we can come to LITE vs. something close enough IMO to the draft-herbert header approach)..
>
> Joe

---

From: Joe Touch <[email-masked]>
Date: Tue, 16 Jul 2019 14:44:44 -0700

On 2019-07-16 14:37, Tom Herbert wrote:

> realities of deployment.

We CAN easily set a limit IF we went. 

We don't need to design that into the protocol structure. And we don't
need to start with something that already impacts things like
fragmentation/reassembly, LITE, etc. where the final user data is inside
the options area. 

Additionally, the comparison to IP is misleading; these are endpoint
options and should not have the same kinds of limits as per-hop. 

Finally, the "realities of deployment" should not be designed or limited
by what is currently available. UDP hasn't been updated in many decades;
I hope we won't need to come back and re-do this in 5 years just because
we're too focused on current technology. 

Joe

---

From: Joe Touch <[email-masked]>
Date: Tue, 16 Jul 2019 14:50:01 -0700

David, 

If we always force the receiver to verify OCS, we will be giving up on
the intent of LITE. I hope that's not necessary - it should be a user
choice, IMO.

> The use of "optional" in this second chunk of text is definitely misleading.   The easy requirement is: 
> 
> I think there's broad agreement on that requirement.

To be clear, this is the case I'm referring to - this would kill LITE
completely. 

I don't think we should do that. If the *user* wants to take this risk,
why are we prohibiting it? This is a user protocol, after all. 

Joe

---

From: Derek Fawcus <[email-masked]>
Date: Tue, 16 Jul 2019 22:52:50 +0100

> of Mike's suggestions in this sub thread.

Something that is percolating, and I need to try to flesh out:

If we take Mikes suggestion of the herbert framing, but with a
16 bit length field, also a fixed position checksum.  Then
treat the length as a "checksum coverage" field ala UDP-Lite.
Also reintroduce the CCO option, and retain the "End of Options" option.

We then declare that the coverage always has to include all
options.

So in the absense of LITE data, the coverage will always amount
to the total length of the surplus, and the CCO option does
not need to be present.  We can use the "End of options" to
mark where normal payload data starts, where such data experiences
the "fate sharing" Mike was discussing.

If the coverage is less than that surplus length, then the sender
should include the CCO option (to compensate for middleboxes),
but the receiver can simply ignore it.  Any data beyond the
coverage length is LITE data.  If we ever get the middlebox fixed
(or following dynamic path probing), one could omit sending the
CCO option.

Would we still need a LITE option?  Would we have any desire to
handle fragmented LITE data?  If we were to send fragmented
LITE data, then a LITE option may be useful as a specialised
LITE-FRAG option.

We should probably recommend not sending both legacy UDP payload
data at the same time as surplus area none-LITE payload data.

DF

---

From: Tom Herbert <[email-masked]>
Date: Tue, 16 Jul 2019 15:06:44 -0700

On Tue, Jul 16, 2019 at 2:44 PM Joe Touch <[email-masked]> wrote:
> We don't need to design that into the protocol structure. And we don't need to start with something that already impacts things like fragmentation/reassembly, LITE, etc. where the final user data is inside the options area.
>
> Additionally, the comparison to IP is misleading; these are endpoint options and should not have the same kinds of limits as per-hop.
>
IPv6 has destination options for which limits are applied by Linux.

> Finally, the "realities of deployment" should not be designed or limited by what is currently available. UDP hasn't been updated in many decades; I hope we won't need to come back and re-do this in 5 years just because we're too focused on current technology.
>

Tell that to the IPv6 guys ;-) It has taken years to derive a
deployable solution and that has included real world considerations
like DOS. IMO, it is wise to learn from that experience.

Consider in the UDP options draft:

 [NOTE: Tom Herbert suggested we declare "more than 3 consecutive
   NOPs" a fatal error to reduce the potential of using NOPs as a DOS
   attack, but IMO there are other equivalent ways (e.g., using
   RESERVED or other UNASSIGNED values) and the "no more than 3"
   creates its own DOS vulnerability)

Okay, so there is no limit to number of NOPs in a packet. So per the
draft we could full up an MTU or even a maximum size IP packet with
NOPs. That's going to wreak havoc on a receiver and hence is a great
DOS attack. If you don't believe, please run the experiment in your
implemenation and see what happens to your CPU utilization.

In any case, if we don't mention practical limits in the protocol
specification for something that is otherwise unlimited, people will
apply their own limits. I will tell you up front, that if you try to
put this in Linux there will be significant pushback to implement
appropriate limits. We've already went down the DOS path and have
learned the lessons. We apply configurable limits with reasonable
defaults for IPv6 HBH and destination options, and they will be
applied to SR options (it's all the same code which UDP options will
use any way). Limits might be configurable, but we're not going to
allow things in the kernel that are known and obvious DOS vectors. Of
course, if people start coming up with their own limits then that
consequently reduces interoperability.

Tom

---

From: Joe Touch <[email-masked]>
Date: Tue, 16 Jul 2019 15:10:47 -0700

Hi, Derek,

On 2019-07-16 14:52, Derek Fawcus wrote: 

> 16 bit length field, also a fixed position checksum.

First, let's skip the type field, i.e., at best add the length field to
udp-opt. We really don't need "classes" of option approaches... 

> Then

That seems like too much and I'm not sure we need the length field in
every packet. 

E.g., if we have EOL, we don't need length. Either one suffices. 

> the "fate sharing" Mike was discussing.

Agreed, but somehow it has to be easy to figure out when NOT to bother
having the receiver check OCS on the full received surplus area quickly.

> (or following dynamic path probing), one could omit sending the

I like that... 

> LITE data, then a LITE option may be useful as a specialised

I hope we can still find a way to make these possible - even "at your
own risk". 

> data at the same time as surplus area none-LITE payload data.

Yes, but that should never be meaningful. It would end up sending
duplicate data to option-aware receivers. 

I.e., of all the things options do, they DO NOT REDEFINE the existing
UDP data area. 

Joe

---

From: Joe Touch <[email-masked]>
Date: Tue, 16 Jul 2019 15:15:17 -0700

> IPv6 has destination options for which limits are applied by Linux.

Linux is not an example of how to follow specs; if anything, it's THE
example to the contrary. 

> Okay, so there is no limit to number of NOPs in a packet.

No, it's still under discussion. 

The problem is the cost of the limit creates its own DOS. 

> So per the

Receivers can do whatever they want to check without us setting protocol
limits. 

A good implementation (of anything) checks resources and limits overuse.
Period. We should not need to set limits to make that happen. 

Besides, such limits *change over time and depend on a particular
system's resources* anyway. 

One size does not fit all for all time. 

Joe

---

From: Tom Herbert <[email-masked]>
Date: Tue, 16 Jul 2019 15:42:23 -0700

On Tue, Jul 16, 2019 at 3:15 PM Joe Touch <[email-masked]> wrote:
> Linux is not an example of how to follow specs; if anything, it's THE example to the contrary.
>
That statement is not at all true nor remotely constructive.

> No, it's still under discussion.

As it has been for over year :-( In any case, I don't think I'll
comment any more on this until someone tries to push UDP Options into
Linux without any reasonable limits for DOS mitigation.

> Receivers can do whatever they want to check without us setting protocol limits.
>
> A good implementation (of anything) checks resources and limits overuse. Period. We should not need to set limits to make that happen.
>
> Besides, such limits *change over time and depend on a particular system's resources* anyway.
>

---

From: Joe Touch <[email-masked]>
Date: Tue, 16 Jul 2019 16:31:20 -0700

On Jul 16, 2019, at 3:42 PM, Tom Herbert <[email-masked]> wrote:

> As it has been for over year

As have many other aspects. 

> :-( In any case, I don't think I'll

That’s an implementation choice. It might be the best way to address the issue. But not as a hard protocol limit. 

Joe

---

From: Joe Touch <[email-masked]>
Date: Tue, 16 Jul 2019 17:31:14 -0700

Getting back to the core design assumptions:

1- support options
2- allow at least some options to be silently ignored by legacy receivers (to enable ‘“optionally enhanced” exchanges)
3- allow at least some options to be required
4- allow the options themselves to be protected
5- support for fragmentation/reassembly
6- support for MTU discovery
7- support (optional?) middlebox checksum/payload length bug traversal
8- support LITE, i.e., where some of the payload is not covered by at least some checksum processing

AFAICT:

#7 requires a CCO-like sum, but it MUST he calculated AFTER all other options are populated (it depends on the value of ALL other options and surplus data)

#8 can depend on everything except CCO (it doesn’t need to protect CCO), but it depends on the value of all other options and needs to be computed next-to-last (or last if CCO isn’t present)

And we need a way to know:
	- for #7, whether CCO is included or not used (at user’s peril, but to allow for transmitters to avoid work)
	- for #8, when to end the options (either a length field OR a EOL flag)

Are there any other design requirements?

Joe

---

From: "C. M. Heard" <[email-masked]>
Date: Tue, 16 Jul 2019 17:42:44 -0700

On Tue, Jul 16, 2019 at 2:33 PM Black, David wrote:
> - If UDP CS == 0, don’t bother doing anything with OCS, even if OCS is
>   non-zero. (I think “don’t bother” winds up being: SHOULD ignore OCS).

What I intended to say might be more accurately expressed by saying
that the default receive behavior is:

- If UDP CS != 0, perform the UDP checksum validation and discard the
  entire packet if the validation fails. In addition, if a surplus area
  is present, perform the OCS validation, and discard the surplus area
  if the validation fails.

- If UDP CS == 0, do not perform either of those checks. Accept or
  discard the UDP datagram and its surplus area depending on whether
  UDP CS == 0 is or is not allowed for the socket in question.

Implementations MAY, but need not, support the following configuration
options: (a) disable checking of OCS when UDP CS != 0 and (b) enable
checking of OCS when UDP CS == 0.

That is more liberal than David's suggestion, which does not allow (a),
and turns (b) from MAY into SHOULD NOT. I could live with either.

I do have an issue with the "expect to find" language because I
explicitly do not want to levy a requirement on an implementation
to have to look around in the trailer to see if an OCS option is
present before doing the surplus area checksum validation. Looping
through the option TLVs is best done AFTER the OCS is validated.

That may be less of a concern if OCS changes from an option to a 16-bit
word at a fixed offset and we gauge the presence/absence of OCS by whether
it is zero or non-zero. However, having OCS at a fixed offset relative to
the beginning of the surplus area interacts badly with LITE as currently
specified, as LITE is required to come first. And if that fixed offset is
relative to end of the surplus area, then in IPv6 we'd be relying on the
IP Payload Length, which owing to the lack of an IPv6 header checksum has
no error checking apart from that provided by the OCS.

> The UDP CS == 0 case is subtle, even if the receiver SHOULD ignore OCS
> as indicated above.  The reason is that OCS increases the likelihood of
> packet delivery, leading to Tom advocating that OCS MUST be used in this
> case.  As noted earlier, I’m ok with a “SHOULD use OCS” requirement here
> based on increased likelihood of packet delivery, but not a “MUST” even
> with a “receiver SHOULD ignore OCS” requirement.

I was aware from slides-103-maprg-a-tale-of-two-checksums-tom-jones-00 of
dramatic improvements in packet delivery with OCS (there called CCO), from
around 50% delivery rate to over 90% (this is based in the results for paths
to HTTP servers, where the tests look for an ICMP error message and so do not
depend upon getting an answer from the application layer). Those measurements
involved packets with a correct UDP checksum and options. I was not aware
that there was evidence that use of OCS/CCO increases the rate of packet
delivery when UDP CS == 0 is combined with options. Can you point to where
I might find that evidence? I am asking because it's counter-intuitive that
OCS/CCO over the options area would make a difference in that case. It is
constructed to make packet delivery possible through middleboxes that do
either UDP Length Checksum, UDP length Pseudoheader
or Full Payload Checksum, IP length Pseudoheader.

FWIW, one of the authors of that work, Raffaele Zullo, recently did some
measurements for UDP CS=0 over IPv6 with and without options, which he
generously shared with me and gave me permission to share with TSVWG. They
are attached below. I do not know whether the options included CCO, but
apart from that, you'll find a fairly complete explanation of the methodology
near the tail of the thread. The short version is that numbers for case B1
represent the chance that a packet with CS=0 will get to the router one hop
away from the destination. The most recent results show a success rate of
between 62% and 75% without options and between 61% and 65% with options.
That's not very good.

Mike Heard

On Tue, Jul 2, 2019 at 5:25 PM Raffaele Zullo wrote:
>
> Hello Mike,
> Sorry for the very late reply.
>
> There was no problem citing those results, I hope you have been in touch
> with Gorry about this.
>
>
> I actually had only one concern since the measurements I provided were
> related to IPv6 UDP CS=0 only and not UDPO.
> AFAICR the discussion at the time was about the general traversal
> chances of UDP CS=0 with IPv6 and not restricted to UDPO.
>
> However I've run the same measurements including UDPO.
> The meaning of A, B and B1 is the same as before, the second column
> reports the cases in which A,B and B1 happen for both UDP CS=0 and UDPO
> CS=0.
>
> The results are not so different: the drop, more visible for DNS, is due
> to paths that are OK with CS=0 but not with UDP Options.
>
> DNS
>       UDP   UDPO
> A)    2%     1%
> B)   88%    72%
> B1)  75%    65%
>
> HTTP
>       UDP   UDPO
> A)   18%    18%
> B)   76%    74%
> B1)  62%    61%
>
>
> Hope this helps.
> (I haven't had yet the chance to follow the most recent discussion on
> TSVWG).
>
>
> Cheers,
> Raffaele Zullo
>
>
> PS
> Please note a 1-2% difference with the results provided in April: I have
> used the same list of addresses but 3-4% of them did not reply in the
> last measurements.
>
>
>
> On 2019-06-23 15:04, C. M. Heard wrote:
> > Raffaele,
> >
> > I am preparing to make some comments to the TSVWG list regarding the
> > use of UDP CS=0 as a means to get packets with UDP options to traverse
> > middleboxes. I would like to cite your B1 figures as showing that
> > 26%-26% of paths in your measurements block IPv6 packets with UDP
> > CS=0; in fact I'd actually like to forward the trailing message to the
> > list with me comments, if you don't mind. Would that be OK?
> >
> > Mike Heard
> >
> > On Sat, Apr 20, 2019 at 2:49 AM Raffaele Zullo
> > <[email-masked]> wrote:
> >
> > > Hello Mike,
> > > Hello Gorry,
> > >
> > > You made quite curious about the traversal of IPv6 UDP CS=0, so I
> > > run
> > > the traceroute measurements from my own host only for this case.
> > > These are the results:
> > > A)  paths working end-to-end (as explained in a previous email)
> > > B)  paths in which last discovered router is the same for UDP with
> > > correct CS and UDP with CS=0
> > > B1) paths in which last discovered router is the same for UDP with
> > > correct CS and UDP with CS=0 and it's 1 TTL from the destination
> > > server
> > >
> > > DNS
> > > A)   2%
> > > B)  87%
> > > B1) 74%
> > >
> > > HTTP
> > > A)  16%
> > > B)  77%
> > > B1) 64%
> > >
> > > Obviously this is not enough to say that these paths are compatible
> > > with
> > > v6 CS=0 but it still suggests that its traversal chances are higher
> > > than
> > > the bare end-to-end percentage.
> > >
> > > Happy Easter,
> > > Raffaele Zullo
...
> > > > On Sun, Apr 7, 2019 at 6:25 AM Raffaele Zullo wrote:
> > > > >
> > > > > Hello Gorry,
> > > > > Hello Mike,
> > > > >
> > > > > Sorry for the late reply.
> > > > > I've got lost in a few other things (basically job hunting is a job).
> > > > >
> > > > > Anyway I finally got my VPN access to the lab network restored so I
> > > > > could retrieve measurements data.
> > > > >
> > > > >
> > > > > We tested a limited number of (paths to) IPv6 servers, obtained from
> > > > > Alexa top-1m:
> > > > > 17110 authoritative DNS servers
> > > > > and
> > > > > 12184 HTTP servers.
> > > > >
> > > > > DNS servers were tested with well-crafted DNS queries.
> > > > > The first packet was a regular UDP packet with correct CS.
> > > > > If the server replied to the query then it was tested with CS=0,
> > > > > Options, etc.
> > > > >
> > > > > Paths to HTTP servers were instead tested with padded packets sent to
> > > > > UDP port 80.
> > > > > The first packet was again a regular UDP packet with correct CS.
> > > > > If the server replied with ICMP Port Unreachable, then it was tested
> > > > > with CS=0, Options, etc.
> > > > >
> > > > > Out of 17110 DNS servers
> > > > > 1.75% replied to UDP CS=0
> > > > > 1.43% replied to UDP+Opt CS=0
> > > > >
> > > > > Out of 12184 HTTP servers
> > > > > 17.21% replied to UDP CS=0
> > > > > 16.67% replied to UDP+Opt CS=0
> > > > >
> > > > >
> > > > > These are the raw data.
> > > > >
> > > > >
> > > > > I would add that the portion of paths OK with IPv6 UDP CS=0 can be
> > > > > underestimated.
> > > > > Since we are measuring paths to servers, the server itself can affect
> > > > > the measurement,
> > > > > for instance if the path is clean but the server's stack discards
> > > > > IPv6 with UDP CS=0, the outcome of the measurement will be negative.
> > > > >
> > > > >
> > > > > Cheers,
> > > > > Raffaele
> > > >
> > >
> >
>

---

From: Tom Herbert <[email-masked]>
Date: Tue, 16 Jul 2019 18:13:33 -0700

Mike,

UDP options should not need to change the specified behavior of UDP
checksum. In IPv4, if the checksum is zero, the packet is accepted. In
IPv6, if the checksum is zero then the packet is dropped with the
narrow exception that that a tunneling protocol is used and the port
is explictly configured to allow zero checksums. That's what the specs
say and that's what's currently implemented.

Also, if you say "is or is not allowed for the socket in question"
that implies that we have to perform a socket lookup in the fast path
just to decide whether to basic packet verification. That's not cheap
and some devices performing the checksum verification might not even
have a way to configure this. It's much better just be deterministic
without needing state or configuration and verify the packet solely
based on its contents and what the protocol specification says to do.

Tom

> ....

---

From: Joe Touch <[email-masked]>
Date: Tue, 16 Jul 2019 21:18:46 -0700

Hi, Tom, et al.,

The checksum behavior as described in Sec 8. These were designed based on WG discussion to achieve the following properties:

- presence of options should NEVER cause behavior different than at a legacy receiver
	this is why if UDP CS passes but OCS fails, the UDP payload should still be passed to the user
	i.e., it should never be a penalty to use options

- individual options can cause the receiver to drop the packet when the option fails ONLY if both sides agree in advance
	when that is NOT the case, the success/failure of the option is indicated to the user but UDP doesn’t made the decision

What changed that makes these incorrect or insufficient?

Joe

---

From: "Black, David" <[email-masked]>
Date: Wed, 17 Jul 2019 13:32:19 +0000

Joe,

This is a convenient message to suggest some structuring of the discussion.

> 8- support LITE, i.e., where some of the payload is not covered by at least

As an individual, I think it would be useful to separate out LITE into a separate discussion topic as that would enable a focus on getting the UDP options functionality framework nailed down for the first 7 design goals, although there may be a need to acknowledge some FRAG/LITE interaction.

Thanks, --David

> Sent: Tuesday, July 16, 2019 8:31 PM
> To: tsvwg
> Subject: [tsvwg] design assumptions - draft-ietf-udp-options
> 
> 
> [EXTERNAL EMAIL]
> 
> #7 requires a CCO-like sum, but it MUST he calculated AFTER all other options
> are populated (it depends on the value of ALL other options and surplus
> data)
> 
> #8 can depend on everything except CCO (it doesn’t need to protect CCO),
> but it depends on the value of all other options and needs to be computed
> next-to-last (or last if CCO isn’t present)
>

---

From: "Black, David" <[email-masked]>
Date: Wed, 17 Jul 2019 13:59:06 +0000

Mike, 

Top-posting on a few items (still as an individual, not a WG chair).

> checking of OCS when UDP CS == 0.

Good - I agree with the two main bullets.   On the two additional items, (a) and (b) for non-LITE cases:

- For (a), I'm convinced that a "MAY disable" requirement is a mistake (too weak) because it provides the network with "enough rope" to hang the less-than-expert endpoint implementer.  I could live with "SHOULD NOT disable" but prefer "MUST NOT disable" for simplicity (as with the exception of mixing of LITE and non-LITE data in the same packet, I've not seen a good reason for not checking OCS when the UDP CS is non-zero).

- For (b), I can see the utility of checking OCS in the non-LITE case where there's another integrity check in the protocol stack that doesn't cover UDP options.  This text is likely to be subtle because RFC 6936 recommends (SHOULD) but does not require (SHALL/MUST) such an integrity check, so both the cases of presence and absence of that integrity check will merit attention.  I remain somewhat unclear about what's going on in the LITE case, in particular about how the service level for LITE data ought to be reflected in the service level for the UDP options.

As I noted previously, I think LITE ought to be split out from the main design and worked through separately, more below ...

> through the option TLVs is best done AFTER the OCS is validated.

I agree - the OCS MUST be present and always be in the same place.  The only exception that I can think of is that LITE might replace OCS, if the OCS position is always at the start of the UDP options, more below ...

> word at a fixed offset and we gauge the presence/absence of OCS by
> whether it is zero or non-zero.

+1 - just do that...

> However, having OCS at a fixed offset relative to
> specified, as LITE is required to come first.

... IMHO, this is one more reason to split out LITE as a separate discussion topic as I suggested in a prior message.

A brute-force straw proposal would be that to define two variants of OCS - plain OCS that MUST NOT be used with LITE, and also define a special OCSLITE that is used only with LITE, and then only if we decide that a UDP options integrity check makes sense with LITE in some situations.  The merit of this design proposal is that if the OCS comes at the start of the UDP options area (which seems to be its preferred location), then this design proposal inflicts the resulting special-case processing of UDP options integrity checks for LITE on LITE implementations, which seems to be where that burden belongs ;-).

> I was not aware
> I might find that evidence?

There isn't any, my mistake.  I confused the LITE case that uses a nonzero UDP CS with the zero UDP CS case - mea culpa.

Thanks, --David

> From: C. M. Heard <[email-masked]>
> Sent: Tuesday, July 16, 2019 8:43 PM
> To: Black, David
> Cc: Joe Touch; tsvwg

---

From: Tom Herbert <[email-masked]>
Date: Wed, 17 Jul 2019 07:26:59 -0700

> Are there any other design requirements?

- Method to disambiguate legacy uses of surplus space
- Specification for options negotiation
- Denial of Service mitigations
- Fallback mechanism to use when intermediate devices drop packets
with UDP surplus space
- Extensibility, i.e. something like version number for the protocol
- Protocol headers versus protocol trailers
- Implementation and deployment considerations
   - Integration into a host stack
   - Interaction with common accelerations for UDP
   - Middleboxes interactions (particularly guidelines for protocols
suggested in https://datatracker.ietf.org/meeting/104/materials/slides-104-wgtlgo-forwarding-plane-realities-00)
   - Running code that implements UDP option to evaluate and guide
above. Preferably open source in Linux since that is the most deployed
end host OS that would support UDP options

---

From: <[email-masked]>
Date: Wed, 17 Jul 2019 14:46:27 +0000

Hi Joe,

IMO, the core requirements are #1, #2, #3, #4, and #7. 

#7 should be mandatory feature. 

5#, 6#, and #8 are nice-to-have. I would even prefer if those are covered in a separate specification. For example, I'm not sure whether FRAG will be useful and how it can interacts with IP fragmentation.  

Cheers,
Med

> -----Message d'origine-----
> De : tsvwg [mailto:[email-masked]] De la part de Joe Touch
> Envoyé : mercredi 17 juillet 2019 02:31
> À : tsvwg
> Objet : [tsvwg] design assumptions - draft-ietf-udp-options
> 
> 2- allow at least some options to be silently ignored by legacy receivers
> (to enable ‘“optionally enhanced” exchanges)
> #7 requires a CCO-like sum, but it MUST he calculated AFTER all other
> options are populated (it depends on the value of ALL other options and
> surplus data)
>

---

From: Joe Touch <[email-masked]>
Date: Wed, 17 Jul 2019 07:58:34 -0700

Tom,

> - Method to disambiguate legacy uses of surplus space

sure.

> - Specification for options negotiation

perhaps? a method to determine which options are supported

> - Denial of Service mitigations

perhaps? ways to use of options for DOS (we’re not trying to design for anti-DOS UDP use in general)

> - Fallback mechanism to use when intermediate devices drop packets

I don’t see a way to make this a design goal; UDP is unreliable and we can’t know where or why packets are dropped per se

> - Extensibility, i.e. something like version number for the protocol

it’s already extensible with the KIND codepoint space; given UDP lacks version numbers, it would not be useful to impose them on options alone

> - Protocol headers versus protocol trailers

that’s a mechanism; what behavior are you describing?

> - Implementation and deployment considerations

what is the design goal here? maximizing initial deployment probability of success? and if so, at what cost to future use?

i.e., this seems useful ONLY if it takes into consideration the ephemeral nature of the current deployment environment

>   - Integration into a host stack

what’s the goal here?

>   - Interaction with common accelerations for UDP

again, what’s the goal? again, would this be maximizing for the short term?

> suggested in https://datatracker.ietf.org/meeting/104/materials/slides-104-wgtlgo-forwarding-plane-realities-00 <https://datatracker.ietf.org/meeting/104/materials/slides-104-wgtlgo-forwarding-plane-realities-00>)

Those are ROUTER guidelines; middleboxes that want to look inside transport protocols would already fail if limited to many of these constraints.

And many don’t apply from the start for us:
- we do not have fixed offset (we’re already at a variable location when used with legacy-compatible data)
- we do not have a fixed header structure (even if we assume we start with either OOC or CCO and even length, there’s not much more; the rest is necessarily a chain)

And what about some of that advice? not being a draft, its advice wasn’t widely discussed or vetted, and in some places is wrong IMO. In fact, I suspect you would agree, because that advice specifically says:
- avoid checksums over the entire packet
- make checksums optional

These are both points you’ve argued against here.

(and, as a side note, another case in point, the idea of not going through the pipeline twice for tunnel exit and entry is basically denying that tunnels are inherently a kind of recursion that needs to be fully reentrant, i.e., it would be much safer to claim “tunnel entry and exit MUST behave EXACTLY as if the packet traverses the pipeline and all its processing EXACTLY twice: once for exit and once for entry, and that information between the two be limited to the context contained in the packet only”)

> end host OS that would support UDP options

There are at many more OSes that are more widely deployed (e.g., in cellphones, etc.). We should not design for the short term.

And Gorry’s team has running code.

Joe

---

From: Joe Touch <[email-masked]>
Date: Wed, 17 Jul 2019 08:02:24 -0700

FWIW, #5 Frag is a driving case for DNSSEC to allow UDP messages larger than the min MTU to traverse NATs. IP

And it should never interact with IP fragmentation any more than TCP segmentation does. 

#6 is required to enable #5.

And thus far, #8 is required to support #5 in a way that satisfies #2.

I.e., the ones I listed can’t be teased out the way you’re suggesting.

Joe

> On Jul 17, 2019, at 7:46 AM, <[email-masked]> <[email-masked]> wrote:
> 
> Hi Joe,
>

---

From: Joe Touch <[email-masked]>
Date: Wed, 17 Jul 2019 08:11:21 -0700

Hi, David,

I would agree with your point on #8 LITE were it not that #8 is the basis for the only way we have to accomplish #5 Frag while satisfying #2.

I.e., these items are not individual; they’re linked together. I haven’t made a full diagram, but:

#5 frag is critical to our driving use cases, e.g., the basis of needing options, i.e., #1 opt
#5 frag depends on #8 LITE to satisfy #2 legacy
#5 frag depends on #6 MTU to determine how to frag
#7 is important - at least for the short term - to get around middleboxes checksum computation bugs
#4 is important whenever UDP CS != 0
#3 is important to the use of some options that either cannot (eg., those that can’t do #2 legacy) or should not (e.g., ACS, AE) be ignored

So:

1 depends on 4, 5 (which depends on 2 (which depends on 3), 6, and 8) and 7

I.e., they’re all tied together.

Joe

> On Jul 17, 2019, at 6:32 AM, Black, David <[email-masked]> wrote:
> 
> Joe,
>

---

From: "C. M. Heard" <[email-masked]>
Date: Wed, 17 Jul 2019 08:33:26 -0700

On Mon, Jul 15, 2019 at 11:47 PM Joe Touch wrote:
> That costs on transmit *but not receive* and allows errors in the LITE
> data *if* the packet doesn’t cross a buggy middlebox afterwards (which
> kills the packet no matter what we do anyway).
>
[...]
> > protocol 136 are not to be, as LITE defeats checksum compensation.

There are actually a couple of live proposals on the table. The trailer
version of LITE that has OCS on transmit only (with OCS covering the LITE
data) has the disadvantage that the protocol control information --
specifically the IP payload length (for IPv6) and the length of the LITE
data in the option (for IPv4 and IPv6) -- is not protected. I think that
these problems can be elegantly solved by by Derek Fawcus's idea to change
the option length field to be a checksum coverage field in the version of
the protocol header format that I proposed, and I'm going to reply to his
message (and your comments) to help flesh out the details.

Note that there is a significant limitation to **any** proposal that has
a compensated checksum on transmit with partial checksum coverage on
receive: packets with errors in the insensitive part will not traverse
meddling middleboxes that actually **need** the compensation. Even with
that limitation, though, it should provide better service to applications
that would want to use it than does conventional UDP, while at the same
time providing better middlebox traversal properties than UDP-Lite.

So, yes, let's continue to explore this idea.

Mike

---

From: Tom Herbert <[email-masked]>
Date: Wed, 17 Jul 2019 08:35:49 -0700

> I don’t see a way to make this a design goal; UDP is unreliable and we can’t know where or why packets are dropped per se
It's a common problem to deal with the situation where the network
drops packets with new protocols or formats it hasn't seen before.
Some kind of falback is generally required to a rudimentary protocol.
See Happy Eyeballs for TCP, QUIC fallback to TCP, etc.

> it’s already extensible with the KIND codepoint space; given UDP lacks version numbers, it would not be useful to impose them on options alone

It's generally useful to have something that allows version or
alternative formats. For instance, five years from now someone may
discover a fatal flaw in UDP Options deployment that makes them
unusable in that format. Or maybe options become so popular that we
run out of option space. For example, TCP option space is quite
limited and has caused problems with extensibility-- if there was a
version number in the protocol we could define a new version to allow
larger options space and still use protocol number 6. Having the
ability to fundamental change the format is very useful for the cost
of a few bits and is common in protocols.
> that’s a mechanism; what behavior are you describing?

The use of protocol trailers is a design decision. The consequences of
taking this path may be significant since it is a departure from
convention for nearly all other IP protocols.

> i.e., this seems useful ONLY if it takes into consideration the ephemeral nature of the current deployment environment
I'm not sure how ephermeral deployment is. Consider that it "only"
took IPv6 twenty years to get to deployment. If you design it a
protocol for an idealized Internet and ignore reality it won't get
deployed.

>   - Integration into a host stack

It's "rough consensus and _running_ code"

> again, what’s the goal? again, would this be maximizing for the short term?
Deployability. Accelerations have existed since the 90's and will be
with us with the next thirty years. There's nothing short term about
them and it's not like it's a stop gap we wait for CPU performance and
power to catch up with network speeds.

> These are both points you’ve argued against here.

Yes, but these are guidelines offered which is more than we've had
before. IMO it's a mistake to just ignore them considering that these
are the devices the will be transporting the packets. And as we've
seen all too often, it routers or middleboxes for whatever reason
decide to arbitrarly drop packets, say a firewall drops packets with
UDP options because it doesn't have visibiity into the surplus space,
then the protocol is doomed.
> There are at many more OSes that are more widely deployed (e.g., in cellphones, etc.). We should not design for the short term.
I'm not how this is designing for the short term. There's over 2.5
billion users of Linux on smart phones for instance, i don't think
it's going away any time soon.

> And Gorry’s team has running code.

Great. Please include a reference to the source code in the next
update of the UDP options draft which I assume is forthcomoing.

Tom

---

From: "C. M. Heard" <[email-masked]>
Date: Wed, 17 Jul 2019 09:06:00 -0700

On Tue, Jul 16, 2019 at 3:15 PM Joe Touch wrote:
> We don't need to design that into the protocol structure.
[...]
> A good implementation (of anything) checks resources and limits
> overuse. Period. We should not need to set limits to make that happen.
>

+1 to all of that.

It may be reasonable to set some lower bounds on what a conforming
implementation is obliged to accept -- something along the lines of
the requirement in RFC 792 that "[a]ll hosts must be prepared to
accept datagrams of up to 576 octets (whether they arrive whole or
in fragments)." But that does not need to be -- and in general should
not be -- baked into the protocol structure. Capabilities change, and
increased limits can find their way over time into new implementations,
but not if the protocol structure imposes an artificial ceiling.

Mike

---

From: Tom Herbert <[email-masked]>
Date: Wed, 17 Jul 2019 09:20:50 -0700

Mike,

See Section 5.3 of RFC8504. This text should be adaptable to any
stateless protocol contaning TLVs.

Side note: UDP options should define both PAD1 and PADN to be
consistent with HBH, Dest, and SR options. Preferably PAD1 would have
type 0 and PADN would have type 1 which would faciliate reuse of the
existing parsing loop.

Tom

Tom

---

From: "C. M. Heard" <[email-masked]>
Date: Wed, 17 Jul 2019 09:36:39 -0700

On Wed, Jul 17, 2019 at 9:21 AM Tom Herbert <[email-masked]> wrote:
> stateless protocol containing TLVs.

Yes, something along those lies was what I had in mind.

> type 0 and PADN would have type 1 which would facilitate reuse of the
> existing parsing loop.

If 8-byte alignment is desired, then PADN saves work. It's not as useful
if 4-byte alignment is all that's wanted; doubtless that's why it was
left out of the IPv4 and TCP option design. I would have no problem with
adopting the more modern IPv6 conventions.

Mike

---

From: "Black, David" <[email-masked]>
Date: Wed, 17 Jul 2019 17:12:40 +0000

Joe,

Still as an individual ..

> I would agree with your point on #8 LITE were it not that #8 is the basis for
> the only way we have to accomplish #5 Frag while satisfying #2.

Hmm - if that's what's going on, could we adopt that as the design goal/scope/rationale for LITE, and focus LITE discussion on that?

In other words, I'm suggesting that LITE's only purpose in this round of design would be to support FRAG, with DNS as a motivating use case for FRAG, so that all the other clever things that LITE might be good for are out of scope.  Is that reasonable?

Thanks, --David

> Sent: Wednesday, July 17, 2019 11:11 AM
> To: Black, David
> Cc: tsvwg
> Subject: Re: [tsvwg] design assumptions - draft-ietf-udp-options
> 
> 
> [EXTERNAL EMAIL]
> 
> Hi, David,
> 
> I would agree with your point on #8 LITE were it not that #8 is the basis for
> the only way we have to accomplish #5 Frag while satisfying #2.
> 
> #3 is important to the use of some options that either cannot (eg., those that
> can’t do #2 legacy) or should not (e.g., ACS, AE) be ignored
> 
> So:
> 
> > As an individual, I think it would be useful to separate out LITE into a
> separate discussion topic as that would enable a focus on getting the UDP
> options functionality framework nailed down for the first 7 design goals,
> although there may be a need to acknowledge some FRAG/LITE interaction.
> >
> > Thanks, --David
> >
> options

---

From: "C. M. Heard" <[email-masked]>
Date: Wed, 17 Jul 2019 10:25:20 -0700

On Tue, Jul 16, 2019 at 6:13 PM Tom Herbert wrote:
> On Tue, Jul 16, 2019 at 5:43 PM C. M. Heard wrote:

I don't think that the words above say that; in any case, I certainly didn't
intend to suggest that. What I did intend to say was the opposite, i.e.,
that the presence or absence of the UDP check should affect how the option
checksum is handled.

> In IPv4, if the checksum is zero, the packet is accepted.

RFC 1122 allows the application to control that behavior:

            If a UDP datagram is received with a checksum that is non-
            zero and invalid, UDP MUST silently discard the datagram.
            An application MAY optionally be able to control whether UDP
            datagrams without checksums should be discarded or passed to
            the application.

My assumption was that would imply a per-socket option.

> In IPv6, if the checksum is zero then the packet is dropped with the

The text was intended to allow (not require) exactly that.

> based on its contents and what the protocol specification says to do.

Should I have used the word "port" rather than "socket"? The assumption
I made is that in IPv6 the application would do the explicit configuration
to allow UDP CS=0 datagrams through the (socket) API.

In any case, it would be best to phrase this in terms of actual behavior
(as RFC 1122 does) instead of in terms of a particular API.

Mike

---

From: Joe Touch <[email-masked]>
Date: Wed, 17 Jul 2019 11:11:26 -0700

Regarding the pad options and option numbers, we have thee choices:

- model TCP codepoints (as we currently do)
- model IPv6 codepoints
- pick our own 

Note that there is no way to do the first two concurrently. 

IMO we should model a transport if we model anything. 

Joe

> On Jul 17, 2019, at 9:36 AM, C. M. Heard <[email-masked]> wrote:
>

---

From: Joe Touch <[email-masked]>
Date: Wed, 17 Jul 2019 11:14:40 -0700

> On Jul 17, 2019, at 10:12 AM, Black, David <[email-masked]> wrote:
> 

I don’t think it matters for frag vs nonfrag user lit data. 

I don’t this it is viable to use lite to force options. I don’t think we should design for anything beyond soft state doing that anyway. 

Is there something left?

Joe

---

From: "C. M. Heard" <[email-masked]>
Date: Wed, 17 Jul 2019 11:20:09 -0700

> > the only way we have to accomplish #5 Frag while satisfying #2.

I emphatically disagree that LITE is needed to support FRAG in an
acceptable way.

I have already put one proposal on the table that divorces FRAG from LITE.
See the thread UDP Options: how to do FRAG without LITE and forced UDP CS=0
<https://mailarchive.ietf.org/arch/msg/tsvwg/Wv--BLVMPAX6g5umok9BQsXAEGg>.

I recently put forward a better proposal, based on a variant of Tom
Herbert.s header.
It's at the end of
https://mailarchive.ietf.org/arch/msg/tsvwg/XZxL29UA-95ReA72mxv5-kEytK0

Derek Fawcus suggested a further improvement that will also support LITE.
It's available at
https://mailarchive.ietf.org/arch/msg/tsvwg/jqctei-uuocYIp1qTJ0p1mIE-HI

I'll post a message fleshing out Derek's ideas but I'll use the other
thread for that.

Getting back to this thread and the core design assumptions that Joe posted:

1- support options
> least some checksum processing

Modulo clarification of  #3, I agree with that list. But I would also add:

9 - any option that affects the handling of payload data must share fate
with that payload data, by all receivers (legacy or otherwise)

The clarification to #3 that I would like to see is whether "required"
means:
(a) "required for any conforming implementation to support" or
(b) "required by the receiver in every packet"

I ask because the term is used both ways in draft-ietf-tsvwg-udp-options-07.

Either way, I think everything on the list is achievable.

Thanks & regards,

Mike

---

From: Tom Herbert <[email-masked]>
Date: Wed, 17 Jul 2019 11:33:40 -0700

Mike,

Yes, the RFC optionally allows it, but I doubt any implementations has
bothered to implement it. In IPv4, accepting UDP datagrams wiht zero
checksums is always protocol compliant. Interestingly, the converse to
true for IPv6; dropping UDPv6 datagrams with a zero checksum is always
protocol compliant.

> to allow UDP CS=0 datagrams through the (socket) API.

Technically, it's a tuple (socket) lookup. The way the Linux
implentation works is a socket lookup is performed, and if a socket is
matched there is a field in the returned structure that indicates if
UDPv6 zero checksum is allowed. Conceptually, this would specific
control over connected sockets (e.g. full 4-tuple is matched), however
the only users of this are UDP encapsulation protocols that just bind
to some socket so it is effectively a lookup by port.

> (as RFC 1122 does) instead of in terms of a particular API.

Yes. For instance, "socket" is not a protocol term, but "port" and
"5-tuple" are.

Tom

---

From: Tom Herbert <[email-masked]>
Date: Wed, 17 Jul 2019 12:02:02 -0700

On Wed, Jul 17, 2019 at 11:20 AM C. M. Heard <[email-masked]> wrote:
> I emphatically disagree that LITE is needed to support FRAG in an acceptable way.
>
+1

The requirement when there are UDP options that can't be ignored, like
FRAG, is that the UDP length is eight and so all the protocol and user
data is in the surplus. The bits in the surplus area can be arranged
as convenient, so then the UDP options can be in a header as opposed
to trailer. Thus the LITE option isn't needed and the awkward moving
around user data to reconstruct the protocol trailer is unneeded.

> See the thread UDP Options: how to do FRAG without LITE and forced UDP CS=0.
>
> I recently put forward a better proposal, based on a variant of Tom Herbert.s header.
> It's at the end of https://mailarchive.ietf.org/arch/msg/tsvwg/XZxL29UA-95ReA72mxv5-kEytK0
>
> It's available at https://mailarchive.ietf.org/arch/msg/tsvwg/jqctei-uuocYIp1qTJ0p1mIE-HI
>
That mentions support for checksum coverage like in UDP-Lite. I would
suggest support for that is not a requirement. Host systems perform
much better processing whole packet checksums than partial checksums
(one reason why UDP-Lite never got any traction). Also, for UDP
checksum compensation to work the checksum has to cover the whole
surplus space.

> I'll post a message fleshing out Derek's ideas but I'll use the other thread for that.
>
> 9 - any option that affects the handling of payload data must share fate with that payload data, by all receivers (legacy or otherwise)
>
> The clarification to #3 that I would like to see is whether "required" means:

---

From: Joe Touch <[email-masked]>
Date: Wed, 17 Jul 2019 12:02:53 -0700

> On Jul 17, 2019, at 11:20 AM, C. M. Heard <[email-masked]> wrote:
> 

The confirming implementations support the core options as listed. We can’t add to that list on the fly. 

Required by receivers ought to be decided by receivers, as is the case for the only current UDP choice (CS=0)

> Either way, I think everything on the list is achievable.

I don’t see how unless the user data is presented twice / once for legacy and a different way for option-aware. 

And that ought to be another design goal to avoid. 

Joe

---

From: Joe Touch <[email-masked]>
Date: Wed, 17 Jul 2019 12:08:43 -0700

> On Jul 17, 2019, at 11:33 AM, Tom Herbert <[email-masked]> wrote:
> 
> For instance, "socket" is not a protocol term, 

It is one of the Internet’s foundational terms, E.g. as in RFC793. It is a port/address tuple. A tcp connection is this defined by such a socket pair. 

Joe

---

From: Joe Touch <[email-masked]>
Date: Wed, 17 Jul 2019 12:16:34 -0700

+1

There were two reasons for the swaps involved: 

1) to enable support for zero-copy (if we're optimizing for any sort of
processing, it ought to be this sort of endpoint issue) 

2) to allow fragments to have both per-fragment options as well as
options over the reassembled whole; this is important to provide both
CCO (per fragment) and AE (over the whole result). And no, doing
security, integrity checks, etc., on fragments is not the same as doing
it over the reassembled whole. 

Joe

---

From: Tom Herbert <[email-masked]>
Date: Wed, 17 Jul 2019 12:30:47 -0700

On Wed, Jul 17, 2019 at 12:16 PM Joe Touch <[email-masked]> wrote:
> 1) to enable support for zero-copy (if we're optimizing for any sort of processing, it ought to be this sort of endpoint issue)
>
That's not needed for zero copy. We already understand how to handle
an optimize variable length headers with options such as TCP. Trailers
do not help that, and in fact they make efficient implemenation much
harder.

> 2) to allow fragments to have both per-fragment options as well as options over the reassembled whole; this is important to provide both CCO (per fragment) and AE (over the whole result). And no, doing security, integrity checks, etc., on fragments is not the same as doing it over the reassembled whole.

That is also not needed. In both IPv4 and IPv6 fragmentation, the
options (extension headers) that are set in the reassembled packet are
taken from the first packet. The same approach can be taken in UDP
options.

As for security and integrity checks being different on fragments than
a whole packet, that is true-- those are going to be much more costly
for operations. For instance, we can easily offload a per packet
checksum to hardware, but not the reassembly checksum. That's
potentially a big performance hit for no value.

Tom

---

From: Joe Touch <[email-masked]>
Date: Wed, 17 Jul 2019 12:39:19 -0700

Tom, 

On 2019-07-17 08:35, Tom Herbert wrote:

>> Some of these are design assumptions; some of these are design *approaches*. Can we please focus on the former? I'll try for a few below:
>> 
>> ...
>> 
>> I don't see a way to make this a design goal; UDP is unreliable and we can't know where or why packets are dropped per se

Those are hard-state protocols; we're dealing with UDP. I think the U
keeps being forgotten. 

>> it's already extensible with the KIND codepoint space; given UDP lacks version numbers, it would not be useful to impose them on options alone
> 
> alternative formats.

Again, that's what the KIND codepoint is for. 

> For instance, five years from now someone may
> run out of option space.

We can't in the current approach. Even if we hit a KIND limit, we can
define one of the last KINDs as "use the next byte as extended KIND". 

> For example, TCP option space is quite

TCP option space is limited exactly because of the fixed header
structure you keep advocating. 

The current udp-opt structure has no such limit, but you complain that
this creates a DOS vector. 

Hmm... 

>> that's a mechanism; what behavior are you describing?
> 

This isn't IP. 

And "trailers" is where we are unless we send packets twice, which we
don't want to do. (there's no such thing as a "header" unless there is
zero data, which means legacy receivers won't see them, which means we
need to send two packets) 

> protocol for an idealized Internet and ignore reality it won't get

Here's an example: 

IP was assigned one Ethernet codepoint, 0x0800 
IP has a version field, which was 4 at the time. 

When IPv6 was created, it complied with IP's generic structure and
version indication, using v6 in that field. 

However, a new Ethernet codepoint was assigned - 0x86dd 

This additional codepoint was assigned because *ethernet hardware at the
time* was parsing IP packets, but not fast enough to do the right thing
for IPv4 and IPv6 at the same time. 

So now we're stuck with two codepoints when we should have had one. 

>> what's the goal here?

I'm being more specific - "integration into the host stack" - are you
asking for running code (Gorry has some). Are you asking for design
advice in the doc (that's not appropriate). Are you asking for something
else? 

>> again, what's the goal? again, would this be maximizing for the short term?

Accelerators change. Common ones today won't be common in 10 years
necessarily. 

Designing for "accelerators" (including zero-copy) is fine, but it
shouldn't be tied to too many details of today's implementations. 

>> And many don't apply from the start for us:
>> - we do not have fixed offset (we're already at a variable location when used with legacy-compatible data)
>> - we do not have a fixed header structure (even if we assume we start with either OOC or CCO and even length, there's not much more; the rest is necessarily a chain)
>> 
>> And what about some of that advice? not being a draft, its advice wasn't widely discussed or vetted, and in some places is wrong IMO. In fact, I suspect you would agree, because that advice specifically says:
>> These are both points you've argued against here.
> 
> before.

I can point to car buying advice sites too, but they're no more relevant
for us. 

> IMO it's a mistake to just ignore them considering that these
> are the devices the will be transporting the packets.

Yes. Transporting. 

Routers shouldn't be this deep into a transport protocol; if they are,
they're not routers and not limited this way anyway. 

> And as we've

Remember that ALL routers dropped IPv6 when it was introduced. 

We should fix what's broken, not continue to try to design around it. 

And as long as we lack compliance enforcement (e.g., via an endorsement
mechanism), we should stop trying so hard to write our way around what
are clearly bugs. 

>> (and, as a side note, another case in point, the idea of not going through the pipeline twice for tunnel exit and entry is basically denying that tunnels are inherently a kind of recursion that needs to be fully reentrant, i.e., it would be much safer to claim "tunnel entry and exit MUST behave EXACTLY as if the packet traverses the pipeline and all its processing EXACTLY twice: once for exit and once for entry, and that information between the two be limited to the context contained in the packet only")
>> 
>> And Gorry's team has running code.
> 

It isn't for several reasons: 

a) there has been no consensus so far on substantive changes 

b) the ID queue is closed 

As to source code, that's Gorry's decision on how and where to post. 

Joe

---

From: Joe Touch <[email-masked]>
Date: Wed, 17 Jul 2019 12:45:13 -0700

On 2019-07-17 12:30, Tom Herbert wrote:

>> ... Thus the LITE option isn't needed and the awkward moving

When the UDP packet arrives, as it comes in the door, where do you put
the data? Remember, you get to put it somewhere ONCE. 

With trailers and the existing approach, you can swap a few bytes and
chop off the end just fine. With headers, not so much. 

> potentially a big performance hit for no value.

By "no value" see "End to End Arguments in System Design". 

Joe

---

From: "C. M. Heard" <[email-masked]>
Date: Wed, 17 Jul 2019 12:51:18 -0700

On Wed, Jul 17, 2019 at 7:27 AM Tom Herbert wrote:
> On Tue, Jul 16, 2019 at 5:31 PM Joe Touch wrote:
> >
> > 2- allow at least some options to be silently ignored by legacy
> >    receivers (to enable ‘“optionally enhanced” exchanges)
[...]

+1

> - Specification for options negotiation

I don't think we need a full specification for options negotiation, e.g.,
something along the lines of what's done for PPP in RFC 1661. There was
a message about a week ago that I think captured what's needed very well:

> application needs.  I think that would solve the question you raise.

In other words, the udp-options spec should be seen as a toolbox for
application protocols, i.e., it should provide a means for application
protocols to carry out UDP option negotiation. This could take the form
of a (set of) general-purpose option negotiation options, or it could be
achieved by having each option specification say how its use is negotiated.

+1

> - Fallback mechanism to use when intermediate devices drop packets

That's out of scope

> - Extensibility, i.e. something like version number for the protocol

Already provided

> - Protocol headers versus protocol trailers

That's an implementation decision, not a design requirement. And it may
not be one or the other; indeed, I believe that BOTH are needed in order
to satisfy all

>    - [ Ease of] Integration into a host stack
>    - [ Ability to take advantage of] with common accelerations for UDP

I see this as a factor to take into account when deciding between
different means to achieve the requirements, or as a consideration
that may lead to a requirement being dropped. IMO, it is not in and
of itself a functional requirement of the protocol.

>    - Middleboxes interactions

Support for middlebox checksum/payload length bug traversal is already
on the list, and I agree that it should NOT be an optional design feature
(though it may be optional to use, turned off at your own risk). Beyond
that, if you mean enabling DPI, that should be an explicit non-goal.

>    - Running code that implements UDP option to evaluate and guide above.

That's a desirable goal -- but it is not not a design requirement.

Mike Heard

---

From: "C. M. Heard" <[email-masked]>
Date: Wed, 17 Jul 2019 13:13:26 -0700

On Tue, Jul 16, 2019 at 9:18 PM Joe Touch <[email-masked]> wrote:

> Hi, Tom, et al.,
>
> The checksum behavior as described in Sec 8. These were designed based on
> WG discussion to achieve the following properties:
>

I have no issue with that action.

> - individual options can cause the receiver to drop the packet when the
> option fails ONLY if both sides agree in advance
>         when that is NOT the case, the success/failure of the option is
> indicated to the user but UDP doesn’t made the decision
>

There is a counter-argument to this: when an option that affects payload
processing is present, and an options-aware implementation does not
understand it, ignoring it and passing the payload data to the application
is clearly incorrect. Such options need (a) to share fate with the payload
(i.e., be covered by the same checksum) and (b) must be marked in the KIND
as requiring that packet to be dropped if they are present. As I have
explained previously, fate-sharing, while unachievable with the protocol
trailer format but can be achieved with a protocol header packet format.

> What changed that makes these incorrect or insufficient?

Additionally, it is highly undesirable to loop though the option TLVs prior
to validating the OCS is undesirable.
draft-ietf-tsvwg-udp-options-07#section-8 doesn't account for that.

Admittedly it's taken me a while to figure this out, sorry that I am so
slow on the uptake. But serious bugs do need to be corrected, even if they
are discovered late.

Mike

---

From: Tom Herbert <[email-masked]>
Date: Wed, 17 Jul 2019 13:19:33 -0700

On Wed, Jul 17, 2019 at 12:45 PM Joe Touch <[email-masked]> wrote:
> When the UDP packet arrives, as it comes in the door, where do you put the data? Remember, you get to put it somewhere ONCE.
>
> With trailers and the existing approach, you can swap a few bytes and chop off the end just fine. With headers, not so much.
>
In the _existing_ approach, any reasonably modern stack has a data
point in packet metadtata (in mbufs, mblock, skbuffs, etc.). The data
pointer that moves as the headers as they are processed and eventually
the data pointer points to the user data that can be copied out,
memory mapped, etc.-- there's no byte swapping. This is shown in
prototype implementation of UDP Extended Header in
https://mailarchive.ietf.org/arch/msg/tsvwg/ZwIwOQKtbM_xXIQ3oC9Vc0mguB8.

But let's assume for a second that you've discovered something new and
protocol trailers are significant implementation improvement. Great!
You're going to need to _prove_ that though. That means you need to
write a real implementation, test it, have peer review. In other
works, you need to show your work. Frankly, academic speculation about
implementation is pointless, we need to see the code it you want to
prove your point.

> No, they can't - exactly as you now note:
Works for IPv4 and IPv6...

---

From: "C. M. Heard" <[email-masked]>
Date: Wed, 17 Jul 2019 14:04:49 -0700

On Wed, Jul 17, 2019 at 12:16 PM Joe Touch <[email-masked]> wrote:

> And no, doing security, integrity checks, etc., on fragments is not the
> same as doing it over the reassembled whole.
>

But that's EXACTLY how TCP works -- both TCP checksums and TCP-AO
cover *individual
segments*.

Mike Heard

---

From: Joe Touch <[email-masked]>
Date: Wed, 17 Jul 2019 14:13:13 -0700

> On Jul 17, 2019, at 2:04 PM, C. M. Heard <[email-masked]> wrote:
> 
> 
> 
>> And no, doing security, integrity checks, etc., on fragments is not the same as doing it over the reassembled whole.
> 
> But that's EXACTLY how TCP works -- both TCP checksums and TCP-AO cover individual segments.
> 
> Mike Heard

TCP delivers fragments in order as soon as they’re ACKd. It can’t do much else because data is used by the app later before other segments arrive. TCP is thus a poor analogy imo.

A better one is the need for file transfer checks using sha or md5 - exactly because it’s lacking in TCP.  There was a proposal to add this to ftp a few years ago fwiw (search for fro hash)

Joe

---

From: Derek Fawcus <[email-masked]>
Date: Wed, 17 Jul 2019 22:22:20 +0100

On Wed, Jul 17, 2019 at 02:04:49PM -0700, C. M. Heard wrote:
> cover *individual segments*.

It is also a lesson which has been hard learnt in the various crypto
protocols.  We should authenticate (and encrypt) individual packets.

Which I suggest in this case means authenticate the UDP fragments.

As to ACS, I'm not sure.  I belive to date it has been defined to be
over the reassembled fragment.

DF

---

From: Tom Herbert <[email-masked]>
Date: Wed, 17 Jul 2019 14:30:47 -0700

Derek,

That's certainly true in the case of QUIC and DTLS. However, in those
cases the crypto is done in the UDP payload under application control.
I'm skeptical there's going to be much push to do this into a lower
layer. For instance, I don't believe TCP-AO has gained much traction.
Also, it's hard to tell where this starts to compete with IPsec.

Tom

---

From: Joe Touch <[email-masked]>
Date: Wed, 17 Jul 2019 17:06:04 -0700

> On Jul 17, 2019, at 1:19 PM, Tom Herbert <[email-masked]> wrote:
> 
>  Frankly, academic speculation about

OK, so you’ve taken both sides, in true academic fashion.

Which is it:
	- fixed offset is needed (I claim this is true for the data, you claim it for the header)
	- fixed offset is irrelevant (I claim this is unavoidable for use with legacy data to legacy receivers, but you didn’t want it because of performance)

Joe

---

From: Joe Touch <[email-masked]>
Date: Wed, 17 Jul 2019 17:09:26 -0700

> On Jul 17, 2019, at 2:30 PM, Tom Herbert <[email-masked]> wrote:
> 
> For instance, I don't believe TCP-AO has gained much traction.

Agreed, but that’s because vendors are continuing to use the deprecated TCP MD5 instead.

> Also, it's hard to tell where this starts to compete with IPsec.

That’s explained in RFC 5925. It matters more for things like BGP for TCP.

For UDP, its use isn’t particularly clear; it’s included only because it’s easy to model after TCP-AO (which is really just a more general version of TCP MD5, except for key change signaling that isn’t applicable for UDP anyway).

Joe

---

From: Joe Touch <[email-masked]>
Date: Wed, 17 Jul 2019 17:18:10 -0700

Hi, Mike,

> On Jul 17, 2019, at 1:13 PM, C. M. Heard <[email-masked]> wrote:
> 
> On Tue, Jul 16, 2019 at 9:18 PM Joe Touch <[email-masked] <mailto:[email-masked]>> wrote:
> Hi, Tom, et al.,
> 
> There is a counter-argument to this: when an option that affects payload processing is present, and an options-aware implementation does not understand it ignoring it and passing the payload data to the application is clearly incorrect.

It would be:
	- pass the user data to the app
	- indicate to the app that this option has failed

It remains up to the user to decide how to proceed. If the user thinks that this message should be handled as legacy, they can. If not, they can ignore / delete it.

> Such options need (a) to share fate with the payload (i.e., be covered by the same checksum) and (b) must be marked in the KIND as requiring that packet to be dropped if they are present.

It’s not the option that makes this determination; it’s the receiver. All the options and their success/failure are passed to the user (if they want to see them). Or a given socket can be configured to handle that discard-if-fail.

But this isn’t the sender’s decision - senders get to add options, but receivers MUST be able to act as legacy if they want. Consider that they could always be running their own legacy stack anyway.

This design rationale is all already explained in the current doc, FWIW.

> As I have explained previously, fate-sharing, while unachievable with the protocol trailer format but can be achieved with a protocol header packet format.

It really can’t - again, you can’t force a receiver to act as options-aware.

> Additionally, it is highly undesirable to loop though the option TLVs prior to validating the OCS is undesirable. draft-ietf-tsvwg-udp-options-07#section-8 doesn't account for that.

It actually does if you consider that OCS MUST come first if present. That can be made more clear in that section, though.

> Admittedly it's taken me a while to figure this out, sorry that I am so slow on the uptake. But serious bugs do need to be corrected, even if they are discovered late.

Agreed, but so far I don’t see a bug so much as revisiting decisions we already had consensus on, e.g., as per above. 

——

I think we do need to (re)converge on the basic assumptions and design goals.

One is the list of 8 items I previously noted, but another is the one highlighted in this note.

Joe

---

From: <[email-masked]>
Date: Thu, 18 Jul 2019 08:47:49 +0000

Hi Joe, 

Your reply confirms my initial thinking. 

5#, 6#, and #8 are targeting a particular case and should not be considered as such as part of the core design requirements for UDP options.

BTWs, I don’t understand well what is specific to DNSSEC fragments here to traverse NATs. Such NATs will fail to handle any incoming fragment, not only DNS. NATs which are compliant with REQ#14 of RFC4787 will handle appropriately fragments (modulo DDoS protection features). 

If a firewall/NAT is explicitly configured to reject such fragments for whatever reason, these devices are likely to strip/reject the FRAG options (or UDP options in general).

We need to set first the foundations, and then build on it for particular usages.

Cheers,
Med

> De : Joe Touch [mailto:[email-masked]]
> Envoyé : mercredi 17 juillet 2019 17:02
> À : BOUCADAIR Mohamed TGI/OLN
> Cc : tsvwg
> Objet : Re: [tsvwg] design assumptions - draft-ietf-udp-options
> 
> > 5#, 6#, and #8 are nice-to-have. I would even prefer if those are
> covered in a separate specification. For example, I'm not sure whether
> FRAG will be useful and how it can interacts with IP fragmentation.
> >
> > Cheers,
> > Med
> >
> receivers

---

From: Derek Fawcus <[email-masked]>
Date: Thu, 18 Jul 2019 11:01:09 +0100

> It really can’t - again, you can’t force a receiver to act as options-aware.

That is not what I understand Mike to mean by fate sharing, and his proposed framing.

I see it as two cases:

1) Packet with legacy UDP data, and UDP options.

In this case there is no fate sharing, a legacy receiver will only see the
UDP data, and the options will be lost.

2) Packet with zero-length UDP data, and UDP options carrying application data and options.

In this case a legacy receiver will simply receive a zero length UDP packet.
However a new receiver will have fate sharing for the application data and options.

Both cases can be used depending upon the application in question.

In a request / response scenario, one can initially send packets in form 1,
and then switch to form 2 if the responder supports such, maintaining soft
state as necessary.

Mike DNS example being one where a request would be send in form 1,
a legacy response would be received as pure UDP,
but a DNS server on a new host could send its response in form 2.

So in general there is no fate sharing between the options within a given
packet (they can be understood or not per option), but there is fate sharing
between the presence of any options and application data in form 2.

Mike was also further suggesting to carve the option id space such that there
is a signalling bit.  Doing so would then allow for fate sharing across options
if at least 1 'discard if not understood' option was present.

DF

---

From: Gorry Fairhurst <[email-masked]>
Date: Thu, 18 Jul 2019 11:11:57 +0100

On 18/07/2019, 11:01, Derek Fawcus wrote:
>>> On Jul 17, 2019, at 1:13 PM, C. M. Heard<[email-masked]>  wrote:
Close to what I assumed.

Gorry

---

From: Gorry Fairhurst <[email-masked]>
Date: Thu, 18 Jul 2019 11:21:53 +0100

On 18/07/2019, 11:11, Gorry Fairhurst wrote:
>>>> As I have explained previously, fate-sharing, while unachievable 
>>>> with the protocol trailer format but can be achieved with a 
>>>> protocol header packet format.
P.S. I like the idea to explicitly allow options longer than 255 bytes? 
- This seems useful for fragmentation and would be an easy addition, 
someone suggested using 0 option length to indicate a 16b length field, 
which could make total sense. As in:

https://mailarchive.ietf.org/arch/msg/tsvwg/B4oeZkpagdl4gkAMDCFW7F1A7_8

Gorry

---

From: Gorry Fairhurst <[email-masked]>
Date: Thu, 18 Jul 2019 11:38:45 +0100

I'm trying to understand LITE and the related topics more...

(1) One use-case is to mimic UDP-Lite for layer2 networks that wish to 
do cross-layer optimisation. The AMR codec would be an example. Such 
networks *NEED* to know
the transport requires them to do partial integrity checks at L2 (if the 
CRC the entire frame
this approach is useless). To get any real advantage, I'd say they have 
to do much more, by
probably changing the waveform (FEC, modulation, etc). UDP-Lite offers 
that, and I have
not seen people wanting more/different features - Magnus provided 
comments recently,
and I would concur this requires strong coordination with the App and 
Link design, UDP-Lite remains a suitable method, I would agree that 
adding this to another protocol is not helpful.

(2) A second use-case is to support router/network device encapsulation 
(i.e. tunnels)
where the network device has no visibility deep into the payload. Such 
devices existed
and continue to exist. This was a motivation in GRE/UDP. However, for
IPv6 a zero UDP checksum is restricted to specific uses-cases, and not a 
host function
- because as Lloyd pointed out this erodes ability to detect rogue UDP 
datagrams.

(3) A third use-case is when the packet contains another checksum (also 
the case
for some tunnels), and we could save the cost of a second checksum.  I 
am not
persauded that this is a real cost when comes to a frag/reassembly 
algorithm.

(4) Some other future use, as yet unknown?

The more I think about LITE the less I really am persauded (1) is needed.

I think (2) could be useful, but I don't yet see someone asking for 
that, and I'd expect hosts to see the entire payload, so at least a host 
implementation does not seem to need that.

I'm hessitant to agree that the complexity of LITE is worth the 
perceived cost saving
and there may be other ways that don't need this:
(*) Hardware  off-load of the checksum (e.g. Tom Herbert commened) means 
the cost can
be minimal.
(*) For a method designed just for reassembly, it is also possible to
"checksum" the fragment checksums in a fragementation protocol to provide
equivalent one-pass checksum.

On the OCS topics:

* DNSSEC or any host could send UDP fragments, would it really cost too much
to send these with a full UDP checksum for IPv6? (and then why not IPv4)?

* I have heard both sides on "middlebox checksum/payload length bug 
traversal"
(#7) which requires a CCO-like sum. I really appreciated the thread, 
which I paraphrase as something like (do I understand):

-- Sender SHOULD add an OCS when the UDP Datagram (Frag) Checksum is 
non-zero - recomended for traversal (and basic integrity protection).
-- Sender MUST check ACS or AE when present and before it process other 
UDP Options, does not then need to check the OCS on reception.
-- Sender MUST check one of the  OCS when the UDP Checksum is non-zero 
when present and no ACS, and before it processes other UDP Options. A 
16b CRC isn't that strong for a full-sized IP packet (but better than 
the Internet checksum), but a CRC32 would be.

-- Sender SHOULD add an OCS when the UDP Checksum is zero - to aide 
traversal.
-- Sender SHOULD allow a LITE option to be processed when the OCS is 
invalid - although that IMHO needs though as to how you decide on the 
validity of the packet.

And so back to Frag:

I'm concerned that DPLPMTU, Frag and Lite have much more complexity in 
their design than other parts of the spec - and more consideration is 
needed on their use.

For DPLPMTU (#6) the complexity is in another spec (which seems correct 
to me). I'd be interested whether a separate spec of Frag is also one 
way to ensure we get experience with the core spec, and then consider 
how to design Frag (#5, #8).

Gorry

On 18/07/2019, 09:47, [email-masked] wrote:
>>> On Jul 17, 2019, at 7:46 AM,<[email-masked]>
>> <[email-masked]>  wrote:
>>> Hi Joe,
>>>

---

From: Joe Touch <[email-masked]>
Date: Thu, 18 Jul 2019 07:24:58 -0700

> On Jul 18, 2019, at 3:01 AM, Derek Fawcus <[email-masked]> wrote:
> 

This is how legacy data is handled in the current draft, per sec 8.

> However a new receiver will have fate sharing for the application data and options.

This is already how LITE data is handled in the current draft, per sec 8.

> ...In a request / response scenario, one can initially send packets in form 1,

Right. And the first packet only needs to include OCS. If OCS is received correctly, then the server can respond using FRAG directly (no need for FRAG+LITE).

> packet (they can be understood or not per option),

Yes, but that opens a new can of worms. Currently, we shut that can by declaring that only a subset of the initially defined apps are ever “required to implement”; everything else MUST be designed to allow options-aware endpoints to silently ignore them if not implemented.

> but there is fate sharing

Yes, but that’s just what LITE already does…

The point I was making is that there is no way to have both forms at the same time.

Either options share fate with data ONLY FOR OPTIONS-AWARE endpoints,
or legacy and options-aware endpoints share fate BUT OPTIONS ARE THEN OPTIONAL TO EVERYONE (technically, including options-aware endpoints).

> if at least 1 'discard if not understood' option was present.

Oh, no - that’s a huge can of worms because there’s another bit to set: “all or none” or “ignore individually”.

Then what do you do if you have three that you want required “all or none as a set”. or ignored that way?

The control bits get out of control very fast…

Also, note that IPv6 included bits to encode whether ICMPs were sent (and whether the source is multicast).  We haven’t addressed that can of worms yet either...

Joe

---

From: Joe Touch <[email-masked]>
Date: Thu, 18 Jul 2019 07:50:50 -0700

Hi, Gorry,

True LITE (not just LITE with no user data) is definitely complex...

> On Jul 18, 2019, at 3:38 AM, Gorry Fairhurst <[email-masked]> wrote:
> 
> (1) One use-case is to mimic UDP-Lite for layer2 networks that wish to do cross-layer optimisation.

This is the case that makes the most complications for us.

> ... I would concur this requires strong coordination with the App and Link design, UDP-Lite remains a suitable method, I would agree that adding this to another protocol is not helpful.

If we take this off the table, things get a little simpler...

> (2) A second use-case is to support router/network device encapsulation (i.e. tunnels)
> where the network device has no visibility deep into the payload. Such devices existed
> IPv6 a zero UDP checksum is restricted to specific uses-cases, and not a host function
> - because as Lloyd pointed out this erodes ability to detect rogue UDP datagrams.

Tunnels are links, and links terminate at both routers and hosts.

This doesn’t erode anything; it follows the IPv6 policy of not duplicating the same kind of check at multiple layers (i.e., it’s odd that we keep citing IPv6 as the reason we can’t set UDP to zero, when in fact the reason we can drop the checksum for tunnels is exactly the same reasoning used to drop the header checksum in IPv6 - thankfully that’s now understood).

> (3) A third use-case is when the packet contains another checksum (also the case
> for some tunnels), and we could save the cost of a second checksum.  I am not
> persauded that this is a real cost when comes to a frag/reassembly algorithm.

> I think (2) could be useful, but I don't yet see someone asking for that,

That’s part of the issue with computing the checksum over the fragment parts vs. whole...

> and I'd expect hosts to see the entire payload, so at least a host implementation does not seem to need that.

Hosts don’t want to touch the data twice to check the fragments AND check the reassembled set.

(and the byte swap in the current LITE is part of RDMA-like zero-copy placement of fragments once received into a single place before making that final check).

> I'm hessitant to agree that the complexity of LITE is worth the perceived cost saving
> (*) Hardware  off-load of the checksum (e.g. Tom Herbert commened) means the cost can
> be minimal.

Hardware offload doesn’t help if checksumming is done multiple times.

> equivalent one-pass checksum.

But it’s not equivalent. Remember that data in the host can have errors once each fragment is received too. TCP gives up on that because it HAS to deliver each fragment once previous ones were received; UDP doesn’t.

> to send these with a full UDP checksum for IPv6? (and then why not IPv4)?

Those are ones I’d be worried about - esp. at root servers, in fact.

> * I have heard both sides on "middlebox checksum/payload length bug traversal"
> (#7) which requires a CCO-like sum. I really appreciated the thread, which I paraphrase as something like (do I understand):
> 
> -- Sender SHOULD add an OCS when the UDP Datagram (Frag) Checksum is non-zero - recomended for traversal (and basic integrity protection).
> -- Sender MUST check ACS or AE when present and before it process other UDP Options, does not then need to check the OCS on reception.

I think you mean receiver, so assuming that…

No, OCS still needs to be checked on receipt (if UDP CS !=0). ACS and AE apply to the UDP user data, not the surplus area.

> -- Sender MUST check one of the  OCS when the UDP Checksum is non-zero when present and no ACS, and before it processes other UDP Options. A 16b CRC isn't that strong for a full-sized IP packet (but better than the Internet checksum), but a CRC32 would be.

(receiver…)

Ignore ACS. Again, ACS is over the user data, not surplus area.

> -- Sender SHOULD add an OCS when the UDP Checksum is zero - to aide traversal.

Yes…let’s call this what it is, though - to aid traversal of middlebox bugs.

> -- Sender SHOULD allow a LITE option to be processed when the OCS is invalid - although that IMHO needs though as to how you decide on the validity of the packet.

(receiver…)

Hmm. Now I’m confused. You’ve just re-enabled everything LITE had in the first place (including UDP-LITE emulation), but then you have to parse into the area to figure out when to ignore OCS...

> And so back to Frag:
> 
> I'm concerned that DPLPMTU, Frag and Lite have much more complexity in their design than other parts of the spec - and more consideration is needed on their use.
> 
> For DPLPMTU (#6) the complexity is in another spec (which seems correct to me). I'd be interested whether a separate spec of Frag is also one way to ensure we get experience with the core spec, and then consider how to design Frag (#5, #8).

We can, but IMO the criticality of fragmentation support means that it’s not particularly relevant to have UDP options that don’t support fragmentation.

Joe

---

From: <[email-masked]>
Date: Thu, 18 Jul 2019 15:00:17 +0000

Re-, 

I think we are in agreement about the splitting of the core functionalities vs. others. 

I have a question about this:

> to me).

I guess you are referring to what is used to be Section 6.2 of DPLPMTU (-07), but that section was removed from your draft for some reason. 

I fail to find the same level of details/discussion in the current udp-options I-D. For example, how nonce (token in DPLPMTU) is defined, used, etc. 

Worse, udp-options I-D states:

==
Their use is described as part of the
   UDP variant of packetization layer path MTU discovery (PLPMTUD)
   [Fa19].
==

But DPLPMTU I-D says the following: 

==
   Support for using DPLPMTUD with UDP-Options is defined in the UDP-
   Options specification [I-D.ietf-tsvwg-udp-options].
==

Cheers,
Med

> De : tsvwg [mailto:[email-masked]] De la part de Gorry Fairhurst
> Envoyé : jeudi 18 juillet 2019 12:39
> À : [email-masked]
> Cc : gorry Fairhurst
> Objet : Re: [tsvwg] design assumptions - draft-ietf-udp-options - trying
> to see the bigger picture
> 
> > 5#, 6#, and #8 are targeting a particular case and should not be
> considered as such as part of the core design requirements for UDP
> options.
> >
> > BTWs, I don’t understand well what is specific to DNSSEC fragments here
> to traverse NATs. Such NATs will fail to handle any incoming fragment, not
> only DNS. NATs which are compliant with REQ#14 of RFC4787 will handle
> appropriately fragments (modulo DDoS protection features).
> >
> > If a firewall/NAT is explicitly configured to reject such fragments for
> whatever reason, these devices are likely to strip/reject the FRAG options
> (or UDP options in general).
> >

---

From: "C. M. Heard" <[email-masked]>
Date: Thu, 18 Jul 2019 08:08:01 -0700

On Thu, Jul 18, 2019 at 7:25 AM Joe Touch wrote:
> > On Jul 18, 2019, at 3:01 AM, Derek Fawcus wrote:
> > Mike was also further suggesting to carve the option id space such that
> > there is a signalling bit.  Doing so would then allow for fate sharing
> > across options if at least 1 'discard if not understood' option was
> > present.
>

I think Derek mis-stated what I had in mind., which was was something
similar to what's in RFC 8200 to specify what happens if an options-aware
implementation encounters an option that it does not recognize:

      1 - discard the packet.

This processing instruction would apply to an individual option. The options
would not be coupled together.

This provides a means to instruct an implementation to discard an option if
the payload would be misunderstood if the option was simply ignores.

You've objected to this before, I expect you still do, so all I am doing
is clarifying what was being proposed.

> Also, note that IPv6 included bits to encode whether ICMPs were sent (and
> whether the source is multicast).  We haven’t addressed that can of worms
> yet either...

Nor will we, presumably, since UDP has no analog to ICMP messages.

Mike

---

From: Joe Touch <[email-masked]>
Date: Thu, 18 Jul 2019 08:26:44 -0700

> On Jul 18, 2019, at 8:08 AM, C. M. Heard <[email-masked]> wrote:
> 

UDP can cause ICMPs to be issued as per RFC1122. 

Joe

---

From: "C. M. Heard" <[email-masked]>
Date: Thu, 18 Jul 2019 08:35:38 -0700

> UDP can cause ICMPs to be issued as per RFC1122.

Correct, it can send Port Unreachable. But that does not
apply for packets discarded because of unknown options,
and I don't see any other ICMP messages listed in Sec 4.1.

Mike

---

From: Tom Herbert <[email-masked]>
Date: Thu, 18 Jul 2019 08:35:41 -0700

> UDP can cause ICMPs to be issued as per RFC1122.
Not the ICMP errors to which Mike is referring to when IPv4 node
detects unrecognized HBH or Destination Option . Those are Parameter
Problem ICMP errors with code 2 "unrecognized IPv6 option encountered"
so they are not applicable here.

A single bit in the option type to indicate that the option can be
skipped if unrecognized should be sufficient.

Tom

---

From: "C. M. Heard" <[email-masked]>
Date: Thu, 18 Jul 2019 08:46:15 -0700

On Thu, Jul 18, 2019 at 3:39 AM Gorry Fairhurst wrote:
> networks *NEED* to know the transport requires them to do partial
> integrity checks at L2 (if the CRC the entire frame this approach is
> useless). To get any real advantage, I'd say they have to do much more,
> by probably changing the waveform (FEC, modulation, etc). UDP-Lite
> offers that, and I have not seen people wanting more/different features
> - Magnus provided comments recently, and I would concur this requires
> strong coordination with the App and Link design, UDP-Lite remains a
> suitable method, I would agree that adding this to another protocol is
> not helpful.
>
> (i.e. tunnels) where the network device has no visibility deep into the
> payload. Such devices existed and continue to exist. This was a
> motivation in GRE/UDP. However, for IPv6 a zero UDP checksum is
> restricted to specific uses-cases, and not a host function - because
> as Lloyd pointed out this erodes ability to detect rogue UDP datagrams.
>
> the case for some tunnels), and we could save the cost of a second
> checksum.  I am not persauded that this is a real cost when comes to a
> frag/reassembly algorithm.
>

Recent discussions of LITE have considered a variant that uses checksum
compensation of the transmit side (which is not checked on the receive
side) to deal with middlebox traversal issues related to the checksum.
It would provide the following services:

(a) delivery of damaged payload data across an Internet where middleboxes
routinely block protocol 136 (UDP-Lite)

(b) additional transport options to accompany such payloads

If I correctly understand the discussion of (1) above, it is argued that
reaping practical benefit from delivery of damaged payload data requires
close coordination between the transport/network layers and the data link
layer. Clearly, there's no way that's going to happen in the general
Internet; such use is confined to controlled environments, where UDP-Lite
works fine. So there really isn't any benefit from (a). Additionally, it
is argued that (b) provides no evident advantage in those environments.

So this variant of LITE would not be helpful for use case (1). And since it
requires visibility into the payload to compute the compensation checksum,
it is not helpful for (2) and provides only marginal help for (3).

> I'm hessitant to agree that the complexity of LITE is worth the perceived
> cost saving and there may be other ways that don't need this:
> the cost can be minimal.

I've argued that last point repeatedly (though Joe does not agree).

If delivery of damaged payloads is adequately covered by UDP-Lite, it
would IMO make sense to strike support for LITE (#8) from the list of
design requirements. That would leave it only as a means to support
FRAG, and I think there are much better ways to do that (I've made two
proposals for FRAG that do not require LITE).

> On the OCS topics:
>
> * DNSSEC or any host could send UDP fragments, would it really cost too
> much to send these with a full UDP checksum for IPv6? (and then why not
> IPv4)?
>
> traversal" (#7) which requires a CCO-like sum. I really appreciated the
> thread, which I paraphrase as something like (do I understand):
>

Yes

> UDP Options, does not then need to check the OCS on reception.

I do not recall that being said. Per the draft, AE is computed first
using a zero-checksum OCS if present, and OCS is computed last before
transmission, over the entire option area, including AE. Validation
on receive would go in the reverse order.

> the Internet checksum), but a CRC32 would be.

During the discussion I did suggest that the receiver should check OCS
before parsing the options (possible if its presence is tied to the UDP
checksum being non-zero, thanks to adopting CCO for the OCS).

> -- Sender SHOULD add an OCS when the UDP Checksum is zero - to aide

I think David Black and I converged on sender MAY add and receiver
MAY check OCS when UDP Checksum is zero. The statement that OCS aids
traversal when UDP Checksum is zero was withdrawn -- we couldn't find
the evidence in the maprg presentation from IETF 103. If that's wrong
please speak up and correct us.

> invalid - although that IMHO needs though as to how you decide on the

I think that is a very controversial statement -- I certainly disagree.

> how to design Frag (#5, #8).

For DPLPMTU the complexity is in another spec, but the tools that UDP
options provide for it are in the base spec. Those tools are:

- padding for probe packets
- request option
- response option

The request and response options are pretty straightforward, but the
details of how padding will be accommodated may well depend on what
packet format we converge to for providing other things, especially
FRAG. So I am concerned that if we take FRAG off the table for now we
might end up with a design that requires major rework to other features
when we go to add it. So I'd advocate keeping it on the list of
design requirements.

Also, there was another design requirement whose addition I requested,
but which (so far) does not seem to have made it onto the list:

9 - any option that affects the handling of payload data must share fate
with that payload data, in all receivers (legacy or otherwise)

This would be to prevent a receiver from dropping something like FRAG or
AE (especially in its encryption variant) because OCS fails but delivers
the payload because its checksum is OK. The same thing could happen if
future options that affect payload handling are defined, and the spec
says ignore unknown options. Dealing with those considerations can
affect the design of the packet format and the encoding of the options.

>> BTWs, I don’t understand well what is specific to DNSSEC fragments
>> here to traverse NATs. Such NATs will fail to handle any incoming
>> fragment, not only DNS. NATs which are compliant with REQ#14 of
>> RFC4787 will handle appropriately fragments (modulo DDoS protection
>> features).
>>
>> If a firewall/NAT is explicitly configured to reject such fragments
>> for whatever reason, these devices are likely to strip/reject the
>> FRAG options (or UDP options in general).
>>
>>>> On Jul 17, 2019, at 7:46 AM,<[email-masked]> wrote:
>>>> Hi Joe,
>>>>
receivers

---

From: Gorry Fairhurst <[email-masked]>
Date: Thu, 18 Jul 2019 17:16:01 +0100

> it is not helpful for (2) and provides only marginal help for (3).
I'm curious as to who in the WG thinks otherwise, and if so why.
> perceived
> > "checksum" the fragment checksums in a fragmentation protocol to provide
It would be interesting to understand the real desire to eliminate the
cost of doing both a UDP and a reassembly checksum - so we can rule
that out. It's going to waste cycles when there is no off-load, but
it also allows full offload, and means you don't need LITE to do Frag.
> on receive would go in the reverse order.
If I'm reading this wrong, sorry.
> checksum being non-zero, thanks to adopting CCO for the OCS).
Possible yes.
> please speak up and correct us.
Right - there are too many exit points for the discussion. The only 
place  I rceall
it traversal is when you have a proxy that reintroduces the checksum - 
which I
think is perverse to start with. So on checking, MAY is fine with me and
"might" improve is a more reasonable note (if we still wish that).
> I think that is a very controversial statement -- I certainly disagree.
Does anybody still think that is a good idea?
> FRAG.
DPLPMTUD works best with *just* padding packets (i.e. that carry no
payloads or fragments). It's just so much easier to run the algorithm.
> So I am concerned that if we take FRAG off the table for now we
Something to indeed get correct.

Gorry
> >
> > Gorry
> >
> >
> >
> > On 18/07/2019, 09:47, [email-masked] 
> <mailto:[email-masked]> wrote:
> >> Hi Joe,
> >>
> >>> De : Joe Touch [mailto:[email-masked] 
> <mailto:[email-masked]>]
> >>> FWIW, #5 Frag is a driving case for DNSSEC to allow UDP messages 
> larger
> >>> than the min MTU to traverse NATs. IP
> >>>
> >>>> On Jul 17, 2019, at 7:46 AM,<[email-masked] 
> <mailto:[email-masked]>> wrote:
> >>>> Hi Joe,
> >>>>
> >>>>> De : tsvwg [mailto:[email-masked] 
> <mailto:[email-masked]>] De la part de Joe Touch
> >>>>> 8- support LITE, i.e., where some of the payload is not covered 
> by at
> >>>>> least some checksum processing
> >>>>>
> >>>>> AFAICT:
> >>>>>
> >>>>>    - for #7, whether CCO is included or not used (at user’s 
> peril, but
> >>>>>      to allow for transmitters to avoid work)

---

From: Gorry Fairhurst <[email-masked]>
Date: Thu, 18 Jul 2019 17:16:08 +0100

Possible yes.

Gorry

---

From: Gorry Fairhurst <[email-masked]>
Date: Thu, 18 Jul 2019 17:31:50 +0100

The thread split, so this is a reply to Joe.

On 18/07/2019, 15:50, Joe Touch wrote:
>> On Jul 18, 2019, at 3:38 AM, Gorry Fairhurst<[email-masked]>  wrote:
>>
Expecting hosts to set zero checksum is something we need to be careful 
about. If Frag did not do this, I'd expect easier acceptance.
> That’s part of the issue with computing the checksum over the fragment parts vs. whole...
Sure - or we compute an *additional* checksum when we have fragments. It 
will cost (a little), but if it solves DNS issues, then I'd already be 
excited.
> Hosts don’t want to touch the data twice to check the fragments AND check the reassembled set.
Right, that's a question I would like tested.
There are other cases: e.g. Frag usually involves a move, doing checksum 
and move can be efficient.
> Hardware offload doesn’t help if checksumming is done multiple times.
Well, it save the per-fragment CPU processing. The host then only needs 
to worry
about other checks.
> But it’s not equivalent. Remember that data in the host can have errors once each fragment is received too. TCP gives up on that because it HAS to deliver each fragment once previous ones were rece…
Yes - what you I proposed would be weaker.  Doing seprate reassembly 
check would be the best thing.
> No, OCS still needs to be checked on receipt (if UDP CS !=0). ACS and AE apply to the UDP user data, not the surplus area.
Ah - understood.
> Ignore ACS. Again, ACS is over the user data, not surplus area.
Same, understood.
> Yes…let’s call this what it is, though - to aid traversal of middlebox bugs.
Fine.
> Hmm. Now I’m confused. You’ve just re-enabled everything LITE had in the first place (including UDP-LITE emulation), but then you have to parse into the area to figure out when to ignore OCS...
Then let's not drill there, becaiuse I really want to know if we need 
LIGHT for Frag.
> We can, but IMO the criticality of fragmentation support means that it’s not particularly relevant to have UDP options that don’t support fragmentation.
I think we are making progress on this and other threads. Let's see how 
far we get by the time of the meeting.
Gorry

---

From: Joe Touch <[email-masked]>
Date: Thu, 18 Jul 2019 09:46:45 -0700

On 2019-07-18 09:31, Gorry Fairhurst wrote:

> On 18/07/2019, 15:50, Joe Touch wrote: Hi, Gorry,
> 
> (1) One use-case is to mimic UDP-Lite for layer2 networks that wish to do cross-layer optimisation. This is the case that makes the most complications for us.
> 
> ... I would concur this requires strong coordination with the App and Link design, UDP-Lite remains a suitable method, I would agree that adding this to another protocol is not helpful. If we take this off the table, things get a little simpler...
> 
> - because as Lloyd pointed out this erodes ability to detect rogue UDP datagrams. Tunnels are links, and links terminate at both routers and hosts.
> 
> This doesn't erode anything; it follows the IPv6 policy of not duplicating the same kind of check at multiple layers (i.e., it's odd that we keep citing IPv6 as the reason we can't set UDP to zero, when in fact the reason we can drop the checksum for tunnels is exactly the same reasoning used to drop the header checksum in IPv6 - thankfully that's now understood).

Any endpoint supporting a tunnel would want to do this. It's nonsensical
to claim that hosts wouldn't do it but routers might. Hosts are tunnel
endpoints too. 

> I think (2) could be useful, but I don't yet see someone asking for that, That's part of the issue with computing the checksum over the fragment parts vs. whole...
 Sure - or we compute an *additional* checksum when we have fragments.
It will cost (a little), but if it solves DNS issues, then I'd already
be excited. 

> Hosts don't want to touch the data twice to check the fragments AND check the reassembled set.

That sort of demand cuts both ways - "show me it DOESN"T cost" too. We
have lots of experience with zero-copy that says it does EXCEPT during
the ONE move in/out to the wire. 

> There are other cases: e.g. Frag usually involves a move, doing checksum and move can be efficient.

The way FRAG+LITE is defined the only move is to the first few bytes
(the swap). That's exactly why it won't be efficient otherwise - it
requires staging the fragments somewhere else which involves a copy. 

> be minimal. Hardware offload doesn't help if checksumming is done multiple times.

> But it's not equivalent. Remember that data in the host can have errors once each fragment is received too. TCP gives up on that because it HAS to deliver each fragment once previous ones were received; UDP doesn't.

That's exactly what's in the current doc - post-reassembly checksum. 

> ... And so back to Frag:
> For DPLPMTU (#6) the complexity is in another spec (which seems correct to me). I'd be interested whether a separate spec of Frag is also one way to ensure we get experience with the core spec, and then consider how to design Frag (#5, #8). We can, but IMO the criticality of fragmentation support means that it's not particularly relevant to have UDP options that don't support fragmentation.

If we want a summary by the meeting, then we need to hit the pause
button please. I don't have time to track the numerous changes if they
continue between now and then. 

And I won't be on the call during th meeting, so perhaps we can focus on
converging on what's been said up to now (which might be useful in of
itself, though). 

Joe

---

From: Joe Touch <[email-masked]>
Date: Thu, 18 Jul 2019 09:56:18 -0700

> skipped if unrecognized should be sufficient.

That bit has two meanings: 

- for legacy receivers, it's simply ignored for legacy data 

- for option-aware, if the flag is set, what does it mean? 
  - it should never mean drop the legacy data, otherwise an
options-aware endpoint can't decide to act as legacy 
  - it could mean drop non-legacy data 

Right now, the onus is on the receiver to decide what to do with the
data. The info is passed to the user - NOT decided by UDP. 

I.e., cuurrently: 

- unknown options are ignored by both legacy and option-aware endpoints 
- UDP processing skips over all such options 
- users can decide/negotiate what to do with those options on a per
endpoint basis 

Note: the first time you send something, if you want a response to help
you know what the endpoint will do, you need to set "ignore" anyway,
which is what the default does. Otherwise you won't get a response and
won't know why (the packet could have been dropped). 

So this only makes sense for negotiated soft-state anyway, at which
point the user can either configure the endpoint UDP socket to drop if
unknown or accept. 

Why would we NOT give the user this choice?

Joe

---

From: Tom Herbert <[email-masked]>
Date: Thu, 18 Jul 2019 10:25:46 -0700

On Thu, Jul 18, 2019 at 9:56 AM Joe Touch <[email-masked]> wrote:
>   - it should never mean drop the legacy data, otherwise an options-aware endpoint can't decide to act as legacy
> Right now, the onus is on the receiver to decide what to do with the data. The info is passed to the user - NOT decided by UDP.
>
> I.e., cuurrently:
>
> - users can decide/negotiate what to do with those options on a per endpoint basis
>
> Note: the first time you send something, if you want a response to help you know what the endpoint will do, you need to set "ignore" anyway, which is what the default does. Otherwise you won't get a response and won't know why (the packet could have been dropped).
>
> So this only makes sense for negotiated soft-state anyway, at which point the user can either configure the endpoint UDP socket to drop if unknown or accept.
>
Joe,

Consider that someone invents the compression option a year from now.
The option cannot be ignored by a receiver since delivering compressed
data to the application would be jibberish. So someone starts using
compression and performs a "soft-state" negotiation with a timeout of
30 seconds in the example algorithm you previously described for
soft-state negotiation. So the sender is sending compression option to
some receiver. But, at some point, say 10 seconds into a timeout
period, the receiving host changes to a different configuration that
doesn't support the compression option-- maybe the host address is
virtual IP and packets are now routed to a different backend, or maybe
someone reboots the receiving hosts with a new configuration. The
sender doesn't know the receiver has changed and continues sending the
compression option. The new receiver sees the options but doesn't
recognize it. If the receiver ignores the option and delivers the data
to the application then that's data corruption. That's not robust.

The "drop packet on unknown option" is the conventional and cheap way
to avoid situations like this and make stateless options robust.

Tom

---

From: Gorry Fairhurst <[email-masked]>
Date: Thu, 18 Jul 2019 18:49:30 +0100

On 18/07/2019, 17:46, Joe Touch wrote:
>>>> On Jul 18, 2019, at 3:38 AM, Gorry Fairhurst<[email-masked] 
>>>> <mailto:[email-masked]>>  wrote:
>>>>
>> Expecting hosts to set zero checksum is something we need to be 
>> careful about. If Frag did not do this, I'd expect easier acceptance.
> Any endpoint supporting a tunnel would want to do this. It's 
> nonsensical to claim that hosts wouldn't do it but routers might. 
> Hosts are tunnel endpoints too.
That's an issue for the WG to think on then.
>>> That's part of the issue with computing the checksum over the fragment parts vs. whole...
>> It will cost (a little), but if it solves DNS issues, then I'd 
>> already be excited.
Well, I'm hoping others will chime-ion.
> requires staging the fragments somewhere else which involves a copy.
Do you moev to reassemble, or just leave a chain of fragments?
>>> Hardware offload doesn't help if checksumming is done multiple times.
>> Well, it save the per-fragment CPU processing. The host then only 
>> needs to worry
>> about other checks.
Yes.
>>> We can, but IMO the criticality of fragmentation support means that it's not particularly relevant to have UDP options that don't support fragmentation.
>> I think we are making progress on this and other threads. Let's see 
>> how far we get by the time of the meeting.
> And I won't be on the call during th meeting, so perhaps we can focus 
> on converging on what's been said up to now (which might be useful in 
> of itself, though).
> Joe
I agree it's time to start banking the outcomes. I think it may be soon 
(already!) time to stop proposing new stuff and let Joe collect issues, 
and find out who thinks he is heading the best direction.

Gorry

---

From: Gorry Fairhurst <[email-masked]>
Date: Thu, 18 Jul 2019 18:59:09 +0100

On 18/07/2019, 18:25, Tom Herbert wrote:
> On Thu, Jul 18, 2019 at 9:56 AM Joe Touch<[email-masked]>  wrote:
>>
>>
>>
>>
>> Right now, the onus is on the receiver to decide what to do with the data.. The info is passed to the user - NOT decided by UDP.
>>
>> I.e., cuurrently:
>>
> recognize it.

> If the receiver ignores the option and delivers the data
That's where I disagree. I think anything that is not a native UDP 
packet MUST be carried in the options space. That's where I would 
suggest we put the compressed payload. if that means the main payload 
area isnothing", then that would be OK for me.

When the remote endpoint fails to process the compressed payload (or 
myseriously the option si stripped/ignored - how, but don't worry), then 
the app doesn't respond. It times out and does something.

Personally, unless there was a strong desire to be unidirectional (then 
you have to live with that), I'd send an option to the other end to 
solicit a little feedback - or do the same at the app layer.

Gorry
> to avoid situations like this and make stateless options robust.
Not sure how that now helps?

The receiver already ignores unknown options.
Gorry

---

From: Tom Herbert <[email-masked]>
Date: Thu, 18 Jul 2019 11:18:37 -0700

> area isnothing", then that would be OK for me.
That's already assumed. If there are any options that can't be ignored
then the UDP Length = 8 format must be used. If processing options
fails then the packet can be dropped.

> the app doesn't respond. It times out and does something.
Remote endpoint or remote application? We can't give an application
the compressed payload.

> solicit a little feedback - or do the same at the app layer.
You can do that, but bear in mind that UDP is a stateless protocol.
The feedback you get from one probe exchange might be completely
invalid on the very next packet sent. Any information gleaned about a
peer can only be considered advisory.

> The receiver already ignores unknown options.

I believe that's the problem. There some options that cannot be
ignored and still maintain correctness. This is already a known issue
with other protocols.

Tom

---

From: Gorry Fairhurst <[email-masked]>
Date: Thu, 18 Jul 2019 19:41:46 +0100

On 18/07/2019, 19:38, Tom Herbert wrote:
> On Thu, Jul 18, 2019 at 11:32 AM Gorry Fairhurst<[email-masked]>  wrote:
>> On 18/07/2019, 19:18, Tom Herbert wrote:
>>> On Thu, Jul 18, 2019 at 10:59 AM Gorry Fairhurst<[email-masked]>   wrote:
>> Remote endpoint.
>> yes. If you build on UDP you need to provide machinery.
>>>> Gorry
>> OK, let's check. The receiver gets a UDP datagram and...
>>
>> (1) Receiver understands options and it understands this specific option
>> - it processes the option - app gets the data.
> Yes, presuming other options were acceptable.
>
>> (2) Receiver understands options and not this specific option - it does
>> not processes the option - no data to the app.
> Yes, that is the correct behavoir. The question is how does the
> protocol provide for this. Right now the draft says "Receivers MUST
> silently ignore unknown options.". That won't work. Either all unknown
> options cause packet to be dropped, or receiver can can determine
> whether the packet should be be dropped based on the bit.
OK, so do you want the app to be able to process several options, but 
make options processing
dependent on other options.

Gorry
>> (3) Receiver does not understand options - it does not processes the
>> option - no data to the app.
> No data app, but app does receive a zero length message. That corner
> case is already noted.
>
>> Gorry
>>> Tom
>>>
>>>>> Tom
>>>>>
>>>>>> Joe
>>>>>>
>>>>>>
>>>> Gorry
>>>>

---

From: Tom Herbert <[email-masked]>
Date: Thu, 18 Jul 2019 11:53:12 -0700

> dependent on other options.
Gorry,

What I want is for the sender to be able to tell the receiver "if you
don't understand this option then drop the packet, because accepting
the packet is incorrect.". As pointed already, unknown options are
already dealt with in IPv6 where two bits indicate what to do with the
packet. The part about ICMP errors isn't needed for UDP options, so
only one bit in type field is needed. Other than that nuance, we
already have the solution for a common protocol problem with
precedence, implementation, and deployment experience and it's really
straightforward and completely stateless. I'm at a loss as to why
there's pushback on this.

Tom

---

From: Tom Herbert <[email-masked]>
Date: Thu, 18 Jul 2019 12:50:51 -0700

Gorry,

It's a question of benefit versus cost.

If the reassembly checksum runs the same algorithm as the packet
checksum then the reassembly checksum does not significantly improve
the integrity check. For instance, if there is an undetected
corruption in a packet that packet checksum misses, the reassembly
checksum misses it also since the corrupted bytes are canceling in
either operation. There is a tiny corner case where the checksums
cover different data (like packet checksum includes options) such that
a corruption could be detected by reassembly checksum but not packet
checksum (conversely though reassembly checksum misses errors in
things it doesn't cover). Checksum is only designed as an check
integrity of transmitted packet and it's generally considered a weak
form of integrity check anyway. For instance, it's not sufficient for
protecting against memory corruption of a received packet in a host,
we have ECC for that. Similary, applications that are sensitive to
corruption, like storage protocols, don't rely on checksum and use
much stronger integrity checks over the data.

The cost of reassembly checksum is CPU utilization. If we need to
compute a packet checksum on host CPU this is a very expensive
operation since every byte of data needs to be touched, offload to
network device avoids this and is a well known win. With some work, we
might be able to offload a reassembly checksum, but there's little
chance we can offload transmit with specialized hardware support (see
UDP Fragmention Offload for analogous case).

I would point out that neither IPv4 nor IPv6 contain a reassembly
checksum. However in the case of fragmenting a UDP packet with the
checksum set the work has to be done to set and receive the UDP
checksum on the full packet. For all the reasons listed above, this is
expensive to do. However, if we perform fragmentation in UDP options,
each UDP packet containing a fragment can have a checksum that is
offloaded so we don't have to any CPU checksum computation over the
reassembled packet. IMO, this would be major benefit over using IPv4
and IPv6 fragmentation for UDP packets.

Tom

---

From: Gorry Fairhurst <[email-masked]>
Date: Thu, 18 Jul 2019 21:09:49 +0100

Thanks Tom,

This thread has proved really helpful to me. The UDP-Options work has 
been through some of this before, but this time around I saw a few new 
things that were unearthed and a few things confirmed, and maybe a few 
things revisited that we'll need to follow-up. Everyone who chimed-in 
has helped, but there is now lots to digest.

For this to imapct the WG draft we need Joe to help us to collect the 
thoughts and try to present a summary to the WG. I'll hang back from 
more comment to allow that to happen.

Gorry

---

From: Joe Touch <[email-masked]>
Date: Thu, 18 Jul 2019 13:37:35 -0700

> On Jul 18, 2019, at 10:25 AM, Tom Herbert <[email-masked]> wrote:
> 
> data to the application would be jibberish. 

Please see sec 7 of the current draft. 

Joe

---

From: Joe Touch <[email-masked]>
Date: Thu, 18 Jul 2019 13:39:17 -0700

> On Jul 18, 2019, at 12:50 PM, Tom Herbert <[email-masked]> wrote:
> 
> checksum

We don’t fragment the area their checksums cover. 

Joe

---

From: Joe Touch <[email-masked]>
Date: Thu, 18 Jul 2019 13:40:51 -0700

On Jul 18, 2019, at 10:49 AM, Gorry Fairhurst <[email-masked]> wrote:

>> only move is to the first few bytes (the swap). That's exactly why it won't be efficient otherwise - it requires staging the fragments somewhere else which involves a copy.

If the fragments come in order there is no need to move anything.  In other cases there may be but that may be unavoidable. 

Joe

---

From: Tom Herbert <[email-masked]>
Date: Thu, 18 Jul 2019 13:49:47 -0700

> Please see sec 7 of the current draft.

I don't see what in section 7 helps, can you point the specific mechanisms.

Tom

---

From: Joe Touch <[email-masked]>
Date: Thu, 18 Jul 2019 13:51:12 -0700

> On Jul 18, 2019, at 1:49 PM, Tom Herbert <[email-masked]> wrote:
> 

Sec 7 says this - and other things- should not be done by options and why.

---

From: Tom Herbert <[email-masked]>
Date: Thu, 18 Jul 2019 14:00:22 -0700

> Sec 7 says this - and other things- should not be done by options and why.
So we can never add a compression option? Or for that matter an
alternate method of fragmentation, or some sort of forward error
correction that might touch the data?  That the reason we don't have
to worry about unknown options that can't be ignored is that we aren't
allowed to create any new options that can't be ignored once the
initial set is published? If I'm interpreting this correctly, it
sounds awfully limiting.

Tom

---

From: "C. M. Heard" <[email-masked]>
Date: Thu, 18 Jul 2019 14:23:21 -0700

On Thu, Jul 18, 2019 at 9:16 AM Gorry Fairhurst wrote:
>>> I'm hesitant to agree that the complexity of LITE is worth the perceived

My position is basically that the per-frgment checksum provided by OCS
should be mandatory, and any additional checksum on the reassembled
should be enabled only by user request. Here's the reasoning.

Having a checksum on each fragment that covers both options and fragment
data is necessary both for middlebox traversal and to ensure robustness
of the reassembly process. It provides protection equivalent to TCP
segment checksums, which should be good enough in most cases. An additional
checksum on the reassembled datagram payload is relatively expensive, so
it should be turned on only for applications that explicitly request it.
ACS is perfect for that purpose since it is defined to be per-datagram
and not per-packet and uses a different checksum than the the fragment OCS.

Consider:

- If fragment data is covered by OCS/CCO, that's comparable protection to
  what a TCP segment gets from the TCP checksum.

- Fragments are kept in sequence (and duplicates are discarded) by means
  of the fragment offset. That accomplished the same thing that TCP's
  sequence number does for segments.

- OCS/CCO on individual fragments can be readily offloaded, provided that
  the checksum is placed early in the packet. A checksum on the reassembled
  datagram, on the other hand, is very likely relegated to software, since
  the reassembled datagram will probably be in scatter-gather storage.

The first two bullets establish that per-fragment protection by OCS is
The third bullet plausibly suggests that the checksum on the reassembled
datagram payload is relatively expensive.

Furthermore, using the Internet checksum to check the reassembled datagram
payload won't add much protection if the ratio of fragment data to
headers+options is large. Every undetectable error pattern that hits the
data area in a single fragment will also go undetected in the reassembled
datagram payload. On the other hand, undetectable error patterns that hit
the data in multiple fragments may well be detected by the individual
fragment checksum. To me, this suggests that the two checksums should be
different, something that ACS will provide.

>>> A 16b CRC isn't that strong for a full-sized IP packet (but better
>>> than the Internet checksum), but a CRC32 would be.

+1

> payloads or fragments). It's just so much easier to run the algorithm.

Yes, and the MTU discovery support requirement should be expanded to
say that. A probe packet consisting of padding data plus the request
and/or response options would fill the bill nicely.

Mike

---

From: "C. M. Heard" <[email-masked]>
Date: Thu, 18 Jul 2019 14:42:50 -0700

> area isnothing", then that would be OK for me.

Yes, exactly, and I say that none of the following qualify as native UDP:

- fragments (option and associated data)

- packets with authentication and encryption (AE)

- packets with alternate checksums (ACS)

- any other option that affects the handling of the payload

I would like to see all of those things behind a UDP header
whose length is 8, with OCS (or equivalent) covering the
surplus area (at least if the checksum in the UDP header
is present). Then the options and their associated data
will both be discarded if OCS fails and will both be
processed if OCS passes (modulo unrecognized options,
more on which anon).

Mike

---

From: "C. M. Heard" <[email-masked]>
Date: Thu, 18 Jul 2019 14:56:56 -0700

On Thu, Jul 18, 2019 at 11:53 AM Tom Herbert wrote:
> the packet is incorrect."

"... if you ignore the option."

> As pointed already, unknown options are

I too fail to understand the pushback. It's really simple, and it's
necessary to ensure correct handling of options that affect the
accompanying data, at least when multiple options can apply to the
same data. Such options have already been specified: any combination
of ACS, AE, and FRAG could be applied to the same data.

Mike

---

From: Joe Touch <[email-masked]>
Date: Thu, 18 Jul 2019 18:29:25 -0700

> On Jul 18, 2019, at 2:00 PM, Tom Herbert <[email-masked]> wrote:
> 
> correction that might touch the data?

Nothing that modifies the UDP user data, ever. No. Not unless you negotiate that, at which point it’s just like any other UDP exchange.

Options do not redefine the user data space.

> That the reason we don't have
> initial set is published?

Basically; the limitation is that a receiver is always legacy until you find out some other way, at some other layer. UDP doesn’t have state and we’re not adding it. At most, there’s only soft state and soft state has its limits.

> If I'm interpreting this correctly, it
> sounds awfully limiting.

We have other transports which  to play with hard state, e.g., TCP.

Joe

---

From: Joe Touch <[email-masked]>
Date: Thu, 18 Jul 2019 18:30:37 -0700

> On Jul 18, 2019, at 11:18 AM, Tom Herbert <[email-masked]> wrote:
> 

Right, but this is UDP. There are inherent limits to what we can do here.

Joe

---

From: Tom Herbert <[email-masked]>
Date: Thu, 18 Jul 2019 19:02:10 -0700

> Right, but this is UDP. There are inherent limits to what we can do here.
IPv6 has HBH and dest options as TLVs. IPv6 allows unrecognized
options. IPv6 has bits in the option type that tell receiver whether
to drop packet if option is unknown. IPv6 is stateless and doesn't
define any negotiation. The solution in IPv6 provides correctness for
options that cannot safely be ingnored. Please explain EXACTLY why UDP
options cannot use this same technique.

Tom

---

From: Joe Touch <[email-masked]>
Date: Thu, 18 Jul 2019 20:02:38 -0700

> On Jul 18, 2019, at 7:02 PM, Tom Herbert <[email-masked]> wrote:
> 

Again, because this is UDP.

IPv6 doesn’t need to deal with legacy receivers that ignore all options anyway - all the time.

So basically “drop if not supported” has no meaning ALREADY for legacy receivers. Anything further - deeper, or involving state - can still be performed by the receiver, at the receiver’s decision already anyway.

Joe

---

From: Tom Herbert <[email-masked]>
Date: Thu, 18 Jul 2019 20:25:13 -0700

> Again, because this is UDP.
This is not UDP. UDP doesn't have options nor any sort of negotiation.
This is a new protocol.

> So basically “drop if not supported” has no meaning ALREADY for legacy receivers. Anything further - deeper, or involving state - can still be performed by the receiver, at the receiver’s decision…

The legacy receiver problem is already resolved by putting options
that can't be ignored in packet with UDP length=8. The problem is when
a non-legacy receiver receives an options that it doesn't recognize.
If the option cannot be safely ignored that packet needs to be
dropped. This is the exact same problem that IPv6 has and hence the
same solution can be applied.

In any case, if you're saying we can't solve this problem and that the
protocol isn't extensible to something as simple as a stateless
compression option, then I raise that a major design flaw with the
protocol.

Tom

---

From: "C. M. Heard" <[email-masked]>
Date: Thu, 18 Jul 2019 20:30:06 -0700

Joe, I'm sorry, but I am having a really, really, hard time understanding
why you are pushing back so hard on this.

On Thu, Jul 18, 2019 at 8:03 PM Joe Touch wrote:

> So basically “drop if not supported” has no meaning ALREADY for legacy
>
receivers.

A bit encoding "drop if not supported" is not there for the benefit of
legacy receivers.

It's there to make it possible to add future "unsafe to ignore" options
without messing
up option-aware receivers that don't understand the new option.

> Anything further - deeper, or involving state - can still be performed by
> the
>
receiver, at the receiver’s decision already anyway.

How is the receiver supposed to make that determination?

And how hard is a totally stateless ignore/discard bit?

Mike Heard

---

From: Joe Touch <[email-masked]>
Date: Thu, 18 Jul 2019 21:04:42 -0700

I’m confused as to why the design rules in sec 7 aren’t being discussed as a coherent whole.

It’s not feasible to selectively pick and edit them. They only make sense as a group - the same is true with many of the mods being discussed.

In this case, we’re talking about whether to make options behave differently in two cases:

- legacy receivers, for which there is no such thing as “drop if you don’t understand this option” for any option

- option-aware receivers

This is why I think of all UDP options as “supplementing” UDP, not changing it.

We’re not starting with a blank page here.

Does that help explain the rationale behind my viewpoint?

Joe

> On Jul 18, 2019, at 8:30 PM, C. M. Heard <[email-masked]> wrote:
> 
> A bit encoding "drop if not supported" is not there for the benefit of legacy receivers.
> 
> It's there to make it possible to add future "unsafe to ignore" options without messing
> Anything further - deeper, or involving state - can still be performed by the

---

From: Joe Touch <[email-masked]>
Date: Thu, 18 Jul 2019 21:10:51 -0700

PS - add to that sec 8’s processing rules  for the same reasons.

Joe

> On Jul 18, 2019, at 9:04 PM, Joe Touch <[email-masked]> wrote:
> 
>> On Jul 18, 2019, at 8:30 PM, C. M. Heard <[email-masked] <mailto:[email-masked]>> wrote:
>>

---

From: Derek Fawcus <[email-masked]>
Date: Sat, 20 Jul 2019 22:56:36 +0100

> So basically “drop if not supported” has no meaning ALREADY for legacy receivers. Anything further - deeper, or involving state - can still be performed by the receiver, at the receiver’s decision…

I suspect we should interpret "drop the packet" here for options marked
as "drop if not known" to actually mean "discard the surplus".

i.e. if a non-legacy receiver in processing the surplus finds such an option,
it then switches to operating in legacy mode for that packet.

Which means that like the legacy receiver, it will either see a zero length
UDP packet (if UDP payload length was zero) or see whatever legacy UDP data
was passed in the non-surplus area.

It might also be possible for a non-legacy receiver to have a configuration
knob to place the system (or just a socket - sockopt) in to an actual
"drop the packet" mode.

DF

---

From: Derek Fawcus <[email-masked]>
Date: Sat, 20 Jul 2019 23:04:38 +0100

> would not be coupled together.

Maybe I'm not understanding something, but I see that as a distinction without a difference.

Assume some packet is received having legacy payload of zero length, options
X, Y, Z, and non-legacy data of length Q.

If options X & Z are understood, but Y is not, and Y has this magic bit set to 1,
I'd expect the whole of the surplus to be discarded.

So depending upon how one implements this, the non-legacy receiver will either
receiver a UDP packet of zero length (as a legacy receiver would),
or simply not receive anything (i.e id discarded the whole packet, not just the surplus).

i.e. the only coupling / fate sharing here is the fact that one of the options
implies the whole of the surplus is to be thrown away.

DF

---

From: "C. M. Heard" <[email-masked]>
Date: Sat, 20 Jul 2019 17:51:16 -0700

On Sat, Jul 20, 2019 at 2:56 PM Derek Fawcus wrote:
> i.e. if a non-legacy receiver in processing the surplus finds such an
> option, it then switches to operating in legacy mode for that packet.

I do not agree.

> was passed in the non-surplus area.

What I was actually advocating was that in the presence of any option marked
"drop if unknown" there MUST NOT be any legacy data, i.e, such options
could not appear in the trailer format. All payload would have to be in the
surplus area (which requires the header format). In that case, "drop the
packet" and "drop the surplus" would amount to the same thing.

Mike

---

From: "Black, David" <[email-masked]>
Date: Sun, 21 Jul 2019 01:47:09 +0000

Still as an individual ...

> packet" and "drop the surplus" would amount to the same thing.

That would "freeze" LITE - if we ever found anything problematic with LITE and wanted to replace it with LITE2, that replacement would be prohibited by such a requirement.

NB: *replace* is crucial in this context - this is not about trying to use both LITE and LITE2 in the same packet.

Thanks, --David

> Sent: Saturday, July 20, 2019 8:51 PM
> To: Derek Fawcus
> Cc: tsvwg

---

From: Tom Herbert <[email-masked]>
Date: Sat, 20 Jul 2019 19:27:34 -0700

David,

This is why having a version number, variant number, or the like in
the surplus area header is so critical.

Tom

---

From: "C. M. Heard" <[email-masked]>
Date: Sat, 20 Jul 2019 20:39:03 -0700

On Sat, Jul 20, 2019 at 6:49 PM Black, David wrote:
> That would "freeze" LITE - if we ever found anything problematic with LITE and
> wanted to replace it with LITE2, that replacement would be prohibited by such
> a requirement.

Forgive me for being thick, but I don't quite get what you are saying. Is this
problem specific to LITE, and in particular LITE in the variant proposed by
Derek? Or would it also be true of other options that affect how the payload
is processed, such as FRAG, AE, and ACS, which are the ones for which I claim
need to be marked as "drop if unknown"? I would imagine that for those a
replacement could be effected with a new option kind (as was done when TCP-AO
replaced TCP-MD5).

Thanks,

Mike

---

From: Derek Fawcus <[email-masked]>
Date: Sun, 21 Jul 2019 09:26:10 +0100

> I'm trying to understand LITE and the related topics more...

Well, I was always interested in use case (2), not (1).  I had a deployment
to design about 7 years ago where I could have used UDP-Lite, but its
general unusability precluded it.  The LITE in UDP as here would have
helped it it was available.

This deployment which was more or less case (3), specifically transporting
an Ethernet packet off box together with metadata.  This was a case where
we sometimes had to send the Ethernet packet fragmented over multiple UDP
packets.  Essentially a tunnel, with metadata, and payload fragementation.

Namely something like:

  [ Uh | meta, frag hdr | Ether packet part 1 ]
  [ Uh | meta, frag hdr | Ether packet part 2 ]

Where Uh is the UDP header, and everything else is in the UDP payload.
I'd have wanted to use UDP-Lite, with the checksum coverage ending just
before the Ethernet payload started.

There are also cases with software implemented encrypted tunnels where
the payload is using AEAD, such that we really do not need to have
the encrypted part covered by a UDP checksum.  But without a CCO scheme
that is only usable across paths across broken middleboxes.

I'd suggest that the scheme I sketched for framing is much simpler than
the existing one, especially assuming we decouple FRAG from LITE.  I do
however need to send a message filling out more of that sketch.

To provide the functionality that FRAG+LITE offers, we could make the order
of options in a FRAG containing packet significant.  i.e. options appearing
before the FRAG option are processed before reassembly, any options appearing
after the FRAG option are copied (in order) in to the final reassembled
packet, then processing of those performed within the context of the reassembled
packet.

For the fragmention issues, I'm inclined towards simply using the checksum
in the fragments to "protect" the reassembled packet, and then if more
assurance is needed, to apply the ACS option to the post assembled result.

I also take the view that I'd like frag to be in the base options spec,
as it seems to be the main use case.

DF

---

From: "Black, David" <[email-masked]>
Date: Sun, 21 Jul 2019 12:05:58 +0000

Mike,

Inline at bottom ...

Thanks, --David (still as an individual)

> Sent: Saturday, July 20, 2019 11:39 PM
> To: Black, David
> Cc: Derek Fawcus; tsvwg
> replacement could be effected with a new option kind (as was done when
> TCP- AO replaced TCP-MD5).
> 
> Thanks,
> 
> Mike
 [David>] I agree - I had recently been focusing on LITE when I wrote that email.

[David>] Responding to Tom, this discussion provides some rationale for per-option versioning (new option has new kind) with a critical bit being preferable to a version number in an option header - bumping the version number prevents an old-version receiver from being able to work with non-critical "ignore if unknown" that aren't affected by the change when the new critical "drop if unknown" option is not present.

The alternative of not using the new version number if not using new critical options adds complexity - subtle rules to only set the new version number when using a new "drop if unknown" option defined in that version are an invitation to subtle sender bugs, with major & minor version numbers being a more likely/robust approach ... which adds complexity over using new option kinds and a critical bit.

---

From: "C. M. Heard" <[email-masked]>
Date: Sun, 21 Jul 2019 08:27:49 -0700

FWIW there is an incompletely edited sentence in the message below.

OLD

NEW
The first two bullets establish that per-fragment protection by OCS is
sufficient to protect reassembled datagrams passed to UDP-based applications
as well as TCP protects user data that is passed to TCP-based applications.

Sorry.

--cmh

On Thu, Jul 18, 2019 at 2:23 PM C. M. Heard <[email-masked]> wrote:

> My position is basically that the per-fragment checksum provided by OCS