Subject: [tsvwg] OCS option in draft-ietf-tsvwg-udp-options-07

From: "C. M. Heard" <[email-masked]>
Date: Sat, 9 Mar 2019 05:23:41 -0800

Greetings,

In an off-list e-mail, Gorry Fairhust brought to my attention the fact that
the errant middleboxes that CCO is designed to work around do not use the
correct length value in their (re)computation of the UDP checksum. Quoting
from draft-fairhurst-udp-options-cco-00:

   These middleboxes use the IP Payload Length (obtained as IP Total
   Length - IP Header Length) to fill UDP pseudo-header Length field and
   also compute the checksum over the all IP Payload bytes.

The genesis of this bug is discussed in the maprg presentation given by the
Tom Jones at IETF 103 (
https://datatracker.ietf.org/meeting/103/materials/slides-103-maprg-a-tale-of-two-checksums-tom-jones-00
).

So, besides compensating for the contents of the surplus area, it is
also necessary to compensate for the incorrect length value in the
UDP pseudo-header. The way that CCO does this is to add a 16-bit
pseudo-header of its own that consists of the length of the surplus
area. This pseudo-header needs to be aligned with the UDP pseudo-header
in order to do its job. Aligning the checksum in the CCO option to a
half-word boundary achieves this.

These are the points that Rafaele Zullo was trying to make in the message at
https://mailarchive.ietf.org/arch/msg/tsvwg/ikg4zLMvejJZERWh4hQTAGUrLGY
that escaped both Joe and me.

So, it would seem that the OCS text in draft-ietf-tsvwg-udp-options-07
needs some more editing work to take this into account. The objectives
can still be achieved with an option that has kind = 2 followed by a
16-bit checksum and no length value, but the calculation has to take
into account the fact that the fact that the 16-bit pseudo-header needs
to be aligned with the UDP header. Rafaele Zullo's message discusses
some of the options. One is to use a NOP to force half-word alignment,
but there are other ways to accomplish that objective.

Another point is that it is necessary either for the OCS to cover the
entire surplus area or else for any padding after the UDP options
to be checksum-neutral (e.g., consist of zeroes) in order to properly
compensate for the middlebox behavior described above.

Besides these matters of substance, there are some editorial matters
that need to be attended to. In Section 4 I see:

   UDP options are defined using a TLV (type, length, and optional
   value) syntax similar to that of TCP [RFC793]. They are typically a
   minimum of two bytes in length as shown in Figure 4, excepting only
   the one byte options "No Operation" (NOP) and "End of Options List"
   (EOL) described below.

If OCS is to have no length field (i.e., have an implicit length of 3),
then some text needs to be added to that paragraph. Also, Figure 7 in
Section 5.3 needs to be updated to something like this:

                   +--------+--------+--------+
                   | Kind=2 |     Checksum    |
                   +--------+--------+--------+

The modifications to OCS requested above would not, AFAICT, cause any
significant additional complication. They would also not address the
fact that errant middle box traversal for LITE or FRAG+LITE remains
unaddressed, but I see those as separate issues.

Mike Heard

---

From: Joe Touch <[email-masked]>
Date: Sat, 9 Mar 2019 11:28:57 -0800

This one’s a bit long - we need to converge quickly if the draft is going to be updated in time for the cutoff.

Please comment asap…

Joe

PS =  yes, we talk about TLV but need to explain that some options are exceptions (OCS, NOP, EOL). I also need to update the OCS to be 3 bytes long (in the table and figure). Those are in addition to the issues below.

-------

Raffaelo and Mike both pointed out:

> On Mar 9, 2019, at 5:23 AM, C. M. Heard <[email-masked]> wrote:
> ...
> correct length value in their (re)computation of the UDP checksum. ...

Well, there are several implications here:

1. we can add in the option length, BUT that also means:
	(a) we need to declare that the option area consumes the whole of the surplus area (no chance for other uses)
	OR
	(b) that the surplus area is always zero
	OR 
	(c) that OCS covers even the surplus area (it doesn’t need to be zero; it does need to be included)

	we need a choice here. AFAICT, the safest (a)

2. we don’t need OCS alignment; what we do need is to “coordinate” the alignment of a 16-bit of the length to the OCS checksum field
	i.e.:
	- calculate the length needed (IPlen - UDP len)
	- if OCS checksum field is not 16-bit aligned to the start of the option area, then swap bytes
	- add the result to the OCS checksum (‘pseudo header’ seems a bit heavy here, but same point)

As Derek noted:
> As I recall the LITE+FRAG use case has no data in the UDP payload
> (i.e UDP header length is zero)?
> 
> For this case, there is another way to work around a broken path (be it
> boxes, NICs, or whatever), that is simply to send with UDP header checksum
> of zero, and contain any necessary checksums in the UDP slack space.

3. we have to deal with LITE/LITE+FRAG 
	we *can* use UDP CS=0, but that also may involve overriding the user’s desires here (i.e., the user API says “use UDP CS” and we’re not doing that)
	which means we have a choice:
	(d) override so that things work, i.e., force UDP CS=0 and then
		(d.1) do OCS as a check on our stuff only
		OR
		(d.2) disallow the use of OCS (if CS=0, what’s the point?)
		OR
		(d.3) use OCS=0 (this seems like a waste)
	OR
	(e) ignore the user setting of CS, then:
		(e.1) always do OCS as a check
		OR
		(e.2) if UDP CS=0, omit OCS (save space), basically like d.2 
		OR
		(e.3) if UDP CS=0, use OCS=0 (basically like e.2

If we do any of the (e) variants, we could either:
	(f.1) require users to disable UDP CS to use OCS
	OR
	(f.2) ignore the user UDP CS setting

So which is it?

I don’t like assuming user correct configuration (f.1).

So to me it’s (f.2) and (d.1). 

Thoughts?

also Derek noted:

> The OCS/CCO case can cover that, except for the LITE (w/o FRAG) case.
> 
> For the latter I guess we could also use UDP checksum of zero, and have
> an option to restore a proper checksum for the UDP data at the receiver,
> however that will still leave legacy receivers experiencing such packets
> as unchecksumed.

5. the LITE only case

The whole point of LITE is that some of the IP payload isn’t checksummed. 

> Given that we're supposed to veryify such receivers can accept
> LITE (w/o FRAG) and hold soft state, maybe this is acceptable?

We don’t have a requirement that such receivers can accept LITE before we start using it, so soft state won’t help here.

> we can send
> LITE (w/o FRAG) using UDP checksum of zero, and the additional
> option encoding the actual checksum for the UDP data.
> 
> This additional checksum could simply be the existing defined
> ACS option.

ACS uses a CRC, not IP checksum. That’s a LOT more work in software, so this isn’t a viable alternative.

IMO, we leave this one alone. The whole point of LITE is to do less work - if that means the LITE data corrupts the entire packet, then so be it. Then LITE basically fails.

------

---

From: Raffaele Zullo <[email-masked]>
Date: Sat, 09 Mar 2019 19:56:56 +0000

Hi,

On 2019-03-09 19:28, Joe Touch wrote:
> We don’t have a requirement that such receivers can accept LITE before
> we start using it, so soft state won’t help here.
> 
>> we can send
> IMO, we leave this one alone. The whole point of LITE is to do less
> work - if that means the LITE data corrupts the entire packet, then so
> be it. Then LITE basically fails.
> 
> ------

The case of LITE or more generally when checksum should not cover the 
full Options area
could be partially solved with a CCO of 4 bytes
(the original CCO proposal had a variable length for further 
flexibility).

the first 2 bytes contains the checksum of UDP Options you want to 
cover,
the other 2 bytes contains the additional compensation value for the 
uncovered bytes, for the traversal only.

The receiver can still validate the covered Options without the effort 
of summing the unnecessary Option bytes (e.g. LITE DATA),
and without risking to drop a packet for an error on the uncovered 
bytes.

Anyway the sender will still need to sum the uncovered bytes.

Thank you,
Raffaele Zullo

---

From: Joe Touch <[email-masked]>
Date: Sat, 9 Mar 2019 12:52:25 -0800

> On Mar 9, 2019, at 11:56 AM, Raffaele Zullo <[email-masked]> wrote:
> 
> Hi,
> 
> The case of LITE or more generally when checksum should not cover the full Options area
> (the original CCO proposal had a variable length for further flexibility).
> 
> the first 2 bytes contains the checksum of UDP Options you want to cover,
> the other 2 bytes contains the additional compensation value for the uncovered bytes, for the traversal only.
> 
> The receiver can still validate the covered Options without the effort of summing the unnecessary Option bytes (e.g. LITE DATA),
> and without risking to drop a packet for an error on the uncovered bytes.
> 

Yes, but there are three points to LITE:

1. less work for the transmitter (which your proposal defeats)
2. less work for the receiver (which your proposal maintains)
3. deliver potentially corrupted data (one of the original points of UDP-lite - it wasn’t just about performance)

These errant middle boxes are also thus defeating #3.

So, IMO, in the presence of those sorts of things, basically LITE with our proposal doesn’t do most of what it was intended.

Again, for that reason, I’d still prefer leaving it alone.

Joe

---

From: "C. M. Heard" <[email-masked]>
Date: Mon, 11 Mar 2019 09:26:04 -0700

On Sat, 9 Mar 2019 11:28:57 -0800 Joe Touch wrote:
> PS =  yes, we talk about TLV but need to explain that some options are
> exceptions (OCS, NOP, EOL). I also need to update the OCS to be 3 bytes
> long (in the table and figure). Those are in addition to the issues below.
>
> -------
>
>         (a) we need to declare that the option area consumes the whole of
>         the surplus area (no chance for other uses)
>         OR

I agree with that.

> 2. we don’t need OCS alignment; what we do need is to “coordinate” the
 alignment of a 16-bit of the length to the OCS checksum field
>           heavy here, but same point)

I believe that is correct. Padding could still be convenient, though,
and I see no reason to disallow it (the current draft does permit
padding for alignment).

>         we *can* use UDP CS=0, but that also may involve overriding the
>         user’s desires here (i.e., the user API says “use UDP CS” and
we’re
>         not doing that) which means we have a choice:

I'm skeptical about this proposal, because it requires making a for
how OCS is calculated for the specific case of LITE+FRAG, and there
are all manner of corner cases that will make that difficult to draft.
In addition, the UDP header itself would not be covered by a checksum,
irrespective of the user UDP CS setting.

I'd like to float a different idea, namely, putting the UDP user data
inside the FRAG option itself. Terminal and non-terminal FRAG options
would need to be distinguished in some manner other than the Length
field; one way is to allocate different Kind code points, as shown below.

                   +--------+--------+--------+--------+
                   | Kind=X |Len=8+k |  Frag. Offset   |
                   +--------+--------+--------+--------+
                   |          Identification           |
                   +--------+--------+--------+--------+

                   ...                               ...
                   |             User Data             |

                   ...                               ...
                   +--------+--------+--------+--------+

UDP non-terminal FRAG option format

                   +--------+--------+--------+--------+
                   | Kind=Y |Len=10+k|  Frag. Offset   |
                   +--------+--------+--------+--------+
                   |          Identification           |
                   +--------+--------+--------+--------+

                   ...                               ...
                   |             User Data             |

                   ...                               ...
                   +--------+--------+--------+--------+

| Checksum | +--------+--------+ UDP terminal FRAG option format

The following requirements would apply:

   >> A host that wishes to signal that it is able to accept and process
   the FRAG option MAY do so by transmitting an unfragmented datagram
   with an empty terminal FRAG option with Offset zero and length 12.

   >> Non-empty FRAG options MUST NOT be present in packets with
   ordinary UDP user data. Any such options MUST be silently
   dropped.

   >> UDP options other than OCS and padding MUST NOT accompany the FRAG
   option in non-terminal fragments. Any such options MUST be silently
   dropped. All other options that apply to a reassembled packet must
   accompany the FRAG header in the terminal fragment.

My thinking is that if user UDP CS setting specifies that the UDP
checksum should be zero, then OCS would not accompany the FRAG
option. Otherwise it would, irrespective of the user OCS setting.

It is true there is the potential for excess work when the user UDP
CS setting is non-zero, because there are checksums that cover both
the individual fragments and the reassembled UDP datagram. However,
most of that work can be avoided by a well-designed implementation
that caches partial checksums during the reassembly process and
sums those to calculate the overall checksum. Alternatively, one
could omit the overall checksum in terminal FRAG options.

Since there is a looming submission deadline, it may be better to
develop this proposal further on the mailing list before putting it
into a draft.

...
>
> IMO, we leave this one alone. [... T]here are three points to LITE:
>
> 3. deliver potentially corrupted data (one of the original points of
> UDP-lite - it wasn’t just about performance)
>
> So, IMO, in the presence of those sorts of things, basically LITE with our
> proposal doesn’t do most of what it was intended.

I agree with this. My take is that if LITE is to remain in the spec, then it
should not be covered by OCS. The workaround would defeat its purpose.

Mike Heard

---

From: Raffaele Zullo <[email-masked]>
Date: Mon, 11 Mar 2019 19:27:08 +0000

Hello,

On 2019-03-11 16:26, C. M. Heard wrote:
>>         (c) that OCS covers even the surplus area (it doesn’t need
> to be
>>         zero; it does need to be included)
>> 

I would add that a Zero padding after UDP Options area is not neutral to 
the (full IP payload) checksum because it still increases the IP Total 
Length (it changes the "pseudo-header").

>>         - if OCS checksum field is not 16-bit aligned to the start
> of the
>>           option area, then swap bytes
>>         - add the result to the OCS checksum (‘pseudo header’
> seems a bit
>>           heavy here, but same point)
> 

Would it be possible to leave this as a choice?
CCO draft, despite on sender's side required alignemnt,
on receiver's side only required that the complement of the sum of 
Options and pseudoheader was zero,
in order to accept also a working CCO computed without alignment.

Thank you,
Raffaele Zullo

---

From: "C. M. Heard" <[email-masked]>
Date: Mon, 11 Mar 2019 13:28:12 -0700

> Length (it changes the "pseudo-header").

Good point; that one seems to keep escaping me :-(

> in order to accept also a working CCO computed without alignment.

+1

Mike Heard

---

From: Derek Fawcus <[email-masked]>
Date: Mon, 11 Mar 2019 23:18:39 +0100

> This one’s a bit long - we need to converge quickly if the draft is going to be updated in time for the cutoff.

Sorry - I've been otherwise occupied; and hence this isn't a complete response...

> 	we need a choice here. AFAICT, the safest (a)

I agree on (a); but then I was never convinced on the possibily of other users,
as future users would still need to be able to parse these options, in which case
why would they not use them?

The only use I could see for leaving surplus for non option use
would be in the case of there being an option indicating
'the rest of the surplus is X', for some X.

So really what we lose is the ability to have an option of more
than 255 bytes.

> So to me it’s (f.2) and (d.1). 

I'd split LITE w/o FRAG from LITE with FRAG.

The latter would not involce overriding the users choice of there being no CS,
as there is no UDP data as such.  The only thing we'd lose is a check on the
pseudo-header, so maybe encode that somewhere in the LITE+FRAG combo, dare I
say as another option?

So for LITE+FRAG, I'd agree (f.2) and (d.1)

[As an aside, the only guarenteed way I've found to send w/o UDP checksum
 using the sockets API is by using RAW sockets.]

> ACS uses a CRC, not IP checksum. That’s a LOT more work in software, so this isn’t a viable alternative.

Actually, my original thought here (which I then culled), was to have another
option, or ACS.  That other option would encode the current style of checksum,
so for concerns about CPU cost of using CRC, we could mitigate it.

What I had in mind was an option containing two 16 bit fields, one would
encode the partial-checksum for the pseudo-header as the originator sent
it, the other the partial-checksum for the original UDP payload.

e.g.:  [ Type = Frag-Chks | Len | part-phdr-chk | part-data-chk ]
  
That way the receiver can recover from any NAT mangling of the packet which
would otherwise mess with the checksum calculation.

i.e. for the UDP portion, assuming the packet was encoded with UDP CS=0

  1. Calculate partial checksum for pseudo-header as received
  2. Compare to option part-phdr-chk
  2a. If equal verify UDP data against part-data-chk
  2b. If differ verify UDP data against part-data-chk + (part-phdr-chk - phdr-chk)

The implementation of that can probably be optimised.

> IMO, we leave this one alone. The whole point of LITE is to do less work - if that means the LITE data corrupts the entire packet, then so be it. Then LITE basically fails.

However, I do agree that the LITE w/o FRAG case (incl. when using the proposed
UDP CS=0 encoding) needs more consideration, and so am happy to leave that
to a later revision of the draft.

Especially as it would be good to avoid having the OCS/CCO option encoding
the LITE data, but I'm not sure how we achieve that.  Maybe this is one of
the cases where we simply have to accept it will not pass through broken
middle devices?  Which would be a pity, but it may still pass through
more than UDP-Lite does.

i.e. maybe LITE w/o FRAG would simply have to be like in the current draft,
and using no clever encoding tricks for CCO discovered problems?

We already have the case that we won't pass through broken devices which
drop packets where there is any slack space.

DF

---

From: Derek Fawcus <[email-masked]>
Date: Tue, 12 Mar 2019 00:28:22 +0100

>   2b. If differ verify UDP data against part-data-chk + (part-phdr-chk - phdr-chk)

It might help to explain my line of reasoning here..

My first idea for using UDP CS=0 for LITE+FRAG was simply to move
the original UDP CS in to an option.

i.e. the host stack would operate as normal, building its UDP header.
Then during building the options area, it would copy the
UDP CS in to an option (call it the Moved Checksum - MCS),
and zero the CS in the UDP header.

So we end up with something like:

  [Kind=MCS|Len=4|16 bit checksum]

Then I realised this would be broken by NATs.

Hence concluding one would have to also carry the partial
checksum for the original pseudo-header
(i.e. src-ip/dst-ip/src-port/dst-port/proto) such that
NAT manipulations can be compensated for:

  [Kind=MCS|Len=6|16 bit part-phdr-chk|16 bit original-chk]

So the processing on the TX side to generate that is no
more costly, the RX side would have a bit more work.

Conceptually generating:

  new-chk = original-chk - (part-phdr-chk - rx-phdr-chk)

Then writing that over the UDP CS field, before passing the
packet in to the original receive / validate routine.

Sending two partial checksums is just another way of achieving
the same effect.  Maybe the moved version is clearer?

Since we know that this checksum is for the pseudo-header
and UDP header alone, with len=0 and no carried data;
I'm just wondering now if that can be improved?

DF

---

From: Joe Touch <[email-masked]>
Date: Mon, 11 Mar 2019 18:41:45 -0700

> On Mar 11, 2019, at 3:18 PM, Derek Fawcus <[email-masked]> wrote:
> 
> than 255 bytes

We can easily declare one final option that is “remainder”.

Note that comes with some caveats:
- it wouldn’t have a length; like NOP, EOL, and OCS, it would just be a flag.
- it would not be defined as being included in the OCS (if it were, it would be a conventional option IMO), which means any use of that area would need to consider a checksum area that can be used like CCO (NOTE - I’m not suggesting including that area by default; the CCO trick is a fix for a pothole, but we should not be designing every mechanism to REQUIRE compensation for potholes that ought to be fixed)

And yes, this (IMO) correctly inverts Tom Herbert’s proposal; it leaves the checksums as first-class and “other” trailer uses as second.

Joe

---

From: Joe Touch <[email-masked]>
Date: Mon, 11 Mar 2019 18:43:02 -0700

FYI - all of the stuff below was considered when designing LITE+FRAG and suggesting the reassembled packet had its own reassembly checksum - that did NOT include the pseudo header. 

I.e., IMO, the LITE+FRAG mechanism we have already does basically all we CAN do.

> On Mar 11, 2019, at 4:28 PM, Derek Fawcus <[email-masked]> wrote:
>

---

From: Joe Touch <[email-masked]>
Date: Mon, 11 Mar 2019 18:51:05 -0700

+1

The current wording was, FWIW, intended to continue to allow OCS alignment if desired.

Joe

> On Mar 11, 2019, at 1:28 PM, C. M. Heard <[email-masked]> wrote:
>

---

From: Raffaele Zullo <[email-masked]>
Date: Tue, 12 Mar 2019 12:10:05 +0000

On 2019-03-09 19:28, Joe Touch wrote:
> user’s desires here (i.e., the user API says “use UDP CS” and we’re
> not doing that)

Maybe I'm missing something but there could be
two other problems when using CS=0 with LITE+FRAG

1) We lose coverage of UDP Length
If the UDP Length due to an error becomes 8 < UDP Length <= 
IPPayloadLength
Options will probably become not parsable
but part of the Options is delivered as UDP Payload to the user.

2) We lose coverage of Source and Dest Ports
If there are several UDP connections ongoing between two hosts (or two 
NATs)
an error on Source/Dest Port can make the packet be delivered to the 
wrong connection.
This issue is very limited since LITE+FRAG carries an identifier.

Raffaele Zullo

---

From: Raffaele Zullo <[email-masked]>
Date: Tue, 12 Mar 2019 16:43:12 +0000

On 2019-03-12 01:41, Joe Touch wrote:
>> On Mar 11, 2019, at 3:18 PM, Derek Fawcus 
>> <[email-masked]> wrote:
>> 

Hello,

To leave it open to Options longer than 255

Why not declare that a Length = 0 means that the 3rd and 4th bytes 
contain the Length of the Option?

> - it would not be defined as being included in the OCS (if it were, it
> would be a conventional option IMO), which means any use of that area
> would need to consider a checksum area that can be used like CCO (NOTE

AFAIU

EOL already works as an Option "remainder"

Raffaele Zullo

---

From: Joe Touch <[email-masked]>
Date: Tue, 12 Mar 2019 15:45:53 -0700

On 2019-03-12 09:43, Raffaele Zullo wrote:

> On 2019-03-12 01:41, Joe Touch wrote: On Mar 11, 2019, at 3:18 PM, Derek Fawcus <[email-masked]> wrote:
> 

Hello,

We could allow that, or we could define a new option of "long option
follows" - or - as you note below - just let EOL do its job ;-) 

Joe 

>> We can easily declare one final option that is "remainder".
>> 
>> - it wouldn't have a length; like NOP, EOL, and OCS, it would just be a flag.

---

From: Joe Touch <[email-masked]>
Date: Tue, 12 Mar 2019 15:53:35 -0700

Yes, when UDP CS=0 we don't cover length or ports. 

Length isn't an issue unless the reassembly checksum is also zero, in
which case we're doing the right thing - leaving it alone. 

As to ports when the frag checksum is NOT zero, I'm much less concerned
given how devices happily recompute that along packet paths for their
own convenience. 

Joe

On 2019-03-12 05:10, Raffaele Zullo wrote:

>> user's desires here (i.e., the user API says "use UDP CS" and we're
>> not doing that)
>> (d.2) disallow the use of OCS (if CS=0, what's the point?)
>> OR
>> I don't like assuming user correct configuration (f.1).
>> 
>> So to me it's (f.2) and (d.1).
>> 
>> Thoughts?
> 
> If the UDP Length due to an error becomes 8 < UDP Length <= IPPayloadLength
> If there are several UDP connections ongoing between two hosts (or two NATs)
> an error on Source/Dest Port can make the packet be delivered to the wrong connection.

---

From: Raffaele Zullo <[email-masked]>
Date: Tue, 12 Mar 2019 23:01:46 +0000

On 2019-03-12 22:53, Joe Touch wrote:

Hello,

> which case we're doing the right thing - leaving it alone.

OK, but
what happens if UDP Length (that was 8) is increased to
8 < UDP Length <= IPPayloadLength?
Will part of the Options area be delivered to user as a regular UDP 
Payload?

Raffaele Zullo

> As to ports when the frag checksum is NOT zero, I'm much less
> concerned given how devices happily recompute that along packet paths
> for their own convenience.
> 
> Joe

---

From: Joe Touch <[email-masked]>
Date: Tue, 12 Mar 2019 16:16:40 -0700

On 2019-03-12 16:01, Raffaele Zullo wrote:

> Will part of the Options area be delivered to user as a regular UDP Payload?

If the reassembly checksum isn't zero, this would be caught (with high
probability) during reassembly. 

If the reassembly checksum is zero, then yes - corrupt data would go to
the user. But that would have been what the user wanted (by setting the
reassembly checksum to zero). 

Joe 

Raffaele Zullo

Joe

3. we have to deal with LITE/LITE+FRAG
we *can* use UDP CS=0, but that also may involve overriding
the
user's desires here (i.e., the user API says "use UDP CS" and
we're
not doing that)
which means we have a choice:
(d) override so that things work, i.e., force UDP CS=0 and
then
(d.1) do OCS as a check on our stuff only
OR
(d.2) disallow the use of OCS (if CS=0, what's the point?)
OR
(d.3) use OCS=0 (this seems like a waste)
OR
(e) ignore the user setting of CS, then:
(e.1) always do OCS as a check
OR
(e.2) if UDP CS=0, omit OCS (save space), basically like
d.2
OR
(e.3) if UDP CS=0, use OCS=0 (basically like e.2

So which is it?

Thoughts?

2) We lose coverage of Source and Dest Ports
If there are several UDP connections ongoing between two hosts (or
two NATs)
an error on Source/Dest Port can make the packet be delivered to the
wrong connection.
This issue is very limited since LITE+FRAG carries an identifier.

Raffaele Zullo

---

From: "C. M. Heard" <[email-masked]>
Date: Tue, 12 Mar 2019 20:26:39 -0700

On Tue, Mar 12, 2019 at 4:16 PM Joe Touch <[email-masked]> wrote:
> > OK, but what happens if UDP Length (that was 8) is increased to

If LITE+FRAG is being used, then the initial LITE option will be corrupted,
as at least one octet will be skipped. With exceedingly high probability,
the LITE option will not be processed. The remaining options won't be
found, and what is there is unlikely to be parsed. Result: the
(UDP Length - 8) octets that appear to be conventional UDP user data
will be delivered, and the trailing octets will be discarded since they
will not appear to be valid UDP options.

Reassembly won't even be invoked.

So, CS=0 does have its risks even for the length field.

Mike Heard

---

From: Joe Touch <[email-masked]>
Date: Tue, 12 Mar 2019 23:39:22 -0400

> On Mar 12, 2019, at 11:26 PM, C. M. Heard <[email-masked]> wrote:
> 

Fair enough. Now show us how to avoid that without recopying the payloads and/or recomputing the checksum twice. 

That’s the issue. A fix for a problem that current UDP already deals with and doesn’t come up vs so much extra work the result won’t be useful. 

Joe

---

From: "C. M. Heard" <[email-masked]>
Date: Tue, 12 Mar 2019 21:36:16 -0700

On Tue, Mar 12, 2019 at 8:39 PM Joe Touch wrote:
>  On Mar 12, 2019, at 11:26 PM, C. M. Heard wrote:
[ ... ]
> Fair enough. Now show us how to avoid that without recopying the
> payloads and/or recomputing the checksum twice.
>
> That’s the issue. A fix for a problem that current UDP already deals
> with and doesn’t come up vs so much extra work the result won’t be
> useful.

That is a fair request, considering that I have sowed ample confusion
by floating three variations of my proposal, and only the last achieved
that result. Here it is, again:

Maintain the format of the FRAG option as currently defined for a
non-terminal fragments, and allocate a new KIND to indicate a
terminal fragment. Both kinds would have the same format:

                   +--------+--------+--------+--------+
                   |  Kind  | Len=10 |  Frag. Offset   |
                   +--------+--------+--------+--------+
                   |          Identification           |
                   +--------+--------+--------+--------+
                   |    Checksum     |
                   +--------+--------+

The following requirements would apply:

   >> When the FRAG option appears, it MUST come last in the UDP options
   list.  All remaining octets in the packet are interpreted as fragment
   data.

   >> OCS, if present, covers both the FRAG option and the trailing
   fragment data.

   the FRAG option MAY do so by transmitting an unfragmented datagram
   with an empty terminal FRAG option whose Offset field is set to zero.

   >> Non-empty FRAG options MUST NOT be present in packets with ordinary
   UDP user data or LITE data. Any such options MUST be silently dropped.

To handle the case when the user UDP CS setting specifies that the UDP
checksum should be zero, then OCS would not accompany the FRAG option.
Otherwise it would, irrespective of the user OCS setting.

This version does not have a distinct overall checksum for the reassembled
packet, and so avoids duplicate work. Having the revised OCS cover the data
in each fragment provides protection that is essentially equivalent to that
provided by the post-reassembly checksum proposed in the current draft. For
in order to deliver the reassembled data, we require all fragments to be
present and to fit together exactly with no gaps and no overlap. Thus,
having each fragment pass its checksum is equivalent to passing an overall
checksum that covers the reassembled data plus the options. Note that ACS
is still available if a stronger post-reassembly checksum is wanted.

I think this answers the important objections. It offers a solution that
provides protection equivalent to conventional IP fragmentation coupled
with the standard UDP checksum, allows for middlebox traversal, and avoids

duplicate checksum computations.

Mike Heard

---

From: Joe Touch <[email-masked]>
Date: Tue, 12 Mar 2019 21:47:53 -0700

> On Mar 12, 2019, at 9:36 PM, C. M. Heard <[email-masked]> wrote:
> 
> Having the revised OCS cover the data in each fragment provides protection that is essentially equivalent to that provided by the post-reassembly checksum proposed in the current draft.

The composition of checksums is not equivalent to a checksum. It does not offer equivalent protection, esp. in errors within the reassembly process itself.

This remains a non starter.

Joe

---

From: "C. M. Heard" <[email-masked]>
Date: Tue, 12 Mar 2019 22:14:10 -0700

On Tue, Mar 12, 2019 at 9:48 PM Joe Touch <[email-masked]> wrote:
> > Having the revised OCS cover the data in each fragment provides protection
> > that is essentially equivalent to that provided by the post-reassembly checksum
> > proposed in the current draft.
>
> The composition of checksums is not equivalent to a checksum. It does not offer
> equivalent protection, esp. in errors within the reassembly process itself.

The equivalence is a mathematical one, assertions to the contrary
notwithstanding.

Heard

---

From: Joe Touch <[email-masked]>
Date: Tue, 12 Mar 2019 22:43:41 -0700

Math isn’t the issue. Reliability is and they’re not the same unless they’re done at the same time. 

Otherwise you’re checking correctness in pieces over time and that may not reflect reality at the time the packet is delivered to the app layer. 

This is a fundamental aspect of E2E. It’s not math. It’s math + time.

Joe

> On Mar 12, 2019, at 10:14 PM, C. M. Heard <[email-masked]> wrote:
>

---

From: "C. M. Heard" <[email-masked]>
Date: Wed, 13 Mar 2019 05:40:14 -0700

On Tue, Mar 12, 2019 at 10:43 PM Joe Touch <[email-masked]> wrote:
>>>> that is essentially equivalent to that provided by the post-reassembly
>>>> checksum proposed in the current draft.
>>>
>>> The composition of checksums is not equivalent to a checksum. It does not
>>> offer equivalent protection, esp. in errors within the reassembly process
>>> itself.
>>
> Otherwise you’re checking correctness in pieces over time and that may not
> reflect reality at the time the packet is delivered to the app layer.
>

I will concede this much: having a separate post-reassembly
checksum does offer some limited protection agains a certain
class of software implementation errors.  Calling that
"fundamental" is IMHO a stretch. After all, TCP's integrity
protection of the delivered data stream relies on checksums
on the individual segments. This would be no different.
To paraphrase some of your own words, we should not be
designing  a “byzantine” protocol that survives arbitrary
implementation errors.

Mike Heard

---

From: Tom Herbert <[email-masked]>
Date: Wed, 13 Mar 2019 07:36:25 -0700

On Wed, Mar 13, 2019 at 5:41 AM C. M. Heard <[email-masked]> wrote:
> class of software implementation errors.

Can you give an example of this? We have be performing reassembly of
TCP segments in GRO and LRO for years without any "reassembly
checksum" and it's never been a problem.

> Calling that

A reassembly checksum is more cost, work, and more consumption of
resources. We can't offload the computation of it and we lose early
drop of bad packets.

---

From: Raffaele Zullo <[email-masked]>
Date: Wed, 13 Mar 2019 14:46:47 +0000

> with and doesn’t come up vs so much extra work the result won’t be

If we're forced to use UDP CS=0 to carry unchecksummed data
I see two ways to cover errors on UDP Length

1) Bring the covered payload inside the Options

An example is a UDPLite Option (with total length, CS coverage and CS) 
instead of Lite

this Option on the first packet implies every payload (covered or not) 
is exchanged inside the UDPLite Option (UDP Length is always 8)

It is basically a UDPLite encapsulation in UDPO.

2) A mandatory Option at beginning of Options area, at least when UDP 
CS=0, to verify UDP Length+1 points correctly to the beginning of 
Options.
(This doesnt cover UDP Length becoming equal to IP Payload Length due to 
an error)

---

From: Tom Herbert <[email-masked]>
Date: Wed, 13 Mar 2019 08:56:58 -0700

On Wed, Mar 13, 2019 at 7:47 AM Raffaele Zullo <[email-masked]> wrote:
> CS=0, to verify UDP Length+1 points correctly to the beginning ofe
> Options.
Raffaele,

The problem with being forced to use UDP CS=0 is that we lose checksum
coverage of the UDP header and IP addresses. Because of NAT, we can't
use a pseudo header that covers addresses or ports in the other
checksums like one in the options area. Without a UDP checksum, the
probability of accepting misdelivered packets increases especially in
IPv6 which doesn't have an IP header checksum (UDP checksums are
required in IPv6 for this reason).

Tom

---

From: Raffaele Zullo <[email-masked]>
Date: Wed, 13 Mar 2019 21:19:23 +0000

> required in IPv6 for this reason).

Hello Tom,
Yes, it was meant as a solution to cover errors on Length only.

Anyway we could still cover the UDP Header and PH with an inner options 
checksum, ICS, at the cost of 1 RTT.

On the first packet A sends to B the checksum of its private addr and 
port and B's public addr and port.
B replies with the checksum of its private addr and port and A's public 
addr and port.

Now A sends packets to B computing ICS as UDP CS but with the value 
provided by B, instead of using addrs and ports that it sees.

No errors
B computes the usual UDP Checksum of the received packet and compare it 
to ICS to validate it.

Errors
The previous connection is on B's port port1.
Let's say A opens another connection for example from the same source 
port that goes to another B port (port2).
It will get from B a different initial value, because now it's the 
checksum of A's public addr and port + B's private addr and port2.
Due to an error on a packet, port2 becomes port1 and this packet is 
delivered to the previous connection.
B computes the usual UDP checksum but the ICS of this packet was 
computing using a different value provided by B, so the ICS is not 
valid.

Am I missing something?
What do you think?

Thank you,
Raffaele Zullo

---

From: Tom Herbert <[email-masked]>
Date: Wed, 13 Mar 2019 14:57:58 -0700

Raffaele,

UDP is a connectionless protocol. Inferring connection semantics is problematic.

> computing using a different value provided by B, so the ICS is not

The fact that some middleboxes don't compute UDP correctly when
surplus area is present is a manifestation of protocol ossification.
Similar to other instances of ossification, we don't change core
protocols to accomodate it. UDP checksum is optional in IPv4, required
IPv6, and AFAIK no IETF protocol has ever required zero UDP checksums.
IMO it should remain so.

If no single solution works in all cases, maybe the best we can do is
to pick one method as the default and make other methods optional. The
most compliant and robust default is to require that the surplus area
sums to zero (works in cases #1 and #2), allow normal usage of UDP
checksum, and have a checksum adjustment option (for cases #3 and #4).

Tom