Subject: [tsvwg] UDP Options API document

From: Tom Jones <[email-masked]>
Date: Fri, 3 Sep 2021 18:38:36 +0100

Hi TSVWG,

It was raised today during the interim (I think by Carsten) that a clear
API description is very important for real world adoption of a protocol.
Michael Tuexen added that during the original SCTP standardisation a
separate API draft made writing portable applications much easier.

Let me start the discussion, should we provide a separate draft for UDP
Options API or should this work happen inside the UDP Options draft?

- Tom

---

From: "[email-masked]" <[email-masked]>
Date: Sat, 4 Sep 2021 09:34:33 -0700

Hi, Tom,

As I mentioned during the meeting, there’s already an abstract API in the spirit of TCP's in section 8 of the draft, with a real-world example in Appendix A based on a preliminary implementation.

It’s been there since draft-touch-tsvwg-udp-options-06, dated March 27, 2017.

I don’t recall anyone commenting on that yet, though.

Joe

—
Joe Touch, temporal epistemologist
www.strayalpha.com

> On Sep 3, 2021, at 10:38 AM, Tom Jones <[email-masked]> wrote:
> 
> Hi TSVWG,
>

---

From: "C. M. Heard" <[email-masked]>
Date: Tue, 14 Sep 2021 11:19:59 -0700

> Options API or should this work happen inside the UDP Options draft?

I would like to see thorough and clear documentation in the base
specification of the ***services*** that are provided by UDP options.to the
application layer (or the higher layer protocol running on top of UDP).

On Sat, Sep 4, 2021 at 9:35 AM Joe Touch replied:

> As I mentioned during the meeting, there’s already an abstract API in the
> spirit of TCP's in section 8 of the draft, with a real-world example in
> Appendix A based on a preliminary implementation.
>

I took Tom's advice and read both RFC 8303 ("On the Usage of Transport
Features Provided by IETF Transport Protocols") and RFC 8304 ("Transport
Features of the User Datagram Protocol (UDP) and Lightweight UDP
(UDP-Lite)"). RFC 8303 (and RFC 8304) relied on existing, documented
application interface requirements and abstract APIs, and that Tom is
proposing to do something similar for UDP options, and asks whether we
should do that in the base spec or as a separate document. Note that RFC
8303, despite its title, deals substantively only with connection-oriented
transport protocols, and simply references RFC 8304 in its sections on
connectionless transport protocols.

I also heeded Joe's admonition to read the abstract API in the current
draft (I indeed have not paid attention to it or commented on it up to now).

My sense is that the existing abstract API is insufficient to serve as a
basis for something like an extended version of RFC 8304. But I also feel
that RFC 8304 somewhat misses the mark, since it does not adequately
distinguish what primitives should actually be per-datagram (i.e., figuring
as input parameters to send(), sendto(), or sendmsg() and output parameters
from recv(), recvfrom(), and recvmsg() or the like) versus what things
should be per-socket (and controlled by setsockopt() or the like).  I would
really like to see a complete laundry list written down, and I will take a
stab at it if I am asked.

Thanks,

Mike Heard

---

From: "[email-masked]" <[email-masked]>
Date: Tue, 14 Sep 2021 12:03:40 -0700

Hi, Mike,

I’m a bit confused by the note below.

UDP is a datagram service. Whether an implementation supports cached parameters on a per socket-pair basis or not is their decision; the abstract API should not need to address this.

I.e., every parameter of UDP is per message and should be available via a ’sendmsg()/recvmesg()’ command (or equivalent). Every parameter can potentially be cached into the OS, with OS defaults, overrides on a OS-wide or per-connection basis, etc. - as desired, but that’s outside the scope of a protocol. That’s for the manual page for that OS.

The UDP service is:
	send a message
	(with options) with the following options:

Or 
	receive a message
	(with options) which returns an indicator of the options received

Everything else, including per-OS, per-socket, etc. configurations, etc - including decisions on how to override UDP option defaults - are implementation optimizations and should never be part of an abstract API.

If there’s something beyond that you’re expecting, can you give us a simple example? Or are you thinking more of an OS manual page than a protocol spec?

Joe

—

> On Sep 14, 2021, at 11:19 AM, C. M. Heard <[email-masked]> wrote:
> 
> I would like to see thorough and clear documentation in the base specification of the ***services*** that are provided by UDP options.to <http://options.to/> the application layer (or the higher layer protocol running on top of UDP).
> 
> 
> I took Tom's advice and read both RFC 8303 ("On the Usage of Transport Features Provided by IETF Transport Protocols") and RFC 8304 ("Transport Features of the User Datagram Protocol (UDP) and Lightweight UDP (UDP-Lite)"). RFC 8303 (and RFC 8304) relied on existing, documented application interface requirements and abstract APIs, and that Tom is proposing to do something similar for UDP options, and asks whether we should do that in the base spec or as a separate document. Note that RFC 8303, despite its title, deals substantively only with connection-oriented transport protocols, and simply references RFC 8304 in its sections on connectionless transport protocols.
> 
> I also heeded Joe's admonition to read the abstract API in the current draft (I indeed have not paid attention to it or commented on it up to now).
> 
> My sense is that the existing abstract API is insufficient to serve as a basis for something like an extended version of RFC 8304. But I also feel that RFC 8304 somewhat misses the mark, since it does not adequately distinguish what primitives should actually be per-datagram (i.e., figuring as input parameters to send(), sendto(), or sendmsg() and output parameters from recv(), recvfrom(), and recvmsg() or the like) versus what things should be per-socket (and controlled by setsockopt() or the like).  I would really like to see a complete laundry list written down, and I will take a stab at it if I am asked.
> 
> Thanks,
> 
> Mike Heard

---

From: Tom Herbert <[email-masked]>
Date: Tue, 14 Sep 2021 12:13:46 -0700

> If there’s something beyond that you’re expecting, can you give us a simple example? Or are you thinking more of an OS manual page than a protocol spec?

Basically, yes. An API is not a necessary part of protocol
specification, it is part of implementation. Instead of an abstract
API description, what would really be most helpful is a reference to
an open source implementation demonstrating the APIs as well as the
rest of the functionality described in the specification. Until there
is an actual implementation that can be evaluated by developers that
use and support the APIs, any discussions of APIs are pretty much an
academic exercise IMO.

Tom

> I would like to see thorough and clear documentation in the base specification of the ***services*** that are provided by UDP options.to the application layer (or the higher layer protocol running on top of UDP).
>
>

---

From: "[email-masked]" <[email-masked]>
Date: Tue, 14 Sep 2021 12:45:40 -0700

Hi, Tom,

> On Sep 14, 2021, at 12:13 PM, Tom Herbert <[email-masked]> wrote:
> 
>> ...
> specification, it is part of implementation.

API has often meant two things for protocols:

- the interface to the layer above
	RFC793, Sec 3.8.

- the implementation of that interface
	see Unix man pages for socket(), bind(), listen(), accept(), etc.

A protocol *specification* is incomplete without the interface definition. 

Implementation information, however, belongs in OS documentation.

(Agreed that using API for both of these is confusing; the spec version is more of an “upper layer interface” than an “application programming interface”)

> Instead of an abstract
> rest of the functionality described in the specification.

Section 8 discussed an example of the ULI and Appendix A gave an example of an API.
It would certainly be useful to update the latter with a more recent implementations’s example.
The former can also be more clear, but doesn’t need a lot of exposition to do so.

Joe

---

From: "C. M. Heard" <[email-masked]>
Date: Tue, 14 Sep 2021 14:18:49 -0700

> I’m a bit confused by the note below.

The confusion was mine :-)

> UDP is a datagram service. Whether an implementation supports cached
> parameters on a per socket-pair basis or not is their decision; the
> abstract API should not need to address this.
>
> I.e., every parameter of UDP is per message and should be available via a
> ’sendmsg()/recvmesg()’ command (or equivalent). Every parameter can
> potentially be cached into the OS, with OS defaults, overrides on a OS-wide
> or per-connection basis, etc. - as desired, but that’s outside the scope of
> a protocol. That’s for the manual page for that OS.
>

Excellent, every parameter of UDP is per message. I assume that in this
context "message" is synonymous  with "user datagram" (pre-fragmentation,
post-reassembly).

> Everything else, including per-OS, per-socket, etc. configurations, etc -
> including decisions on how to override UDP option defaults - are
> implementation optimizations and should never be part of an abstract API.
>
> If there’s something beyond that you’re expecting, can you give us a
> simple example? Or are you thinking more of an OS manual page than a
> protocol spec?
>

Yes, what I am looking for is a complete list of the per-datagram services
offered by UDP options and how they are invoked. Note that the services are
not always mapped in a direct one-to-one way to the on-the-wire options.
Two prominent ones that come to mind are fragmentation and padding. On
send, I would expect to be able to specify whether UDP fragmentation is, or
is not, allowed for a particular message, and I would expect to be able to
specify the minimum padded segment/fragment size (likely most useful in
combination with fragmentation not allowed). Specifying "fragmentation
allowed" causes UDP to use the FRAG header as specified in the protocol
spec, but that process (and the subsequent reassembly on the other side)
take place "under the hood." Similarly, specifying a minimum padded size
would be implemented using EOL followed by zero fill as necessary, but the
application wouldn't specifically request that.

I will offer to make up a complete list, based on what comes out in the -14
draft, if you wish.

As for the question from Tom Jones, maybe the right approach is to write up
a separate API document along the lines of RFC 8304 (perhaps even an RFC
8304bis) after the abstract API is fully fleshed out and some
implementations of the completed spec exist.

Mike Heard

---

From: "[email-masked]" <[email-masked]>
Date: Tue, 14 Sep 2021 22:56:43 -0700

Notes below...
—

> On Sep 14, 2021, at 2:18 PM, C. M. Heard <[email-masked]> wrote:
> 
> Excellent, every parameter of UDP is per message. I assume that in this context "message" is synonymous  with "user datagram" (pre-fragmentation, post-reassembly).

Yes.

> Yes, what I am looking for is a complete list of the per-datagram services offered by UDP options and how they are invoked. Note that the services are not always mapped in a direct one-to-one way to the on-the-wire options. Two prominent ones that come to mind are fragmentation and padding.

See below...

> On send, I would expect to be able to specify whether UDP fragmentation is, or is not, allowed for a particular message,

Yes, with the same types of optimizations typically offered for other UDP parameters such as port and address - i.e., they might be set/overridden by OS defaults or cached/overridden on a per-socketpair basis, and the specific API command may vary (e.g., send after bind vs sendto).

> and I would expect to be able to specify the minimum padded segment/fragment size (likely most useful in combination with fragmentation not allowed).

That one might not be clear; I would expect that the max fragment size would be derived from the interface (less room for IP and UDP options), as with TCP.  I can imagine direct write control the same way as some TCP implementations might afford, but not as an expected part of the ULI. 

There is no “padding” currently defined. I checked Gorry's UDP MTU doc, which mentions using this but doesn’t explain how. 

*IF* we add a padding request, then AFAICT it would act like an option - i.e., specific “PAD N” would extend the packet to at least N bytes (or the option, or simply add those bytes - depends on what UDP MTU needs). But that would be to non-fragmented packets; there’s no point in padding fragmented ones.

> Specifying "fragmentation allowed" causes UDP to use the FRAG header as specified in the protocol spec, but that process (and the subsequent reassembly on the other side) take place "under the hood." Similarly, specifying a minimum padded size would be implemented using EOL followed by zero fill as necessary, but the application wouldn't specifically request that.

There’s no utility in a generic “min packet size”; the probes happen as needed and so should be triggered per-datagram only - and likely only with non-fragmented data anyway.

> I will offer to make up a complete list, based on what comes out in the -14 draft, if you wish.

Yeah - I’ll take a pass for the ULI…

> As for the question from Tom Jones, maybe the right approach is to write up a separate API document along the lines of RFC 8304 (perhaps even an RFC 8304bis) after the abstract API is fully fleshed out and some implementations of the completed spec exist.

There could be a simpler way, IMO - one that more closely mimics RFC793 and avoids the need to list every possible configurable parameter.

E.g., IMO, the ULI for UDP is:

	send(sadder, daddr, sport, sport, user data, options)
		where, for 768, options was basically just “CS on/off”

There’s no good reason to state those options now or ever in a ULI description; they should be part of the option definition.

I also think it ought to be sufficient to also claim, as per RFC1122 Sec 3.4., that IP parameters configurable by transport or signaled to transport should be passed as options; that would then cover DSCP, ECN, etc. I see no reason to state those in detail as requirements either.

I’ll take a shot at this in the update as an ULI; it’s going to be very brief, as IMO it should be. 

Joe

---

From: "C. M. Heard" <[email-masked]>
Date: Wed, 15 Sep 2021 05:52:40 -0700

On Tue, Sep 14, 2021 at 10:57 PM Joe Touch wrote:

> and I would expect to be able to specify the minimum padded
> segment/fragment size (likely most useful in combination with fragmentation
> not allowed).
>
>
> That one might not be clear; I would expect that the max fragment size
> would be derived from the interface (less room for IP and UDP options), as
> with TCP.  I can imagine direct write control the same way as some TCP
> implementations might afford, but not as an expected part of the ULI.
>
> There is no “padding” currently defined. I checked Gorry's UDP MTU doc,
> which mentions using this but doesn’t explain how.
>

No, it's not explicitly defined, but there is an obvious and simple method
to do so that do not require definition of any new "on-the-wire" options:
after all user data, and options are in place, insert zeroes (the first of
which serves as an EOL marker) as needed to get the segment size up to that
requested by the upper layer. For this to work, the upper layer interface
(ULI) has to have a method written down for the upper layer to make such a
request.

*IF* we add a padding request, then AFAICT it would act like an option -
> i.e., specific “PAD N” would extend the packet to at least N bytes (or the
> option, or simply add those bytes - depends on what UDP MTU needs). But
> that would be to non-fragmented packets; there’s no point in padding
> fragmented ones.
>

Actually, I agree with that. It would ***act*** like an option, but there
is no need to define another TLV in order to accomplish this.

There’s no utility in a generic “min packet size”; the probes happen as
> needed and so should be triggered per-datagram only - and likely only with
> non-fragmented data anyway.
>

There seems to be an implicit assumption in that last sentence that DLPMTUD
is implemented "under the hood." That is certainly one way to do so, but
that is not what I have inferred as the intent of the specifications (if
there is something that I missed, I invite correction). I have been
implicitly assuming that DLPMTUD is implemented on top of the UDP with
options, using its per-datagram ULI.

And, FWIW, I am fine if a "minimum padded size" ULI option applies only to
datagrams and not to fragments. I made the argument before that
DLPMTUD probes should not be fragmented at all (either at the IP layer or
the UDP layer) but in
https://mailarchive.ietf.org/arch/msg/tsvwg/whbnTz0LG8NVY6C4OelcNpU2cpE/ you
argued against that position (I'm still waiting for the DLPMTUD authors to
weigh in on that).

There could be a simpler way, IMO - one that more closely mimics RFC793 and
> avoids the need to list every possible configurable parameter.
>
> There’s no good reason to state those options now or ever in a ULI
> description; they should be part of the option definition.
>
> I also think it ought to be sufficient to also claim, as per RFC1122 Sec
> 3.4., that IP parameters configurable by transport or signaled to transport
> should be passed as options; that would then cover DSCP, ECN, etc. I see no
> reason to state those in detail as requirements either.
>

OK, I will make my comments on that update when it appears. Brief is good,
but underspecified is not. One of the issues that the ULI specification
should address is what level of upper layer control (if any) exists for
per-fragment options on fragmented datagrams. It has become clear to me
that a good part of the (IMO unresolved) controversy regarding the utility
of per-fragment options (and whether any other than OCS and FRAG are even
needed at all) centers around precisely this point.

Thanks,

Mike Heard