Subject: Re: [Taps] draft-ietf-taps-interface-05 review

From: Michael Welzl <[email-masked]>
Date: Wed, 4 Mar 2020 15:41:12 +0100

Dear Kyle,

Thanks so much for all your comments - and sorry for taking so long to get back to you!

I decided to begin attacking them now with a PR. Please see:
https://github.com/ietf-tapswg/api-drafts/pull/514
for the comments that I tried to address, my answers to them, and the changes that I propose to our draft.

For the record, if this PR can be merged, this still leaves us with the unresolved comments below. I inserted some answers below, prefixed with "MW:”. I think that some of them can be addressed relatively easily by others, whereas some should probably best be filed as an issue for further discussion… let’s take this step by step. Easy ones first, to make the list a bit more digestable  :-)

Cheers,
Michael

-------

REMAINING UNADDRESSED COMMENTS FROM KYLE ROSE:

2. Terminology and Notation

 * The notation "Sender -> Event" says nothing about who receives the event. Is it just assumed to be the application in the context of the particular event (e.g., a thread processing a particular connection)?

MW: We have the following statement in this section: "Events could be implemented via event queues, handler functions or classes, communicating sequential processes, or other asynchronous calling conventions."
I wonder why this isn't enough information? E.g., if handlers are used, then the recipient of the event is a pre-registered handler -  and handlers must therefore be registered before events can occur; this function isn't visible in the API because this is platform- and language-specific.

4.2.1. Transport Property Names

 * The use of dashes in property names means they will necessarily look different in almost every widely-used language, in which dash is an operator.

 * If we're not currently asking IANA to create a registry of property names or namespaces, should we provide a recommendation that such symbols not listed explcitly in this document be prefixed with some experimental identifier?

5.2. Specifying Transport Properties

 * There are a lot of choices being made about how users will want to prioritize transport protocol selection. How confident are we that, for instance, path selection should take priority over protocol selection? I think that's right, but I wonder if it might not make more sense to have two interfaces: one that provides a purely numeric priority ordering of preferences, and one (based on the existing 5-level preferences) that maps into it.

 * Using the same enum (Require(d[sic])/Avoid/Ignore) for the queried output of selected properties seems like a shortcut that will lead to some confusion.

MW: I agree;  partially addressed: s/Required/Require wherever it's a Preference level.

5.2.5. Use 0-RTT Session Establishment with an Idempotent Message

 * I'll repeat the same statement I made at the mic a few years ago: idempotency != replay-safe. DELETE is idempotent, but not safe for replay because someone might have done a PUT or POST in the meantime.

MW: I remember you saying this, sorry that we haven't addressed it yet! I didn't understand - is your concern about replaying that a Message may arrive later, i.e. as Message X, Messages Y and Z in between, then Message X again?  => if so, are you saying that this could happen with TFO or QUIC?  (I didn't think so)

5.2.10. Interface Instance or Type

 * These type symbols really deserve an actual registry, or at least the start of one. Otherwise, we are likely to end up with a mess.

5.2.11. Provisioning Domain Instance or Type

 * What about ordering of similar interfaces? I have a 2-SIM cellphone with wifi.

5.3.2. Connection Establishment Callbacks

 * What constitutes trust verification prior to the handshake?

7.3.1. Sent

 * It seems like an abstract API would be helpful for the reference to the Message to which a Sent event applies: this seems like something many, many applications would need to do. With callbacks, the application can always curry in a reference to the original message (that's what my event system from ~2001 did), so maybe that should be the recommendation and the message reference removed...? I don't have a strong feeling here other than that if something is included then its use should be more well-defined.

7.3.3. SendError

 * Is there a reason why a single messageContext object is used for both sends and receives? I probably missed the original discussion about this, but I'd like to understand the reasoning.

MW: I don't get this, where does it say that it's only a single object? Or do you mean that it's the same type? Why is that problematic?

7.4.7. Reliable Data Transfer (Message)

 * The default isn't "true", it's whatever the underlying connection had, right?

8. Receiving Data

 * A connection can be used to receive data if it is not configured as unidirectional transmit.

MW: that's obviously correct - but I can't find a statement that would contradict what you say here.

8.2. Receive Events

 * "A call to Receive will be paired with a single Receive Event" or possibly multiple ReceivedPartial Events?

 * Also, can the draft be consistent about Receive vs. Received, et al.?

11. General comments

 * There's a lot of UNIXisms here that should probably be excised in favor of a more abstract presentation. The most obvious is the use of -1 to mean something other than the integer -1. In Python, for instance, you might want to use None for this purpose. I would specify non-numeric values symbolically, and maybe indicate in the appendix that a UNIX C implementation might want to use -1 to represent such values.

MW: agreed... TODO.

11.1.10. Capacity Profile

 * Very little of this section is about capacity. Traffic Profile?

 * "High Throughput Data" might be better phrased as "Capacity-Seeking".

---

From: "Philipp S. Tiesel" <[email-masked]>
Date: Wed, 4 Mar 2020 17:51:42 +0100

Hi,

Some of the remaining issues had already been discussed, but either did not lead to text or the text was lost. Let me comment on these issues:

>> On 4. Mar 2020, at 15:38, Michael Welzl <[email-masked]> wrote:
> 

The solution for this issue was to allow implementations to change the property names in consistent ways, e.g., by replacing the dashes with underscores or removing them and use CamelCasing.
The text was removed as over-specific.

> * If we're not currently asking IANA to create a registry of property names or namespaces, should we provide a recommendation that such symbols not listed explcitly in this document be prefixed wit…

The IANA registry question was explicitly postponed, be maybe we should re-iterate it now.
The idea of having an „x“ namespace was rejected in fear we end up with quasi-standard properties that have an x prefix then. 
> * There are a lot of choices being made about how users will want to prioritize transport protocol selection. How confident are we that, for instance, path selection should take priority over proto…

We had a lengthy discussion about this and rejected it as too complex and limiting for the implementation.
> * Using the same enum (Require(d[sic])/Avoid/Ignore) for the queried output of selected properties seems like a shortcut that will lead to some confusion.

We had text saying that these should turn into Boolean when queried and should reflect whether the protocol stack chosen provides the feature. I will have to double-check whether this text is still in the description of the preference type.

> * These type symbols really deserve an actual registry, or at least the start of one. Otherwise, we are likely to end up with a mess.

There is already one, but that one was not useful for our matters. 
> * What about ordering of similar interfaces? I have a 2-SIM cellphone with wifi.

Each cellular provider will have a unique PVD. 
> * The default isn't "true", it's whatever the underlying connection had, right?

Ack
> _______________________________________________
> Taps mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/taps

---

From: Michael Welzl <[email-masked]>
Date: Thu, 5 Mar 2020 08:30:43 +0100

Hi,

A few comments in line:

> On Mar 4, 2020, at 5:51 PM, Philipp S. Tiesel <[email-masked]> wrote:
> 
> Hi,
> 

I remember that. However, the proposal here would be to just replace the property names instead of adding new text.
Let me propose: can we just update them all to CamelCasing?

> We had a lengthy discussion about this and rejected it as too complex and limiting for the implementation.

Right. Maybe we should just remove this item from this list.

>> * The default isn't "true", it's whatever the underlying connection had, right?

I agree too, but doesn’t this warrant some more discussion?
I think our defaults all translate into things that can work in some way when TCP is used - with “Reliable Data Transfer” being an exception, as it just can’t work in a UDP-only system. Are there others?  I think I’ll open an issue for this one.

>> [email-masked] <mailto:[email-masked]>
>> https://www.ietf.org/mailman/listinfo/taps <https://www.ietf.org/mailman/listinfo/taps>

Cheers,
Michael

---

From: Gorry Fairhurst <[email-masked]>
Date: Thu, 5 Mar 2020 07:55:27 +0000

See in-line

On 05/03/2020 07:30, Michael Welzl wrote:
> Hi,
>
> A few comments in line:
>
>> On Mar 4, 2020, at 5:51 PM, Philipp S. Tiesel <[email-masked] 
>> <mailto:[email-masked]>> wrote:
>>
>> Hi,
>>
>> Some of the remaining issues had already been discussed, but either 
>> did not lead to text or the text was lost. Let me comment on these 
>> issues:
>>
>>>> On 4. Mar 2020, at 15:38, Michael Welzl <[email-masked] 
>>>> <mailto:[email-masked]>> wrote:
>>>
>>> * The notation "Sender -> Event" says nothing about who receives the 
>>> event. Is it just assumed to be the application in the context of 
>>> the particular event (e.g., a thread processing a particular 
>>> connection)?
>>>
>>> MW: We have the following statement in this section: "Events could 
>>> be implemented via event queues, handler functions or classes, 
>>> communicating sequential processes, or other asynchronous calling 
>>> conventions."
>>> I wonder why this isn't enough information? E.g., if handlers are 
>>> used, then the recipient of the event is a pre-registered handler - 
>>> and handlers must therefore be registered before events can occur; 
>>> this function isn't visible in the API because this is platform- and 
>>> language-specific.
>>>
>>>
>>> * The use of dashes in property names means they will necessarily 
>>> look different in almost every widely-used language, in which dash 
>>> is an operator.
>>
>> The solution for this issue was to allow implementations to change 
>> the property names in consistent ways, e.g., by replacing the dashes 
>> with underscores or removing them and use CamelCasing.
> I remember that. However, the proposal here would be to just replace 
> the property names instead of adding new text.
I like this idea.
>
>>> * If we're not currently asking IANA to create a registry of 
>>> property names or namespaces, should we provide a recommendation 
>>> that such symbols not listed explcitly in this document be prefixed 
>>> with some experimental identifier?
>>
>> The idea of having an „x“ namespace was rejected in fear we end up 
>> with quasi-standard properties that have an x prefix then.
>>>
>>> * There are a lot of choices being made about how users will want to 
>>> prioritize transport protocol selection. How confident are we that, 
>>> for instance, path selection should take priority over protocol 
>>> selection? I think that's right, but I wonder if it might not make 
>>> more sense to have two interfaces: one that provides a purely 
>>> numeric priority ordering of preferences, and one (based on the 
>>> existing 5-level preferences) that maps into it.
>>
>>> * Using the same enum (Require(d[sic])/Avoid/Ignore) for the queried 
>>> output of selected properties seems like a shortcut that will lead 
>>> to some confusion.
>>
>> We had text saying that these should turn into Boolean when queried 
>> and should reflect whether the protocol stack chosen provides the 
>> feature. I will have to double-check whether this text is still in 
>> the description of the preference type.
>>
>>> * I'll repeat the same statement I made at the mic a few years ago: 
>>> idempotency != replay-safe. DELETE is idempotent, but not safe for 
>>> replay because someone might have done a PUT or POST in the meantime.
>>>
>>> MW: I remember you saying this, sorry that we haven't addressed it 
>>> yet! I didn't understand - is your concern about replaying that a 
>>> Message may arrive later, i.e. as Message X, Messages Y and Z in 
>>> between, then Message X again?  => if so, are you saying that this 
>>> could happen with TFO or QUIC?  (I didn't think so)
>>>
>>>
>>> * These type symbols really deserve an actual registry, or at least 
>>> the start of one. Otherwise, we are likely to end up with a mess.
>>
>>> * It seems like an abstract API would be helpful for the reference 
>>> to the Message to which a Sent event applies: this seems like 
>>> something many, many applications would need to do. With callbacks, 
>>> the application can always curry in a reference to the original 
>>> message (that's what my event system from ~2001 did), so maybe that 
>>> should be the recommendation and the message reference removed...? I 
>>> don't have a strong feeling here other than that if something is 
>>> included then its use should be more well-defined.
>>>
>>> 7.3.3. SendError
>>>
>>> * Is there a reason why a single messageContext object is used for 
>>> both sends and receives? I probably missed the original discussion 
>>> about this, but I'd like to understand the reasoning.
>>>
>>> MW: I don't get this, where does it say that it's only a single 
>>> object? Or do you mean that it's the same type? Why is that problematic?
>>>
>>>
> I think our defaults all translate into things that can work in some 
> way when TCP is used - with “Reliable Data Transfer” being an 
> exception, as it just can’t work in a UDP-only system. Are there 
> others?  I think I’ll open an issue for this one.
>
I don't agree at all - To me this was just the default behaviour of the API.

I think the default needs to be "reliable" and an app will have to 
over-ride that to get datagram/unreliable operation. That doesn't mean 
if you have only UDP (for instance) that you can't offer TAPS, to me it 
simply means you always need to say that the app wants this service.

>>> * "A call to Receive will be paired with a single Receive Event" or 
>>> possibly multiple ReceivedPartial Events?
>>>
>>> * There's a lot of UNIXisms here that should probably be excised in 
>>> favor of a more abstract presentation. The most obvious is the use 
>>> of -1 to mean something other than the integer -1. In Python, for 
>>> instance, you might want to use None for this purpose. I would 
>>> specify non-numeric values symbolically, and maybe indicate in the 
>>> appendix that a UNIX C implementation might want to use -1 to 
>>> represent such values.
>>>
>>> MW: agreed... TODO.
>>>
>>>
I don't much like Traffic Profile - to a traffic profile specifies the 
characteristics of the traffic the sender is transmitting, not the 
treatment that traffic expects from the network service.

I also object to /bandwidth efficiency/ - since sounds like it should be 
measured in b/Hz which isn't at all what we are talking about. In fact, 
I did not see anything in the text about the efficiency or the use of 
bandwith ... as I read, it is all about capacity sharing tradeoffs.
>>> https://www.ietf.org/mailman/listinfo/taps
Best wishes,

Gorry

---

From: Kyle Rose <[email-masked]>
Date: Tue, 10 Mar 2020 10:06:45 -0400

On Wed, Mar 4, 2020 at 9:41 AM Michael Welzl <[email-masked]> wrote:

> event. Is it just assumed to be the application in the context of the
> particular event (e.g., a thread processing a particular connection)?
>
> MW: We have the following statement in this section: "Events could be
> implemented via event queues, handler functions or classes, communicating
> sequential processes, or other asynchronous calling conventions."
> I wonder why this isn't enough information? E.g., if handlers are used,
> then the recipient of the event is a pre-registered handler -  and handlers
> must therefore be registered before events can occur; this function isn't
> visible in the API because this is platform- and language-specific.
>

After some thought, I think I'm okay with this now. It feels like one of
those things that would make perfect sense if I were actually implementing
a TAPS interface.

>  * The use of dashes in property names means they will necessarily look
> different in almost every widely-used language, in which dash is an
> operator.
>

Following on to the discussion later in the thread, CamelCase would
probably be an improvement from this perspective. Underscores would also be
fine.

>   * If we're not currently asking IANA to create a registry of property
> names or namespaces, should we provide a recommendation that such symbols
> not listed explcitly in this document be prefixed with some experimental
> identifier?
>

Can't remember where I saw objection that this creates the possibility of a
bunch of long-lived X-* properties. To that I respond: which is worse? A
complete free-for-all, or a recommendation to have the free-for-all
confined to an X- namespace? Do properties starting with X- just offend
engineering sensibilities?

One "right" alternative is a documentation-required registry, but does that
raise the bar too high? How about a TAPS wiki as a compromise? The
important outcome is that properties be well-defined and not conflict with
each other. Since these properties have entirely local effects (e.g., they
don't get encoded as options and transmitted to the peer), we probably
don't need the level of precision motivating option registries, for
instance. Is there precedent at the IETF for a registry mechanism less
formal that what IANA provides?

> prioritize transport protocol selection. How confident are we that, for
> instance, path selection should take priority over protocol selection? I
> think that's right, but I wonder if it might not make more sense to have
> two interfaces: one that provides a purely numeric priority ordering of
> preferences, and one (based on the existing 5-level preferences) that maps
> into it.
>

I'm okay with whoever said that this is unworkable. I'm sure I missed the
original discussion.

> output of selected properties seems like a shortcut that will lead to some
> confusion.
>

FWIW, my objection to the other issue (using a normative term as an
indication of what was chosen) isn't that it's underspecified, only that
it's confusing. I put it roughly in the same category as specifying -1 as
an error indicator in an abstract API. Is the idea that the output
parameters could be fed into the protocol selection logic for a subsequent
connection to select the same protocol? Is that useful?

> idempotency != replay-safe. DELETE is idempotent, but not safe for replay
> because someone might have done a PUT or POST in the meantime.
>
> MW: I remember you saying this, sorry that we haven't addressed it yet! I
> didn't understand - is your concern about replaying that a Message may
> arrive later, i.e. as Message X, Messages Y and Z in between, then Message
> X again?  => if so, are you saying that this could happen with TFO or
> QUIC?  (I didn't think so)
>

What I mean is maybe best illustrated by 0RTT in TLS or QUIC. If a user
sends a 0RTT DELETE /object, followed by a PUT /object, then within the
replay window and without any replay prevention at the application layer,
an attacker could replay the 0RTT DELETE /object and cause the object to be
deleted again. DELETE is idempotent but it's not safe to replay.

7.3.3. SendError
>
>  * Is there a reason why a single messageContext object is used for both
> sends and receives? I probably missed the original discussion about this,
> but I'd like to understand the reasoning.
>
> MW: I don't get this, where does it say that it's only a single object? Or
> do you mean that it's the same type? Why is that problematic?
>

Same type. (Maybe I should have said "class" instead of "object".) For
example, the "lifetime" property makes sense as metadata for sends, but is
completely irrelevant to the receiver. The same is true of several other
message properties. It might make sense for an implementation to use a
single type to represent metadata both for sends and receives, but is that
also true for an abstract API?

> contradict what you say here.

"Once a Connection is established, it can be used for receiving data." Not
true if it's a unidirectional connection.

I'll take a look at the subsequent replies and respond at the end of the
thread to anything that I didn't cover here.