Subject: Re: [tcpm] draft-ietf-tcpm-urgent-data-06

From: Lars Eggert <[email-masked]>
Date: Tue, 5 Oct 2010 10:33:00 +0200

Hi, folks,

where are we with resolving the DISCUSSes on this document? 

Thanks,
Lars

---

From: Ronald Bonica <[email-masked]>
Date: Tue, 5 Oct 2010 10:17:28 -0400

I cleared, assuming that Jari would hold my discuss.

                               Ron

> -----Original Message-----
> From: Lars Eggert [mailto:[email-masked]]
> Sent: Tuesday, October 05, 2010 4:33 AM
> To: Ronald Bonica; Jari Arkko
> Cc: [email-masked]; [email-masked]
> Extensions
> Subject: Re: [tcpm] draft-ietf-tcpm-urgent-data-06
> 
> Hi, folks,
>

---

From: Lars Eggert <[email-masked]>
Date: Mon, 11 Oct 2010 11:17:46 +0300

Hi,

> I cleared, assuming that Jari would hold my discuss.

thanks, Ron.

Editors, chairs, WG: are we clear on what needs to happen to the ID to address the discuss? When will we see a revision?

Thanks,
Lars

---

From: Fernando Gont <[email-masked]>
Date: Mon, 11 Oct 2010 11:27:27 -0300

Hi, Lars,

Thanks so much for following this one closely. Comments inline....

On 11/10/2010 05:17 a.m., Lars Eggert wrote:
> Editors, chairs, WG: are we clear on what needs to happen to the ID
> to address the discuss? When will we see a revision?

It's clear that there are two different positions on this DISCUSS:
David's, and Jari/Ron's (probably shared by Joe, too). But it's not
clear to me whether I should simply modify the document as indicated by
Jari/Ron, or whether there's a middle-ground that could possibly make
both parties happy (I'm assuming that sticking to David's position is
not an option).

(FWIW, me, I agree with David's position... but for obvious reasons I
might be biased... hence my request for advice on how to proceed).

Thoughts?

Thanks!

Kind regards,
-- 
Fernando Gont
e-mail: [email-masked] || [email-masked]
PGP Fingerprint: 7809 84F5 322E 45C7 F1C9 3945 96EE A9EF D076 FFF1

---

From: Lars Eggert <[email-masked]>
Date: Mon, 11 Oct 2010 18:51:25 +0300

Hi,

> both parties happy (I'm assuming that sticking to David's position is

if you can find a middle ground that addresses the discusses, that would obviously also be a way forward. Please suggest one.

> might be biased... hence my request for advice on how to proceed).

I'm mostly agreeing with David as well, but a significant fraction of the IESG does not. 

Lars

---

From: "Eddy, Wesley M. (GRC-MS00)[ASRC AEROSPACE CORP]" <[email-masked]>
Date: Mon, 11 Oct 2010 19:24:40 -0500

For what it's worth, I also agree with the position that David presented.  I don't think it's necessarily worth spending a lot of time and energy to debate, though.

________________________________________
From: [email-masked] [[email-masked]] On Behalf Of Lars Eggert [[email-masked]]
Sent: Monday, October 11, 2010 11:51 AM
To: Fernando Gont
Cc: [email-masked]; [email-masked] Extensions
Subject: Re: [tcpm] draft-ietf-tcpm-urgent-data-06

...

Lars

---

From: Joe Touch <[email-masked]>
Date: Mon, 11 Oct 2010 19:49:31 -0700

Hi, all,

I can't speak for the IESG, but I can explain my reasoning:

The doc presents two pieces of info:

- update the spec
- change use to SHOULD NOT

Given we don't have apps breaking all over the place, the second one 
seems to be more dramatic in its impact to the community. The document 
presents the second one in an almost ancillary way.

IMO, if this doc is really about updating the spec, and we don't really 
have issue with new uses of the feature, then the SHOULD NOT should be 
dropped.

If the SHOULD NOT is kept, I can't see how it's not more prominent (and 
thus warrants doc restructuring as suggested).

Joe

On 10/11/2010 5:24 PM, Eddy, Wesley M. (GRC-MS00)[ASRC AEROSPACE CORP] 
wrote:
> _______________________________________________
> tcpm mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/tcpm

---

From: David Borman <[email-masked]>
Date: Tue, 12 Oct 2010 09:58:49 -0500

On Oct 11, 2010, at 9:49 PM, Joe Touch wrote:

> Given we don't have apps breaking all over the place, the second one seems to be more dramatic in its impact to the community. The document presents the second one in an almost ancillary way.
> 
> IMO, if this doc is really about updating the spec, and we don't really have issue with new uses of the feature, then the SHOULD NOT should be dropped.

From my viewpoint, the issue is to 1) make application developers aware of the problems cause by the potential of OS's having different definitions of the urgent pointer, and 2) tell them NOT to use the broken bsd socket() OOB mechanism, by setting SO_OOBINLINE.  The goal isn't to say that applications can't use the Urgent pointer, but to warn of the potential issues in a generic environment.

As you said, we don't have applications breaking all over the place.  Also, the goal of this document isn't to change all the current Urgent pointer implementations, but to match the spec to the majority of implementations.

The WG did not put forth this document to deprecate the Urgent pointer, but to clarify it and warn of the interoperability issues.

So perhaps the "SHOULD NOT" should be changed to a "MAY", meaning you can still use it, but be aware of the issues and make sure you design around them.  It is entirely possible to reliably make use of the urgent pointer in an application, even with different OS definitions.  The application can the test urgent pointer at the beginning of the connection by sending a known piece of data and observing how it gets delivered to the receiver, the application can then adjust accordingly and use it reliably.

Perhaps combine section 5 & 6 into a single section like:

5.  Advice on using and implementing the TCP urgent mechanism

  As a result of the issues discussed in Section 3.2 and Section 3.4,
  using the TCP urgent mechanism has the potential to be unreliable.
  New applications MAY use the TCP urgent mechanism, but should be
  robust against these issues.  In addition, applications MUST set
  the SO_OOBINLINE socket option, such that "urgent data" is delivered
  inline, as intended by the IETF specifications.

  TCP implementations MUST include support for the urgent mechanism
  such that existing applications can still use it.

			-David Borman

> 
> If the SHOULD NOT is kept, I can't see how it's not more prominent (and thus warrants doc restructuring as suggested).
> 
> Joe
> 
> On 10/11/2010 5:24 PM, Eddy, Wesley M. (GRC-MS00)[ASRC AEROSPACE CORP] wrote:

---

From: Joe Touch <[email-masked]>
Date: Tue, 12 Oct 2010 09:12:34 -0700

Hi, Dave,

On 10/12/2010 7:58 AM, David Borman wrote:

> As you said, we don't have applications breaking all over the place.
> Also, the goal of this document isn't to change all the current Urgent
> pointer implementations, but to match the spec to the majority of
> implementations.
>
> The WG did not put forth this document to deprecate the Urgent
> pointer, but to clarify it and warn of the interoperability issues.
>
> So perhaps the "SHOULD NOT" should be changed to a "MAY", meaning
> you can still use it, but be aware of the issues and make sure you
> design around them.

It's sufficient to delete Sec 5 then. It's already a "MAY" for users in 
RFC793 (though 2119 language isn't used, the intent is there). We 
definitely should not restate requirements already sufficiently stated.

> It is entirely possible to reliably make use of
> the urgent pointer in an application, even with different OS
> definitions. The application can the test urgent pointer at the
> beginning of the connection by sending a known piece of data and
> observing how it gets delivered to the receiver, the application can
> then adjust accordingly and use it reliably.

That's not in the doc anywhere. No current app I've ever heard of does 
this, and it doesn't seem practical for most apps (esp. since it's not 
backward compatible; the test data would appear in the data stream). I'm 
not sure this is a useful consideration for this discussion at all.

> Perhaps combine section 5& 6 into a single section like:
>    robust against these issues.

The above simply isn't required; it's already part of RFC793.

>    In addition, applications MUST set

SO_OOBINLINE and its behavior isn't defined in RFC793.

The fact that urgent data is inline needs to be defined and explained. 
Then the requirement stated that urgent pointer data is inline, as is 
all TCP data.

It may be useful to note that the way to do this in some implementations 
is to set this socket option, and that this particular option should 
default to ON (and note that it isn't, so apps should set it), but 2119 
language isn't appropriate. The discussion shouldn't use 2119 language 
in reference to an implementation of the API that isn't required.

>    such that existing applications can still use it.

That's not necessary either; it's in 793, and it's already required 
therein. Again, I don't think this doc should restate existing requirements.

Joe

---

From: Joe Touch <[email-masked]>
Date: Tue, 12 Oct 2010 09:33:18 -0700

PS - to summarize my suggestions to wrap this up:

- delete section 5

- revise section 6 to explain what inline vs. out-of-band data is, and 
remind users that urgent is NOT an out-of-band signal -- that the data 
indicated by the urgent pointer MUST NOT be extricated from the data 
byte stream presented to the user via means indicated in RFC793 (e.g., 
read calls). This would also note that in some implementations this is 
requires overriding the default, and setting SO_OOBINLINE=1.

Joe

---

From: Jari Arkko <[email-masked]>
Date: Sat, 16 Oct 2010 00:04:15 +0300

All,

We discussed this yesterday in the IESG conference call. I would like to 
summarize my thinking about it and suggest a way forward. (There are a 
few other ADs who have similar but perhaps not exactly the same 
thoughts, but I'm the one who representing these thoughts with the Discuss.)

The main concern is that at least in my view, urgent data seems pretty 
badly broken. It has potential interop problems if out of band mode is 
used, mismatch of recommendations wrt real world implementations that 
mostly use out of band mode as the default, etc. As far as I understand, 
when out of band mode is used the two end points have to agree about the 
semantics of the urgent pointer. It is reassuring that most 
implementations that you tested actually use the same semantics, but we 
an implementation that we do not know about or a middlebox could disrupt 
an application's understanding of the correct order and place of TCP 
data from the other end. With this background, I felt that the document 
stopped short of making clear enough recommendations about the use of 
this feature. In the IESG we talked about what the right level of 
"health warning" should be, and there were different opinions ranging 
from text at the beginning of the document to formal deprecation of the 
feature.

I have a proposal below that takes the warning text approach. Note that 
I'm not really changing anything substantive from what the document was 
already saying, but if we publish a new RFC on urgent indications, it 
seems reasonable to make it very clear that there are issues. If these 
changes were made to the document I would be happy to clear my Discuss, 
though I do know that Ron would probably would like to see an even more 
drastic action. What does the working group think?

Jari

Change Abstract:

OLD:
   This document analyzes how current TCP implementations process TCP
   urgent indications, and how the behavior of some widely-deployed
   middle-boxes affect how urgent indications are processed by end
   systems.  This document updates the relevant specifications such that
   they accommodate current practice in processing TCP urgent
   indications, provides advice to applications that make use of the
   urgent mechanism, and raises awareness about the reliability of TCP
   urgent indications in the current Internet.
NEW:
   This document analyzes how current TCP implementations process TCP
   urgent indications, and how the behavior of some widely-deployed
   middle-boxes affect how urgent indications are processed by end
   systems. This document updates the relevant specifications such that
   they accommodate current practice in processing TCP urgent
   indications, raises awareness about the reliability of TCP
   urgent indications in the Internet and recommends against the use of the
   urgent indications (but provides advice to applications in case that 
they do).

Change Introduction:

OLD:
   This document analyzes how some current TCP implementations process
   TCP urgent indications, and how the behavior of some widely-deployed
   middle-boxes affect the processing of urgent indications by hosts.
   This document updates RFC 793 [RFC0793], RFC 1011 [RFC1011], and RFC
   1122 [RFC1122] such that they accommodate current practice in
   processing TCP urgent indications, provides advice to applications
   using the urgent mechanism, and raises awareness about the
   reliability of TCP urgent indications in the current Internet.
NEW:
   This document analyzes how some current TCP implementations process
   TCP urgent indications, and how the behavior of some widely-deployed
   middle-boxes affect the processing of urgent indications by hosts.
   This document updates RFC 793 [RFC0793], RFC 1011 [RFC1011], and RFC
   1122 [RFC1122] such that they accommodate current practice in
   processing TCP urgent indications, provides advice to applications
   using the urgent mechanism, and raises awareness about the
   reliability of TCP urgent indications in the current Internet.

   Given the above issues and potential interoperability issues with 
respect to the
   currently common default mode operation, it is strongly recommended that
   applications do not employ urgent indications. Nevertheless, urgent 
indications
   are still retained as a mandatory part of the TCP protocol to support the
   few legacy applications that employ them. However, it is expected 
that even
   these applications will have difficulties in environments with 
middle-boxes.

---

From: Ronald Bonica <[email-masked]>
Date: Fri, 15 Oct 2010 17:07:25 -0400

Looks good to me.

> From: [email-masked] [mailto:[email-masked]] On Behalf Of
> Jari Arkko
> Sent: Friday, October 15, 2010 5:04 PM
> To: [email-masked] Extensions
> Cc: David Borman; Fernando Gont; Joe Touch

---

From: Peter Saint-Andre <[email-masked]>
Date: Fri, 15 Oct 2010 15:12:52 -0600

+1 -- that seems to be an accurate summary of what folks think this
document is saying or should be saying.

On 10/15/10 3:04 PM, Jari Arkko wrote:
>   few legacy applications that employ them. However, it is expected that
> even

---

From: David Borman <[email-masked]>
Date: Fri, 15 Oct 2010 16:41:28 -0500

Jari,
This seems to be a reasonable compromise for the differing points of view, and a way to get past this issue.  Thanks.
			-David

On Oct 15, 2010, at 4:04 PM, Jari Arkko wrote:

> All,
> 
> We discussed this yesterday in the IESG conference call. I would like to summarize my thinking about it and suggest a way forward. (There are a few other ADs who have similar but perhaps not exactly the same thoughts, but I'm the one who representing these thoughts with the Discuss.)
> 
> The main concern is that at least in my view, urgent data seems pretty badly broken. It has potential interop problems if out of band mode is used, mismatch of recommendations wrt real world implementations that mostly use out of band mode as the default, etc. As far as I understand, when out of band mode is used the two end points have to agree about the semantics of the urgent pointer. It is reassuring that most implementations that you tested actually use the same semantics, but we an implementation that we do not know about or a middlebox could disrupt an application's understanding of the correct order and place of TCP data from the other end. With this background, I felt that the document stopped short of making clear enough recommendations about the use of this feature. In the IESG we talked about what the right level of "health warning" should be, and there were different opinions ranging from text at the beginning of the document to formal deprecation of the feature.
> 
> I have a proposal below that takes the warning text approach. Note that I'm not really changing anything substantive from what the document was already saying, but if we publish a new RFC on urgent indications, it seems reasonable to make it very clear that there are issues. If these changes were made to the document I would be happy to clear my Discuss, though I do know that Ron would probably would like to see an even more drastic action. What does the working group think?
> 
> Jari
> 
> Change Abstract:
> 
> OLD:
>  urgent indications (but provides advice to applications in case that they do).
> 
> Change Introduction:
> 
> OLD:
>  Given the above issues and potential interoperability issues with respect to the
>  applications do not employ urgent indications. Nevertheless, urgent indications
>  few legacy applications that employ them. However, it is expected that even
>  these applications will have difficulties in environments with middle-boxes.
>

---

From: Joe Touch <[email-masked]>
Date: Fri, 15 Oct 2010 14:48:51 -0700

Hi, all,

FWIW, this skirts the key questions:

1) should this doc include any particular 2119 language on the urgent 
option?

	- IMO, it should not repeat or restate existing requirements
	- based on the note from Jari, I can't tell whether the
	doc is recommending SHOULD NOT for new apps or not

	if it is, then the doc should definitely be revised to
	lead with that issue; if not, then such revision isn't needed

2) how should this doc address the in/out of band issue?
Right now, it explains it solely by reference to the implementation of a 
Unix API. That doesn't seem sufficient, as per the note I already posted.

It'd be useful to understand what the note below says to these two points.

Joe

---

From: Jari Arkko <[email-masked]>
Date: Sat, 16 Oct 2010 01:00:01 +0300

Joe,

>     doc is recommending SHOULD NOT for new apps or not

My text did not change that aspect in any way. The new text is in the 
introduction and has no keywords. The formal requirement is later in the 
document. IIRC the current text there says SHOULD NOT for new apps.

Jari

---

From: Joe Touch <[email-masked]>
Date: Tue, 19 Oct 2010 15:10:01 -0700

> document. IIRC the current text there says SHOULD NOT for new apps.

Correct. In that case, the note was not responsive to the thread I 
raised discussing the point that this is a new and substantial change, 
one that I feel suggests that the title, abstract, and overall structure 
should be inverted to focus on SHOULD NOT as the key point.

It's hard to understand a doc that basically focuses on a change to the 
spec, and includes the SHOULD NOT almost parenthetically.

Joe

---

From: Jari Arkko <[email-masked]>
Date: Wed, 20 Oct 2010 01:14:43 +0300

Joe, I think I agree with you. I was not planning on holding on to a 
Discuss for that part, however.

Jari

Joe Touch kirjoitti:
> one that I feel suggests that the title, abstract, and overall 
> structure should be inverted to focus on SHOULD NOT as the key point.
>
> It's hard to understand a doc that basically focuses on a change to 
> the spec, and includes the SHOULD NOT almost parenthetically.
>
> Joe
>

---

From: Joe Touch <[email-masked]>
Date: Tue, 19 Oct 2010 15:22:19 -0700

Hmm. That was the issue that the discuss I thought that Ron was holding 
was focused on (see attached)

Ron?

Joe

>> the spec, and includes the SHOULD NOT almost parenthetically.

Subject: RE: [tcpm] draft-ietf-tcpm-urgent-data-06
From: Ronald Bonica <[email-masked]>
To: Joe Touch <[email-masked]>, John Heffner <[email-masked]>
Cc: "[email-masked]" <[email-masked]>
Date: Mon, 27 Sep 2010 15:49:05 -0400

Joe,

I think that you have hit the nail on the head. Reading through Sections 1-5, I thought that we were trying to fix the feature. Only in Section 6 did I realize that we really wanted to drive a stake through its heart.

Maybe if we changed the title and added some test to the abstract and introduction, this would be better. Otherwise, people may spend time and money fixing the TCP Urgent option instead of just avoiding it.

> From: Joe Touch [mailto:[email-masked]]
> Sent: Thursday, September 23, 2010 10:18 PM
> To: John Heffner
> Cc: Ronald Bonica; [email-masked]
> On 9/23/2010 6:00 PM, John Heffner wrote:
> > My understanding is that deprecating urgent data is exactly what this
> > document is doing.  In section 5: "... new applications SHOULD NOT
> > employ the TCP urgent mechanism."  The rest of the document is
> largely
> > a justification for deprecating the mechanism.
> 
> And a clarification as to what byte the urgent data actually starts at.
> The spec doesn't match any of the implementations, and since there's no
> benefit to either variant, the WG decided to change the spec to match
> the extant implementations. See section 4.
> 
> This is NOT an enhancement, and is NOT intended to make future apps
> more
> likely to use the urgent mechanism. See section 6, which deprecates the
> feature.
> 
> Is there a question that, e.g., the title be updated to reflect the
> primary intent, i.e.,"Deprecating TCP's Urgent Mechanism", with a few
> other related changes (e.g., update the abstract/intro, move section 6
> forward, and then explain that the remainder of the doc is to document
> how existing apps work?)?
> 
> I could see how the current doc reads more like "we're updating the
> spec
> first. oh yeah, and don't use this", which seems inverted; the WG
> conclusion was more like "deprecate this. and oh yeah, here's how it
> really works in the few apps that we know of that actually use it"
> 
> Joe
> 
> >
> >    -John
> >
> >
> > On Thu, Sep 23, 2010 at 5:10 PM, Ronald Bonica<[email-masked]>
> wrote:
> >> Folks,
> >>
> >> The purpose of this email is to encourage the TCPM working group to
> deprecate the feature described in draft-ietf-tcpm-urgent-data-06.
> >>
> >> I understand the following statements to be true:
> >>
> >> - The WG discourages new applications from using the feature.
> >> - Many middle-boxes defeat the feature by removing the urgent data
> indicator. Therefore, when an application indicates that data is
> urgent, it is never sure that that indication will reach the remote
> TCP.
> >> - Only three applications have been identified as using the feature.
> These are a proprietary application, telnet and ftp. More elegant
> mechanisms are available to Telnet and FTP.
> >> - The feature is rarely implemented as specified.
> >>
> >> Finally, there are more elegant ways to bring urgent data to
> applications.
> >>
> >> IMHO, this feature should be deprecated, not enhanced. Let's not
> pass up this opportunity to simplify the TCP stack.
> >>
> >>                                                            Ron
> >>

---

From: Ronald Bonica <[email-masked]>
Date: Tue, 19 Oct 2010 20:45:58 -0400

I cleared, letting Jari hold the DISCUSS.

             Ron

> Sent: Tuesday, October 19, 2010 6:22 PM
> To: Jari Arkko
> Cc: David Borman; [email-masked] Extensions; Fernando Gont
> >>>> - based on the note from Jari, I can't tell whether the doc is
> >>>> recommending SHOULD NOT for new apps or not
> >>>
> >>> My text did not change that aspect in any way. The new text is in
> >>> the introduction and has no keywords. The formal requirement is
> >>> later in the document. IIRC the current text there says SHOULD NOT
> for new apps.
> >>
> >> raised discussing the point that this is a new and substantial
> >> change, one that I feel suggests that the title, abstract, and
> >> overall structure should be inverted to focus on SHOULD NOT as the
> key point.
> >>

---

From: David Borman <[email-masked]>
Date: Tue, 19 Oct 2010 21:18:48 -0500

On Oct 19, 2010, at 5:10 PM, Joe Touch wrote:

> Correct. In that case, the note was not responsive to the thread I raised discussing the point that this is a new and substantial change, one that I feel suggests that the title, abstract, and overall structure should be inverted to focus on SHOULD NOT as the key point.
> 
> It's hard to understand a doc that basically focuses on a change to the spec, and includes the SHOULD NOT almost parenthetically.

We're not trying to deprecate the URGENT pointer.  The goal is to first align the spec with the implementations, and then point out issues with the URGENT pointer, which is how the doc is currently structured.  I don't want the "SHOULD NOT" to be the main point of this document, it is a secondary point due to the issues listed.  The SHOULD NOT applies to new applications, and it means "If you really want to make use of the URGENT pointer, make sure you understand and have addressed all the issues pointed out in this document."

The SHOULD NOT for application writers using the URGENT pointer, not for the URGENT pointer support in the TCP implementations.  I have no objections to the additional highlighting of this, but that doesn't make it the key point of the document.  I say leave the document as it is.

			-David Borman

---

From: Joe Touch <[email-masked]>
Date: Tue, 19 Oct 2010 21:43:10 -0700

Hi, Dave,

One additional point (while I wait to see if Ron agrees with your 
summary); the current discussion of in-line data still needs to be 
revised to avoid relying primarily on the definition of a particular 
interface (SO_OOBINLINE).

As to your point, IMO *every* aspect of a protocol is SHOULD NOT by your 
definition, i.e., every capability should not be used unless its 
implications, side-effects, and nuances are understood.

To me, SHOULD NOT on an existing, required protocol capability 
effectively relegates it to "deprecated", i.e., there to support legacy 
apps. There's no point in a MUST NOT on such a capability - MUST NOT 
would mistakenly suggest that the capability need not be implemented 
anymore.

Joe

---

From: Jari Arkko <[email-masked]>
Date: Wed, 20 Oct 2010 09:17:06 +0300

Joe,

Yes, there were several concerned ADs. After a discussion on the list 
and in the IESG it was my opinion that the changes that I suggested were 
a sufficient compromise to make the document acceptable. I would 
personally (and Ron too) wanted to go further, indeed make the SHOULD 
NOT a major point in the document. But I chose not to make that a part 
of a blocking comment. It is of course always tricky when one AD is 
representing concerns from others as well as himself. It was my 
interpretation that what we did is an acceptable way forward, but it 
could of course happen that Ron or others feel strongly and want to 
Discuss further.

Jari