Subject: [Taps] RFC 6458 etc. in draft-welzl-taps-transports

From: Michael Welzl <[email-masked]>
Date: Tue, 3 Nov 2015 13:40:37 +0100

Dear all,

Sorry for not being able to attend the TAPS meeting on site or even remotely. I just finished watching the recording, and I noticed that the question of RFC 6458  - "why is the SCTP part of draft-welzl- ..  based on only RFC 4960 and not on RFC 6458?" - was brought up several times. I'd like to provide an answer and start a discussion about this.

There are two reasons why RFC 6458 wasn't used in draft-welzl-taps-transports-00: a very mundane one, and a more serious one. I'll list them both and hope we can discuss the second reason.

1) Reason one: RFC 6458 is quite long, and I wanted to limit the amount of work I'm putting into the -00 version, given that the point was to show people the procedure and the idea and see what they think, and not to fully cover everything 100% correctly yet. Basically, I didn't want to risk writing out all the stuff from RFC 6458 and then have people tell me to go away  :-)

2) Reason two, more serious: RFC 6458 is Informational, and my understanding was that this is just one description of one API implementation, not a recommendation or even prescription of what all SCTP implementations must provide. However, my decision for draft-welzl-taps-transports was to *exclude* things that are only optional to implement - I don't think we want to end up with a TAPS system that provides services that, alas, on Operating System XY are not available because here only a subset of the API is implemented. Therefore I went with a minimal set of functions that I thought we can assume are implemented everywhere (well, at least in every system that claims to "follow the spec").  Can we assume that every system that claims to implement SCTP in accordance with the spec fully implements RFC 6458?

A side note about TCP, because Karen made a comment about the TCP API too:  a similar logic applies here, irrespective of whether the API is old or not: I think we should cover whatever a system claiming to "implement the protocol in accordance with the spec" has to cover. Going down the path of looking at actual socket API implementations is dangerous in that we end up in "only implemented here, not there" - land. We want a minimal set of mechanisms that are (or at least really should be! for that, what else can we use as a basis than our own recommendations?!) available everywhere. Karen, you specifically mentioned URG and PSH and how they are implemented; what is it in draft-welzl-.. about these two mechanisms that you don't agree with?

Cheers,
Michael

---

From: [email-masked]
Date: Tue, 3 Nov 2015 13:33:04 -0000

A note just on the INFO status of SCTP APIs (below):
>
> Dear all,
>
> Sorry for not being able to attend the TAPS meeting on site or even
> remotely. I just finished watching the recording, and I noticed that the
> question of RFC 6458  - "why is the SCTP part of draft-welzl- ..  based on
> only RFC 4960 and not on RFC 6458?" - was brought up several times. I'd
> like to provide an answer and start a discussion about this.
>
> There are two reasons why RFC 6458 wasn't used in
> draft-welzl-taps-transports-00: a very mundane one, and a more serious
> one. I'll list them both and hope we can discuss the second reason.
>
> 1) Reason one: RFC 6458 is quite long, and I wanted to limit the amount of
> work I'm putting into the -00 version, given that the point was to show
> people the procedure and the idea and see what they think, and not to
> fully cover everything 100% correctly yet. Basically, I didn't want to
> risk writing out all the stuff from RFC 6458 and then have people tell me
> to go away  :-)
>
> 2) Reason two, more serious: RFC 6458 is Informational, and my
> understanding was that this is just one description of one API
> implementation, not a recommendation or even prescription of what all SCTP
> implementations must provide. However, my decision for
> draft-welzl-taps-transports was to *exclude* things that are only optional
> to implement - I don't think we want to end up with a TAPS system that
> provides services that, alas, on Operating System XY are not available
> because here only a subset of the API is implemented. Therefore I went
> with a minimal set of functions that I thought we can assume are
> implemented everywhere (well, at least in every system that claims to
> "follow the spec").  Can we assume that every system that claims to
> implement SCTP in accordance with the spec fully implements RFC 6458?
>
GF: From a TSVWG Chair perspective, beware here...  *ALL* more recent IETF
SCTP API work from TSVWG is INFO.  Each SCTP RFC is expected to have an
informative section that describes the API together with the normative
protocol spec. That is not because there are expected to be alternatives
to choose from:  It's because, as I understand, the IETF is not the formal
standards body for specification of such normative APIs.
>
>
> A side note about TCP, because Karen made a comment about the TCP API too:
>  a similar logic applies here, irrespective of whether the API is old or
> not: I think we should cover whatever a system claiming to "implement the
> protocol in accordance with the spec" has to cover. Going down the path of
> looking at actual socket API implementations is dangerous in that we end
> up in "only implemented here, not there" - land. We want a minimal set of
> mechanisms that are (or at least really should be! for that, what else can
> we use as a basis than our own recommendations?!) available everywhere.
> Karen, you specifically mentioned URG and PSH and how they are
> implemented; what is it in draft-welzl-.. about these two mechanisms that
> you don't agree with?
>
> Cheers,
> Michael
>
> _______________________________________________
> Taps mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/taps
>
Gorry

---

From: Joe Touch <[email-masked]>
Date: Tue, 3 Nov 2015 09:44:18 -0800

> standards body for specification of such normative APIs.

That has been a serious misinterpretation of how a protocol definition
works, which the IETF has propagated over the years.

The abstract APIs - above and below - of a protocol are a key part of a
protocol specification. More directly, a protocol definition is a FSM
that consists of:

	- a finite state machine
	- upper layer events (in/out)
		i.e., the upper layer abstract API
		the services that a protocol "creates"
	- lower layer events (out/in)
		i.e., the services on which the protocol relies
	- time events
	- rules that relate the items above

The way in which an abstract API is implemented as Unix sockets might be
informative to the IETF (but not, e.g., to the POSIX community), but the
abstract API cannot be. It has to be a normative part of the definition
of the protocol.

Otherwise, you end up with a protocol with no upper layer events or
actions, i.e., a tree falling in the woods with nobody to hear.

Joe

---

From: Karen Elisabeth Egede Nielsen <[email-masked]>
Date: Wed, 4 Nov 2015 10:27:58 +0900

HI,

As a general comment then I believe that when describing what is supported
by TCP/SCTP (or UDP) as standard then it does not suffice to look into
IETF RFCs.
One need at least to relate to the *basic functions* of the POSIX/Berkeley
socket api standard.

My understanding of the TCP API, for example, is that while RFC793 did
specify an abstract API for TCP, then
the true defacto standard for the socket api is the Berkeley socket api
from .?. around 1990.
Not saying the different socket api standards don't differ and that there
is *one* standard to look at.
But making is be RFC793 rather then what emerged as defacto in the 1990's
seems a "bit odd" to me.

Especially since we have RFCs RFC1122 dating back to 1989 already
clarifying part of RFC793 (namely the PUSH bit)
And one, much more recent, RFC6093, clarifying the urgent bit.

Please see below.

> -----Original Message-----
> From: Taps [mailto:[email-masked]] On Behalf Of
> [email-masked]
> Sent: 3. november 2015 22:33
> To: Michael Welzl <[email-masked]>
> Cc: taps WG <[email-masked]>
> Subject: Re: [Taps] RFC 6458 etc. in draft-welzl-taps-transports
>
> > remotely. I just finished watching the recording, and I noticed that
> > the question of RFC 6458  - "why is the SCTP part of draft-welzl- ..
> > based on only RFC 4960 and not on RFC 6458?" - was brought up several
> > times. I'd like to provide an answer and start a discussion about
this.
> > 1) Reason one: RFC 6458 is quite long, and I wanted to limit the
> > amount of work I'm putting into the -00 version, given that the point
> > was to show people the procedure and the idea and see what they think,
> > and not to fully cover everything 100% correctly yet. Basically, I
> > didn't want to risk writing out all the stuff from RFC 6458 and then
> > have people tell me to go away  :-)
> >
[Karen Elisabeth Egede Nielsen] I have no problem with this approach. Only
now that you're
in, I would like you to look at RFC6458.

> > implementation, not a recommendation or even prescription of what all
> > SCTP implementations must provide. However, my decision for
> > draft-welzl-taps-transports was to *exclude* things that are only
> > optional to implement - I don't think we want to end up with a TAPS
> > system that provides services that, alas, on Operating System XY are
> > not available because here only a subset of the API is implemented.
> > Therefore I went with a minimal set of functions that I thought we can
> > assume are implemented everywhere (well, at least in every system that
> > claims to "follow the spec").  Can we assume that every system that
> > claims to implement SCTP in accordance with the spec fully implements
RFC
> 6458?
> >
> GF: From a TSVWG Chair perspective, beware here...  *ALL* more recent
> IETF SCTP API work from TSVWG is INFO.  Each SCTP RFC is expected to
have
> an informative section that describes the API together with the
normative
> protocol spec. That is not because there are expected to be alternatives
to
> choose from:  It's because, as I understand, the IETF is not the formal
[Karen Elisabeth Egede Nielsen] Not only that. RFC6458 was around as a
tsvwg  wg draft
long before RFC4960 work was started. My understanding is that while going
from RFC2960 to RFC4960 one did
not bother "too much" with updating the API section of RFC2960 -->
RFC4960, as the finer API details were being
worked on in this wg draft.

However I was not personally involved in IETF SCTP back then, so I like to
ask others to confirm this.

Something else:

The implementation that I have worked with was initiated in the early 2000
and at that time we
looked for RFC2960bis (RFC4960) for protocol specification and the socket
api for api features.
Yes there are particulars of the RFC6458 which represent implementations
aspects. For example
our API implementation implements certain calls in the format they were
specified in ver 14/15 of the socket api from
draft from around 2007  and not the exact form the calls have in RFC6458
(version 30-some of the draft) from 2011,
but the features available as controllable has not changed much during the
years. And it is the features available,
not the implementation aspects that we are focused on in Taps.

NB:
But I have to admit that in our SCTP API implementation we implements the
basic POSIX api functions of TCP/IP for SCTP simply
when they makes as much sense for SCTP as they do for TCP/IP. For example
DSCP is settable via a socket option per association,
even if I think this cannot be found in RFC6458 nor in RFC4960 (which is a
bit of a surprise to me). So my view
on the basic API functions of SCTP (as for TCP) is influenced by this.
Which I have to recognize.

> > not: I think we should cover whatever a system claiming to "implement
> > the protocol in accordance with the spec" has to cover. Going down the
> > path of looking at actual socket API implementations is dangerous in
> > that we end up in "only implemented here, not there" - land. We want a
> > minimal set of mechanisms that are (or at least really should be! for
> > that, what else can we use as a basis than our own recommendations?!)
> available everywhere.

[Karen Elisabeth Egede Nielsen] Yes I agree completely ! The difficult
exercise is where from
to gather this minimal set of mechanisms in a qualified manner.

> > implemented; what is it in draft-welzl-.. about these two mechanisms
> > that you don't agree with?
> >
[Karen Elisabeth Egede Nielsen]

The PUSH bit as settable by application was recognized as being optional
by RFC1122 (1989).

Enforcing the PUSH bit set via Nagle off, does not mean that one can
control the PUSH function (one do get Nagle off at the same time).

Possible certain implementations have certain tweeks so that one may
control the PUSH bit from ULP. If that is consider to be the
Defacto standard, even if RFC1122 says that it is optional, then I do not
disagree, but is it so ?

The PUSH bit makes the data be available for read by the application (when
in-line) but I am not sure this, as defacto, makes
the data be pushed to the application in most common api's exposed to
applications today. In some implementations, a socket becomes readable
based on the amount of data that has arrived correlated with when the
application has asked to be awaken (in number of bytes), but the PUSH flag
makes no difference. Possibly that is a deficiency of such
implementations. Not sure ?
I know that in some implementations then a socket becomes readable only if
the receive buffer is full or the PUSH flag has arrived.

Perhaps I should ask in tcpm, if TCP implementations generally pushes the
data to the application (or notifies the application) when the PUSH bit is
seen ? Or perhaps I will get an overwhelming - YES SURE !! - response this
to email, and then at least  the receive part of the PUSH is then inline
with RFC1122/RFC793.

BR, Karen

---

From: Karen Elisabeth Egede Nielsen <[email-masked]>
Date: Wed, 4 Nov 2015 15:37:44 +0900

HI Joe,

Yes I agree. But still there are finer features and BCP for api
and protocol implementations that have appeared from the api's defined
outside of the IETF and for which one need to look outside of RFC docs.

Or for PUSH bit one can also look at RFC1122 which describes it as
optional.

BR, Karen

> From: Taps [mailto:[email-masked]] On Behalf Of Joe Touch
> Sent: 4. november 2015 02:44
> To: [email-masked]; Michael Welzl <[email-masked]>
> Cc: taps WG <[email-masked]>; [email-masked]
> > have an informative section that describes the API together with the
> > normative protocol spec. That is not because there are expected to be
> > alternatives to choose from:  It's because, as I understand, the IETF
> > is not the formal standards body for specification of such normative
APIs.
> That has been a serious misinterpretation of how a protocol definition
works,
> which the IETF has propagated over the years.
>
that
> consists of:
of the
> protocol.
actions,
> i.e., a tree falling in the woods with nobody to hear.
>
> Joe
>

---

From: Joe Touch <[email-masked]>
Date: Tue, 3 Nov 2015 22:50:14 -0800

> outside of the IETF and for which one need to look outside of RFC docs.

Oh, I was certainly not claiming that IETF specifications are always
complete. The key is to derive the abstract API requirement from the
implementation extensions (and update the spec, ultimately).

> Or for PUSH bit one can also look at RFC1122 which describes it as

RFC1122 is a standards track doc that clearly updates RFC793, even
though that sort of marking was not part of IETF process when it was issued.

Joe

---

From: Michael Welzl <[email-masked]>
Date: Wed, 4 Nov 2015 12:06:16 +0100

Hi,

Thanks Karen for your long email and helpful comments!

I re-order it a bit and cut some parts to get my answer more structured:

> to gather this minimal set of mechanisms in a qualified manner.

So I think we really do need to rely on RFCs.

1)  TCP:
-------------

I think there is a misunderstanding about draft-welzl-taps-transports:

> On 04 Nov 2015, at 02:27, Karen Elisabeth Egede Nielsen <[email-masked]> wrote:
> 
> HI,
> 

I only used RFC4960 for SCTP in this -00 version, but I did try to use all of the TCP spec as far as possible... never easy with all these RFCs - but at least I constantly checked what I'm doing against the Roadmap RFC 7414.
So I did include RFC 1122 and RFC 6093. Specifically, the text at the end of Pass 1, under "Excluded Services", says the following about URG:

***
   The 'send' and 'receive' commands include usage of an "URGENT"
   mechanism, which SHOULD NOT be implemented according to [RFC6093]
   and is therefore not described here.  This also concerns the notification
   "Urgent pointer advance" in the ERROR_REPORT described in 
   Section 4.2.4.1 of [RFC1122].
***

and about PUSH:

***
The 'receive'
   command can (under some conditions) yield the status of the PUSH flag
   according to [RFC0793], but this TCP functionality is made optional
   in [RFC1122] and hence not considered here.  Generally, section
   4.2.2.2 of [RFC1122] says that PUSH on send calls MAY be implemented,
   which could be a reason not to consider it here.  However, the text
   then explains that "an interactive application protocol must set the
   PUSH flag at least in the last SEND call in each command or response
   sequence", and most implementations provide some option to cause a
   behavior that is in some way similar to PUSH.  Therefore PUSH is
   described as a part of SEND here.
***

We can debate these decisions, but I did base them on RFC 1122 and RFC 6093.

> control the PUSH function (one do get Nagle off at the same time).

Technically, I don't see a connection between Nagle and PUSH. It's kind of obvious though that if you want small messages to be delivered fast, you don't want Nagle and you do want PUSH, but still these are separate functions.
This sounds exactly like the type of confusion we may end up with if we follow actual API implementations and not what the spec says!

> Defacto standard, even if RFC1122 says that it is optional, then I do not

See my interpretation above: I think RFC 1122 is a bit ambiguous here, once saying it MAY be implemented, then saying "an interactive application protocol must set the PUSH flag ...."  ... though this isn't a capitalized MUST.

> the receive buffer is full or the PUSH flag has arrived.

Yes, it's certainly implementation-dependent. The idea of PUSH is that, when not using it, the receiving process doesn't need to be woken up every time a new handful of bytes arrive.

> to email, and then at least  the receive part of the PUSH is then inline

It's a mechanism of TCP, it's described in the spec, and it comes with some SHOULD's and MAY's. I think one can create a very efficient TCP implementation that completely ignores PUSH, but, depending on your system, it can also be helpful. Based on my analysis above, I thought it makes sense to assume that at least the sender can set it  (I think it's not explicit in today's APIs but implicit... I did take a look (just out of curiosity, not breaking my own rules when writing the document!  ;-)  ) and I think I found it in the BSD API or in some older version thereof).

See, that's the point of trying to rely only on RFCs: we end up with a view of what really should be implemented by the APIs out there. If it isn't, well, that's bad, but then it's really a breach of the spec.

2)  SCTP:
--------------

> not the implementation aspects that we are focused on in Taps.

I agree, I don't care much about the call format

> bit of a surprise to me).

Hm, this is about the only piece from RFC 6458 that I did cite in draft-welzl-taps-transports, just as a comment, because I found it disturbingly incoherent that TCP provides this but it's missing in RFC 4960:

***
   o  Change DSCP
      Protocols: TCP
      Comments: This is described to be changeable for SCTP too in
      [RFC6458].
***

In RFC 6458, "uint8_t  spp_dscp;" is a part of the per-adress parameters, struct sctp_paddrparams.

> So my view

Hm. This statement makes me feel a bit uneasy...

So, if we have consensus that RFC 6458 reflects the services (not necessarily the precise call format) that are supposed to be provided by any SCTP implementation, I'm happy to include it. Let's hear some more voices perhaps...

Cheers,
Michael

---

From: Karen Elisabeth Egede Nielsen <[email-masked]>
Date: Wed, 4 Nov 2015 21:28:32 +0900

HI Michael,

Thanks a lot.
Please see inline below.

BR, Karen

> From: Michael Welzl [mailto:[email-masked]]
> Sent: 4. november 2015 20:06
> To: Karen Elisabeth Egede Nielsen <[email-masked]>
> Cc: <[email-masked]> Fairhurst <[email-masked]>;
> [email-masked]
> >>> A side note about TCP, because Karen made a comment about the TCP
> >>> API
> >> too:
> >>> not: I think we should cover whatever a system claiming to
> >>> "implement the protocol in accordance with the spec" has to cover.
> >>> Going down the path of looking at actual socket API implementations
> >>> is dangerous in that we end up in "only implemented here, not there"
> >>> - land. We want a minimal set of mechanisms that are (or at least
> >>> really should be! for that, what else can we use as a basis than our
> >>> own recommendations?!)
> >> available everywhere.
> >
> > exercise is where from to gather this minimal set of mechanisms in a
> > qualified manner.
>
> > As a general comment then I believe that when describing what is
> > supported by TCP/SCTP (or UDP) as standard then it does not suffice to
> > look into IETF RFCs.
> > One need at least to relate to the *basic functions* of the
> > POSIX/Berkeley socket api standard.
> >
> > specify an abstract API for TCP, then the true defacto standard for
> > the socket api is the Berkeley socket api from .?. around 1990.
> > Not saying the different socket api standards don't differ and that
> > there is *one* standard to look at.
> > But making is be RFC793 rather then what emerged as defacto in the
> > 1990's seems a "bit odd" to me.
> >
> > clarifying part of RFC793 (namely the PUSH bit) And one, much more
> > recent, RFC6093, clarifying the urgent bit.
>
>
> I only used RFC4960 for SCTP in this -00 version, but I did try to use
all of the
> TCP spec as far as possible... never easy with all these RFCs - but at
least I
> constantly checked what I'm doing against the Roadmap RFC 7414.
> So I did include RFC 1122 and RFC 6093. Specifically, the text at the
end of
> Pass 1, under "Excluded Services", says the following about URG:
>
> ***
[Karen Elisabeth Egede Nielsen] Yes I am happy with this.
I may have more problems with the transport 07 draft in this context.
This comment I have given there.

Sorry for referring to it here in the context of your draft. It just
belonged
in the overall discussion.

>    described as a part of SEND here.
[Karen Elisabeth Egede Nielsen] There is a very clear distinction in
between
capitalized Keywords and non-capitalized must's and should's in section
4.2.2.2 of rfc1122.

 I have no problem with assuming that TCP implementations follow
           MUST set the PSH bit in
            the last buffered segment (i.e., when there is no more
            queued data to be sent).

I have no problem with assuming that applications, if the MAY option is
supported at the api, s
Is assumed to set the PUSH flag at the occasions described in RFC1122.

But I do have a problem  with the following text - repeated from your
paragraph above;:

>  and most implementations provide some option to cause a

You are referring to "most implementations provide some option"

What do you mean by "most implementations" ?

What do you mean by "option" ?

My problem particularly with "the option" part, is that is to me in the
context of
an API description sounds as if this "option" is a function in the API
(read an "API option")
 that the ULP can invoke.

> > The PUSH bit as settable by application was recognized as being
> > optional by RFC1122 (1989).
> >
> Technically, I don't see a connection between Nagle and PUSH. It's kind
of
> obvious though that if you want small messages to be delivered fast, you
> don't want Nagle and you do want PUSH, but still these are separate
> functions.

[Karen Elisabeth Egede Nielsen] Perfect. I cound't agree more !!
I was trying to understand which "API option" that you might be referring
to,
and as I have had many (in my opinion confused) discussions with
certain TCP users which looking for the "API option" for setting of the
PUSH,
thought they had to use Nagle off for this. While in fact they didn't have
to do
anything because the stack will set the PUSH bit following the RFC1122
MUST.

> This sounds exactly like the type of confusion we may end up with if we
> follow actual API implementations and not what the spec says!
>
[Karen Elisabeth Egede Nielsen] GOOD. I am happy that we avoid this.
So could we loose the text about that the TCP API provides an option to
control the PUSH flag - :-) :-) !!
> > control the PUSH bit from ULP. If that is consider to be the Defacto
> > standard, even if RFC1122 says that it is optional, then I do not
> > disagree, but is it so ?
>
> See my interpretation above: I think RFC 1122 is a bit ambiguous here,
once
> saying it MAY be implemented, then saying "an interactive application
> protocol must set the PUSH flag ...."  ... though this isn't a
capitalized MUST.
[Karen Elisabeth Egede Nielsen] For what it is worth (if anything)  this
is not how I interpret the text. I understand
the text as saying that if the MAY option to set PUSH in the apu
is implemented then applications must do what the text says. But as
RFC1122 is not really specifying what applications should or must do.,
these musts or should are not  capitalized.

But I should assume that we could get the people working with RFC793bis in
tcpm
provide the "right" interpretation ?!

> > in-line) but I am not sure this, as defacto, makes the data be pushed
> > to the application in most common api's exposed to applications today.
> > In some implementations, a socket becomes readable based on the
> amount
> > of data that has arrived correlated with when the application has
> > asked to be awaken (in number of bytes), but the PUSH flag makes no
> > difference. Possibly that is a deficiency of such implementations. Not
> > sure ?
> > I know that in some implementations then a socket becomes readable
> > only if the receive buffer is full or the PUSH flag has arrived.
>
> Yes, it's certainly implementation-dependent. The idea of PUSH is that,
when
> not using it, the receiving process doesn't need to be woken up every
time a
> new handful of bytes arrive.
>
[Karen Elisabeth Egede Nielsen] Yes and certainly, as many TCP
implementations simply
always sets the PUSH flag following the MUST of rfc1122, section 4.2.2.2,

then the PUSH flag ends up being set on a lot of segments just because the
application is writing in
small chunks....

Or even worse - the PUSH flag  it is lost when the SEND buffer gets
congested,
which is terribly when the receiving side read operation has implemented
trust on the PUSH
for being waken up. This actually is something that we have seen resulting
in
outages (signaling network collapse)  even recently.

>
> > Perhaps I should ask in tcpm, if TCP implementations generally pushes
> > the data to the application (or notifies the application) when the
> > PUSH bit is seen ? Or perhaps I will get an overwhelming - YES SURE !!
> > - response this to email, and then at least  the receive part of the
> > PUSH is then inline with RFC1122/RFC793.
>
> It's a mechanism of TCP, it's described in the spec, and it comes with
some
> SHOULD's and MAY's. I think one can create a very efficient TCP
> implementation that completely ignores PUSH,
but, depending on your
> system, it can also be helpful.
[Karen Elisabeth Egede Nielsen] Perhaps - but it can also be very harmful
to put trust in the PUSH
flag, if the two ends to not agree on what controls it - the ULP or the
TCP SND buffer congestion level...

Based on my analysis above, I thought it makes
> sense to assume that at least the sender can set it  (I think it's not
explicit in
> today's APIs but implicit... I did take a look (just out of curiosity,
not breaking
> my own rules when writing the document!  ;-)  ) and I think I found it
in the
> BSD API or in some older version thereof).
>
[Karen Elisabeth Egede Nielsen] But the implicit is what gets us into the
confusion !
How is it implicitly implemented in API  implementations ?

(Despite that rfc1122 say that it need not be).

> See, that's the point of trying to rely only on RFCs: we end up with a
view of
> what really should be implemented by the APIs out there. If it isn't,
well,
> that's bad, but then it's really a breach of the spec.
>
[Karen Elisabeth Egede Nielsen] In this case then I am afraid that it is
the spec that allowed the
implementations to distroy the functions as it (rfc793 to rfc1122) has
given some freedom in implementations.
The uncertainty of which has made the function unreliable/not trustworthy.
(Or perhaps the function was never a good idea to begin with..)

> >>> implementation, not a recommendation or even prescription of what
> >>> all SCTP implementations must provide. However, my decision for
> >>> Therefore I went with a minimal set of functions that I thought we
> >>> can assume are implemented everywhere (well, at least in every
> >>> system that claims to "follow the spec").  Can we assume that every
> >>> system that claims to implement SCTP in accordance with the spec
> >>> fully implements
> > RFC
> >> 6458?
> >>>
> >> choose from:  It's because, as I understand, the IETF is not the
> >> formal standards body for specification of such normative APIs.
> >>>
> > tsvwg  wg draft long before RFC4960 work was started. My understanding
> > is that while going from RFC2960 to RFC4960 one did not bother "too
> > much" with updating the API section of RFC2960 --> RFC4960, as the
> > finer API details were being worked on in this wg draft.
> >
> > However I was not personally involved in IETF SCTP back then, so I
> > like to ask others to confirm this.
> >
> > Something else:
> >
> > The implementation that I have worked with was initiated in the early
> > 2000 and at that time we looked for RFC2960bis (RFC4960) for protocol
> > specification and the socket api for api features.
> > Yes there are particulars of the RFC6458 which represent
> > implementations aspects. For example our API implementation
> implements
> > certain calls in the format they were specified in ver 14/15 of the
> > socket api from draft from around 2007  and not the exact form the
> > calls have in RFC6458 (version 30-some of the draft) from 2011, but
> > the features available as controllable has not changed much during the
> > years. And it is the features available, not the implementation
> > aspects that we are focused on in Taps.
>
> > But I have to admit that in our SCTP API implementation we implements
> > the basic POSIX api functions of TCP/IP for SCTP simply when they
> > makes as much sense for SCTP as they do for TCP/IP. For example DSCP
> > is settable via a socket option per association, even if I think this
> > cannot be found in RFC6458 nor in RFC4960 (which is a bit of a
> > surprise to me).
>
>
> Hm, this is about the only piece from RFC 6458 that I did cite in
draft-welzl-
> taps-transports, just as a comment, because I found it disturbingly
> incoherent that TCP provides this but it's missing in RFC 4960:
>
> ***
>    o  Change DSCP
>       Protocols: TCP
> In RFC 6458, "uint8_t  spp_dscp;" is a part of the per-adress
parameters,
> struct sctp_paddrparams.
[Karen Elisabeth Egede Nielsen] Great. I missed it yesterday. Couldn't
understand it either.
Perhaps I can blame the jetlag...

> Hm. This statement makes me feel a bit uneasy...

[Karen Elisabeth Egede Nielsen] Yes. :-(.

>
> So, if we have consensus that RFC 6458 reflects the services (not
necessarily
> the precise call format) that are supposed to be provided by any SCTP
> implementation, I'm happy to include it. Let's hear some more voices
> perhaps...
>
[Karen Elisabeth Egede Nielsen] Yes definitively.

---

From: Michael Welzl <[email-masked]>
Date: Wed, 4 Nov 2015 13:39:10 +0100

Hi,

I'm cutting the part about URG, because here we agree. I think we agree about PUSH too, but I'll say it in line below  :-)

> capitalized Keywords and non-capitalized must's and should's in section

I know....

>            the last buffered segment (i.e., when there is no more

but it isn't a MUST, it's a "must".

> What do you mean by "option" ?

GAAAAAAH !!!!!!   Here you did catch me diverging from my own rules!!!!!!!
That must have come from looking at actual API implementations, and no it REALLY shouldn't be there. I did intend to strictly only follow RFCs. Big mistake, very sorry!

> an API description sounds as if this "option" is a function in the API

Based on what you said about capitalization and on the mistake in talking about actual implementations I saw, I tend to erase PUSH from SEND because it's only qualified as MAY, period.
Ok?

> control the PUSH flag - :-) :-) !!

See above - we're in agreement. I'll drop it.

> these musts or should are not  capitalized.

- right.

> provide the "right" interpretation ?!

Nah I think this is clear enough already. I'll drop PUSH from SEND.

> (Despite that rfc1122 say that it need not be).

... we agree already.

> (Or perhaps the function was never a good idea to begin with..)

- which I why I want to drop parts of the protocol implementation that are not at least qualified as SHOULD.

Cheers,
Michael

---

From: Karen Elisabeth Egede Nielsen <[email-masked]>
Date: Wed, 4 Nov 2015 23:09:56 +0900

Yes we agree ! :-) .
Glad that ended here.
Thanks a lot for the patience.

BR, Karen

> Sent: 4. november 2015 21:39
> I'm cutting the part about URG, because here we agree. I think we agree
> about PUSH too, but I'll say it in line below  :-)
>
>
> >> and about PUSH:
> >>
> >> ***
> >> The 'receive'
> > between capitalized Keywords and non-capitalized must's and should's
> > in section
> > 4.2.2.2 of rfc1122.
>
> I know....
>
>
> > I have no problem with assuming that applications, if the MAY option
> > is supported at the api, s Is assumed to set the PUSH flag at the
> > occasions described in RFC1122.
> >
> That must have come from looking at actual API implementations, and no
it
> REALLY shouldn't be there. I did intend to strictly only follow RFCs.
Big
> mistake, very sorry!
>
>
> > My problem particularly with "the option" part, is that is to me in
> > the context of an API description sounds as if this "option" is a
> > function in the API (read an "API option") that the ULP can invoke.
>
> Based on what you said about capitalization and on the mistake in
talking
> about actual implementations I saw, I tend to erase PUSH from SEND
> because it's only qualified as MAY, period.
> Ok?
>
>
> >> obvious though that if you want small messages to be delivered fast,
> >> you don't want Nagle and you do want PUSH, but still these are
> >> separate functions.
> >
> > I was trying to understand which "API option" that you might be
> > referring to, and as I have had many (in my opinion confused)
> > discussions with certain TCP users which looking for the "API option"
> > for setting of the PUSH, thought they had to use Nagle off for this.
> > While in fact they didn't have to do anything because the stack will
> > set the PUSH bit following the RFC1122 MUST.
> >
> >> This sounds exactly like the type of confusion we may end up with if
> >> we follow actual API implementations and not what the spec says!
> >>
> > So could we loose the text about that the TCP API provides an option
> > to control the PUSH flag - :-) :-) !!
>
> > [Karen Elisabeth Egede Nielsen] For what it is worth (if anything)
> > this is not how I interpret the text. I understand the text as saying
> > that if the MAY option to set PUSH in the apu is implemented then
> > applications must do what the text says. But as
> > But I should assume that we could get the people working with
> > RFC793bis in tcpm provide the "right" interpretation ?!
>
> >>> in-line) but I am not sure this, as defacto, makes the data be
> >>> pushed to the application in most common api's exposed to
applications
> today.
> >>> difference. Possibly that is a deficiency of such implementations.
> >>> Not sure ?
> > implementations simply always sets the PUSH flag following the MUST of
> > rfc1122, section 4.2.2.2,
> >
> > then the PUSH flag ends up being set on a lot of segments just because
> > the application is writing in small chunks....
> >
> > congested, which is terribly when the receiving side read operation
> > has implemented trust on the PUSH for being waken up. This actually is
> > something that we have seen resulting in outages (signaling network
> > collapse)  even recently.
> >
> >
> >>
> >>> Perhaps I should ask in tcpm, if TCP implementations generally
> >>> pushes the data to the application (or notifies the application)
> >>> when the PUSH bit is seen ? Or perhaps I will get an overwhelming -
YES
> SURE !!
> > [Karen Elisabeth Egede Nielsen] Perhaps - but it can also be very
> > harmful to put trust in the PUSH flag, if the two ends to not agree on
> > what controls it - the ULP or the TCP SND buffer congestion level...
> >
> > [Karen Elisabeth Egede Nielsen] But the implicit is what gets us into
> > the confusion !
> > [Karen Elisabeth Egede Nielsen] In this case then I am afraid that it
> > is the spec that allowed the implementations to distroy the functions
> > as it (rfc793 to rfc1122) has given some freedom in implementations.
> - which I why I want to drop parts of the protocol implementation that
are
> not at least qualified as SHOULD.
>
>
> >> 2)  SCTP:
> >> --------------
> >>
> >>
> >>>>> optional to implement - I don't think we want to end up with a
> >>>>> TAPS system that provides services that, alas, on Operating System
> >>>>> XY are not available because here only a subset of the API is
> implemented.
> >>>>> system that claims to "follow the spec").  Can we assume that
> >>>>> every system that claims to implement SCTP in accordance with the
> >>>>> spec fully implements
> >>> RFC
> >>>> 6458?
> >>>>>
> >>>> GF: From a TSVWG Chair perspective, beware here...  *ALL* more
> >>>> recent IETF SCTP API work from TSVWG is INFO.  Each SCTP RFC is
> >>>> expected to
> >>> have
> >>> [Karen Elisabeth Egede Nielsen] Not only that. RFC6458 was around as
> >>> a tsvwg  wg draft long before RFC4960 work was started. My
> >>> understanding is that while going from RFC2960 to RFC4960 one did
> >>> RFC4960, as the finer API details were being worked on in this wg
draft.
> >>> 2000 and at that time we looked for RFC2960bis (RFC4960) for
> >>> protocol specification and the socket api for api features.
> >>> the features available as controllable has not changed much during
> >>> the years. And it is the features available, not the implementation
> >>> But I have to admit that in our SCTP API implementation we
> >>> implements the basic POSIX api functions of TCP/IP for SCTP simply
> >>> when they makes as much sense for SCTP as they do for TCP/IP. For
> >>> example DSCP is settable via a socket option per association, even
> >>> if I think this cannot be found in RFC6458 nor in RFC4960 (which is
> >>> a bit of a surprise to me).
> >>
> >>

---

From: Joe Touch <[email-masked]>
Date: Wed, 4 Nov 2015 10:11:26 -0800

> But making is be RFC793 rather then what emerged as defacto in the 1990's

There are implementations of TCP that do not use the Berkeley sockets
model. Note, though, that the concept of a socket for communications
itself comes from RFC793, not Unix.

> And one, much more recent, RFC6093, clarifying the urgent bit.

When we talk about TCP, we certainly mean more than just RFC793 - we
include all updates to that spec, which include these documents. In some
cases, the updates came from implementation experience; in other cases,
they came from the standards community because of more fundamental concerns.

Joe

---

From: Michael Welzl <[email-masked]>
Date: Wed, 4 Nov 2015 20:31:23 +0100

> On 4. nov. 2015, at 19.11, Joe Touch <[email-masked]> wrote:
> 
> 
> 

I believe that was just a misunderstanding: Karen thought that I had only used RFC793 when writing draft-welzl-taps-transports-00, but really I did try to use all relevant RFCs.

Cheers,
Michael

---

From: Karen Elisabeth Egede Nielsen <[email-masked]>
Date: Thu, 5 Nov 2015 08:02:09 +0900

HI,

>
> I believe that was just a misunderstanding: Karen thought that I had
only
> used RFC793 when writing draft-welzl-taps-transports-00, but really I
did try
> to use all relevant RFCs.
>
[Karen Elisabeth Egede Nielsen] ......well, I did observe that you were
referring to rfc1122 (and also RFC6093).
but the result and the emphasis on the PUSH bit made me think that weight
only was put on what was written in RFC 793. ;-)

BTW,  something which I didn't mention yesterday, but now got the "energy"
to in and find is the following:

Send:  This sends a message of a certain length in bytes over an
      association.  A number can be provided to later refer to the
      correct message when reporting an error and a stream id is
      provided to specify the stream to be used inside an association
      (we consider this as a mandatory parameter here for simplicity: if
      not provided, the stream id defaults to 0).  An optional maximum
      life time can specify the time after which the message should be
      discarded rather than sent.  A choice (advisory, i.e. not
      guaranteed) of the preferred path can be made by providing a
      destination transport address, and the message can be delivered
      out-of-order if the unordered flag is set.  Another advisory flag
      indicates the ULP's preference to avoid bundling user data with
      other outbound DATA chunks (i.e., in the same packet).  The

^^^^
      handling of this no-bundle flags is similar to the sender side
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
      handling of the TCP PUSH flag.
      ^^^^^^^^^^^^^^^^^^^^^^^^
      A payload protocol-id can be
      provided to pass a value that indicates the type of payload
      protocol data to the peer.

Can we agree to have this be removed also :-).
Bundling in SCTP is similar to Nagle. Indeed it is often implemented as
Nagle exactly
(though with add-ons). Comparing it to PUSH is NOT helpful.

Then, there is the additional discussion about RFC4960 and RFC6458 (socket
api) in that
bundling control is not often implemented as a flag on a message, but as
in TCP on a per association/connection level.
This again made me think emphasis was put on RFC793/RFC4960 and not much
else ;-).

BR, Karen

---

From: Michael Welzl <[email-masked]>
Date: Thu, 5 Nov 2015 08:31:21 +0100

> On 5. nov. 2015, at 00.02, Karen Elisabeth Egede Nielsen <[email-masked]> wrote:
> 
> HI,
> 
>> 

Agreed. Actually I think that was not by intention, just a nit that happened as I wrote it: I don’t think I wanted to compare it with PUSH but indeed with Nagle.

> This again made me think emphasis was put on RFC793/RFC4960 and not much

Things like “[not] often implemented” really shouldn’t be there. All mistakes, all to be fixed. Apologies!

Michael