Subject: AW: [Tsvwg] Re: SCTP PAD chunk

From: "Meyknecht, Bernward" <[email-masked]>
Date: Mon, 10 Apr 2006 11:38:20 +0200

Hi all,

only one remark: please ensure that the introduction/usage of PAD 
is compatible with all existing implementations,
thinking e.g. about the thing with control chunks following a
DATA chunk.
Even a test at an interop will only give you a hint, what could
happen, since not all implementations may take part or 
there are 'older' code versions deployed, not being 
tested at the interop.

Kind regards,

Bernward

-----Ursprüngliche Nachricht-----
Von: Michael Tuexen [mailto:[email-masked]] 
Gesendet: Samstag, 8. April 2006 00:05
An: [email-masked]
Cc: Randall Stewart; [email-masked]
Betreff: Re: [Tsvwg] Re: SCTP PAD chunk

Dear all,

all SCTP implementations consider only DATA chunks (ie chunks with  
chunk type == 0)
as data chunks and all others as control chunks. This is also what  
RFC 2960 states and
I think we should not change that.

Looking at the point Mark raises of aligning data not only to 32 bits  
but possibly also to
64 or even 128 bits, I think this is not only relevant inside the  
kernel but also on
the kernel/user land. Having in mind that the DATA chunk header is 16  
bytes (128 bits)
the padding can be done at the application layer layer. At the  
transport layer only
an initial padding before the DATA chunks may be required, which is  
possible with the
PAD chunk.

Best regards
Michael

On Apr 7, 2006, at 11:41 PM, Brian F. G. Bidulock wrote:

> Mark,
>
> Mark Butler wrote:                         (Fri, 07 Apr 2006 15:24:35)
>
> -X--snip--X-
>>
>> Speaking of what things are good for - what is the point of the  
>> control
>> chunk before data chunk rule anyway?  It has little or no  
>> advantages -
>> the primary effect of the rule is to increase implementation  
>> complexity
>> and the number of packets that have to be transmitted.
>
> Well, I found the rule afforded the reverse: that it decreases  
> implementation
> complexity and allows for processing optimizations.  Control chunks  
> other than
> SACK are rare during data transfer phase, whereas DATA chunks are  
> not.  Only
> one SACK normally occurs in a packet.  Placing the SACK in the  
> first chunks
> locates it nicely and a hard loop on DATA chunks can be performed  
> after that.
> Placing the SACK ahead of all the DATA chunks allows data buffers  
> in the
> retransmission buffer to be cleared before allocating any for the  
> received
> data chunks.
>
> Ask yourself the reverse question: what's the good in burying a  
> control chunk
> in the middle of a bunch of data chunks?
>
> Why not just make the padding chunk a "data" chunk.
>
> --brian
>
> -- 
> Brian F. G. Bidulock
> [email-masked]
> http://www.openss7.org/
>

---

From: Randall Stewart <[email-masked]>
Date: Mon, 10 Apr 2006 05:55:49 -0400

Bernward:

Yes, this is a real concern I have about putting a
pad chunk between two data chunks... I am not
to sure that implementations will support this... not
that they should not.. but it is something that we must
use care if we define this new chunk...

If we make it able to go anywhere.. then maybe a restriction
on not sending it unless your peer announces support
for it... with the supported chunk types parameters... this
would then make it so an implemenation would say.. yes I
support this.. and in so doing .. say it may fall anywhere
in the packet.

Michael, what do you think?

R

-- 
Randall Stewart
ITD - Cisco Systems Inc.
803-345-0369 <or> 815-342-5222 (cell)

---

From: Mark Butler <[email-masked]>
Date: Mon, 10 Apr 2006 09:24:19 -0600

> support this.. and in so doing .. say it may fall anywhere

I would prefer a parameter that indicates the sender allows control and 
data chunks to be bundled in any reasonable order, as if they were in 
separate packets.  Any compliant implementation implicitly supports a 
PAD chunk that occurs before any DATA chunks.

 - Mark B.

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Mon, 10 Apr 2006 11:26:05 -0600

Mark,

While your defining the cool new data chunks that includes fragment
numbers, why not have it specify the length and offset of the data
within the chunk to allow for internal padding.

I really question the use of this ISO like feature.  Everything of
interest (including data) is already padded to a 32-bit alignment
boundary.  Did you have some 128-bit architecture for which that was
insufficient?

--brian

Mark Butler wrote:                             (Mon, 10 Apr 2006 09:24:19)

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: Mark Butler <[email-masked]>
Date: Mon, 10 Apr 2006 16:17:52 -0600

>within the chunk to allow for internal padding.
That would be unnecessarily wasteful for every application, not just the
rare ones that prefer a larger grained alignment.

>boundary.  Did you have some 128-bit architecture for which that was
Communication bandwidth on 64-bit machines is improved when the data is
64 bit aligned because memcpy and checksum functions are often optimized
for that case.  I doubt there are going to be any true 128-bit
architectures (in the contemporary sense) for several decades, although
some machines might still benefit from larger alignments.

- Mark B.

---

From: Mark Butler <[email-masked]>
Date: Mon, 10 Apr 2006 17:47:53 -0600

Brian F. G. Bidulock wrote:

>Oh, go on.  memcpy is optimized to transfer the first unaligned bytes and
>then long word copy from there.  The little stagger at the beginning is
>far less complex than adding padding chunks to a protocol (and calculating
>the alignment at the sender).
>  
>
memcpy cannot use 64 bit operations on many 64 bit architectures (say 
IA-64 or Power) unless both source and destination have the same 
relative alignment.  An initial adjustment can 64 bit align either the 
source or the destination pointers, but in general not both.

In any case, this behavior is strictly optional.  Broken network PMTU 
discovery does not need the receiver to recognize a PAD chunk at all.  
It just needs to know packets of a specified size arrive intact, (a 
purpose which would be aided by placing some random data in the PAD 
chunk, or putting the PAD chunk at the start of the packet, and the DATA 
or HEARTBEAT at the end).

I am just saying that if one is going to create a PAD chunk, it ought to 
be general rather than special purpose, and that an option to relax a 
unnecessary ordering requirement is the simplest way to that end.  
Unless someone else expresses interest in such an option, it is not 
likely to get standardized anyway. 

- Mark

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Mon, 10 Apr 2006 18:16:39 -0600

Mark,

IA-64 has instructions more suited for copying than reading 64-bit
words and writing them out.  I originally thought you meant a RISC
architecture.  OS's must deal with non-aligned copies all day long:
we don't need to make the protocol worry about it too.  Placing
architecture specific optimizations in the protocol is certainly
not a priority.

--brian

Mark Butler wrote:                                                    (Mon, 10 Apr 2006 17:47:53)
>    chunk,  or  putting  the PAD chunk at the start of the packet, and the
>    DATA or HEARTBEAT at the end).
>    I  am just saying that if one is going to create a PAD chunk, it ought
>    to be general rather than special purpose, and that an option to relax
>    a  unnecessary  ordering  requirement is the simplest way to that end.

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Mon, 10 Apr 2006 17:03:59 -0600

Mark,

Mark Butler wrote:                                                    (Mon, 10 Apr 2006 16:17:52)

I say dump the requirement.

--brian

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: Mark Butler <[email-masked]>
Date: Mon, 10 Apr 2006 20:23:09 -0600

>architecture specific optimizations in the protocol is certainly
Sure they do - and they do it *slower*.  But that is beside the point,  
I am suggesting an option to simplify the protocol.  Perhaps someday it 
might even be the default.  And rather than coming up with a germane 
objection, you are objecting to an obscure implementation possibility 
enabled by, but certainly not required by the option.  It is as classic 
a case of "worse is better" as I have ever seen.  Opposition to all 
changes no matter how inconsequential just because they are changes.  
i.e. we want SCTP to stay Specially Contrained Telecom Protocol, and not 
lift a finger to make it useful for anything else.

 - Mark B.

  ...

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Mon, 10 Apr 2006 23:21:48 -0600

Mark,

Since when does avoiding in a general purpose protocol architecture
specific implementation issues (for specific processors yet) make it
specially constrained for telecom???

If your specific processor doesn't process the protocol fast enough, buy
one that does.  Don't ask for the protocol to change to optimize your
specific processor at the cost of most others.

Perhaps you could consider a network processor that is better suited to
the task.

If it is so important to you, I respectfully suggest you write a I-D.

--brian

Mark Butler wrote:                           (Mon, 10 Apr 2006 20:23:09)

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: Mark Butler <[email-masked]>
Date: Tue, 11 Apr 2006 03:41:50 -0600

>specially constrained for telecom???
It doesn't.  It is a the general practice of avoiding all protocol / API 
generalizations, harmonizations, and normalizations - even strictly 
optional ones - for trivial and petty reasons that I find disturbing.  
The contrast to the environment in many working groups is striking.  
With DCCP for example, one gets the impression that they are almost too 
flexible.  It is more like - "that could be useful, lets add an option" 
vs. the "over my dead body" attitude that dominates discussions here, 
amid a background of general indifference - indifference to weaknesses 
both large and small.  The general  impression one gets is that SCTP is 
a victim of premature and zealously guarded fossilization.

>If it is so important to you, I respectfully suggest you write a I-D.
I am not inclined to spend time writing I-Ds that are likely to be 
rejected for reasons having little or nothing to do with actual merit.  
The only purpose it might serve would be to reserve a parameter slot or 
two.

 - Mark B.

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Tue, 11 Apr 2006 07:29:42 -0600

Mark,

I think you will need to scratch your own itch.  If you are not ready
for the hard work of specification writing and consensus building, you
are likely in the wrong forum.

--brian

Mark Butler wrote:                          (Tue, 11 Apr 2006 03:41:50)

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: Mark Butler <[email-masked]>
Date: Tue, 11 Apr 2006 11:26:38 -0600

>are likely in the wrong forum.
As if there has been anything discussed here for months that you were in 
favor of... On the contrary you practice little other than arrogant, 
conceited, and insincere indirection, distraction, and disruption.

 - Mark B.

---

From: Randall Stewart <[email-masked]>
Date: Wed, 12 Apr 2006 06:50:09 -0400

Brian:

I would ask that Mark go ahead and write his own
draft... If he wants to have some cool features
to have new types of data chunks and special padding..
thats wonderful.. but lets not put it in the base spec.

He can go ahead and write a draft that has a data chunk
that can have pad and all that stuff in it..

Lets leave the PAD chunk for control and what it
was intended.. PMTU probing for Matt Mathis's stuff..

That way this whole thing can be negotiated at startup
that you "support the advanced data chunk" and then
the implementations that support this can use it instead
of the normal data chunk... If he is defining a new
data chunk he can surely add a "pad bytes" field to it
to indicate the number of bytes of trailing pad..

Thus a control chunk pad.. and this new data chunk is
all we need to handle all padding scenarios :-D

R

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Wed, 12 Apr 2006 05:32:12 -0600

Randall,

Yes, I agree.  The protocol is quite extensible and it is not
all that difficult to write an extension draft; however, I also
agree with Mark that writing a draft without some interest
within the WG might be a wasted effort.

Also, I am concerned with the base protocol in that it appears
that, as Mark put it, all future chunks will be considered
control chunks by a base implementation because they do not have
the precise tag value of a DATA chunk.  This is not so good for
future extensibility.  This is one of the considerations I made
when I changed my loop to not just check for DATA chunks.  If
there is an unrecognized chunk in the sequence of DATA chunks,
it should be reported and aborted or continued in accordance
with the bits in the chunk type.  That is, unrecognized chunk
types should be considered as possibly being of a "DATA" type.

I think that the passage should be considered for revision in
the 2960bis document, if nothing more, but to pave the way for
future DATA-chunk-like extensions.

Are there comments on altering the passage in question?

--brian

Randall Stewart wrote:                         (Wed, 12 Apr 2006 06:50:09)

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: Randall Stewart <[email-masked]>
Date: Wed, 12 Apr 2006 07:44:51 -0400

> types should be considered as possibly being of a "DATA" type.

I agree, and I think Michael and I will be adding tests to this
nature for the next inter-op..

I am not really to sure what should be done to valid CONTROL
chunks appearing inter-mixed with DATA.. but for sure the
upper bits need to be checked... One can envision at least
3 handlings of valid control in the wrong place:

a) process it anyway
b) skip it and continue
c) stop processing all together at that point.

I am thinking that BSD will change to <b> for the default
and then have <a> and <c> as sysctl options... but I have
not fully decided..

In any event BSD's processing loop will be changing soon to
at a minimum handle the upper bits :-D

> Are there comments on altering the passage in question?
I don't know if we need to change 2960 on this.. it already
says you must handle the upper bits.. I think this is more
a matter of we have just failed to check this  :-D

R

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Wed, 12 Apr 2006 06:21:21 -0600

Randall,

Randall Stewart wrote:                                                             (Wed, 12 Apr 2006 07:44:51)
> at a minimum handle the upper bits :-D

Should the behavior differ from unrecognized to valid?  When a chunk
changes from unrecognized to valid do you really want its handling
between DATA chunks to change?

I really lean towards a) for both.  Bundling logic can be complicated
at times, and non-SCTP savy middle boxes could concatenate or split
packets at arbitrary chunk boundaries.  Also, when assembling very
large packets, either the packet must be terminated early or a
control chunk delayed until after the packet is assembled.  I agree
with Mark in that it should not make any difference if the control
chunk starts the next packet, or continues in the current packet.

> a matter of we have just failed to check this  :-D

I meant reference to control chunks (being non-true-DATA chunks)
having to appear before DATA in the packet.  I think that sending
control chunks first could be a SHOULD (with consideration for
implementations that might not be able to handle them out of place).

--brian

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: Randall Stewart <[email-masked]>
Date: Fri, 14 Apr 2006 07:25:44 -0400

Comments in-line..

> chunk starts the next packet, or continues in the current packet.

And you can do any of these.. and also a d) I did
not mention.. abort the association with a
"protocol violation".

This I will bet some implementations will do....

I am leaning towards making <b> the standard procedure in
BSD with sysctl options to do both <a> and <d>.

That way the default is modest.. but the sysadmin has
the ability to be liberal or very conservative... which
ever it chooses..

> implementations that might not be able to handle them out of place).

So let me see.. you are advocating a signifigant change to
2960 that:

a) Has no real advantage... in fact your first email argued
    as to why control should be first...

b) Will surely cause implementation incompatability... since I
    know that there are implementations out there that will probably
    do <d>. So new implementations that do this change would NOT
    inter-operate...

c) Has never come up at any inter-op in the last 6 years.

Sorry I just can't buy it...

At the last inter-op we had 14 implentations, of those 13
of them out-of-the box inter-operated.. only one, yours, had
issues and could not talk to anyone else without signifigant
code changes... you of all people must know how frustrating
this can be... I would not want any such change to the spec
that would only make things worse for developers with no
apparent advantage....

It is NOT hard to make control come first. On top of that
your previous talk of large packets is just not valid.. if
you reach the PMTU or close to it.. you send the all control
bundled packet.. and then continue bundling your next
control in the next packet... this is not hard... especially
since you must always be watching the PMTU has you bundle
and send when you have a full packet (or close to full).

R

---

From: "Brian F. G. Bidulock" <[email-masked]>
Date: Sat, 15 Apr 2006 06:45:16 -0600

Randall,

Randall Stewart wrote:                  (Fri, 14 Apr 2006 07:25:44)
> that would only make things worse for developers with no

As you have singled out a specific implementation, some
statements are in order:

The OpenSS7 implementation that was tested at the last interop
was experimental and was tested by a student.  I had asked
Professor Tuexen if we could sponsor one of his graduate
students to attend the event and perform interoperability
testing on an alpha implementation.

Although the alpha implementation did not show as well as our
production implementations tested at previous interops, I was
very pleased with Professor Tuexen's student's ability to patch
kernels and ferret out problems with the alpha implementation.

I was not frustrated in the least, and I believe that the
exercise was a very good learning experience for the student, and
the problems with the alpha implementation that were found more
than compensated for the small fee that was paid to sponsor the
student.  I am thankful that Professor Tuexen was able to make
the arrangement possible.

But this begs a question: do you intend to later identify
specific implementations in a negative light after the interop
in August as you have done here?

--brian

-- 
Brian F. G. Bidulock
[email-masked]
http://www.openss7.org/

---

From: Randall Stewart <[email-masked]>
Date: Sat, 15 Apr 2006 19:32:26 -0400

> the arrangement possible.

My apologies..  I was NOT trying to single your implementation
out... in fact if you were not frustrated ... I was.. I dislike
seeing any new implementation's NOT have 100% success....

I want every implemenation to succeed...

This is why we MUST be very very careful in changing MUST's -> SHOULD
or SHOULD -> MUST..

If we change a MUST to a SHOULD .. we may well end up breaking
many implemenations that exist and only find it when a new
implemenation comes along.. which sets a very very bad precedent...

If we change a SHOULD to a MUST we can be in the same boat..

Changing the chunk ordering rules is a crutial thing that need
be considered with extreme care...

There may be no good reason for having Control come before Data..
I can't remember where or why that came into the protocol.. but
there is equally NO good reason not to have it be that way..
(except of course the reason you gave in your first response
  to Mark which I thought was very good :-D) especially when
many implementations are out there..

R
-- 
Randall Stewart
803-345-0369 <or> 815-342-5222(cell)

---

From: Michael Tuexen <[email-masked]>
Date: Sun, 16 Apr 2006 12:21:28 +0200

Dear all,

RFC 2960 states that control chunks MUST put before DATA chunks
in SCTP packets.
Using the typographical conventions used in RFC 2960 this means only
that all DATA chunks (these are chunks with chunk type 0) MUST be
put before control chunks. Also the set of control chunks is mentioned,
but RFC 2960 does NOT define what control chunks are.
So we should fix this, by explicitly defining the notion control chunk.
 From my point of view it is clear that all chunk types defined in  
RFC 2960
except for DATA chunks are control chunks and as far as I know all  
implementations
have like this. Unfortunately, most implementations will consider all  
chunks
except DATA chunks as control chunks, but this is not true. One  
simply does not
know if an unknown chunk is a control chunks or not (it is definitely  
not
a DATA chunk). So an implementation should not consider an unknown  
chunk after
a DATA a protocol violation or so. I think this needs to be clarified.

If someone, like Mark, wants to define a new chunk which carries user  
data,
call it a N-DATA chunk, one has to define if it is a control chunk  
(probably not)
and if not, what kind of ordering restrictions exist. One could even  
state that
N-DATA can be anywhere in the packet. Using the supported chunk type  
parameter
and only sending them if the peer has listed the chunk type N-DATA in  
its
supported chunk types parameter will make sure that there are no interop
problems. Even in the case the peer does not understand it and  
consider it a control
chunk (which is wrong, but most implementation do today) this will  
not break
interoperability with current implementations.

So let us make clear in RFC 2960bis or in an IG what the notion of  
control chunk
is and what the correct behaviour on reception of unsupported chunks  
is. We will
test this at the next interop.

Now coming to the subject of the e-mail... SCTP PAD chunk. It is  
intended to
support PMTU with current implementations with as little impact as  
possible. So
I think the best is to define it as a control chunk. This makes it  
impossible
to perform Marks idea of 64/128 bit alignment.
I (personally) think that this is not critical, because I can get the  
64/128 bit
alignment by
- negotiating at the association setup what kind of alignment I want
   (the default 32bit, 64 bit, 128 bit) or whatever.
- if both sides supports this, all chunks (not only DATA) will be  
padded to
   this length. Fr 64 bit this would mean that we use 0-7 padding bytes.
- If we need to put the first chunk on some boundary, we can use the PAD
   chunk for this (except we want to AUTH it).

Mark, wouldn't give this you the alignment you want?

The only this we need to specify for this is that the PAD chunk will  
not the padded. Such
that it can be 32 bit, even if 64 bit alignment is wanted.

Best regards
Michael

---

From: Randall Stewart <[email-masked]>
Date: Sun, 16 Apr 2006 08:23:37 -0400

Michael:

Though maybe a bit fuzzy.. I think your defintion is already
in 2960..

------------------------

    o  SCTP packet (or packet): The unit of data delivery across the
       interface between SCTP and the connectionless packet network
       (e.g., IP).  An SCTP packet includes the common SCTP header,
       possible SCTP control chunks, and user data encapsulated within
       SCTP DATA chunks.

----------

Notice.. it classify's all chunks as control except data.

Also there is a picture (in text) later on showing data
chunks coming in on one side and control on the other
(page 68)..

I think these defintions are good enough.... no implementor
has been confused by these...

I would agree that:
a) we need to test for this at inter-op's aka unknown chunks
    amongst the data.
b) we should probably test valid control amongst data

and

c) All future (including current drafts) extensions should declare
    that the chunk is either control or data... or as you
    any special appearance rules..

R

Michael Tuexen wrote:
>  From my point of view it is clear that all chunk types defined in  RFC 
> 2960
> except DATA chunks as control chunks, but this is not true. One  simply 
> does not
> know if an unknown chunk is a control chunks or not (it is definitely  not
> and only sending them if the peer has listed the chunk type N-DATA in  its
> problems. Even in the case the peer does not understand it and  consider 
> it a control
> chunk (which is wrong, but most implementation do today) this will  not 
> break

---

From: Mark Butler <[email-masked]>
Date: Sun, 16 Apr 2006 10:19:21 -0600

> Mark, wouldn't give this you the alignment you want?

With the qualifier that I readily concede the issue with regard to 
backward compatibility, would not have brought this up in the first 
place if I had known about it, and do not want to cause any more 
distress about a relatively obscure application of arbitrary chunk 
ordering, I would add the following:

1. A negotiated alignment of that type aligns the chunk header, not the 
payload.
2. Even if (1) weren't a problem, supporting arbitrary 2^N chunk 
alignment is a rather intrusive change, where simply skipping over an 
embedded pad chunk is trivial.
3. Consequently, in the environments where support for this would be 
beneficial, an option that simply indicates that an endpoint supports 
arbitrary chunk ordering would be adequate.  No conditional code on the 
receive side processing path required.

But as far as the PAD chunk is concerned here and now, I have no 
objection to dropping the issue.

Thanks,

   Mark B.

---

From: speakeasy <[email-masked]>
Date: Sun, 16 Apr 2006 15:26:57 -0700

> many implementations are out there..

In that context, why couldn't a Pad chunk be defined as a data
chunk that is delivered to the bit bucket rather than as a control
chunk that had no effect?