Subject: RE: [Tsvwg] SCTP Checksum Change to the IESG

From: [email-masked]
Date: Thu, 2 May 2002 11:06:13 +0300

Hi Scott & Doug,

I think that we need to get this over & done with it.  If this is as
important as Doug thinks it is (so I am willing to say it may be),
then lets just put it to a working group 'vote' - have both 
propsosed texts & have the WG decide which is appropriate.

best regards,
john

> While I agree with this sentiment, the introduction of a hardware
> implementation requirement is new text in conflict with previously raised
> concerns.  It appears Jonathan does not agree there is a need to allow
> implementations where the packet is not modified during the checking
> process.  This is a real problem.  This is true for hardware where an
> unmodified packet allows far greater software compatibility and should be
> the recommended approach.  I was happy with the draft prior to these last
> minute changes that appeared without review.  I was not the one twiddling.
> This is a wrong approach, if to standardize the hardware interface as is the
> intent of this language.
> 
> Presenting an unmodified packet would allow an Alder scheme to be introduced
> in software, as example, without any changes to existing code.  The same is
> equally true for a software CRC algorithm.  The scheme, as defined now, will
> require specialized code to handle such a situation.  The current
> recommendation is clumsy and wrong.  More generalized language would be
> acceptable such as- For hardware checking schemes, software MUST be able to
> confirm the integrity of the packet and not rely exclusively on a status
> bit.  An unmodified packet accomplishes this goal.  This added language has
> compounded an inappropriate algorithm interpretation.

_______________________________________________
tsvwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tsvwg

_______________________________________________
tsvwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tsvwg

---

From: [email-masked]
Date: Thu, 2 May 2002 14:38:38 +0300

Hi Scott,

> if others think that Doug has a point that is important enough to
> delay this document even longer please speak up - if there is not
> support it is time to move on - if there is support (and more than 
> one or two people) then we will delay yet again

That sounds like a reasonable way forward.

John

---

From: Tuexen Michael <[email-masked]>
Date: Thu, 2 May 2002 17:57:12 +0200

Doug,

I agree 100% with Randy. Now an author of the ID says that
you can do what you want to do and that is covered by the
ID there is no reason for objection.

Best regards
Michael

Michael Tuexen    Tel.:   +49 89 722 47210
Siemens AG        Fax:    +49 89 722 48212
ICN WN CC SE 7    E-mail: [email-masked]

> -----Original Message-----
> From:	Randall Stewart [SMTP:[email-masked]]
> Sent:	Thursday, May 02, 2002 4:13 PM
> To:	Douglas Otis
> Cc:	[email-masked]
> Subject:	Re: [Tsvwg] SCTP Checksum Change to the IESG
> 
> Douglas Otis wrote:
> > 
> >  "An implementation using hardware assist is encouraged to follow the
> >   exact procedure specified above.  Whatever implementation strategy
> >   an implementation chooses, it MUST arrange to provide both the
> >   initial value of the checksum field, and the value computed for the
> >   SCTP checksum, to the software stack."
> > 
> 
> Doug the only requirement above is that the hardware must let the
> software see both the initial value in the checksum field .. i.e.
> what came off the wire AND the value computed by the hardware..
> 
> I think what Jonathan was after here is so that you don't get a
> GO/NO-GO from the hardware..
> 
> This does NOT discourage you from doing non-alter techniques to
> the data packet.. since by non altering the packet you are letting
> the software see the value off the wire..
> 
> I don't understand your objection.. the wording lets you do
> what you want and just recommends a specific implementation...
> 
> R
> 
> -- 
> Randall R. Stewart
> [email-masked] 815-342-5222 (cell phone)
>

---

From: [email-masked]
Date: Thu, 2 May 2002 15:02:32 -0600

Scott,

I realized I made an error in what I sent earlier in regards to how the
magic number (the constant result of the good CRC check) is derived. It only
occurs because the CRC appended to the data is inverted. Therefore when one
runs a CRC computation over the data stream plus the inverted CRC, one ends
up with a remainder of dividing the polynomial FFFF0000 (for a 32-bit CRC)
by the CRC polynomial which is the magic number. If the CRC was not
inverted, the result would be zeros.

In verifying that the CRC computation for SCTP does invert the appended CRC
value, I noticed that the description of coefficient mapping to bits is
incorrect in the detailed description.

The paragraph at the bottom of page 4 states clearly that the first byte of
the payload has the eight highest coefficients which is good. It also says
that the least significant bit within a byte is the most significant
polynomial quotient within the byte.

However, when the calculation is described in detail on the next page, it
says: 

data bytes are numbered from 0 to N-1 which implies that the first byte is
byte 0 and the last byte is byte N-1 and
bit 0 of byte j is coefficient of x^(8j-1) and bit 7 is coefficient of
x^(8j-8). bit 0 is the most significant bit in the usual IETF numbering so
this results in the most significant bit being the most significant
coefficient of the byte and byte 0 - the first byte - containing the least
significant coefficients. (Actually the caret is in the wrong place in the
equation for bit 7.)

This is backwards from earlier description both for bit order and byte order
and also results in terms with negative exponents. Byte 0 would have
coefficients for x^(8*0-1) and x^(8*0-8) or x^-1 to x^-8.

the second bullet item should be:

           - the transport-level byte-stream is mapped to a polynomial
value.
	     An N-byte PDU with bytes 0 to N-1, is  considered as
	     coefficients of a polynomial M(x) of order 8N-1, with
	     bit 0 of byte j being coefficient x^(8(N-j)-8), bit 7 of  byte
	     0 being coefficient x^(8(N-j)-1). 

Regards,
Pat

-----Original Message-----
From: THALER,PAT (A-Roseville,ex1) 
Sent: Thursday, May 02, 2002 11:28 AM
To: 'Scott Bradner'; [email-masked]; [email-masked];
[email-masked]
Subject: RE: [Tsvwg] SCTP Checksum Change to the IESG

Scott,

The existing text does not reflect the way most hardware performs CRC
checks. The normal approach does not modify the packet passing through the
CRC checker at all. It does not zero out the CRC field. The most commonly
implemented hardware CRC generator/checker uses the simultaneous multiply
and divide (SMD) method which does an operation which is equivalent to
multiplying by x^32 (for a 32 bit CRC polynomial) and dividing by the CRC.
When this done in the checker by running the covered data and the received
CRC through the checker, the final value in the CRC checker will be a
constant.

This method is normally used because it uses the same circuit for the
checker as for the generator, it doesn't require handling any part of the
received message specially. You just run the checker until the data runs out
and look at the result.

(Because this method multiplies the checked packet by x^32, a polynomial
with a correct CRC produces a result equal to the remainder of x^32 divided
by the CRC polynomial.)

The common hardware implementation would be:
Compute the CRC with SMD over the covered data including the received CRC.
Compare the result to the magic number.

This type of implementation can either strip the original CRC or leave it on
when passing the data to memory. It normally does not retain the CRC it
computed.

It makes reasonable sense to pass up the received CRC, but recommending
passing up a computed new CRC would be an unreasonable constraint on CRC
implementations. Because following the recommendation would require a
different checker than the kind that is normally implemented, it could
increase the likelyhood of bad implementations.

Regards,
Pat Thaler

-----Original Message-----
From: Scott Bradner [mailto:[email-masked]]
Sent: Thursday, May 02, 2002 4:23 AM
To: [email-masked]; [email-masked]; [email-masked]
Subject: RE: [Tsvwg] SCTP Checksum Change to the IESG

I disagree

Scott

---

From: "THALER,PAT (A-Roseville,ex1)" <[email-masked]>
Date: Thu, 2 May 2002 15:37:38 -0600

The text says that "Any implementation using hardware assists for checksum
computation MUST make the final CRC value available to the software portion
of the stack.  Such implementations SHOULD also make the buffer size used by
the hardware unit available to software."

That looks like a requirement to pass up the computed CRC (assuming "final"
CRC means the one computed by the receiving hardware). The problem I have
with such a requirement is that different valid types of hardware CRC
checkers produce different kinds of results which are not necessarily the
same as the value in the CRC at the end of the good packet so giving the
result of the CRC check process doesn't seem useful to software unless the
hardware checker is overconstrained. It isn't quite clear to me what "make
the final CRC value available" means. I also don't understand why the buffer
size of the hardware unit should be made available or what you mean by
buffer size. Various hardware implementations organize their memory in a
variety of ways. For instance, PDUs are often stored in a linked list of
short buffers to optimize memory utilization. Does the software want to know
that the hardware used 64 bytes or 128 bytes for that purpose? 

I suspect you may mean that the hardware SHOULD make available the size of
the data that it ran the check over which might be useful in debugging.
Generally hardware implementations have some way to access a wide variety of
state information for debugging but that is an interface specialized to the
particular hardware and I don't see where it is useful to put some of that
type of items into this standard. 

Getting the data and the received CRC allows software to run a check on
hardware and that is all that should be required or recommended.

Pat 

-----Original Message-----
From: Scott Bradner [mailto:[email-masked]]
Sent: Thursday, May 02, 2002 11:29 AM
To: [email-masked]; [email-masked]; [email-masked];
[email-masked]; [email-masked]
Subject: RE: [Tsvwg] SCTP Checksum Change to the IESG

> implementations. 

I don't quite get this objection

the text in question makes a recommendation it does not mandate
and it does not claim that its the only way

Scott

---

From: "THALER,PAT (A-Roseville,ex1)" <[email-masked]>
Date: Mon, 6 May 2002 12:02:04 -0600

Dear Jonathan,

Getting data to and from a NIC generally involves going through a FIFO
crossing a clock boundary and some engineers design this crossing poorly
resulting in a circuit that occasionally passes the wrong data. It is
certainly possible to design the circuit so that it performs well and does
not exhibit this behavior. I've seen plenty of hardware that does it right.

I don't think that the statistic of 1 in 10,000 internet frames having a TCP
checksum error and a good Ethernet CRC indicates anything about the
prevalence of this kind of NIC error. Frames pass through middle boxes such
as routers where they are not covered by CRC and some portion of the errors
are occurring there. I know of cases where a middle box had internal bugs
that occasionally corrupted the data.

Even if this kind of error is a concern, I don't see how it is helped by
returning a CRC calculated by the checker. 

The concern in the existing draft seems to center on the checker, but the
hardware calculating the CRC to send with the packet is equally likely to
make an error as the checker receiving the packet. Whatever text is
developed should deal with both equally.

Probability of an undetected error due to an error by the CRC checker in the
NIC is:

 Pneterr * Pnicerr * Pcrcmatch

where 
Pneterr is the probability of an error to the packet on the network
Pnicerr is the probability that the NIC makes an error in the data over
which it calculates the CRC
Pcrcmatch is the probability that the CRC calculated over data with errors
matches the received CRC. As long as the probability of an error in the NIC
CRC checker is much less than one, this is much smaller than the probability
of an undetected error due to errors on the network alone which would be:

 Pneterr * Pcrcmatch

Since undetected errors by the CRC checker will be quite rare, an audit is
highly unlikely to catch the problem.

The most significant problem caused by errors in CRC offload hardware would
be errors in CRC generation.  If the CRC generator makes an error and the
retransmissions continue to use the same CRC rather than recalculating it,
then the frame will continue to get rejected. The secondary problem is that
errors in CRC checking will occasionally cause good frames to fail the CRC
check. 

I can see retaining the received CRC because it allows software to do a
double check that a packet which the hardware says failed CRC check really
did fail. I can see making the number of bytes processed through the CRC
generator and checker available to software (which I assume is what was
meant by "buffer size" in the draft), because a synchronizer error normally
results in processing extra bytes or dropping some bytes. If so, "buffer
size" should be changed to something like "number of bytes over which the
CRC was computed not including the received CRC" and should also be applied
to the generator. Buffer size in hardware often has little or no
relationship to frame size.

I can't see any purpose served by retaining the result calculated by the CRC
checker especially since that result will vary unless CRC checker
implementations are needlessly constrained.

Regards,
Pat Thaler

-----Original Message-----
From: Jonathan Stone [mailto:[email-masked]]
Sent: Friday, May 03, 2002 2:02 PM
To: Tuexen Michael; 'Scott Bradner'; [email-masked];
[email-masked]'; [email-masked]; [email-masked]
Cc: [email-masked]
Subject: Re: [Tsvwg] SCTP Checksum Change to the IESG 

Dear all,

Michael Tuexen and I have discussed the contentious text offline at
some length.  One example we addressed is a NIC with TCP checksum
assist in hardware.  The checksum assist in that NIC will occasionally
deliver incorrect TCP checksum indications for packets smaller than
the minimum Ethernet segment.

I hesitate to say such bugs are commonplace, but they certainly
aren't' rare: both in current, shipping server-class hardware and in the
previous generation of bugs. The open-source community device drivers
show the history of first enabling TCP checksum assist, then disabling,
then finding workarounds or diagnostic tests which are cheaper than
the alternative: deciding the checksum-assist hardware is busted,
and ignoring it altogether.

That is what the contentious text is saying. It is not saying
*anything* about how a hardware CRC assist should compute the CRC.  It
*is* saying that hardware assists for SCTP must provide some
additional ``audit-trail'' information on what the hardware really
did.  We can call that information ``CRC assist framing'': since it
leaves a record of where the CRC hw took its checksum value; the
length of the CRC computation; and the final result.

That said, here's an attempt at some text which explains when, and
where, and why, that audit information is necessary:

-----
   Errors in NICs and in outboard checksum acceleration have,
   historically, been commonplace. This document requires that
   hardware assists for the SCTP CRC provide more information than
   a single CRC-GOOD/CRC-NOT-GOOD bit.  Suppose that some particular
   NIC (or model of NIC) will occasionally give incorrect `CRC-GOOD'
   status for packets that are in fact damaged. In this scenario,
   GOOD/NOT-GOOD implementations give the SCTP software no recourse but
   to ignore the CRC assist entirely, and recompute the CRC of
   every packet in software.

   This extra ``CRC-assist framing'' information  gives
   software the option to apply a cheaper audit of the CRC assist
   result based on the framing, before resorting to a software CRC.
   Such checks are commonplace in the installed base of TCP stacks
   using hardware- assisted TCP checksum.

   Moreover, the CRC hardware assist, its associated NIC,
   and the NIC  device-driver will  in practice often
   be provided by a different vendor than is the SCTP stack.
   To ensure that the desired CRC-assist audit trail crosses the
   NIC-vendor/SCTP-stack-vendor boundary, this additional
   information is [currently] specified as a MUST.

-----

That isn't quite up to snuff for an I-D, but it will be soon:) and it
gets the idea across.  I hope Michael Tuexen will repost (or give me
permission to repost) an excerpt of our discussion.  I'm hoping that
together, those will resolve most of the contentiousness.

An anecdote: when I gave a talk on the background data at SIGCOMM 2000
in Stockholm, it was a hardware person (not a software guy) who
commented that there was a `right' way and a `wrong' way to do
checksum-offload assists; and that the one-bit GOOD/NOT-GOOD result
was a bad way which should be discouraged.  For pretty much the
reasons I give here, and of which I walked through an explicit example
with Michael.  (To be fair, this was in a context of presenting data
showing that some 1 in 10,00 frame in the Internet pass their Ethernet
CRC, but fail the TCP checksum -- often in circumstances which point
squarely at bugs in NICs.)

Even so, I'm surprised that this has been so contentious.

--Jonathan