Subject: Re: [dtn] CRC options

From: "Burleigh, Scott C (312B)" <[email-masked]>
Date: Tue, 1 Mar 2016 17:40:45 +0000

I think we have approached something like general (though not specific) consensus on our CRC question, and I've seen no objection to the proposed resolution to our other open bpbis technical issue (moving the bundle block inventory/manifest to an extension block and signaling the existence of that block with a flag in the primary block).  Since the deadline for posting Internet Drafts prior to the Buenos Aires meeting is only three weeks away, I propose to post one more edition of the bpbis I-D shortly, differing from bpbis-02 as follows:

1.	Valid values of the "CRC type code" identified in section 4.1 will be 0 (block contains no CRC), 1 (block contains a CRC-16, a.k.a. CRC-16-ANSI), and 2 (block contains a CRC-32), exclusively.

2.	All blocks, not just the primary block, will have CRC type codes and may therefore include CRCs.

3.	An ASCII-art diagram illustrating the interaction of the various components of the BPA will be added.

4.	A flag will be added to the primary block indicating the presence or absence of a "manifest" extension block (TBD in a separate document) that identifies the blocks that were present in the bundle at the time it was created.

If you object to these resolutions, please speak up.  I hope to post the new I-D by the end of this week.

Scott

-----Original Message-----
From: Carlo Caini [mailto:[email-masked]] 
Sent: Saturday, January 23, 2016 1:50 PM
To: Sebastian Schildt <[email-masked]>; Burleigh, Scott C (312B) <[email-masked]>
Cc: [email-masked]; [email-masked]; [email-masked]
Subject: RE: [dtn] CRC options

Dear Sebastian,
     I agree with you on the fact that CRC varieties should not be enlarged without real necessity. Let me just to point out that the peculiar value of CRC at BP is to protect the bunle data against possible flipped bits in the processing or storing inside a machine, where they cannot be protected by lower layers integrity mechanisms. Of course these BP CRCs offers also an additional protection against bits that are occasionally flipped in the actual Tx from a node to another, i.e. when their integrity may be protected by lower layers CRC. If this integrity is protected mduring Tx, an additional protection is not bad as checksums, as CRC, are not infallible. If not, the BP protection becomes essential. This is for instance the case of LTP green-part, where losses of LTP segments do not prevent the delivery of corrupted bundles. This could be also the case of UDP with optional checksum switched off. 
More generally, BP source and destination cannot have control of all DTN hops between them, therefore they cannot take for granted that in all hops in between integrity mechanisms of lower layers are applied, or that intermediate nodes' hardware is error free. 
This is why I am in favor of CRC-32 (however, I suppose that one 32 bit option be enough) as an option for all blocks. It is just better than nothing or CRC-16.
Yours,
   Carlo

________________________________________
Da: dtn [[email-masked]] per conto di Sebastian Schildt [[email-masked]]
Inviato: sabato 23 gennaio 2016 21.58
A: Burleigh, Scott C (312B)
Cc: [email-masked]; [email-masked]; [email-masked]
Oggetto: Re: [dtn] CRC options

Hi Scott,

that is valid point, and I know you are always eager to shave off a few bits of overhead at the lower end of what the BP can do :) .

While I am not sure it is really neccessary to make this difference, I could accept two CRCs. However, my feeling is, that most blocks will be rather small, so maybe even a 16bit CRC is enough. As for  payload blocks, they might conceivably be so large (think multiple TB for extreme cases.), that even a "large" CRC to secure it all would not be adequate (and a small 16 bit CRC would still catch a flipped bit)

So for the record: In my opinion 16 bit or 32 bit (as some have argued for in this thread), or two specific CRCs with a size threshold would all be ok for me (although I would prefer a single CRC). Only if we are planning to use multiple different CRC families and lengths I would strongly oppose.

Sebastian

> On 23 Jan 2016, at 19:12, Burleigh, Scott C (312B) <[email-masked]> wrote:
>
> Hi, Sebastian.  I agree with you in concept, but I would suggest that we specify two supported CRCs for BP rather than one.  You are right that there is no way to identify a perfect CRC for BP because of the very broad range of possible variations in the underlying convergence layer stack.  But the CRCs we are talking about would be applied to individual blocks of the bundle rather than to the bundle as a whole, and block size is the one relevant parameter that will always be exposed to the BP implementation at the time a CRC is to be applied.  It makes sense to me to specify one CRC type for very small blocks (the primary block, most or all extension blocks, and small payloads) and one other for large payload blocks.
>
> Scott
>
> -----Original Message-----
> From: Sebastian Schildt [mailto:[email-masked]]
> Sent: Saturday, January 23, 2016 1:58 AM
> To: [email-masked]
> Cc: [email-masked]; Burleigh, Scott C (312B); [email-masked]
> Subject: Re: [dtn] CRC options
>
> Hello,
>
> I haven't followed the CRC discussion (f there as any) closely in the past. Therefore, regarding the specific CRC or the length I am fine with everything.
>
> However, if this has not been discussed before, I would strongly suggest specifying only ONE CRC valid for the BP: As a highler layer protocol a CRC in the BP is only a sort of "catch-all" indicating whether something went wrong during transmission/storage or processing, similarly in scope to for example the relatively braindead TCP checksum. Therefore, there is no way to identify a CRC checksum with  the "perfect properties" for the BP. Only in the protocols employed by convergence layers  can error detection and correction be tailored to the specific challengs of that concrete channel.
>
> Therefore, to ease implementation and interoperability hurdles, I 
> strongly suggest specifying only one kind of acceptable CRC for the BP 
> (maybe that was already the plan, but reading this thread it seems 
> people are converging towards a "suite" of CRCs)
>
>
>
> Sebastian
>
>
>> On 19 Jan 2016, at 10:56, [email-masked] wrote:
>>
>> Hey everyone,
>> I would strongly suggest allowing CRC-32, as well as CRC-16. Hardware support for CRC-32 is fairly widespread, and (as Tomaso said) used widely within industry.  As for which algorithms are utilized, I think we should keep it limited (as much as is reasonable) to a small set of strong, well-used, and frequently implemented CRC's, such as CRC-16 CCITT, CRC-16 ANSI, and CRC-32(C). Keeping the number small obviously simplifies implementation, and also allows (for example) a single byte of mask data for checksumming ID information.
>>
>> Thanks,
>> Jeremy
>> -----Original Message-----
>> From: dtn [mailto:[email-masked]] On Behalf Of 
>> [email-masked]
>> Sent: Dienstag, 19. Januar 2016 09:10
>> To: [email-masked]
>> Cc: [email-masked]
>> Subject: Re: [dtn] CRC options
>>
>> Hi Scott,
>>
>> I see you are mentioning only 16-bits CRC: does it mean that we should exclude 32-bits CRC? Since its use is quite widespread, I would not drop it, unless there are strong arguments against it. Alternatively, I was wondering whether we could support different  CRC-lengths and provide an extra bit (or a mask) to indicate which CRC class is being used.
>>
>> Regards,
>>
>> Tomaso
>>
>> ------------------------
>> Deutsches Zentrum für Luft- und Raumfahrt (DLR) German Aerospace Center Institute of Communications and Navigation | Satellite Networks | Oberpfaffenhofen | 82234 Wessling | Germany Tomaso de Cola, Ph.D.
>> Telefon +49 8153 28-2156 | Telefax  +49 8153 28-2844 | 
>> [email-masked] http://www.dlr.de/kn/institut/abteilungen/san
>>
>>> -----Original Message-----
>>> From: dtn [mailto:[email-masked]] On Behalf Of Burleigh, Scott 
>>> C
>>> (312B)
>>> Sent: Monday, January 18, 2016 18:45
>>> To: [email-masked]
>>> Subject: [dtn] CRC options
>>>
>>> Hi.  As discussed in this morning's virtual meeting, I think we are 
>>> down to just two open technical issues in the new Bundle Protocol specification ("bpbis").
>>> One of those issues is the question of providing a bundle manifest (a.k.a.
>>> "inventory") indicating what blocks were originally present in the 
>>> bundle; I'll start a separate email thread on that topic in a few 
>>> minutes.  The other issue is, I think, simpler: which specific 
>>> mechanisms for constructing and validating the optional CRC in the primary block should the specification identify as valid?
>>> CRC-16-ARINC, CRC-16-CCITT, CRC-16-CDMA2000, CRC-16-DECT, etc.?
>>>
>>> Your opinion is welcome.
>>>
>>> Scott
>>>
>>> _______________________________________________
>>> dtn mailing list
>>> [email-masked]
>>> https://www.ietf.org/mailman/listinfo/dtn
>>
>> _______________________________________________
>> dtn mailing list
>> [email-masked]
>> https://www.ietf.org/mailman/listinfo/dtn
>>
>> _______________________________________________
>> dtn mailing list
>> [email-masked]
>> https://www.ietf.org/mailman/listinfo/dtn
>

---

From: "Burleigh, Scott C (312B)" <[email-masked]>
Date: Fri, 4 Mar 2016 22:34:22 +0000

Hi.  I just posted version 03 of the bpbis Internet Draft; it includes all the changes listed below plus a few editorial tweaks, plus one change in the language for bundle status reports: I came across an additional case in which a status report appears to be mandatory, and an additional restriction is needed on the block processing flag that produces that case.

I also took a first cut at an initial "representation specification", describing the use of CBOR in encoding/decoding bundles.  The I-D is draft-burleigh-dtn-rs-cbor-00.txt.

Scott

-----Original Message-----
From: dtn [mailto:[email-masked]] On Behalf Of Burleigh, Scott C (312B)
Sent: Tuesday, March 1, 2016 9:41 AM
To: [email-masked]
Subject: Re: [dtn] CRC options

Scott

-----Original Message-----
From: Carlo Caini [mailto:[email-masked]]
Sent: Saturday, January 23, 2016 1:50 PM
To: Sebastian Schildt <[email-masked]>; Burleigh, Scott C (312B) <[email-masked]>
Cc: [email-masked]; [email-masked]; [email-masked]
Subject: RE: [dtn] CRC options

Dear Sebastian,

Hi Scott,

Sebastian

---

From: Rick Taylor <[email-masked]>
Date: Wed, 23 Mar 2016 15:56:38 +0000

> I also took a first cut at an initial "representation specification", describing the use of CBOR in encoding/decoding bundles.  The I-D is draft-burleigh-dtn-rs-cbor-00.txt.

Thanks for doing the additional CBOR representation, Scott - I 
completely missed your notification of the posting. This is great work.

My one query is whether the informational info (tag types) need to be 
MUSTs. Perhaps some boilerplate text way of saying "XXX SHALL be 
represented as a CBOR unsigned integer (major type 0) item, and if tags 
are in use, SHALL use tag value 24" (I don't think type 0 and tag 24 is 
valid, but it's an example).

 From my understanding of CBOR, tags are always optional, hence my 
concern about SHALL.

Regards,

Rick

---

From: Carsten Bormann <[email-masked]>
Date: Wed, 23 Mar 2016 21:01:10 +0100

Rick Taylor wrote:
>  From my understanding of CBOR, tags are always optional,

The CBOR format does not prescribe the use of tags.
However, if there are datatypes (such as the bignum datatypes) that
benefit from using tags, an application can very much say that you are
supposed to use them.  E.g., COSE will define a number of tags that you
need to use to tell apart different types of crypto unless those are
implied from the environment (e.g., the media type specified in the CoAP
Content-Format Option).

I hadn't noticed draft-burleigh-dtn-rs-cbor-00.txt -- interesting.
I just had a quick scan of it.  I'm not a big fan of specifying details
such as "Each DTN time SHALL be represented as a CBOR unsigned integer
(major type 0) item with additional info 26" -- this specific example of
course will blow up 2136-02-07, but more generally, I prefer it left up
to the generic encoder/decoder to choose these details; if you specify
them here, you might not be able to use a generic encoder that just
happens to choose different.  (More annoyingly, it generates unneeded
text such as "with additional info and supplementary bytes (as needed)
indicating the sequence number value" where no such specification is
intended.)  URIs are text strings and should probably be using major type 3.

I might try writing up the content of draft-burleigh-dtn-rs-cbor-00.txt
in CDDL later...

(BTW: Does DTN really use UTC, or shouldn't this be TAI?  Probably this
has been discussed to death, sorry about that.)

Grüße, Carsten

---

From: "Burleigh, Scott C (312B)" <[email-masked]>
Date: Thu, 24 Mar 2016 00:00:22 +0000

Carsten, do I understand correctly that you'd advise just saying (for example) "Each DTN time SHALL be represented as a CBOR unsigned integer (major type 0) item", omitting the additional info specification?  Or would you advocate not specifying the major type either?

Scott

(Yes, DTN really does use UTC, and it has indeed been discussed to death.)

-----Original Message-----
From: Carsten Bormann [mailto:[email-masked]] 
Sent: Wednesday, March 23, 2016 1:01 PM
To: Rick Taylor <[email-masked]>
Cc: Burleigh, Scott C (312B) <[email-masked]>; [email-masked]
Subject: Re: [dtn] CBOR representation

The CBOR format does not prescribe the use of tags.
However, if there are datatypes (such as the bignum datatypes) that benefit from using tags, an application can very much say that you are supposed to use them.  E.g., COSE will define a number of tags that you need to use to tell apart different types of crypto unless those are implied from the environment (e.g., the media type specified in the CoAP Content-Format Option).

I hadn't noticed draft-burleigh-dtn-rs-cbor-00.txt -- interesting.
I just had a quick scan of it.  I'm not a big fan of specifying details such as "Each DTN time SHALL be represented as a CBOR unsigned integer (major type 0) item with additional info 26" -- this specific example of course will blow up 2136-02-07, but more generally, I prefer it left up to the generic encoder/decoder to choose these details; if you specify them here, you might not be able to use a generic encoder that just happens to choose different.  (More annoyingly, it generates unneeded text such as "with additional info and supplementary bytes (as needed) indicating the sequence number value" where no such specification is
intended.)  URIs are text strings and should probably be using major type 3.

I might try writing up the content of draft-burleigh-dtn-rs-cbor-00.txt in CDDL later...

(BTW: Does DTN really use UTC, or shouldn't this be TAI?  Probably this has been discussed to death, sorry about that.)

Grüße, Carsten

---

From: "Burleigh, Scott C (312B)" <[email-masked]>
Date: Thu, 24 Mar 2016 02:34:17 +0000

Rick, maybe I don't have as good a grasp of CBOR as I thought I did: my understanding was that "additional information" was distinct from tags.  But in any case, if I understood Carsten properly, I'm fine with dialing back to just specifying major types in most cases, as in "XXX SHALL be represented as a CBOR unsigned integer (major type 0) item."

Scott

-----Original Message-----
From: Rick Taylor [mailto:[email-masked]] 
Sent: Wednesday, March 23, 2016 8:57 AM
To: Burleigh, Scott C (312B); [email-masked]
Subject: Re: [dtn] CBOR representation

> I also took a first cut at an initial "representation specification", describing the use of CBOR in encoding/decoding bundles.  The I-D is draft-burleigh-dtn-rs-cbor-00.txt.

Thanks for doing the additional CBOR representation, Scott - I completely missed your notification of the posting. This is great work.

My one query is whether the informational info (tag types) need to be MUSTs. Perhaps some boilerplate text way of saying "XXX SHALL be represented as a CBOR unsigned integer (major type 0) item, and if tags are in use, SHALL use tag value 24" (I don't think type 0 and tag 24 is valid, but it's an example).

 From my understanding of CBOR, tags are always optional, hence my concern about SHALL.

Regards,

Rick

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 24 Mar 2016 05:53:28 +0100

> Carsten, do I understand correctly that you'd advise just saying (for example) "Each DTN time SHALL be represented as a CBOR unsigned integer (major type 0) item", omitting the additional info spec…

Specifying the major type is fine, although this is implied by saying
"unsigned integer" or "byte string".

> (Yes, DTN really does use UTC, and it has indeed been discussed to death.)

Thanks...

Grüße, Carsten

---

From: Rick Taylor <[email-masked]>
Date: Thu, 24 Mar 2016 10:14:46 +0000

> Rick, maybe I don't have as good a grasp of CBOR as I thought I did: my understanding was that "additional information" was distinct from tags.  But in any case, if I understood Carsten properly, I…

I'm not sure I'm right either, I just wanted to flag up the issue.

I think CBOR is one of those elegantly simple encodings, with some 
subtleties.  If anyone knows the best way forward here, it's probably 
Carsten.

Watching with interest!

Rick