Subject: Re: [manet] New Version Notification for draft-clausen-manet-rfc5444-usage-00.txt

From: Lotte Steenbrink <[email-masked]>
Date: Tue, 6 Oct 2015 20:15:43 +0200

Hi all,
I realize vacation season and/or other circumstances have slowed most of us down at one point or another, but I was wondering if one of the authors could give us a very brief update on how the work on this draft is doing? 

I'm particularly interested in an answer to my earlier question concerning section 4.6., i.e.: if a message needs to be regenerated to be forwarded, and this message contains TLVs that the regenerating 5444 parser/builder doesn't recognize, should it add the unknown TLVs to the message (which would mean that “MUST NOT remove suchTLVs or values” applies, which should imo be clarified in the draft) or should the unknown TLVs be discarded (which would need clarification too, imo)?

If the answer is “we're working on it, hold on for a few more days”, that's completely fine, I'd just like to know if/when I can find out how to do this correctly. :)

Regards,
Lotte

Am 09.06.2015 um 16:26 schrieb Lotte Steenbrink <[email-masked]>:

> Hi Ulrich, Thomas, Chris and Henning,
> 
> Am 31.05.2015 um 13:33 schrieb Ulrich Herberg <[email-masked]>:
> 
>> As we announced previously, we have been working on a document specifying rules for how to use RFC5444 in MANET protocols. Well, here it is (at least in a -00 version). 
>> We'd appreciate reviews and will soon ask for WG adoption.
> 
> first of all, thank you so much for publishing this much-needed draft. I'm also in favor of WG adoption. While reading through the document, I wrote down some nitpicks and suggestions that I'd like to share.
> (Just in case my writing sounds harsh, I fear that's due to language/cultural differences. I'm really happy about this document and would like to see it flourish.)
> 
> == General notes ==
> - If I didn’t misunderstand it, the document addresses two groups of people: Those who want to design RFC5444 messages for their protocol, and those wo want to implement a RCF 5444 builder/parser. Imo, The former group needs more of an outside view (i.e. what can/should I put where), while the latter needs the inside view (i.e. how do I arrange and process everything efficiently). To my mind, these are two very different goals, but while reading the document, it wasn’t always instantly clear to me which bit addresses which group… I realize the two can’t always be separated completely, but would it be possible to make it more clear which group is addressed currently? Maybe the document could be organized into two big sections (one for each group), or there could be a recurring pattern of “first outside then inside view” for each (sub)section?
> 
> - A lot of the examples are very NHDP/OLSR-specific. While I think it’s great to have pointers to where a certain technique has already been used, from an outsider’s perspective, the examples don’t always help me understand the technique that’s being illustrated, since I have to familiarize myself with the details referenced first… I’m wondering if it would be possible to keep the examples which illustrate a point more generic, and then briefly point to existing instances of this situation in case the reader wants to follow up on that?
> 
> == Section by section ==
> Section 1.2., Page 4:
>      “A packet is designed to as travel […]” I think the “as” doesn’t belong here.
> 
> Section 4.2., Page 8:
>      “Outgoing messages MAY be created by owning protocol” I think there’s a “the” missing.
> 
> Section 4.3., Page 8:
>      If I understand it correctly, the last paragraph on this page is talking about multi-value TLVs without actually saying “multi-value TLV”. Maybe that could be stated explicitly for clarity and in case somebody’s searching for advice by this keyword?
> 
> Section 4.4., Page 9/10:
>      The last paragraph on page and the first two on page 10, more specifically these parts:
>      
>           Such an example already exists (but within the basic specification, rather  
>           than as an extension) in the use of LOST values in the LINK_STATUS  
>           and OTHER_NEIGHB TLVs to report that an address is of a router known  
>           not to be a neighbor.  A future example might be to list an address  
>           to be added to a "blacklist" of addresses not to be used.
>      
>           This example can be taken further.  NHDP must also not reject a HELLO  
>           message because it contains an unrecognized TLV. This also applies
>           to unrecognized TLV values, where a TLV supports only a limited set
>           of values.
> 
>      were a bit hard to read for me. Imho, the example is very specific and contains details which aren’t necessary to understand the issue at hand (such as the fact that this extension is technically not a completely independent extension, or possible future other extensions). It took me about 4 re-reads to really grasp what’s going on there...
> 
>      Also, I would’ve expected the last paragraph of this section to clearly state that relying on address order *will* break your messages if you don’t have control over all 5444 builders that are used to implement your protocol ever.
> 
> Section 4.5.:
>      I'm assuming that this is the response to our (the AODVv2 Author team and possibly others) complaints that RFC5444 lacks a clear "API" which tells it what we can and cannot set manually in an RFC 5444 packet, and that there's no (at the risk of sounding really pretentious now) "visual language" to describe the contents of a 5444 message and their relationship to each other. To be honest, I'm a bit puzzled by this section.... The mapping that is presented is a bit vague and introduced very promptly. And while I strongly agree with the statement that writing down an actual API is too language- and implementation-specific to belong in a document like this, it would be really helpful to have a comprehensive "these are the things that your protocol can set in a RFC5444 message. Your packet builder/parser will figure out the rest." List in this section rather than some examples.
> Along with this list, it would be great to have a clear specification on how to specify which Addresses and TLVs a specific message has, and how their relationship to each other are. (i.e. how to make clear which address TLVs are associated with which addresses). I don’t care if this ends up being a very visual approach (like the debugging output of Henning’s oonf API [1]), or the tables that OLSRv2 uses, or something completely different, but it would be helpful to be able to read and write messages down in a way that is clear to everyone.
> 
> Section 4.6:
>      “[…] and MUST NOT remove such TLVs or values”
>      A question I’ve been wanting to ask for a while now: Does this rule also apply for message regeneration? AODVv2 regenerates all messages per hop. Should implementations keep track which unknown TLVs of a message are associated with which addresses, and put them back in their place when regenerating the message?
> 
> Section 6.1.:
>      (I’m not terribly familiar with compression, so my questions are probably very naive, but maybe they can help clear things up)
>      Would this very straightforward to implement compression algorithm still produce interoperable messages? The rest of the section reads as if the “compression algorithm” mentioned in the introduction is mainly a way to cleverly arrange addresses so that they are still understood by all parsers, but take up a minimal amount of space. I think that’s a very good tip in itself, but is that really all there is to compression? Would it make sense to say “Here’s how you should arrange addresses cleverly, you can also implement a more elaborate compression algorithm, but that’s out of the scope of this document”?
> Also, any mention of the word “straightforward” makes me very suspicious, but that might just be me.
> 
> Section 6.1.:
>      “RECOMMNDED” in the first paragraph is missing an E.
> 
> Regards,
> Lotte
> 
> 
> [1] Random one pulled out of my logfiles for illustration:
> 
>      ,------------------
>      |  PACKET
>      |------------------
>      | * Packet version:    0
>      | * Packet flags:      0x0
>      |    ,-------------------
>      |    |  MESSAGE
>      |    |-------------------
>      |    | * Message type:       11
>      |    | * Message flags:      0x40
>      |    | * Address length:     16
>      |    | * Hop limit:          20
>      |    |    ,-------------------
>      |    |    |  Address: fe80::ff:fe00:1719
>      |    |    |    | - TLV
>      |    |    |    |     Flags = 0x50
>      |    |    |    |     Type = 0
>      |    |    |    |     Value length: 2
>      |    |    |    |       0000: 0100
>      |    |    `-------------------
>      |    |    ,-------------------
>      |    |    |  Address: fe80::ff:fe00:16fe
>      |    |    |    | - TLV
>      |    |    |    |     Flags = 0x50
>      |    |    |    |     Type = 1
>      |    |    |    |     Value length: 2
>      |    |    |    |       0000: 0100
>      |    |    |    | - TLV
>      |    |    |    |     Flags = 0xd0
>      |    |    |    |     Type = 3; Type ext. = 3
>      |    |    |    |     Value length: 1
>      |    |    |    |       0000: 00
>      |    |    `-------------------
>      |    `-------------------
>      `------------------
> 
>> 
>> Best regards
>> Ulrich 
>> 
>> Begin forwarded message:
>> 
>>> From: [email-masked]
>>> Date: May 28, 2015 at 11:12:35 GMT+3
>>> To: "Christopher Dearlove" <[email-masked]>, "Henning Rogge" <[email-masked]>, "Thomas Clausen" <[email-masked]>, "Henning Rogge" <[email-masked]>, "Ulrich Herberg" <[email-masked]>, "Thomas H. Clausen" <[email-masked]>, "Christopher Dearlove" <[email-masked]>, "Ulrich Herberg" <[email-masked]>
>>> Subject: New Version Notification for draft-clausen-manet-rfc5444-usage-00.txt
>>> 
>>> 
>>> A new version of I-D, draft-clausen-manet-rfc5444-usage-00.txt
>>> has been successfully submitted by Thomas Clausen and posted to the
>>> IETF repository.
>>> 
>>> Name:        draft-clausen-manet-rfc5444-usage
>>> Revision:    00
>>> Title:        Rules For Designing Protocols Using the RFC5444 Generalized Packet/ Message Format
>>> Document date:    2015-05-28
>>> Group:        Individual Submission
>>> Pages:        17
>>> URL:            https://www.ietf.org/internet-drafts/draft-clausen-manet-rfc5444-usage-00.txt
>>> Status:         https://datatracker.ietf.org/doc/draft-clausen-manet-rfc5444-usage/
>>> Htmlized:       https://tools.ietf.org/html/draft-clausen-manet-rfc5444-usage-00
>>> 
>>> 
>>> Abstract:
>>>   This document updates the generalized MANET packet/message format,
>>>   specified in RFC5444, by providing prescriptive guidelines for how
>>>   protocols can use that packet/message format.  In particular, these
>>>   mandatory guidelines prohibit a number of uses of RFC5444 that have
>>>   been suggested in various proposals, and which would have lead to
>>>   interoperability problems, to impediment of protocol extension
>>>   development, and to inability to use generic RFC5444 parsers.
>>> 
>>> 
>>> 
>>> 
>>> Please note that it may take a couple of minutes from the time of submission
>>> until the htmlized version and diff are available at tools.ietf.org.
>>> 
>>> The IETF Secretariat
>>> 
>> _______________________________________________
>> manet mailing list
>> [email-masked]
>> https://www.ietf.org/mailman/listinfo/manet
> 
> _______________________________________________
> manet mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/manet

---

From: Ulrich Herberg <[email-masked]>
Date: Sat, 10 Oct 2015 17:01:55 -0700

Dear Lotte,

I apologize for the delay responding to your email. Life always seems
to get in the way of IETF work... ;-) See below:

> [...]

> I'm particularly interested in an answer to my earlier question concerning section 4.6., i.e.: if a message needs to be regenerated to be forwarded, and this message contains TLVs that the regenera…

First of all, I think that "regenerating" is the wrong word. That would
imply that you consume the incoming message, erase it, allocate new
memory and create a new message with the same content. While that is
certainly possible, it does not seem a very good idea doing that; in
addition, it is also counter-intuitive to understand. The idea of
RFC5444 is really:
- packet: container for zero or more messages; travels exactly one
hop; is not "owned" by any particular protocol; may contain TLVs for,
e.g., security credentials
- message: owned by a particular protocol; travels multiple hops
(end-to-end), contains information about the message itself (message
TLV), as well as a list of addresses with associated information
(address block TLV)

In my implementation of RFC5444 (actually, I built multiple, since I
was changing employer at some point and didn't have the IP to reuse
the code), I would have a general RFC5444 generator and parser that
can communicate via an API with one more protocols (e.g., NHDP).
Protocols would register themselves with the parser, and so when the
parser receives an incoming packet, it checks for each contained
message if the message type is "owned" by one of the protocols. If
not, the message is discarded. If it is, then parse the message to
interpret the information and hand up to the protocol (but keep the
actual byte array). The protocol can then do whatever it likes to do
with the message, including dropping, consuming or forwarding. In that
process, the byte array may be modified (but again, not regenerated,
since that would be very inefficient). After the protocol is done
modifying the message, it hands the message back down to the RFC5444
generator/parser, which in turn forwards it to the specified
(multicast) IP address.

One more important argument why "regenerate" is not a good word (IMO) is
that it could imply that end-to-end securing a message (e.g., with RFC7182)
is not possible. In order to allow to use digital signatures to verify integrity
of a message multiple hops away, it must not be modified along the path
towards its destination (that is modulo fields such as hop-count/limit,
which can be set to 0, as done for example in OLSRv2). If a router regenerates
a message this would for example imply that it could change order of TLVs.
However, that would break such a security mechanism.

Now, regarding dropping the TLVs, the process is similar to the way
the protocols register with the RFC5444 module; the protocol will have
a list of "known" TLVs that it recognizes. Additional TLVs that it
does not know about will be ignored (but not dropped). Other
extensions may be able to parse the information of these TLVs. Also,
in terms of end-to-end integrity protection, it is evident that we
cannot drop these TLVs without breaking said protection.

I re-read the section 4.6, and I think the paragraph is concise and
specifies exactly what I wrote above (a little less verbose and using
different words).

One other thing that I wanted to mention, since I used the word "API"
in the above description: I remember that there were some requests to
have an "API" document for RFC5444. This is clearly not the intention
of the RFC5444-usage document, and it would be outside of IETF scope
(as well as a bad idea, in my personal opinion). The API I mentioned
above in my implementation was written in Java and had a particular
signature. Writing that down in a document would be unhelpful for,
say, Thomas' COBOL60 implementation. He may have completely different
constraints of the language as well as way of calling the API. Also,
since the bytes sent between these two software modules do not go over
the network, there are no interoperability problems (which is the
whole point of having the IETF), and it unnecessarily constrain
vendors in designing efficient APIs.

Best regards
Ulrich

On Tue, Oct 6, 2015 at 11:15 AM, Lotte Steenbrink
<[email-masked]> wrote:
> Hi all,
> I realize vacation season and/or other circumstances have slowed most of us
> down at one point or another, but I was wondering if one of the authors
> could give us a very brief update on how the work on this draft is doing?
>
> I'm particularly interested in an answer to my earlier question concerning
> section 4.6., i.e.: if a message needs to be regenerated to be forwarded,
> and this message contains TLVs that the regenerating 5444 parser/builder
> doesn't recognize, should it add the unknown TLVs to the message (which
> would mean that “MUST NOT remove suchTLVs or values” applies, which should
> imo be clarified in the draft) or should the unknown TLVs be discarded
> (which would need clarification too, imo)?
>
> If the answer is “we're working on it, hold on for a few more days”, that's
> completely fine, I'd just like to know if/when I can find out how to do this
> correctly. :)
>
> Regards,
> Lotte
>
> As we announced previously, we have been working on a document specifying
> rules for how to use RFC5444 in MANET protocols. Well, here it is (at least
> in a -00 version).
> first of all, thank you so much for publishing this much-needed draft. I'm
> also in favor of WG adoption. While reading through the document, I wrote
> down some nitpicks and suggestions that I'd like to share.
> (Just in case my writing sounds harsh, I fear that's due to
> language/cultural differences. I'm really happy about this document and
> would like to see it flourish.)
>
> == General notes ==
> - If I didn’t misunderstand it, the document addresses two groups of people:
> Those who want to design RFC5444 messages for their protocol, and those wo
> want to implement a RCF 5444 builder/parser. Imo, The former group needs
> more of an outside view (i.e. what can/should I put where), while the latter
> needs the inside view (i.e. how do I arrange and process everything
> efficiently). To my mind, these are two very different goals, but while
> reading the document, it wasn’t always instantly clear to me which bit
> addresses which group… I realize the two can’t always be separated
> completely, but would it be possible to make it more clear which group is
> addressed currently? Maybe the document could be organized into two big
> sections (one for each group), or there could be a recurring pattern of
> “first outside then inside view” for each (sub)section?
>
> - A lot of the examples are very NHDP/OLSR-specific. While I think it’s
> great to have pointers to where a certain technique has already been used,
> from an outsider’s perspective, the examples don’t always help me understand
> the technique that’s being illustrated, since I have to familiarize myself
> with the details referenced first… I’m wondering if it would be possible to
> keep the examples which illustrate a point more generic, and then briefly
> point to existing instances of this situation in case the reader wants to
> follow up on that?
>
> == Section by section ==
> Section 1.2., Page 4:
>      If I understand it correctly, the last paragraph on this page is
> talking about multi-value TLVs without actually saying “multi-value TLV”.
> Maybe that could be stated explicitly for clarity and in case somebody’s
> searching for advice by this keyword?
>
> Section 4.4., Page 9/10:
>      were a bit hard to read for me. Imho, the example is very specific and
> contains details which aren’t necessary to understand the issue at hand
> (such as the fact that this extension is technically not a completely
> independent extension, or possible future other extensions). It took me
> about 4 re-reads to really grasp what’s going on there...
>
>      Also, I would’ve expected the last paragraph of this section to clearly
> state that relying on address order *will* break your messages if you don’t
> have control over all 5444 builders that are used to implement your protocol
> ever.
>
> Section 4.5.:
>      I'm assuming that this is the response to our (the AODVv2 Author team
> and possibly others) complaints that RFC5444 lacks a clear "API" which tells
> it what we can and cannot set manually in an RFC 5444 packet, and that
> there's no (at the risk of sounding really pretentious now) "visual
> language" to describe the contents of a 5444 message and their relationship
> to each other. To be honest, I'm a bit puzzled by this section.... The
> mapping that is presented is a bit vague and introduced very promptly. And
> while I strongly agree with the statement that writing down an actual API is
> too language- and implementation-specific to belong in a document like this,
> it would be really helpful to have a comprehensive "these are the things
> that your protocol can set in a RFC5444 message. Your packet builder/parser
> will figure out the rest." List in this section rather than some examples.
> Along with this list, it would be great to have a clear specification on how
> to specify which Addresses and TLVs a specific message has, and how their
> relationship to each other are. (i.e. how to make clear which address TLVs
> are associated with which addresses). I don’t care if this ends up being a
> very visual approach (like the debugging output of Henning’s oonf API [1]),
> or the tables that OLSRv2 uses, or something completely different, but it
> would be helpful to be able to read and write messages down in a way that is
> clear to everyone.
>
> Section 4.6:
>      A question I’ve been wanting to ask for a while now: Does this rule
> also apply for message regeneration? AODVv2 regenerates all messages per
> hop. Should implementations keep track which unknown TLVs of a message are
> associated with which addresses, and put them back in their place when
> regenerating the message?
>
> Section 6.1.:
>      (I’m not terribly familiar with compression, so my questions are
> probably very naive, but maybe they can help clear things up)
>      Would this very straightforward to implement compression algorithm
> still produce interoperable messages? The rest of the section reads as if
> the “compression algorithm” mentioned in the introduction is mainly a way to
> cleverly arrange addresses so that they are still understood by all parsers,
> but take up a minimal amount of space. I think that’s a very good tip in
> itself, but is that really all there is to compression? Would it make sense
> to say “Here’s how you should arrange addresses cleverly, you can also
> implement a more elaborate compression algorithm, but that’s out of the
> scope of this document”?
> To: "Christopher Dearlove" <[email-masked]>, "Henning Rogge"
> <[email-masked]>, "Thomas Clausen"
> <[email-masked]>, "Henning Rogge"
> <[email-masked]>, "Ulrich Herberg" <[email-masked]>,
> "Thomas H. Clausen" <[email-masked]>, "Christopher Dearlove"
> <[email-masked]>, "Ulrich Herberg" <[email-masked]>
> URL:
> https://www.ietf.org/internet-drafts/draft-clausen-manet-rfc5444-usage-00.txt
> Status:
> https://datatracker.ietf.org/doc/draft-clausen-manet-rfc5444-usage/
> Htmlized:
> https://tools.ietf.org/html/draft-clausen-manet-rfc5444-usage-00
>
>
> Abstract:

---

From: Lotte Steenbrink <[email-masked]>
Date: Mon, 12 Oct 2015 23:00:13 +0200

Hello Ulrich,

> to get in the way of IETF work... ;-) See below:

No worries, thank you for the very detailed and helpful response!

> generator/parser, which in turn forwards it to the specified

💡 That makes much more sense... For some reason I'd just assumed that
manipulating any part of a message other than the header was illegal,
thank you for clearing that up!

> specifies exactly what I wrote above (a little less verbose and using

Agreed.

> vendors in designing efficient APIs.

I'm afraid my use of the term "API" was a poor choice of words... As I've
said in my initial E-Mail and in Prague, I really didn't mean to say "tell
me which functions I can implement/rely on being implemented, and which
signatures they have". I agree that that request would be silly in
many ways.
What I meant by my request was that, when working on the packet format for
AODVv2, we had a very specific perspective: we needed to tell people who
want to implement AODVv2 what to put in their packets *without getting
into too much detail about what happens under the bonnet in the RFC5444
generator/parser*, since that is RFC5444's realm, and not AODVv2's.

In other words, if someone with a working RFC5444 generator/parser (that they
didn't implement themselves) and a rough understanding of RFC5444 wants
to implement AODVv2, they need to know which Addresses and TLVs to feed
their generator/parser, and which header fields to set, but they really
don't need to care about header flags and multi-value TLVs and all the
nifty tricks that their RFC5444 generator/parser does internally.

It was really hard to write this down in the draft, because after careful
inspection of the RFC, all of us had a different idea of which bits our
potential implementor has to care about and which are handled by the
generator/parser and thus don't have to (or can't) be specified manually
by the protocol. If we had had a list of things that we, as protocol authors,
were allowed to tell other people to set, that would've been very very
helpful and saved us a lot of time. Again, I'm not asking for this this list to
be in code form. It could for example be something along the lines of:

   Protocols using RFC5444 MAY define the following 
   properties of a RFC5444 message.

       <msg-header>    := <msg-type>
                          <msg-orig-addr>?
                          <msg-hop-limit>?
                          <msg-hop-count>?
                          <msg-seq-num>?
       <tlv>           := <tlv-type>
                          <tlv-type-ext>
                          (<length><value>?)?
       <address-block> := <mid>*
                          <prefix-length>*

   All other values and properties are to be set by the 
   RFC5444 generator/parser handling the message, and 
   SHOULD NOT be tampered with by any protocol handling
      the message.

Now, I'm sure there are many ways to do this more elegantly[1], but maybe
this sketch helps to illustrate what I'm trying to say...

Best Regards,
Lotte

[1] And correctly. At the very least, my address block sketch is a bit...
dodgy. But you get the idea.