Subject: Re: [Anima-signaling] More GRASP in CDDL

From: Brian E Carpenter <[email-masked]>
Date: Mon, 9 Nov 2015 16:34:35 +1300

Hi Carsten,

I've hit a bit of a coding problem (in Python) with one of your suggestions
for the syntax. At the moment we have:

locator-option /= ipv4-locator-option
ipv4-locator-option = bytes .size 4
; this is simpler than [O_IPv4_LOCATOR, bytes .size 4]

locator-option /= ipv6-locator-option
ipv6-locator-option = bytes .size 16

locator-option /= fqdn-locator-option
fqdn-locator-option = [O_FQDN_LOCATOR, text]

locator-option /= url-locator-option
url-locator-option = [O_URL_LOCATOR, text]

The problem is that when an incoming message contains (say)
an IPv6 address it looks like this after applying cbor.loads():

[2, 1234, 319018505059631000003079090149510920874]

whereas an incoming message containing an FQDN looks like this:

[2, 1234, [105, 'host.example.net']]

(These were generated by a Python fragment, not by me.)

The only way I know how to distinguish these when parsing the incoming
message is using type(), and even that does not distinguish the IPv6
and IPv4 cases. Now you can argue that this is a defect in Python,
but the problem would certainly go away if we put a type code
in the messages. Any thoughts? Am I missing something?

> By the way, the first element of the array that represents an option can
> now be an integer (for the non-objective options), a text string (for
> the objective options), or a {PEN: pen} map; in the third case, that is
> not the whole identification of the option kind.
> 
> A small simplification might be to use
> 
> vendor-obj = [[pen, objective-name], objective-flags, loop-count, any]
> 
> so in the third case the first element also becomes the whole option
> kind identifier and not just gives the PEN (followed by the
> objective-name later).

That would give me a similar difficulty in Python. I'm thinking that this
would be simpler all round:

objective = [objective-name, pen, objective-flags, loop-count, ?any]

where by convention pen=0 for a generic option. (Zero is reserved by IANA.)

Regards
    Brian

---

From: Carsten Bormann <[email-masked]>
Date: Mon, 09 Nov 2015 17:58:44 +0800

Hi Brian,

a CBOR implementation should allow you to access information about
representation types.

> an IPv6 address it looks like this after applying cbor.loads():

That looks like a large number -- the representation type should have
been a byte string.

> and IPv4 cases. 

No, it doesn't, since both are represented as byte strings; you really
have to look at the length to make that distinction.

> Now you can argue that this is a defect in Python,

Of course, you can put a type code there; I just have been arguing that
this would be redundant.

> where by convention pen=0 for a generic option. (Zero is reserved by IANA.)

If you want to go that route, nil (null) would be another way to
represent an absent PEN.
(I was assuming that most of the objective-names were going to be
PEN-free, and that it would be useful to keep PEN and objective-name
together in one location if both are used.)

None of these questions have a "right" answer, but it seems to me the
Python CBOR implementation you are using is a bit short-changing you.
Which one is it?

Grüße, Carsten

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 10 Nov 2015 08:14:48 +1300

Hi Carsten,

> That looks like a large number -- the representation type should have

Well, yes, I just used an integer. I am still learning Python. So, now
I have learned about bytestrings. Unfortunately that leads to an issue
wthe the package:

M_RESPONSE = 2
import cbor
v6loc = bytearray.fromhex('f000baaaf000baaaf000baaaf000baaa')
payload = cbor.dumps([M_RESPONSE, 1234, v6loc])

Exception: ("don't know how to cbor serialize object of type %s", <class 'bytearray'>)

That's using https://bitbucket.org/bodhisnarkva/cbor (in fact, I installed
it simply with 'pip install cbor'). I'm not quite sure what the procedure is
for feature requests. Is there a better package?

Thanks
   Brian

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 10 Nov 2015 13:13:10 +1300

Well, I found a work-around. If I use a Python 'bytes' object,
the CBOR module knows how to endcode it. It still seems odd
that it doesn't know how to encode a 'bytearray', which is the
same thing but with mutable values. However, I can code round that.

Regards
   Brian

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 10 Nov 2015 08:18:17 +0800

Which version of Python are you using?
(Python has been officially at version 3 for half a decade, but the tool-smithing community seems stuck at 2.
The handling of bytes vs. text has changed radically between 2 and 3 — that actually is the main difference, since 3 finally fully embraced Unicode.)
I recently tried the cbor module with Python 2.7, and it deserialized byte strings into “str” and text strings into “unicode”.
(That sounds vaguely right to this non-Python-speaker.)

Grüße, Carsten

On 10 November 2015 at 08:13:28, Brian E Carpenter ([email-masked]) wrote:

Regards  
Brian  

>> Python CBOR implementation you are using is a bit short-changing you.  

_______________________________________________  
Anima-signaling mailing list  
[email-masked]  
https://www.ietf.org/mailman/listinfo/anima-signaling

---

From: Brian E Carpenter <[email-masked]>
Date: Tue, 10 Nov 2015 15:00:54 +1300

I'm using Python 3.4; there doesn't seem to be any excuse for using
an old version. At some point I will be able to check if the actual
bits coming out of the serialisation are what we want; for the moment
I'm happy enough that I can round-trip GRASP-like objects through
dumps/loads.

Regards
   Brian

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 10 Nov 2015 23:02:15 +0800

There is an issue tracker:

https://bitbucket.org/bodhisnarkva/cbor/issues?status=new&status=open

(I’m having a bit trouble accessing this from China right now.)

This is indeed the Python CBOR implementation I would currently recommend.

Grüße, Carsten