Subject: [VCARDDAV] New Version of the iCalendar in JSON (jCal) draft, feedback requested

From: Philipp Kewisch <[email-masked]>
Date: Tue, 15 Jan 2013 09:24:47 +0100

Hello Folks,

Based on discussion we have had within the calconnect group, I have just 
posted a new version of draft-kewisch-et-al-icalendar-in-json with the 
following changes. You can find the draft here 
<http://www.ietf.org/id/draft-kewisch-et-al-icalendar-in-json-01.txt>.

       *  Added information on how to handle multi-value parameter.  The
          decision leads to a cleaner draft for a similar proposal for
          vcard.

       *  Removed the open discussion point section regarding the mime
          media type in favor of adding one.

I would appreciate your feedback on the draft in order to advance on 
this topic.

Also, especially or the vcarddav folks, please also take a look at my 
previous email 
<http://www.ietf.org/mail-archive/web/vcarddav/current/msg02678.html> to 
get a comparison between the vcarddav-json draft and the format we 
propose for iCalendar. I would like to push forward with this and come 
to a consensus so we can provide a compatible draft for vcards.

I am also happy to announce that Mozilla is already testing this draft 
in the field with the Firefox OS calendaring application. To do so, we 
are using the ical.js library I have been working on, see 
<https://github.com/kewisch/ical.js/>.

Regards,
Philipp

---

From: Michael Angstadt <[email-masked]>
Date: Tue, 15 Jan 2013 20:24:15 -0500

Here are a few of my thoughts about the two specs:

Summary:

(1) jCal doesn't support grouping.
(2) jCal eliminates the possibility for naming conflicts that
vcarddav-json makes possible.
(3) vcarddav-json allows property values to be encoded as objects,
while jCal does not.

(1) Grouping

One problem with jCal is that it doesn't support grouping, which is
problematic for vCards.  For example, the vCard below can't be
JSON-encoded according to the jCal specs because the EMAIL property
belongs to a group.

BEGIN:VCARD
VERSION:4.0
FN:John Doe
group.EMAIL:[email-masked]
END:VCARD

A simple solution would be to prepend the group to the property name,
separating it from the property name with some special character, just
like in vCard.  For example:

["group.email", {}, "text", "[email-masked]"]

The JSON syntax could also be modified to accomodate a group name (for
example, by adding a new array element), but some of the benefits to
the above approach that I see are:

(a) Greater compatibility with jCal.
(b) Grouping is a fairly infrequently-used feature.  Changing the
syntax would add extra complexity which won't be used in most
situations.

I must say, I'm not a big fan of the grouping mechanism in
vcarddav-json.  Like Bert Greevenbosch mentioned previously, naming
conflicts will occur if a group shares the same name as a property.
It does, however, make it so that the group name only has to be
declared once, which is a plus (like in xCard).

(2) Naming conflicts

I think one major plus about jCal is its elimination of the potential
for naming conflicts between value type names ("text", "uri", etc) and
parameter names.  Philipp explained this in a previous email.  Quoting
Philipp:

[[quote]]
Another issue here might be extensibility. What happens if an
extension draft specifies a new value type that is the same as a
parameter, or vice versa?

X-THING;VALUE=URI;URI="urn:foo":urn:bar
vcarddav-json: { "x-thing": { "uri": ??? } }
jCal: ["x-thing", { "uri": "urn:foo" }, "uri", "urn:bar"]
[[/quote]]

jCal prevents these naming conflicts from occurring because parameters
and property values are stored separately.

(3) JSON objects for property values

On the flip-side, I think one nice feature about vcarddav-json is that
property values are encoded as JSON objects.  This is great for
multi-valued properties like ADR and N because each value can be
indexed by a human-readable name.  jCal, however, requires that the
values be appended onto the end of the property array, which means
that they must be retrieved by index.  It also means that a
placeholder element (like a "null" value) must be included for any
missing components of the property value in order to preserve the
array indexing.  The ADR property is a good example:

vcarddav-json:

"adr": {
  "type": "work",
  "street": "2875 boul. Laurier, suite D2-630",
  "locality": "Quebec",
  "region": "QC",
  "code": "G1V 2M2",
  "country": "CA"
}

jCal:

["adr", {"type":"work"}, "text", null, "2875 boul. Laurier, suite
D2-630", "Quebec", "QC", "G1V 2M2", "CA"]

The "null" in the jCal example is for the missing "extended address" field.

One possible solution to this (if this is deemed a "problem" in the
first place) would be to add an "object" value type to the jCal spec
that allows you to specify a JSON object as the value.  For example:

["adr", {"type":"work"}, "object", {"street": "2875 boul. Laurier,
suite D2-630", "locality":"Quebec", "region":"QC", "code": "G1V 2M2",
"country":"CA"}]

jCard & jCal in 2013!! :)

Regards,
Mike Angstadt
http://code.google.com/p/ez-vcard

On Tue, Jan 15, 2013 at 3:24 AM, Philipp Kewisch <[email-masked]> wrote:
> I would appreciate your feedback on the draft in order to advance on this
> topic.
>
> <http://www.ietf.org/mail-archive/web/vcarddav/current/msg02678.html> to get
> a comparison between the vcarddav-json draft and the format we propose for
> iCalendar. I would like to push forward with this and come to a consensus so
> we can provide a compatible draft for vcards.
>
> I am also happy to announce that Mozilla is already testing this draft in
> the field with the Firefox OS calendaring application. To do so, we are
> using the ical.js library I have been working on, see
> _______________________________________________
> VCARDDAV mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/vcarddav
>

---

From: Michael Angstadt <[email-masked]>
Date: Thu, 17 Jan 2013 21:35:17 -0500

Another issue is how to handle embedded vCards (i.e. AGENT).

vcarddav-json currently does not say anything about this.  One
solution would be to add a "vcard" data type, whose value would be a
fully-formed jCard (which is a JSON object).  This could be done for
both versions 2.1 and 3.0, even though 3.0 embeds its vCards by
encoding the vCard as a text value.  For example:

{
  "version":"3.0",
  "fn": "John Doe",
  "agent": {
    "vcard": {
      "version":"3.0",
      "fn": "Agent Smith"
    }
  }
}

iCalendar supports a similar data structure, called a "component".
>From what I understand, components are just like embedded vCards,
except that components are not associated with a property.  To
explain, an AGENT property in a 2.1 vCard looks like this:

BEGIN:VCARD
VERSION:2.1
FN:John Doe
AGENT:
BEGIN:VCARD
VERSION:2.1
FN:Agent Smith
END:VCARD
END:VCARD

The embedded vCard comes after the blank AGENT property.  iCalendar's
components, however, are not associated with a property.  For
instance, in the vCal below, the "VTIMEZONE" component exists by
itself, without "belonging" to a property:

BEGIN:VCALENDAR
VERSION:2.0
BEGIN:VTIMEZONE
TZID:US-Eastern
END:VTIMEZONE
END:VCALENDAR

jCal separates its properties and components, which would not work for
encoding an AGENT property because the link between the property and
the embedded vCard would be lost:

["vcalendar",
  [ /* properties */ ],
  [ /* components */ ]
]

Regards,
Mike Angstadt

---

From: Simon Perreault <[email-masked]>
Date: Fri, 18 Jan 2013 15:16:50 +0100

> Another issue is how to handle embedded vCards (i.e. AGENT).

Embedded vCards don't exist anymore in vCard 4. xCard does not support 
them. jCard should not support them.

Simon

---

From: Michael Angstadt <[email-masked]>
Date: Fri, 18 Jan 2013 09:44:40 -0500

According to the specs, jCard supports 2.1 and 3.0, not just version 4.

On Fri, Jan 18, 2013 at 9:16 AM, Simon Perreault
<[email-masked]> wrote:
> Embedded vCards don't exist anymore in vCard 4. xCard does not support them.
> jCard should not support them.
>
> Simon

---

From: Simon Perreault <[email-masked]>
Date: Fri, 18 Jan 2013 15:47:43 +0100

> According to the specs, jCard supports 2.1 and 3.0, not just version 4.

This is wrong, it should only support version 4. Doing otherwise would 
be asking for a lot of trouble.

Simon

---

From: Cyrus Daboo <[email-masked]>
Date: Fri, 18 Jan 2013 09:51:51 -0500

Hi Simon,

--On January 18, 2013 3:47:43 PM +0100 Simon Perreault 
<[email-masked]> wrote:

> This is wrong, it should only support version 4. Doing otherwise would be
> asking for a lot of trouble.

That would be one way to encourage more adoption of version 4, which would 
be a good thing IMHO.

-- 
Cyrus Daboo

---

From: Michael Angstadt <[email-masked]>
Date: Fri, 18 Jan 2013 10:13:21 -0500

+1

---

From: Philipp Kewisch <[email-masked]>
Date: Sat, 26 Jan 2013 09:43:10 +0100

Hi Scotty,

I understand that the JSON you see here is slightly different than what 
you often see (root being an object), but the data we have to handle 
here is slightly different too. What we have here is definitly ON, as in 
Javascript, Arrays are also Objects and Arrays are also part of the JSON 
spec.

The JSON you provide looks very much like what I started with when I was 
working on my library to parse iCalendar data, but as we progressed with 
the spec, more and more flaws came up. Remember we are designing a data 
format, not a library to process it. Let me show you some examples of 
what is not working:

>
> {
>      "prodid": "-//Example Corp.//Example Client//EN"],
>      "version": "2.0",
>      "vtimezones": [
What rule would this go by? Use the component name and add 's' ?
>          {
>              "tzid": "US/Eastern",
>              "last-modified": "2004-01-10T03:28:45Z",
>              "daylight": {
What happens if there are multiple daylight components? Even if thats 
not possible in this case, it could for other components so there would 
have to be lots of per-component exceptions which is neither consistent 
nor future proof.

>              "dtstamp": "2006-02-06T00:11:21Z",
>              "dtstart": {
>                  "tzid": "US/Eastern",
>                  "type": "date-time",
>                  "value": "2006-01-02T12:00:00"
>              },
So dates can have two different formats? What to use when? What if there 
is a parameter named TYPE ?

>              "summary": "Event #2",
>              "description": "We are having ... 12 pm meetings.",
>              "uid": "[email-masked]"
What happens with properties that can occur multiple times or have 
multiple values (like CATEGORIES)?

My next idea back then was to turn it into an array, i.e "category": 
["abc", "def"], but then you run into consistency/performance issues 
again. Add an array for everything, i.e "summary": ["Event #2"] or do 
type checking each time?
>       
>
> Now the data can be processed with code like:
>
>      var jcal = getJCalData(…);
>
>      jcal.vevents.forEach(function (vevent) {
While I understand it would be nice to process jCal without a library 
(and you can, even with our draft), in many cases you will just end up 
duplicating code that a js library will provide to you. The easier we 
make this, the more likely it is that people will produce wrong jCal, 
because their code evolves from something simple.

I hope this give you a better feeling with the format we propose, let me 
know what you think!
Philipp

---

From: Philipp Kewisch <[email-masked]>
Date: Sat, 26 Jan 2013 09:43:50 +0100

TL;DR; version at the end of this email.

> syntax would add extra complexity which won't be used in most
I agree, I had thought about grouping, but I preferred a similar syntax 
so a library that could parse jCal could also parse jCard. For a simple 
solution, I'd go with just using the whole thing as a property name like 
you posted:

["group.email", {}, "text", "[email-masked]"]

Another option that would allow same property names as group names is to 
add a special type "group" in the jCard spec.

["groupname", {}, "group", [
     ["email", {}, "text", "[email-masked]"]
   ]
]

Which would serialize to:

groupname.EMAIL:[email-masked]

This would differ from a property named groupname:

["groupname", {}, "text", "foo"]

and due to the fact that properties are in an array and not in an 
object, there is no problem keeping both. The downside for this 
alternative is that there is an extra set of parameters that does not 
serialize back to iCalendar/vCard. This would "look" correct but be invalid:

["groupname", { "x-bogus": "param" }, "group", [
     ["email", { "x-ok": "param" }, "text", "[email-masked]"]
   ]
]

And still serialize to the following, dropping x-bogus.

groupname.EMAIL;X-OK=param:[email-masked]

> array indexing.  The ADR property is a good example:
We've discussed this in the last TC-XML-JSON call, related to a 
different issue. I think this is something we could find a consensus on 
and put it into the jCal spec. On the one hand I would like to avoid 
making property values too diverse and just stay with not splitting the 
text at all:

["adr", {"type":"work"}, "text", ";2875 boul. Laurier, suite 
D2-630;Quebec;QC;G1V 2M2;CA"]

This would leave the parsing to a library. For jCal this would be 
sufficient, since the allowed text in an RRULE property value is 
limited, so its just a matter of val.split(";").forEach(function(part) { 
let k,v = part.split("="); map[k] = v }); if the library needs to access 
the parts in a map. Reasons that speak for against using objects (or 
even arrays) for structured values:

* Slight performance loss, which might add up, due to:
    ** Need for either special-casing of type per property or, to be 
safe, for every property.
    ** Additional Objects per property, which might not be needed every 
time.
* Breaks consistency. Which properties are allowed to have objects?
* What about unknown properties added via extension? What would this 
parse to:  X-MYTHING:LoOKS=;LIKE=;A=;STrUCTURED=;VAlUE=
   ** ["x-mything", {}, "text", { "looks": "", "like":"", 
"a":"","structured":"","value":"" }]
   ** ["x-mything", {}, "text", "LoOKS=;LIKE=;A=;STrUCTURED=;VAlUE="]
* If unknown objects should have a default, what separator to choose?
* For jCal, this is only valid for RRULE, which you will need a library 
anyway to process gracefully (calculate occurrences)

On the other hand, especially with vcard in mind, I can see this is 
useful. Specifically for jCal, we have discussed that it would be good 
to change GEO and REQUEST-STATUS a single-value property where the value 
is an array, i.e ["geo", {}, "float", [12.34, 56.78]]. In line with 
this, we could also allow using an object for RRULE since it has its own 
type.

 From your vCard example it seems to me that /any/ "text" value can be 
structured, so we would need to make using objects as property values 
something that is generally allowed for and warn parser implementers 
that they need to check the value type. To handle the issue with unknown 
properties, we could assert that if the parser does not know about the 
property that it should treat it as opaque text.

So let me summarize whats left to decide:

(1) Grouping:
   a) Leave "groupname.email" as the property name, or
   b) Add a pseudo-property with the special type "group" and make its 
value be an array of group members

(2) Naming conflicts: We agree here that jCal's mechanism works better

(3) Objects for structured property values
   a) Use an array, possibly with a null value for any structured property
   b) Use an object, avoids null values and makes access easier
   c) Do not parse value at all, leave it as an opaque text

My personal preferences are 1a and either 3b or 3c.

Philipp

---

From: Scotty Logan <[email-masked]>
Date: Tue, 29 Jan 2013 13:35:27 -0800

On Sat, Jan 26, 2013 at 12:43 AM, Philipp Kewisch <[email-masked]> wrote:
> I understand that the JSON you see here is slightly different than what you
> often see (root being an object),

I started out with a response to each of your points, but then it
occurred to me that the problem isn't in the naming ("vtimezone" vs
"vtimezones"), or representing values as arrays or strings:  the
problem, as I see it, is that the JSON draft appears to be an attempt
to convert the iCalendar format into JSON, rather than creating a JSON
representation of the iCalendar data model.

  Scotty

---

From: Michael Angstadt <[email-masked]>
Date: Thu, 31 Jan 2013 07:26:23 -0500

Philipp,

> What rule would this go by? Use the component name and add 's' ?

jCard doesn't require that the property name be plural when there are
multiple instances of the property.  In this example, you'd simply use
"vtimezone".

>   b) Add a pseudo-property with the special type "group" and make its
value
> be an array of group members
>

I think I'd go with (1a).  (1b) adheres to DRY principles, but has a lot of
"fluff" (an empty parameter object, the "group" string, and complex array
nesting for the properties).  (1a) is simpler and groups aren't used that
much anyway.

Agree on (2).

I like (3b), but one problem with this, which I think you alluded to, is
how extended properties with object values should be parsed.  If an
extended property uses an object as its value, then the parser would have
no way of getting the value because it doesn't know how to interpret the
fields in the object.  However, if all properties are required to have a
"simple" value, like a string or an array of strings (3c), then extended
properties can still be parsed.

But then again, xCard basically takes the (3b) approach.  In xCard,
extended properties are rendered as XML elements with their own
namespaces.  They can have whatever child XML elements they want and aren't
limited to the standard xCard elements.  If the parser encounters such a
property, there's no way for it to parse it, so the XML gets stored in the
"XML" vCard property.  If the (3b) approach is used for JSON, maybe there
would have to be a similar "JSON" vCard property for storing the raw JSON
code of such extended types.

-Mike

>             "last-modified": "2004-01-10T03:28:45Z",

jCard doesn't require that the property name be plural when there are
multiple instances of the property.  In this example, you'd simply use
"vtimezone".

> I agree, I had thought about grouping, but I preferred a similar syntax
so a
> library that could parse jCal could also parse jCard. For a simple
solution,
> I'd go with just using the whole thing as a property name like you posted:
>
add
> a special type "group" in the jCard spec.
>

Not sure which is better.  Option 1 is simpler, but the 2nd option is
better in terms of adhering to DRY principles.  Like you alluded to, option
2 adds a lot of "fluff" to the JSON (an empty parameter object, the "group"
string, and complex array nesting for the properties).  But option 2 also
makes it easier to retrieve all properties that belong to a group.

> sufficient, since the allowed text in an RRULE property value is limited,
so
> its just a matter of val.split(";").forEach(function(part) { let k,v =
> part.split("="); map[k] = v }); if the library needs to access the parts
in
> a map. Reasons that speak for against using objects (or even arrays) for
> structured values:

split(";") won't work if there are escaped semi-colons.

> From your vCard example it seems to me that any "text" value can be
> something that is generally allowed for and warn parser implementers that
> they need to check the value type. To handle the issue with unknown

No, most properties that have text values are *not* structured like ADR is.
 Most just store a regular text value.  The only structured properties are
ADR and N (and GEO in 2.1/3.0).  There are many properties that take
multiple text values though, like CATEGORIES, ORG, and NICKNAME.

That made me think of a problem with using JSON objects as property values,
which you alluded to.  If an extended property uses an object as its value,
then the parser would have no way of getting the value because it doesn't
know how to interpret the fields in the object.  However, if all properties
are required to have a "simple" value, like a string or an array of
strings, then extended properties can still be parsed.

-Mike

On Saturday, January 26, 2013, Philipp Kewisch wrote:

> ["group.email", {}, "text", "[email-masked]" <javascript:_e({}, 'cvml', [email-masked]');>]
>
> ["group.email", {}, "text", "[email-masked]" <javascript:_e({}, 'cvml', [email-masked]');>]
>
>
>     ["email", {}, "text", "[email-masked]" <javascript:_e({},
> 'cvml', [email-masked]');>]
>   ]
> ]
>
> groupname.EMAIL:[email-masked] <javascript:_e({}, 'cvml',
> 'groupname.EMAIL:[email-masked]');>
>
> and due to the fact that properties are in an array and not in an object,
> there is no problem keeping both. The downside for this alternative is that
> there is an extra set of parameters that does not serialize back to
> iCalendar/vCard. This would "look" correct but be invalid:
>
>     ["email", { "x-ok": "param" }, "text", "[email-masked]"<javascript:_e({}, 'cvml', [email-masked]');>
> ]
>   ]
> ]
>
> groupname.EMAIL;X-OK=param:[email-masked] <javascript:_e({}, 'cvml',
> 'groupname.EMAIL;X-OK\x3dparam:[email-masked]');>
>
>
>  We've discussed this in the last TC-XML-JSON call, related to a different
> issue. I think this is something we could find a consensus on and put it
> into the jCal spec. On the one hand I would like to avoid making property
> values too diverse and just stay with not splitting the text at all:
>
> sufficient, since the allowed text in an RRULE property value is limited,
> so its just a matter of val.split(";").forEach(function(part) { let k,v =
> part.split("="); map[k] = v }); if the library needs to access the parts in
> a map. Reasons that speak for against using objects (or even arrays) for
> structured values:
>
>    ** Need for either special-casing of type per property or, to be safe,
> for every property.
> * What about unknown properties added via extension? What would this parse
> to:  X-MYTHING:LoOKS=;LIKE=;A=;STrUCTURED=;VAlUE=
> useful. Specifically for jCal, we have discussed that it would be good to
> change GEO and REQUEST-STATUS a single-value property where the value is an
> array, i.e ["geo", {}, "float", [12.34, 56.78]]. In line with this, we
> could also allow using an object for RRULE since it has its own type.
>
> From your vCard example it seems to me that *any* "text" value can be
> something that is generally allowed for and warn parser implementers that
> they need to check the value type. To handle the issue with unknown

---

From: Michael Angstadt <[email-masked]>
Date: Thu, 31 Jan 2013 09:04:47 -0500

Oops sorry, I sent part of my draft email that I didn't mean to send.
You can ignore everything after my first signature.  This is what
happens when you try to write detailed emails on an iPad. >:(

-Mike

On Thu, Jan 31, 2013 at 7:26 AM, Michael Angstadt
<[email-masked]> wrote:
> I like (3b), but one problem with this, which I think you alluded to, is how
> extended properties with object values should be parsed.  If an extended
> property uses an object as its value, then the parser would have no way of
> getting the value because it doesn't know how to interpret the fields in the
> object.  However, if all properties are required to have a "simple" value,
> like a string or an array of strings (3c), then extended properties can
> still be parsed.
>
> But then again, xCard basically takes the (3b) approach.  In xCard, extended
> properties are rendered as XML elements with their own namespaces.  They can
> have whatever child XML elements they want and aren't limited to the
> standard xCard elements.  If the parser encounters such a property, there's
> no way for it to parse it, so the XML gets stored in the "XML" vCard
> property.  If the (3b) approach is used for JSON, maybe there would have to
> be a similar "JSON" vCard property for storing the raw JSON code of such
> extended types.
>
> -Mike
>> I agree, I had thought about grouping, but I preferred a similar syntax so
>> a
> Not sure which is better.  Option 1 is simpler, but the 2nd option is better
> in terms of adhering to DRY principles.  Like you alluded to, option 2 adds
> a lot of "fluff" to the JSON (an empty parameter object, the "group" string,
> and complex array nesting for the properties).  But option 2 also makes it
> easier to retrieve all properties that belong to a group.
>
>
> are required to have a "simple" value, like a string or an array of strings,
> then extended properties can still be parsed.
>
> -Mike
>> I agree, I had thought about grouping, but I preferred a similar syntax so
>> a library that could parse jCal could also parse jCard. For a simple
>> solution, I'd go with just using the whole thing as a property name like you
>> posted:
>>
>> sufficient, since the allowed text in an RRULE property value is limited, so
>> array, i.e ["geo", {}, "float", [12.34, 56.78]]. In line with this, we could
>> also allow using an object for RRULE since it has its own type.
>>