Subject: Media Type "text/csv": new draft (-02) and Last Call
From: clyde.ingram at edl.uk.eds.com ([email-masked])
Date: Thu Mar 31 22:07:11 2005

Yakov,

That draft wording sounds excellent.

Now, if you were to ALSO mandate the presence of the header record, that
would mitigate my concern that computer applications should be able to
exchange CSV files without worrying about same field order (simply because
the header conveys the field names, rather than needing an out-of-band,
bilateral agreement).  And, of course, presence of a header record lets a
computer application determine immediately whether the supplied fields are
likely to be what it expects.   All good aids for careful validation.

Thank-you,
Clyde

-----Original Message-----
From: Yakov Shafranovich [mailto:[email-masked]]
Sent: Wednesday, March 30, 2005 8:08 PM
To: [email-masked]
Cc: [email-masked]; [email-masked]
Subject: Re: Media Type "text/csv": new draft (-02) and Last Call

[email-masked] wrote:
> 
> But the same cannot be said of automated computer-based applications, 
> where maintaining a strict count of generated and expected 
> Comma-Separated-Values per record is not only easy, but also allows for 
> an extra level of data validation: namely that a received record is 
> corrupt if it has too few or too many fields.  This is where 
> standardisation in the format of the CSV records becomes appropriate 
> material for an RFC.
> 

The draft as it is written now (-03) does not mandate that the same 
number of fields need to appear on each line, mainly due to the fact 
that the draft is focusing on the MIME type registration. Would the 
following change to section 2, subsection 4 be sufficient to address 
your concerns:

"Within the header and each record there may be one or more fields, 
separated by commas. Each line should contain the same number of fields 
throughout the file. The last field in the record may not be followed by 
a comma. For example:"

Yakov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://eikenes.alvestrand.no/pipermail/ietf-types/attachments/20050331/73e44478/attachment.html
>From [email-masked]  Thu Mar 31 13:16:41 2005
From: GK-lists at ninebynine.org (Graham Klyne)
Date: Mon Apr  4 12:14:11 2005
Subject: Media Type "text/csv": new draft (-02) and Last Call
In-Reply-To: <[email-masked]>
References: <[email-masked]>
	<[email-masked]>
Message-ID: <[email-masked]>

At 14:08 30/03/05 -0500, Yakov Shafranovich wrote:
>[email-masked] wrote:
>>But the same cannot be said of automated computer-based applications, 
>>where maintaining a strict count of generated and expected 
>>Comma-Separated-Values per record is not only easy, but also allows for 
>>an extra level of data validation: namely that a received record is 
>>corrupt if it has too few or too many fields.  This is where 
>>standardisation in the format of the CSV records becomes appropriate 
>>material for an RFC.
>
>The draft as it is written now (-03) does not mandate that the same number 
>of fields need to appear on each line, mainly due to the fact that the 
>draft is focusing on the MIME type registration. Would the following 
>change to section 2, subsection 4 be sufficient to address your concerns:
>
>"Within the header and each record there may be one or more fields, 
>separated by commas. Each line should contain the same number of fields 
>throughout the file. The last field in the record may not be followed by a 
>comma. For example:"

I think this change would *decrease* the utility of the proposed MIME type 
registration, which I see as being to document existing practice so that 
new applications can interoperate with it.  I think the benefits of 
interworking with existing practice greatly outweigh the purported 
advantage of (weak) integrity checking.  In this case, I really think that 
less [specification] is more.

#g

------------
Graham Klyne
For email:
http://www.ninebynine.org/#Contact

From: clyde.ingram at edl.uk.eds.com ([email-masked])
Date: Tue Jan  3 16:22:57 2006
Subject: Media Type "text/csv": new draft (-02) and Last Call
Message-ID: <[email-masked]>

Yakov,

That draft wording sounds excellent.

Now, if you were to ALSO mandate the presence of the header record, that
would mitigate my concern that computer applications should be able to
exchange CSV files without worrying about same field order (simply because
the header conveys the field names, rather than needing an out-of-band,
bilateral agreement).  And, of course, presence of a header record lets a
computer application determine immediately whether the supplied fields are
likely to be what it expects.   All good aids for careful validation.

Thank-you,
Clyde

-----Original Message-----
From: Yakov Shafranovich [mailto:[email-masked]]
Sent: Wednesday, March 30, 2005 8:08 PM
To: [email-masked]
Cc: [email-masked]; [email-masked]
Subject: Re: Media Type "text/csv": new draft (-02) and Last Call

[email-masked] wrote:
> 
> But the same cannot be said of automated computer-based applications, 
> where maintaining a strict count of generated and expected 
> Comma-Separated-Values per record is not only easy, but also allows for 
> an extra level of data validation: namely that a received record is 
> corrupt if it has too few or too many fields.  This is where 
> standardisation in the format of the CSV records becomes appropriate 
> material for an RFC.
> 

The draft as it is written now (-03) does not mandate that the same 
number of fields need to appear on each line, mainly due to the fact 
that the draft is focusing on the MIME type registration. Would the 
following change to section 2, subsection 4 be sufficient to address 
your concerns:

"Within the header and each record there may be one or more fields, 
separated by commas. Each line should contain the same number of fields 
throughout the file. The last field in the record may not be followed by 
a comma. For example:"

Yakov
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.alvestrand.no/pipermail/ietf-types/attachments/20050331/73e44478/attachment.html
>From [email-masked]  Thu Mar 31 13:16:41 2005
From: GK-lists at ninebynine.org (Graham Klyne)
Date: Tue Jan  3 16:22:57 2006
Subject: Media Type "text/csv": new draft (-02) and Last Call
In-Reply-To: <[email-masked]>
References: <[email-masked]>
	<[email-masked]>
Message-ID: <[email-masked]>

At 14:08 30/03/05 -0500, Yakov Shafranovich wrote:
>[email-masked] wrote:
>>But the same cannot be said of automated computer-based applications, 
>>where maintaining a strict count of generated and expected 
>>Comma-Separated-Values per record is not only easy, but also allows for 
>>an extra level of data validation: namely that a received record is 
>>corrupt if it has too few or too many fields.  This is where 
>>standardisation in the format of the CSV records becomes appropriate 
>>material for an RFC.
>
>The draft as it is written now (-03) does not mandate that the same number 
>of fields need to appear on each line, mainly due to the fact that the 
>draft is focusing on the MIME type registration. Would the following 
>change to section 2, subsection 4 be sufficient to address your concerns:
>
>"Within the header and each record there may be one or more fields, 
>separated by commas. Each line should contain the same number of fields 
>throughout the file. The last field in the record may not be followed by a 
>comma. For example:"

I think this change would *decrease* the utility of the proposed MIME type 
registration, which I see as being to document existing practice so that 
new applications can interoperate with it.  I think the benefits of 
interworking with existing practice greatly outweigh the purported 
advantage of (weak) integrity checking.  In this case, I really think that 
less [specification] is more.

#g

------------
Graham Klyne
For email:
http://www.ninebynine.org/#Contact

From: clyde.ingram at edl.uk.eds.com ([email-masked])
Date: Tue Jan  3 16:22:57 2006
Subject: Media Type "text/csv": new draft (-02) and Last Call
Message-ID: <[email-masked]>

Graham & Yakov,

Where a Comma-Separated-Value format is used by peer computer applications
attempting to communicate with each other in an open fashion, it is very
simple for them to produce a fixed number of Comma-Separated-Values.
It sounds like your production of a variable number of
Comma-Separated-Values is an artefact of how you are manually driving one
proprietary spreadsheet program from the keyboard/mouse.  If such manually
generated output is to be read by a similarly manually driven mechanism,
then it may be acceptable to have variable numbers of Comma-Separated-Values
per record.  Standardisation in numbers of fieldds is then unnecessary, so
an RFC need not cater for the uncontrolled nature of manually handled data.

But the same cannot be said of automated computer-based applications, where
maintaining a strict count of generated and expected Comma-Separated-Values
per record is not only easy, but also allows for an extra level of data
validation: namely that a received record is corrupt if it has too few or
too many fields.  This is where standardisation in the format of the CSV
records becomes appropriate material for an RFC.

Regards,
Clyde Ingram

-----Original Message-----
From: Graham Klyne [mailto:[email-masked]]
Sent: Tuesday, March 29, 2005 12:21 PM
To: [email-masked]; [email-masked]
Cc: [email-masked]
Subject: RE: Media Type "text/csv": new draft (-02) and Last Call

At 15:19 23/03/05 +0000, [email-masked] wrote:
>Please clarify whether the trailing commas that your Excel export 
>generates are there to mark the end of the last field, or to mark the 
>start of a last field which currently has no value.

I'm not sure how to tell the difference in an Excel spreadsheet.

In the case where this arose for me, I had created a speadsheet with 
varying numbers of values in different rows, and many of the rows were 
output by Excel with *multiple* trailing commas.  Some rows were generated 
without any trailimng commas.  My point would be that if this happens with 
reasonable data then is must be permitted.  Whether it's interpreted as a 
field terminator as start of field with no value is, I think, moot.

#g
--

>Graham,
>
>-----Original Message-----
>From: Graham Klyne 
>[<mailto:[email-masked]>mailto:[email-masked]]
>Sent: Wednesday, March 23, 2005 9:55 AM
>To: Yakov Shafranovich; [email-masked]
>Cc: [email-masked]
>Subject: Re: Media Type "text/csv": new draft (-02) and Last Call
>
>At 01:14 23/03/05 -0500, Yakov Shafranovich wrote:
>
> >Clyde,
> >
> >Thanks for pointing this out. I personally think that instead of making
> >the header record mandatory which is something that most CSV applications
> >do not have, I would rather take the comma out of the end of the record
> >and have the last field end with a CRLF instead of an optional COMMA. Do
> >you think that is a plausible solution?
>
>No.  Some of the Excel data I process has trailing commas.  This must be
>allowed.
>
>I also don't think it's necessary to say anything (other than maybe as a
>comment) about any special status for the first line:  such use is
>accommodated quite reasonably within the basic CSV format.
>
>For example, having such a line when exporting Excel as CSV depends
>entirely upon how the user constructs the original spreadsheet.  Column
>headings are common, but not mandatory.  In some cases, there may be a more
>complex heading structure -- this is an application issue, not a dataset
>format issue, and as such does not belong in the dataset format 
>specification.
>
>#g
>--
>------------
>
>Please clarify whether the trailing commas that your Excel export 
>generates are there to mark the end of the last field, or to mark the 
>start of a last field which currently has no value.
>
>To take a concrete example, I would expect a CSV of sibling relationships 
>in a mythical family to look like this, assuming the siblings are one 
>brother (Bart) and 2 sisters (Lisa & Maggie):
>
>     child,sisters,brothers<CR-LF>
>     Bart,Lisa & Maggie,<CR-LF>
>     Lisa,Maggie,Bart<CR-LF>
>     Maggie,Lisa,Bart<CR-LF>
>
>where the trailing comma for the record of child=Bart signifies that the 
>"brothers" field is null, so that Bart has no brothers.  In my view this 
>is a logical conclusion, and in fact stripping that one trailing comma 
>would be an error, as that record would only have 2 fields, not 3.
>
>Would you, however, expect the CSV file to use comma as a 
>field-terminator, rather than a field-separator, as follows?:
>
>     child,sisters,brothers,<CR-LF>
>     Bart,Lisa & Maggie,,<CR-LF>
>     Lisa,Maggie,Bart,<CR-LF>
>     Maggie,Lisa,Bart,<CR-LF>
>
>Note that parsers that split data records on unprotected comma would 
>detect one field too many in this latter case.
>
>In a Comma SEPARATED Value file format, can you configure Excel to use 
>comma as a SEPARATOR between values, rather than a TERMINATOR (at the end 
>of values)?
>
>
>Regarding your remarks on the header record being "an application issue, 
>not a dataset format issue, and as such does not belong in the dataset 
>format specification": XML, ASN.1, and other (application-independent) 
>data interchange formats,  explicitly tag individual fields so that their 
>type is unambiguously defined within a context.  In contrast, CSV conveys 
>no tags per field in a data record.  Hence, to help with 
>application-independent data interchange, the CSV format should convey 
>field titles in a header record.
>
>Here is an example of lack of application-independence: if my application 
>sends yours this CSV file:
>
>     ,Bart,Lisa & Maggie<CR-LF>
>     Bart,Lisa,Maggie<CR-LF>
>     Bart,Maggie,Lisa<CR-LF>
>
>and your application depends on the assumption that the fields are the 
>sequence:
>
>     child
>     sisters
>     brothers
>
>then your application will mis-interpret the data.
>But if my application precedes this with a header record, like so:
>
>     brothers,child,sisters<CR-LF>
>     ,Bart,Lisa & Maggie<CR-LF>
>     Bart,Lisa,Maggie<CR-LF>
>     Bart,Maggie,Lisa<CR-LF>
>
>then your application can maintain independence from the change by my 
>application, because the CSV file conveys the corresponding new field 
>sequence (the columns "brother" and "child" have swapped).
>
>Regards,
>Clyde

------------
Graham Klyne
For email:
http://www.ninebynine.org/#Contact
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.alvestrand.no/pipermail/ietf-types/attachments/20050330/ed9b78ae/attachment.html
>From [email-masked]  Wed Mar 30 21:08:25 2005
From: YakovS at solidmatrix.com (Yakov Shafranovich)
Date: Tue Jan  3 16:22:57 2006
Subject: Media Type "text/csv": new draft (-02) and Last Call
In-Reply-To: <[email-masked]>
References: <[email-masked]>
Message-ID: <[email-masked]>

[email-masked] wrote:
> 
> But the same cannot be said of automated computer-based applications, 
> where maintaining a strict count of generated and expected 
> Comma-Separated-Values per record is not only easy, but also allows for 
> an extra level of data validation: namely that a received record is 
> corrupt if it has too few or too many fields.  This is where 
> standardisation in the format of the CSV records becomes appropriate 
> material for an RFC.
> 

The draft as it is written now (-03) does not mandate that the same
number of fields need to appear on each line, mainly due to the fact
that the draft is focusing on the MIME type registration. Would the
following change to section 2, subsection 4 be sufficient to address
your concerns:

"Within the header and each record there may be one or more fields,
separated by commas. Each line should contain the same number of fields
throughout the file. The last field in the record may not be followed by
a comma. For example:"

Yakov

From: ned.freed at mrochek.com ([email-masked])
Date: Wed Mar 30 04:19:20 2005
Subject: Media Type "text/csv": new draft (-02) and Last Call
In-Reply-To: "Your message dated Tue, 29 Mar 2005 12:16:30 +0100" <[email-masked]>
References: <[email-masked]> <[email-masked]> <[email-masked]>
Message-ID: <[email-masked]>

> At 09:35 23/03/05 -0500, Yakov Shafranovich wrote:
> >Graham Klyne wrote:
> >>I feel compelled to ask the question:  should this really be a text/csv
> >>type rather than application/csv?  Past comments by Ned Freed with
> >>respect to text/html suggest possibly not, but similarity with
> >>text/tab-separated-values suggests maybe yes.  I don't personally feel
> >>strongly about this.
> >
> >Can you point to Ned's message?

> OK, it's here:
>    http://www.imc.org/ietf-xml-mime/mail-archive/msg00622.html

These are actually fairly different cases. HTML is typically littered with tons
of formatting gunk that nobody wants to see or muck with. (HTML where the
actual content beings huundreds of lines from the beginning of the object is
fairly common.) CSV formats are not cluttered up this way, can and frequently
are read or written by relative novices with no special tools.

I really can see this one going either way.

				Ned
>From [email-masked]  Wed Mar 30 20:44:57 2005
From: clyde.ingram at edl.uk.eds.com ([email-masked])
Date: Wed Mar 30 20:47:22 2005
Subject: Media Type "text/csv": new draft (-02) and Last Call
Message-ID: <[email-masked]>

Graham & Yakov,

Where a Comma-Separated-Value format is used by peer computer applications
attempting to communicate with each other in an open fashion, it is very
simple for them to produce a fixed number of Comma-Separated-Values.
It sounds like your production of a variable number of
Comma-Separated-Values is an artefact of how you are manually driving one
proprietary spreadsheet program from the keyboard/mouse.  If such manually
generated output is to be read by a similarly manually driven mechanism,
then it may be acceptable to have variable numbers of Comma-Separated-Values
per record.  Standardisation in numbers of fieldds is then unnecessary, so
an RFC need not cater for the uncontrolled nature of manually handled data.

But the same cannot be said of automated computer-based applications, where
maintaining a strict count of generated and expected Comma-Separated-Values
per record is not only easy, but also allows for an extra level of data
validation: namely that a received record is corrupt if it has too few or
too many fields.  This is where standardisation in the format of the CSV
records becomes appropriate material for an RFC.

Regards,
Clyde Ingram

-----Original Message-----
From: Graham Klyne [mailto:[email-masked]]
Sent: Tuesday, March 29, 2005 12:21 PM
To: [email-masked]; [email-masked]
Cc: [email-masked]
Subject: RE: Media Type "text/csv": new draft (-02) and Last Call

At 15:19 23/03/05 +0000, [email-masked] wrote:
>Please clarify whether the trailing commas that your Excel export 
>generates are there to mark the end of the last field, or to mark the 
>start of a last field which currently has no value.

I'm not sure how to tell the difference in an Excel spreadsheet.

In the case where this arose for me, I had created a speadsheet with 
varying numbers of values in different rows, and many of the rows were 
output by Excel with *multiple* trailing commas.  Some rows were generated 
without any trailimng commas.  My point would be that if this happens with 
reasonable data then is must be permitted.  Whether it's interpreted as a 
field terminator as start of field with no value is, I think, moot.

#g
--

>Graham,
>
>-----Original Message-----
>From: Graham Klyne 
>[<mailto:[email-masked]>mailto:[email-masked]]
>Sent: Wednesday, March 23, 2005 9:55 AM
>To: Yakov Shafranovich; [email-masked]
>Cc: [email-masked]
>Subject: Re: Media Type "text/csv": new draft (-02) and Last Call
>
>At 01:14 23/03/05 -0500, Yakov Shafranovich wrote:
>
> >Clyde,
> >
> >Thanks for pointing this out. I personally think that instead of making
> >the header record mandatory which is something that most CSV applications
> >do not have, I would rather take the comma out of the end of the record
> >and have the last field end with a CRLF instead of an optional COMMA. Do
> >you think that is a plausible solution?
>
>No.  Some of the Excel data I process has trailing commas.  This must be
>allowed.
>
>I also don't think it's necessary to say anything (other than maybe as a
>comment) about any special status for the first line:  such use is
>accommodated quite reasonably within the basic CSV format.
>
>For example, having such a line when exporting Excel as CSV depends
>entirely upon how the user constructs the original spreadsheet.  Column
>headings are common, but not mandatory.  In some cases, there may be a more
>complex heading structure -- this is an application issue, not a dataset
>format issue, and as such does not belong in the dataset format 
>specification.
>
>#g
>--
>------------
>
>Please clarify whether the trailing commas that your Excel export 
>generates are there to mark the end of the last field, or to mark the 
>start of a last field which currently has no value.
>
>To take a concrete example, I would expect a CSV of sibling relationships 
>in a mythical family to look like this, assuming the siblings are one 
>brother (Bart) and 2 sisters (Lisa & Maggie):
>
>     child,sisters,brothers<CR-LF>
>     Bart,Lisa & Maggie,<CR-LF>
>     Lisa,Maggie,Bart<CR-LF>
>     Maggie,Lisa,Bart<CR-LF>
>
>where the trailing comma for the record of child=Bart signifies that the 
>"brothers" field is null, so that Bart has no brothers.  In my view this 
>is a logical conclusion, and in fact stripping that one trailing comma 
>would be an error, as that record would only have 2 fields, not 3.
>
>Would you, however, expect the CSV file to use comma as a 
>field-terminator, rather than a field-separator, as follows?:
>
>     child,sisters,brothers,<CR-LF>
>     Bart,Lisa & Maggie,,<CR-LF>
>     Lisa,Maggie,Bart,<CR-LF>
>     Maggie,Lisa,Bart,<CR-LF>
>
>Note that parsers that split data records on unprotected comma would 
>detect one field too many in this latter case.
>
>In a Comma SEPARATED Value file format, can you configure Excel to use 
>comma as a SEPARATOR between values, rather than a TERMINATOR (at the end 
>of values)?
>
>
>Regarding your remarks on the header record being "an application issue, 
>not a dataset format issue, and as such does not belong in the dataset 
>format specification": XML, ASN.1, and other (application-independent) 
>data interchange formats,  explicitly tag individual fields so that their 
>type is unambiguously defined within a context.  In contrast, CSV conveys 
>no tags per field in a data record.  Hence, to help with 
>application-independent data interchange, the CSV format should convey 
>field titles in a header record.
>
>Here is an example of lack of application-independence: if my application 
>sends yours this CSV file:
>
>     ,Bart,Lisa & Maggie<CR-LF>
>     Bart,Lisa,Maggie<CR-LF>
>     Bart,Maggie,Lisa<CR-LF>
>
>and your application depends on the assumption that the fields are the 
>sequence:
>
>     child
>     sisters
>     brothers
>
>then your application will mis-interpret the data.
>But if my application precedes this with a header record, like so:
>
>     brothers,child,sisters<CR-LF>
>     ,Bart,Lisa & Maggie<CR-LF>
>     Bart,Lisa,Maggie<CR-LF>
>     Bart,Maggie,Lisa<CR-LF>
>
>then your application can maintain independence from the change by my 
>application, because the CSV file conveys the corresponding new field 
>sequence (the columns "brother" and "child" have swapped).
>
>Regards,
>Clyde

------------
Graham Klyne
For email:
http://www.ninebynine.org/#Contact
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://eikenes.alvestrand.no/pipermail/ietf-types/attachments/20050330/ed9b78ae/attachment.html
>From [email-masked]  Wed Mar 30 21:08:25 2005
From: YakovS at solidmatrix.com (Yakov Shafranovich)
Date: Wed Mar 30 21:08:42 2005
Subject: Media Type "text/csv": new draft (-02) and Last Call
In-Reply-To: <[email-masked]>
References: <[email-masked]>
Message-ID: <[email-masked]>

[email-masked] wrote:
> 
> But the same cannot be said of automated computer-based applications, 
> where maintaining a strict count of generated and expected 
> Comma-Separated-Values per record is not only easy, but also allows for 
> an extra level of data validation: namely that a received record is 
> corrupt if it has too few or too many fields.  This is where 
> standardisation in the format of the CSV records becomes appropriate 
> material for an RFC.
> 

The draft as it is written now (-03) does not mandate that the same
number of fields need to appear on each line, mainly due to the fact
that the draft is focusing on the MIME type registration. Would the
following change to section 2, subsection 4 be sufficient to address
your concerns:

"Within the header and each record there may be one or more fields,
separated by commas. Each line should contain the same number of fields
throughout the file. The last field in the record may not be followed by
a comma. For example:"

Yakov

From: ned.freed at mrochek.com ([email-masked])
Date: Tue Jan  3 16:22:57 2006
Subject: Media Type "text/csv": new draft (-02) and Last Call
In-Reply-To: "Your message dated Tue, 29 Mar 2005 12:16:30 +0100" <[email-masked]>
References: <[email-masked]> <[email-masked]> <[email-masked]>
Message-ID: <[email-masked]>

> At 09:35 23/03/05 -0500, Yakov Shafranovich wrote:
> >Graham Klyne wrote:
> >>I feel compelled to ask the question:  should this really be a text/csv
> >>type rather than application/csv?  Past comments by Ned Freed with
> >>respect to text/html suggest possibly not, but similarity with
> >>text/tab-separated-values suggests maybe yes.  I don't personally feel
> >>strongly about this.
> >
> >Can you point to Ned's message?

> OK, it's here:
>    http://www.imc.org/ietf-xml-mime/mail-archive/msg00622.html

These are actually fairly different cases. HTML is typically littered with tons
of formatting gunk that nobody wants to see or muck with. (HTML where the
actual content beings huundreds of lines from the beginning of the object is
fairly common.) CSV formats are not cluttered up this way, can and frequently
are read or written by relative novices with no special tools.

I really can see this one going either way.

				Ned

From: GK-lists at ninebynine.org (Graham Klyne)
Date: Wed Mar 30 03:40:21 2005
Subject: Media Type "text/csv": new draft (-02) and Last Call
In-Reply-To: <[email-masked] dl.uk.eds.com>
Message-ID: <[email-masked]>

At 15:19 23/03/05 +0000, [email-masked] wrote:
>Please clarify whether the trailing commas that your Excel export 
>generates are there to mark the end of the last field, or to mark the 
>start of a last field which currently has no value.

I'm not sure how to tell the difference in an Excel spreadsheet.

In the case where this arose for me, I had created a speadsheet with 
varying numbers of values in different rows, and many of the rows were 
output by Excel with *multiple* trailing commas.  Some rows were generated 
without any trailimng commas.  My point would be that if this happens with 
reasonable data then is must be permitted.  Whether it's interpreted as a 
field terminator as start of field with no value is, I think, moot.

#g
--

>Graham,
>
>-----Original Message-----
>From: Graham Klyne 
>[<mailto:[email-masked]>mailto:[email-masked]]
>Sent: Wednesday, March 23, 2005 9:55 AM
>To: Yakov Shafranovich; [email-masked]
>Cc: [email-masked]
>Subject: Re: Media Type "text/csv": new draft (-02) and Last Call
>
>At 01:14 23/03/05 -0500, Yakov Shafranovich wrote:
>
> >Clyde,
> >
> >Thanks for pointing this out. I personally think that instead of making
> >the header record mandatory which is something that most CSV applications
> >do not have, I would rather take the comma out of the end of the record
> >and have the last field end with a CRLF instead of an optional COMMA. Do
> >you think that is a plausible solution?
>
>No.  Some of the Excel data I process has trailing commas.  This must be
>allowed.
>
>I also don't think it's necessary to say anything (other than maybe as a
>comment) about any special status for the first line:  such use is
>accommodated quite reasonably within the basic CSV format.
>
>For example, having such a line when exporting Excel as CSV depends
>entirely upon how the user constructs the original spreadsheet.  Column
>headings are common, but not mandatory.  In some cases, there may be a more
>complex heading structure -- this is an application issue, not a dataset
>format issue, and as such does not belong in the dataset format 
>specification.
>
>#g
>--
>------------
>
>Please clarify whether the trailing commas that your Excel export 
>generates are there to mark the end of the last field, or to mark the 
>start of a last field which currently has no value.
>
>To take a concrete example, I would expect a CSV of sibling relationships 
>in a mythical family to look like this, assuming the siblings are one 
>brother (Bart) and 2 sisters (Lisa & Maggie):
>
>     child,sisters,brothers<CR-LF>
>     Bart,Lisa & Maggie,<CR-LF>
>     Lisa,Maggie,Bart<CR-LF>
>     Maggie,Lisa,Bart<CR-LF>
>
>where the trailing comma for the record of child=Bart signifies that the 
>"brothers" field is null, so that Bart has no brothers.  In my view this 
>is a logical conclusion, and in fact stripping that one trailing comma 
>would be an error, as that record would only have 2 fields, not 3.
>
>Would you, however, expect the CSV file to use comma as a 
>field-terminator, rather than a field-separator, as follows?:
>
>     child,sisters,brothers,<CR-LF>
>     Bart,Lisa & Maggie,,<CR-LF>
>     Lisa,Maggie,Bart,<CR-LF>
>     Maggie,Lisa,Bart,<CR-LF>
>
>Note that parsers that split data records on unprotected comma would 
>detect one field too many in this latter case.
>
>In a Comma SEPARATED Value file format, can you configure Excel to use 
>comma as a SEPARATOR between values, rather than a TERMINATOR (at the end 
>of values)?
>
>
>Regarding your remarks on the header record being "an application issue, 
>not a dataset format issue, and as such does not belong in the dataset 
>format specification": XML, ASN.1, and other (application-independent) 
>data interchange formats,  explicitly tag individual fields so that their 
>type is unambiguously defined within a context.  In contrast, CSV conveys 
>no tags per field in a data record.  Hence, to help with 
>application-independent data interchange, the CSV format should convey 
>field titles in a header record.
>
>Here is an example of lack of application-independence: if my application 
>sends yours this CSV file:
>
>     ,Bart,Lisa & Maggie<CR-LF>
>     Bart,Lisa,Maggie<CR-LF>
>     Bart,Maggie,Lisa<CR-LF>
>
>and your application depends on the assumption that the fields are the 
>sequence:
>
>     child
>     sisters
>     brothers
>
>then your application will mis-interpret the data.
>But if my application precedes this with a header record, like so:
>
>     brothers,child,sisters<CR-LF>
>     ,Bart,Lisa & Maggie<CR-LF>
>     Bart,Lisa,Maggie<CR-LF>
>     Bart,Maggie,Lisa<CR-LF>
>
>then your application can maintain independence from the change by my 
>application, because the CSV file conveys the corresponding new field 
>sequence (the columns "brother" and "child" have swapped).
>
>Regards,
>Clyde

------------
Graham Klyne
For email:
http://www.ninebynine.org/#Contact

From: GK-lists at ninebynine.org (Graham Klyne)
Date: Tue Jan  3 16:22:57 2006
Subject: Media Type "text/csv": new draft (-02) and Last Call
In-Reply-To: <[email-masked] dl.uk.eds.com>
Message-ID: <[email-masked]>

At 15:19 23/03/05 +0000, [email-masked] wrote:
>Please clarify whether the trailing commas that your Excel export 
>generates are there to mark the end of the last field, or to mark the 
>start of a last field which currently has no value.

I'm not sure how to tell the difference in an Excel spreadsheet.

In the case where this arose for me, I had created a speadsheet with 
varying numbers of values in different rows, and many of the rows were 
output by Excel with *multiple* trailing commas.  Some rows were generated 
without any trailimng commas.  My point would be that if this happens with 
reasonable data then is must be permitted.  Whether it's interpreted as a 
field terminator as start of field with no value is, I think, moot.

#g
--

>Graham,
>
>-----Original Message-----
>From: Graham Klyne 
>[<mailto:[email-masked]>mailto:[email-masked]]
>Sent: Wednesday, March 23, 2005 9:55 AM
>To: Yakov Shafranovich; [email-masked]
>Cc: [email-masked]
>Subject: Re: Media Type "text/csv": new draft (-02) and Last Call
>
>At 01:14 23/03/05 -0500, Yakov Shafranovich wrote:
>
> >Clyde,
> >
> >Thanks for pointing this out. I personally think that instead of making
> >the header record mandatory which is something that most CSV applications
> >do not have, I would rather take the comma out of the end of the record
> >and have the last field end with a CRLF instead of an optional COMMA. Do
> >you think that is a plausible solution?
>
>No.  Some of the Excel data I process has trailing commas.  This must be
>allowed.
>
>I also don't think it's necessary to say anything (other than maybe as a
>comment) about any special status for the first line:  such use is
>accommodated quite reasonably within the basic CSV format.
>
>For example, having such a line when exporting Excel as CSV depends
>entirely upon how the user constructs the original spreadsheet.  Column
>headings are common, but not mandatory.  In some cases, there may be a more
>complex heading structure -- this is an application issue, not a dataset
>format issue, and as such does not belong in the dataset format 
>specification.
>
>#g
>--
>------------
>
>Please clarify whether the trailing commas that your Excel export 
>generates are there to mark the end of the last field, or to mark the 
>start of a last field which currently has no value.
>
>To take a concrete example, I would expect a CSV of sibling relationships 
>in a mythical family to look like this, assuming the siblings are one 
>brother (Bart) and 2 sisters (Lisa & Maggie):
>
>     child,sisters,brothers<CR-LF>
>     Bart,Lisa & Maggie,<CR-LF>
>     Lisa,Maggie,Bart<CR-LF>
>     Maggie,Lisa,Bart<CR-LF>
>
>where the trailing comma for the record of child=Bart signifies that the 
>"brothers" field is null, so that Bart has no brothers.  In my view this 
>is a logical conclusion, and in fact stripping that one trailing comma 
>would be an error, as that record would only have 2 fields, not 3.
>
>Would you, however, expect the CSV file to use comma as a 
>field-terminator, rather than a field-separator, as follows?:
>
>     child,sisters,brothers,<CR-LF>
>     Bart,Lisa & Maggie,,<CR-LF>
>     Lisa,Maggie,Bart,<CR-LF>
>     Maggie,Lisa,Bart,<CR-LF>
>
>Note that parsers that split data records on unprotected comma would 
>detect one field too many in this latter case.
>
>In a Comma SEPARATED Value file format, can you configure Excel to use 
>comma as a SEPARATOR between values, rather than a TERMINATOR (at the end 
>of values)?
>
>
>Regarding your remarks on the header record being "an application issue, 
>not a dataset format issue, and as such does not belong in the dataset 
>format specification": XML, ASN.1, and other (application-independent) 
>data interchange formats,  explicitly tag individual fields so that their 
>type is unambiguously defined within a context.  In contrast, CSV conveys 
>no tags per field in a data record.  Hence, to help with 
>application-independent data interchange, the CSV format should convey 
>field titles in a header record.
>
>Here is an example of lack of application-independence: if my application 
>sends yours this CSV file:
>
>     ,Bart,Lisa & Maggie<CR-LF>
>     Bart,Lisa,Maggie<CR-LF>
>     Bart,Maggie,Lisa<CR-LF>
>
>and your application depends on the assumption that the fields are the 
>sequence:
>
>     child
>     sisters
>     brothers
>
>then your application will mis-interpret the data.
>But if my application precedes this with a header record, like so:
>
>     brothers,child,sisters<CR-LF>
>     ,Bart,Lisa & Maggie<CR-LF>
>     Bart,Lisa,Maggie<CR-LF>
>     Bart,Maggie,Lisa<CR-LF>
>
>then your application can maintain independence from the change by my 
>application, because the CSV file conveys the corresponding new field 
>sequence (the columns "brother" and "child" have swapped).
>
>Regards,
>Clyde

------------
Graham Klyne
For email:
http://www.ninebynine.org/#Contact

From: GK-lists at ninebynine.org (Graham Klyne)
Date: Tue Jan  3 16:22:57 2006
Subject: Media Type "text/csv": new draft (-02) and Last Call
In-Reply-To: <[email-masked]>
References: <[email-masked]> <[email-masked]>
Message-ID: <[email-masked]>

At 09:35 23/03/05 -0500, Yakov Shafranovich wrote:
>Graham Klyne wrote:
>>I feel compelled to ask the question:  should this really be a text/csv 
>>type rather than application/csv?  Past comments by Ned Freed with 
>>respect to text/html suggest possibly not, but similarity with 
>>text/tab-separated-values suggests maybe yes.  I don't personally feel 
>>strongly about this.
>
>Can you point to Ned's message?

OK, it's here:
   http://www.imc.org/ietf-xml-mime/mail-archive/msg00622.html

#g

------------
Graham Klyne
For email:
http://www.ninebynine.org/#Contact

From: mccobb at us.ibm.com (Gerald McCobb)
Date: Tue Jan  3 16:22:57 2006
Subject: Request for comments regarding draft-mccobb-xplusv-media-type-01
Message-ID: <[email-masked]>

This is a request for comments regarding the internet draft
draft-mccobb-xplusv-media-type-01 for media type 
"application/xhtml+voice+xml".
This media type is for XHTML+Voice documents.  The XHTML+Voice 
specification
is located at http://www.voicexml.org/specs/multimodal/x+v/12/ .

Network Working Group                                          G. McCobb
Internet-Draft                                           IBM Corporation
Expires: May 15, 2005                                      Nov. 15, 2004

                XHTML+Voice - application/xhtml+voice+xml
                    draft-mccobb-xplusv-media-type-01

Status of this Memo

   By submitting this Internet-Draft, I certify that any applicable
   patent or other IPR claims of which I am aware have been disclosed,
   and any of which I become aware will be disclosed, in accordance 
   with RFC 3668.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as
   Internet-Drafts.

   Internet-Drafts are draft documents valid for a maximum of six 
   months and may be updated, replaced, or obsoleted by other documents
   at any time.  It is inappropriate to use Internet-Drafts as 
   reference material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt.

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.

Copyright Notice

   Copyright (C) The Internet Society (2004).  All Rights Reserved

Abstract

   This document describes the registration of the MIME sub-type
   application/xhtml+voice+xml.  This sub-type is intended for use as a
   media descriptor for XHTML+Voice multimodal language documents.  The
   XHTML+Voice 1.2 language specification is maintained by the VoiceXML
   Forum at <http://www.voicexml.org/specs/multimodal/x+v/12/>.

1.  Conventions used in this document

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in BCP 14, RFC 2119
   [REQ].

Mccobb                   Expires May 15, 2005                   [Page 1]

Internet-Draft   XHTML+Voice - application/xhtml+voice+xml Nov. 15, 2004

2. Introduction

   XHTML+VOice is a member of the XHTML family of document types, as
   specified by XHTML Modularization.  XHTML+Voice extends XHTML 1.1
   [XHTML11] with a modularized subset of VoiceXML 2.0 [VXML20], XML
   Events [XMLEVNTS], and a few extensions to both XHTML and VoiceXML
   2.0.  XHTML 1.1, VoiceXML 2.0 and XML Events are W3C Recommendations.

   The language integration defined by XHTML+Voice supports all modules
   defined by XHTML Modularization, and adds voice interaction to XHTML
   elements to enable multimodal applications.  The defined document
   type for XHTML+VOice is XHTML Host language document type conformant.

   XHTML+Voice 1.2 [XPLUSV12] is maintained by the VoiceXML Forum, at
   URI location <http://www.voicexml.org/specs/multimodal/x+v/12/>.

2.1 application/xhtml+voice+xml Usage

   The application/xhtml+voice+xml media type is intended to be a media
   descriptor for XHTML+Voice documents.

   This media type registration is not intended for email usage.

3. IANA Registration

      To: [email-masked]
      Subject: Registration of Standard MIME media type 
               application/xhtml+voice+xml

      MIME media type name: application

      MIME subtype name: xhtml+voice+xml

      Required parameters: none

      Optional parameters:
   version:  refers to the XHTML+Voice language version in the document.
             Acceptable values are 1.0, 1.1, and 1.2 (default).

   charset:  has the same meaning as the text/html media type.  See
   section 2 of [RFC 2854].

      Encoding considerations: 7bit.  See section 4 of [RFC 2854].

      Security considerations:
 
   XHTML+Voice is an extension of XHTML and has the same security issues
   as XHTML.  These include interpreting anchors and forms in HTML

Mccobb                   Expires May 15, 2005                   [Page 2]

Internet-Draft   XHTML+Voice - application/xhtml+voice+xml Nov. 15, 2004

   documents, and scripting languages and other dynamic interactive
   capabilities.  See section 7 of [RFC 2854].

      Interoperability considerations :

   Because XHTML+Voice is built upon W3C standard recommendations, it is
   designed to be interoperable across a wide range of platforms and
   client devices.  Because the extensions to XHTML are identified by
   their namespaces, all browsers that have namespace support can run an
   XHTML+Voice document as an XHTML document without voice interaction.
 
     Published specification:

         The latest published version of XHTML+Voice is [XPLUSV12].

      Applications which use this media type:

         XHTML+Voice documents are intended to be deployed on the World
         Wide Web and rendered by multimodal browsers that support the
         visual and voice modes of interaction.  See section 2 of
         [RFC 2854]. 

      Additional information:

         Magic number(s):  There is no single string that is always
             present.
         File extension(s): html, htm, mxml, xvml
         Macintosh File Type Code(s): TEXT

      Person & email address to contact for further information:

         Gerald M. McCobb
         [email-masked]

      Intended usage: COMMON

      Author/Change controller: Gerald McCobb

      Further information:

4. Fragment Identifiers

   See section 3 of [RFC 2854].

5. Recognizing XHTML+Voice files

   Because XHTML+Voice is XML, an XHTML+Voice document [optionally]
   starts with an XML declaration which begins with "<?xml" and has a
   DOCTYPE declaration "<!DOCTYPE html".  XHTML+Voice 1.2 has the

Mccobb                   Expires May 15, 2005                   [Page 3]

Internet-Draft   XHTML+Voice - application/xhtml+voice+xml Nov. 15, 2004

   following DOCTYPE declaration:
 
   <!DOCTYPE html PUBLIC "-//VoiceXML Forum//DTD XHTML+Voice 1.2//EN"
"http://www.voicexml.org/specs/multimodal/x+v/12/dtd/xhtml+voice12.dtd">
 
   Because XHTML+Voice is in the XHTML family of languages, the root
   element of an XHTML+Voice document is 'html' and '<html' can be found
   near the top of the document. 

6. Security Considerations

   Security considerations for this media type are discussed in the MIME
   type registration that appears in section 4.

7. References

   [REQ]      Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119, March 1997.

   [DOM2EV]   "Document Object Model Level 2 Events Specification," 
              Tom Pixley, 2000.  W3C Recommendation,
              http://www.w3.org/TR/DOM-Level-2-Events/.

   [HTML401]  Raggett, D., et al., "HTML 4.01 Specification", W3C
              Recommendation, December 1999.
              Available at <http://www.w3.org/TR/html401>.

   [MIME4]    Freed, N. and N. Borenstein, "Multipurpose Internet Mail
              Extensions (MIME) Part Four: Registration Procedures", BCP
              13, RFC 2048, November 1996.

   [RFC 2854] RFC 2854: The 'text/html' Media Type, Connolly and
              Masinter, June 2000, http://www.ietf.org/rfc/rfc2854.txt.

   [XHTMLMOD] "Modularization of XHTML," 10 April, 2001, Murray Altheim,
              Frank Boumphrey, Sam Dooley, et al, W3C Recommendation, 
              http://www.w3.org/TR/xhtml-modularization/

   [XHTML1]   "XHTML 1.0: The Extensible HyperText Markup Language: A
              Reformulation of HTML 4 in XML 1.0", W3C Recommendation,
              January 2000. Available at <http://www.w3.org/TR/xhtml1>.

   [XHTML11]  "XHTML 1.1 - Module-based XHTML," 31 May 2001, Murray 
              Altheim, Shane McCarron, W3C Recommendation,
              http://www.w3.org/TR/xhtml11/.

   [XMLEVNTS] "XML Events - An events syntax for XML", Steven
              Pemberton, T. V. Raman, and Shane McCarron, 2002.  W3C
              Recommendation, http://www.w3.org/TR/xml-events/.

Mccobb                   Expires May 15, 2005                   [Page 4]

Internet-Draft   XHTML+Voice - application/xhtml+voice+xml Nov. 15, 2004

   [XPLUSV12] "XHTML+Voice Profile 1.2," 16 March 2004, J. Axelsson,
              et al, http://www.voicexml.org/specs/multimodal/x+v/12/

   [VXML20]   "Voice Extensible Markup Language (VoiceXML)," 16 March
              2004, Scott McGlashan et al, W3C Recommendation,
              http://www.w3.org/TR/voicexml20/.

8. Authors' Address

   Gerald M. McCobb
   IBM Corporation
   8051 Congress Avenue, Office 2019
   Boca Raton, Florida 33487
   USA

   Phone: +1-561-862-2109
   Fax:   +1-561-862-3922
   EMail: [email-masked]

9. Full Copyright Statement

   Copyright (C) The Internet Society (2004).  This document is subject
   to the rights, licenses and restrictions contained in BCP 78, and
   except as set forth therein, the authors retain all their rights.

   This document and the information contained herein are provided on 
   an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE 
   REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE
   INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR 
   IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF 
   THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED 
   WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Intellectual Property

   The IETF takes no position regarding the validity or scope of any
   Intellectual Property Rights or other rights that might be claimed 
   to pertain to the implementation or use of the technology described 
   in this document or the extent to which any license under such 
   rights might or might not be available; nor does it represent that 
   it has made any independent effort to identify any such rights. 
   Information on the procedures with respect to rights in RFC 
   documents can be found in BCP 78 and BCP 79.

   Copies of IPR disclosures made to the IETF Secretariat and any
   assurances of licenses to be made available, or the result of an
   attempt made to obtain a general license or permission for the use

Mccobb                   Expires May 15, 2005                   [Page 5]

Internet-Draft   XHTML+Voice - application/xhtml+voice+xml Nov. 15, 2004

   of such proprietary rights by implementers or users of this 
   specification can be obtained from the IETF on-line IPR repository 
   at http://www.ietf.org/ipr. 

   The IETF invites any interested party to bring to its attention any
   copyrights, patents or patent applications, or other proprietary
   rights that may cover technology that may be required to implement
   this standard.  Please address the information to the IETF at ietf-
   [email-masked].

Acknowledgement

   Funding for the RFC Editor function is currently provided by the
   Internet Society.

Mccobb                   Expires May 15, 2005                   [Page 6]

Regards,
Gerald McCobb
IBM
8051 Congress Avenue
Boca Raton, FL 33487
Tel. # 561-862-2109 T/L 975-2109
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.alvestrand.no/pipermail/ietf-types/attachments/20050325/20b185e7/attachment.html
>From [email-masked]  Sun Mar 27 00:06:48 2005
From: derhoermi at gmx.net (Bjoern Hoehrmann)
Date: Tue Jan  3 16:22:57 2006
Subject: Request for comments regarding draft-mccobb-xplusv-media-type-01
In-Reply-To: <[email-masked]>
References: <[email-masked]>
Message-ID: <[email-masked]>

* Gerald McCobb wrote:
>2.1 application/xhtml+voice+xml Usage
>
>   The application/xhtml+voice+xml media type is intended to be a media
>   descriptor for XHTML+Voice documents.
>
>   This media type registration is not intended for email usage.

The draft does not really explain why there is a need for such a
type in the first place; why don't you use application/xhtml+xml?

>      Optional parameters:
>   version:  refers to the XHTML+Voice language version in the document.
>             Acceptable values are 1.0, 1.1, and 1.2 (default).

What are the processing requirements for this parameter?

>   charset:  has the same meaning as the text/html media type.  See
>   section 2 of [RFC 2854].

A word seems to be missing here. It is not really clear why the draft
refers to RFC 2854 rather than RFC 3023 or RFC 3236. 

>      Security considerations:
> 
>   XHTML+Voice is an extension of XHTML and has the same security issues
>   as XHTML.  These include interpreting anchors and forms in HTML
>   documents, and scripting languages and other dynamic interactive
>   capabilities.  See section 7 of [RFC 2854].

So the extensions to XHTML have no security considerations?

>         File extension(s): html, htm, mxml, xvml

I don't think .html and .htm should be listed here, they are already in
common use for application/xhtml+xml and text/html.

>4. Fragment Identifiers
>
>   See section 3 of [RFC 2854].

It is unclear what this means, RFC 2854 and RFC 3236 have inconsistent
rules for fragment identifiers and XHTML+Voice adds new ID attributes
so that the fragment identifier namespaces are different.

>6. Security Considerations
>
>   Security considerations for this media type are discussed in the MIME
>   type registration that appears in section 4.

Section 4 has no security considerations...

>7. References

The draft should be clear about which of these references are
informative and which normative.
-- 
Bj?rn H?hrmann ? mailto:[email-masked] ? http://bjoern.hoehrmann.de
Weinh. Str. 22 ? Telefon: +49(0)621/4309674 ? http://www.bjoernsworld.de
68309 Mannheim ? PGP Pub. KeyID: 0xA4357E78 ? http://www.websitedev.de/ 

From: YakovS at solidmatrix.com (Yakov Shafranovich)
Date: Thu Mar 24 23:24:44 2005
Subject: Media Type "text/csv" - trailing nullcols
In-Reply-To: <[email-masked]>
References: <[email-masked]>
Message-ID: <[email-masked]>

Tak,

Thanks for your comments. My intent with this draft was to document the 
current convention and register the MIME type. I would rather stay away 
from requiring or allowing features explicetly that might not be 
followed by all implementations.

Yakov

Yakov Shafranovich wrote:
> From: "Tak Tang" <[email-masked]>
> To: <[email-masked]>
> Date: Thu, 24 Mar 2005 11:30:43 +0000
> Subj: Media Type "text/csv" - trailing nullcols
> 
> Hello,
> 
> This is my first post, and I've entered the discussion late, so please 
> excuse me if this has already been discussed . . .
> 
> The draft (03) does not require that each record/line have the same number 
> of values/commas.
> I think it may be benefitial to explicitly allow csv writers to omit 
> trailing commas if there are no further non-null values.
> 
> eg (all of these lines are valid)
> aaa,bbb,ccc,ddd<CR-LF>
> aaa,,,<CR-LF>
> aaa<CR-LF>
> ,bbb<CR-LF>
> ,,ccc<CR-LF>
> ,,,ddd<CR-LF>
> 
> The second and third line are the same - they just have a value in the 
> first field.
> The csv reader should assume that if there are missing columns (ie commas) 
> then the values are null.
> Leading null values still require the commas to be included (lines 4-6).
> 
> This is useful when
> a) the dataset is sparse.
> b) the dataset has new columns added - the reader can still accept old 
> datafiles without them being considered 'malformed'.
> 
> Microsoft Excel and Oracle SQL*Loader can already accept data with missing 
> (trailing) commas.
> 
> 
> Tak Tang
> 
> 
> 
> 
>From [email-masked]  Fri Mar 25 17:19:09 2005
From: mccobb at us.ibm.com (Gerald McCobb)
Date: Sat Mar 26 23:48:40 2005
Subject: Request for comments regarding draft-mccobb-xplusv-media-type-01
Message-ID: <[email-masked]>

This is a request for comments regarding the internet draft
draft-mccobb-xplusv-media-type-01 for media type 
"application/xhtml+voice+xml".
This media type is for XHTML+Voice documents.  The XHTML+Voice 
specification
is located at http://www.voicexml.org/specs/multimodal/x+v/12/ .

Network Working Group                                          G. McCobb
Internet-Draft                                           IBM Corporation
Expires: May 15, 2005                                      Nov. 15, 2004

                XHTML+Voice - application/xhtml+voice+xml
                    draft-mccobb-xplusv-media-type-01

Status of this Memo

   By submitting this Internet-Draft, I certify that any applicable
   patent or other IPR claims of which I am aware have been disclosed,
   and any of which I become aware will be disclosed, in accordance 
   with RFC 3668.

   Internet-Drafts are working documents of the Internet Engineering
   Task Force (IETF), its areas, and its working groups.  Note that
   other groups may also distribute working documents as
   Internet-Drafts.

   Internet-Drafts are draft documents valid for a maximum of six 
   months and may be updated, replaced, or obsoleted by other documents
   at any time.  It is inappropriate to use Internet-Drafts as 
   reference material or to cite them other than as "work in progress."

   The list of current Internet-Drafts can be accessed at
   http://www.ietf.org/ietf/1id-abstracts.txt.

   The list of Internet-Draft Shadow Directories can be accessed at
   http://www.ietf.org/shadow.html.

Copyright Notice

   Copyright (C) The Internet Society (2004).  All Rights Reserved

Abstract

   This document describes the registration of the MIME sub-type
   application/xhtml+voice+xml.  This sub-type is intended for use as a
   media descriptor for XHTML+Voice multimodal language documents.  The
   XHTML+Voice 1.2 language specification is maintained by the VoiceXML
   Forum at <http://www.voicexml.org/specs/multimodal/x+v/12/>.

1.  Conventions used in this document

   The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT",
   "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this
   document are to be interpreted as described in BCP 14, RFC 2119
   [REQ].

Mccobb                   Expires May 15, 2005                   [Page 1]

Internet-Draft   XHTML+Voice - application/xhtml+voice+xml Nov. 15, 2004

2. Introduction

   XHTML+VOice is a member of the XHTML family of document types, as
   specified by XHTML Modularization.  XHTML+Voice extends XHTML 1.1
   [XHTML11] with a modularized subset of VoiceXML 2.0 [VXML20], XML
   Events [XMLEVNTS], and a few extensions to both XHTML and VoiceXML
   2.0.  XHTML 1.1, VoiceXML 2.0 and XML Events are W3C Recommendations.

   The language integration defined by XHTML+Voice supports all modules
   defined by XHTML Modularization, and adds voice interaction to XHTML
   elements to enable multimodal applications.  The defined document
   type for XHTML+VOice is XHTML Host language document type conformant.

   XHTML+Voice 1.2 [XPLUSV12] is maintained by the VoiceXML Forum, at
   URI location <http://www.voicexml.org/specs/multimodal/x+v/12/>.

2.1 application/xhtml+voice+xml Usage

   The application/xhtml+voice+xml media type is intended to be a media
   descriptor for XHTML+Voice documents.

   This media type registration is not intended for email usage.

3. IANA Registration

      To: [email-masked]
      Subject: Registration of Standard MIME media type 
               application/xhtml+voice+xml

      MIME media type name: application

      MIME subtype name: xhtml+voice+xml

      Required parameters: none

      Optional parameters:
   version:  refers to the XHTML+Voice language version in the document.
             Acceptable values are 1.0, 1.1, and 1.2 (default).

   charset:  has the same meaning as the text/html media type.  See
   section 2 of [RFC 2854].

      Encoding considerations: 7bit.  See section 4 of [RFC 2854].

      Security considerations:
 
   XHTML+Voice is an extension of XHTML and has the same security issues
   as XHTML.  These include interpreting anchors and forms in HTML

Mccobb                   Expires May 15, 2005                   [Page 2]

Internet-Draft   XHTML+Voice - application/xhtml+voice+xml Nov. 15, 2004

   documents, and scripting languages and other dynamic interactive
   capabilities.  See section 7 of [RFC 2854].

      Interoperability considerations :

   Because XHTML+Voice is built upon W3C standard recommendations, it is
   designed to be interoperable across a wide range of platforms and
   client devices.  Because the extensions to XHTML are identified by
   their namespaces, all browsers that have namespace support can run an
   XHTML+Voice document as an XHTML document without voice interaction.
 
     Published specification:

         The latest published version of XHTML+Voice is [XPLUSV12].

      Applications which use this media type:

         XHTML+Voice documents are intended to be deployed on the World
         Wide Web and rendered by multimodal browsers that support the
         visual and voice modes of interaction.  See section 2 of
         [RFC 2854]. 

      Additional information:

         Magic number(s):  There is no single string that is always
             present.
         File extension(s): html, htm, mxml, xvml
         Macintosh File Type Code(s): TEXT

      Person & email address to contact for further information:

         Gerald M. McCobb
         [email-masked]

      Intended usage: COMMON

      Author/Change controller: Gerald McCobb

      Further information:

4. Fragment Identifiers

   See section 3 of [RFC 2854].

5. Recognizing XHTML+Voice files

   Because XHTML+Voice is XML, an XHTML+Voice document [optionally]
   starts with an XML declaration which begins with "<?xml" and has a
   DOCTYPE declaration "<!DOCTYPE html".  XHTML+Voice 1.2 has the

Mccobb                   Expires May 15, 2005                   [Page 3]

Internet-Draft   XHTML+Voice - application/xhtml+voice+xml Nov. 15, 2004

   following DOCTYPE declaration:
 
   <!DOCTYPE html PUBLIC "-//VoiceXML Forum//DTD XHTML+Voice 1.2//EN"
"http://www.voicexml.org/specs/multimodal/x+v/12/dtd/xhtml+voice12.dtd">
 
   Because XHTML+Voice is in the XHTML family of languages, the root
   element of an XHTML+Voice document is 'html' and '<html' can be found
   near the top of the document. 

6. Security Considerations

   Security considerations for this media type are discussed in the MIME
   type registration that appears in section 4.

7. References

   [REQ]      Bradner, S., "Key words for use in RFCs to Indicate
              Requirement Levels", BCP 14, RFC 2119, March 1997.

   [DOM2EV]   "Document Object Model Level 2 Events Specification," 
              Tom Pixley, 2000.  W3C Recommendation,
              http://www.w3.org/TR/DOM-Level-2-Events/.

   [HTML401]  Raggett, D., et al., "HTML 4.01 Specification", W3C
              Recommendation, December 1999.
              Available at <http://www.w3.org/TR/html401>.

   [MIME4]    Freed, N. and N. Borenstein, "Multipurpose Internet Mail
              Extensions (MIME) Part Four: Registration Procedures", BCP
              13, RFC 2048, November 1996.

   [RFC 2854] RFC 2854: The 'text/html' Media Type, Connolly and
              Masinter, June 2000, http://www.ietf.org/rfc/rfc2854.txt.

   [XHTMLMOD] "Modularization of XHTML," 10 April, 2001, Murray Altheim,
              Frank Boumphrey, Sam Dooley, et al, W3C Recommendation, 
              http://www.w3.org/TR/xhtml-modularization/

   [XHTML1]   "XHTML 1.0: The Extensible HyperText Markup Language: A
              Reformulation of HTML 4 in XML 1.0", W3C Recommendation,
              January 2000. Available at <http://www.w3.org/TR/xhtml1>.

   [XHTML11]  "XHTML 1.1 - Module-based XHTML," 31 May 2001, Murray 
              Altheim, Shane McCarron, W3C Recommendation,
              http://www.w3.org/TR/xhtml11/.

   [XMLEVNTS] "XML Events - An events syntax for XML", Steven
              Pemberton, T. V. Raman, and Shane McCarron, 2002.  W3C
              Recommendation, http://www.w3.org/TR/xml-events/.

Mccobb                   Expires May 15, 2005                   [Page 4]

Internet-Draft   XHTML+Voice - application/xhtml+voice+xml Nov. 15, 2004

   [XPLUSV12] "XHTML+Voice Profile 1.2," 16 March 2004, J. Axelsson,
              et al, http://www.voicexml.org/specs/multimodal/x+v/12/

   [VXML20]   "Voice Extensible Markup Language (VoiceXML)," 16 March
              2004, Scott McGlashan et al, W3C Recommendation,
              http://www.w3.org/TR/voicexml20/.

8. Authors' Address

   Gerald M. McCobb
   IBM Corporation
   8051 Congress Avenue, Office 2019
   Boca Raton, Florida 33487
   USA

   Phone: +1-561-862-2109
   Fax:   +1-561-862-3922
   EMail: [email-masked]

9. Full Copyright Statement

   Copyright (C) The Internet Society (2004).  This document is subject
   to the rights, licenses and restrictions contained in BCP 78, and
   except as set forth therein, the authors retain all their rights.

   This document and the information contained herein are provided on 
   an "AS IS" basis and THE CONTRIBUTOR, THE ORGANIZATION HE/SHE 
   REPRESENTS OR IS SPONSORED BY (IF ANY), THE INTERNET SOCIETY AND THE
   INTERNET ENGINEERING TASK FORCE DISCLAIM ALL WARRANTIES, EXPRESS OR 
   IMPLIED, INCLUDING BUT NOT LIMITED TO ANY WARRANTY THAT THE USE OF 
   THE INFORMATION HEREIN WILL NOT INFRINGE ANY RIGHTS OR ANY IMPLIED 
   WARRANTIES OF MERCHANTABILITY OR FITNESS FOR A PARTICULAR PURPOSE.

Intellectual Property

   The IETF takes no position regarding the validity or scope of any
   Intellectual Property Rights or other rights that might be claimed 
   to pertain to the implementation or use of the technology described 
   in this document or the extent to which any license under such 
   rights might or might not be available; nor does it represent that 
   it has made any independent effort to identify any such rights. 
   Information on the procedures with respect to rights in RFC 
   documents can be found in BCP 78 and BCP 79.

   Copies of IPR disclosures made to the IETF Secretariat and any
   assurances of licenses to be made available, or the result of an
   attempt made to obtain a general license or permission for the use

Mccobb                   Expires May 15, 2005                   [Page 5]

Internet-Draft   XHTML+Voice - application/xhtml+voice+xml Nov. 15, 2004

   of such proprietary rights by implementers or users of this 
   specification can be obtained from the IETF on-line IPR repository 
   at http://www.ietf.org/ipr. 

   The IETF invites any interested party to bring to its attention any
   copyrights, patents or patent applications, or other proprietary
   rights that may cover technology that may be required to implement
   this standard.  Please address the information to the IETF at ietf-
   [email-masked].

Acknowledgement

   Funding for the RFC Editor function is currently provided by the
   Internet Society.

Mccobb                   Expires May 15, 2005                   [Page 6]

Regards,
Gerald McCobb
IBM
8051 Congress Avenue
Boca Raton, FL 33487
Tel. # 561-862-2109 T/L 975-2109
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://eikenes.alvestrand.no/pipermail/ietf-types/attachments/20050325/20b185e7/attachment-0001.html
>From [email-masked]  Sun Mar 27 00:06:48 2005
From: derhoermi at gmx.net (Bjoern Hoehrmann)
Date: Sun Mar 27 00:06:46 2005
Subject: Request for comments regarding draft-mccobb-xplusv-media-type-01
In-Reply-To: <[email-masked]>
References: <[email-masked]>
Message-ID: <[email-masked]>

* Gerald McCobb wrote:
>2.1 application/xhtml+voice+xml Usage
>
>   The application/xhtml+voice+xml media type is intended to be a media
>   descriptor for XHTML+Voice documents.
>
>   This media type registration is not intended for email usage.

The draft does not really explain why there is a need for such a
type in the first place; why don't you use application/xhtml+xml?

>      Optional parameters:
>   version:  refers to the XHTML+Voice language version in the document.
>             Acceptable values are 1.0, 1.1, and 1.2 (default).

What are the processing requirements for this parameter?

>   charset:  has the same meaning as the text/html media type.  See
>   section 2 of [RFC 2854].

A word seems to be missing here. It is not really clear why the draft
refers to RFC 2854 rather than RFC 3023 or RFC 3236. 

>      Security considerations:
> 
>   XHTML+Voice is an extension of XHTML and has the same security issues
>   as XHTML.  These include interpreting anchors and forms in HTML
>   documents, and scripting languages and other dynamic interactive
>   capabilities.  See section 7 of [RFC 2854].

So the extensions to XHTML have no security considerations?

>         File extension(s): html, htm, mxml, xvml

I don't think .html and .htm should be listed here, they are already in
common use for application/xhtml+xml and text/html.

>4. Fragment Identifiers
>
>   See section 3 of [RFC 2854].

It is unclear what this means, RFC 2854 and RFC 3236 have inconsistent
rules for fragment identifiers and XHTML+Voice adds new ID attributes
so that the fragment identifier namespaces are different.

>6. Security Considerations
>
>   Security considerations for this media type are discussed in the MIME
>   type registration that appears in section 4.

Section 4 has no security considerations...

>7. References

The draft should be clear about which of these references are
informative and which normative.
-- 
Bj?rn H?hrmann ? mailto:[email-masked] ? http://bjoern.hoehrmann.de
Weinh. Str. 22 ? Telefon: +49(0)621/4309674 ? http://www.bjoernsworld.de
68309 Mannheim ? PGP Pub. KeyID: 0xA4357E78 ? http://www.websitedev.de/ 
>From [email-masked]  Tue Mar 29 13:16:30 2005
From: GK-lists at ninebynine.org (Graham Klyne)
Date: Wed Mar 30 03:40:20 2005
Subject: Media Type "text/csv": new draft (-02) and Last Call
In-Reply-To: <[email-masked]>
References: <[email-masked]>
	<[email-masked]>
Message-ID: <[email-masked]>

At 09:35 23/03/05 -0500, Yakov Shafranovich wrote:
>Graham Klyne wrote:
>>I feel compelled to ask the question:  should this really be a text/csv 
>>type rather than application/csv?  Past comments by Ned Freed with 
>>respect to text/html suggest possibly not, but similarity with 
>>text/tab-separated-values suggests maybe yes.  I don't personally feel 
>>strongly about this.
>
>Can you point to Ned's message?

OK, it's here:
   http://www.imc.org/ietf-xml-mime/mail-archive/msg00622.html

#g

------------
Graham Klyne
For email:
http://www.ninebynine.org/#Contact

From: YakovS at solidmatrix.com (Yakov Shafranovich)
Date: Tue Jan  3 16:22:57 2006
Subject: Media Type "text/csv" - trailing nullcols
In-Reply-To: <[email-masked]>
References: <[email-masked]>
Message-ID: <[email-masked]>

Tak,

Thanks for your comments. My intent with this draft was to document the 
current convention and register the MIME type. I would rather stay away 
from requiring or allowing features explicetly that might not be 
followed by all implementations.

Yakov

Yakov Shafranovich wrote:
> From: "Tak Tang" <[email-masked]>
> To: <[email-masked]>
> Date: Thu, 24 Mar 2005 11:30:43 +0000
> Subj: Media Type "text/csv" - trailing nullcols
> 
> Hello,
> 
> This is my first post, and I've entered the discussion late, so please 
> excuse me if this has already been discussed . . .
> 
> The draft (03) does not require that each record/line have the same number 
> of values/commas.
> I think it may be benefitial to explicitly allow csv writers to omit 
> trailing commas if there are no further non-null values.
> 
> eg (all of these lines are valid)
> aaa,bbb,ccc,ddd<CR-LF>
> aaa,,,<CR-LF>
> aaa<CR-LF>
> ,bbb<CR-LF>
> ,,ccc<CR-LF>
> ,,,ddd<CR-LF>
> 
> The second and third line are the same - they just have a value in the 
> first field.
> The csv reader should assume that if there are missing columns (ie commas) 
> then the values are null.
> Leading null values still require the commas to be included (lines 4-6).
> 
> This is useful when
> a) the dataset is sparse.
> b) the dataset has new columns added - the reader can still accept old 
> datafiles without them being considered 'malformed'.
> 
> Microsoft Excel and Oracle SQL*Loader can already accept data with missing 
> (trailing) commas.
> 
> 
> Tak Tang
> 
> 
> 
> 

From: Tak.Tang at suht.swest.nhs.uk (Tak Tang)
Date: Thu Mar 24 20:49:42 2005
Subject: Media Type "text/csv" - trailing nullcols
Message-ID: <[email-masked]>

Hello,

This is my first post, and I've entered the discussion late, so please excuse me if this has already been discussed . . .

The draft (03) does not require that each record/line have the same number of values/commas.
I think it may be benefitial to explicitly allow csv writers to omit trailing commas if there are no further non-null values.

eg (all of these lines are valid)
aaa,bbb,ccc,ddd<CR-LF>
aaa,,,<CR-LF>
aaa<CR-LF>
,bbb<CR-LF>
,,ccc<CR-LF>
,,,ddd<CR-LF>

The second and third line are the same - they just have a value in the first field.
The csv reader should assume that if there are missing columns (ie commas) then the values are null.
Leading null values still require the commas to be included (lines 4-6).

This is useful when
a) the dataset is sparse.
b) the dataset has new columns added - the reader can still accept old datafiles without them being considered 'malformed'.

Microsoft Excel and Oracle SQL*Loader can already accept data with missing (trailing) commas.

Tak Tang

D I S C L A I M E R
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. Any views or opinions presented are solely those of
the author and do not necessarily represent those of the 
Trust unless explicitly stated otherwise.  If you have received this 
e-mail in error please delete the e-mail and contact the
Southampton University Hospitals NHS Trust Helpdesk on:-
023 80796000

The information contained in this e-mail may be subject to public 
disclosure under the Freedom of Information Act 2000. Unless the 
Information is legally exempt from disclosure, the confidentiality
of this e-mail and your reply cannot be guaranteed.

This footnote also confirms that this email message has been swept 
by MIMEsweeper for the presence of computer viruses.

From: Tak.Tang at suht.swest.nhs.uk (Tak Tang)
Date: Tue Jan  3 16:22:57 2006
Subject: Media Type "text/csv" - trailing nullcols
Message-ID: <[email-masked]>

Hello,

This is my first post, and I've entered the discussion late, so please excuse me if this has already been discussed . . .

The draft (03) does not require that each record/line have the same number of values/commas.
I think it may be benefitial to explicitly allow csv writers to omit trailing commas if there are no further non-null values.

eg (all of these lines are valid)
aaa,bbb,ccc,ddd<CR-LF>
aaa,,,<CR-LF>
aaa<CR-LF>
,bbb<CR-LF>
,,ccc<CR-LF>
,,,ddd<CR-LF>

The second and third line are the same - they just have a value in the first field.
The csv reader should assume that if there are missing columns (ie commas) then the values are null.
Leading null values still require the commas to be included (lines 4-6).

This is useful when
a) the dataset is sparse.
b) the dataset has new columns added - the reader can still accept old datafiles without them being considered 'malformed'.

Microsoft Excel and Oracle SQL*Loader can already accept data with missing (trailing) commas.

Tak Tang

D I S C L A I M E R
This email and any files transmitted with it are confidential and
intended solely for the use of the individual or entity to whom they
are addressed. Any views or opinions presented are solely those of
the author and do not necessarily represent those of the 
Trust unless explicitly stated otherwise.  If you have received this 
e-mail in error please delete the e-mail and contact the
Southampton University Hospitals NHS Trust Helpdesk on:-
023 80796000

The information contained in this e-mail may be subject to public 
disclosure under the Freedom of Information Act 2000. Unless the 
Information is legally exempt from disclosure, the confidentiality
of this e-mail and your reply cannot be guaranteed.

This footnote also confirms that this email message has been swept 
by MIMEsweeper for the presence of computer viruses.

From: mankin at psg.com (Allison Mankin)
Date: Tue Jan  3 16:22:57 2006
Subject: Transport Area requests new/additional Media Type review fortext/red 
In-Reply-To: Your message of Thu, 24 Mar 2005 07:35:26 +0900. <6.0.0.20.2.20050324073019.072e20a0@localhost> 
Message-ID: <[email-masked]>

Martin,

A good point - yes, "red" stands for redundant.

The full specification discusses the use of this media
type to provide redundancy, but some context provision in
the registration is a good idea.  Will add.

Allison

From: YakovS at solidmatrix.com (Yakov Shafranovich)
Date: Thu Mar 24 03:03:57 2005
Subject: Type 'text/csv', draft-03
Message-ID: <[email-masked]>

I posted a new version (-03) correcting the issue pointed out by Clyde 
Ingram. As a matter of fact while editing the draft, I noticed that the 
ABNF grammar did not allow commas as end of field markers for the last 
field and I changed the text to match. Copies of the draft can be found 
here (before it posts to the repository):

http://www.shaftek.org/publications/drafts/mime-csv/draft-shafranovich-mime-csv-03.html
http://www.shaftek.org/publications/drafts/mime-csv/draft-shafranovich-mime-csv-03.txt

HTML and text diff of changes:

http://www.shaftek.org/publications/drafts/mime-csv/rfcdiff-02-to-03.html
http://www.shaftek.org/publications/drafts/mime-csv/diff-02-to-03.txt

Yakov
>From [email-masked]  Thu Mar 24 03:16:03 2005
From: mankin at psg.com (Allison Mankin)
Date: Thu Mar 24 03:16:10 2005
Subject: Transport Area requests new/additional Media Type review
	fortext/red 
In-Reply-To: Your message of Thu, 24 Mar 2005 07:35:26 +0900.
	<6.0.0.20.2.20050324073019.072e20a0@localhost> 
Message-ID: <[email-masked]>

Martin,

A good point - yes, "red" stands for redundant.

The full specification discusses the use of this media
type to provide redundancy, but some context provision in
the registration is a good idea.  Will add.

Allison

From: YakovS at solidmatrix.com (Yakov Shafranovich)
Date: Tue Jan  3 16:22:57 2006
Subject: Type 'text/csv', draft-03
Message-ID: <[email-masked]>

I posted a new version (-03) correcting the issue pointed out by Clyde 
Ingram. As a matter of fact while editing the draft, I noticed that the 
ABNF grammar did not allow commas as end of field markers for the last 
field and I changed the text to match. Copies of the draft can be found 
here (before it posts to the repository):

http://www.shaftek.org/publications/drafts/mime-csv/draft-shafranovich-mime-csv-03.html
http://www.shaftek.org/publications/drafts/mime-csv/draft-shafranovich-mime-csv-03.txt

HTML and text diff of changes:

http://www.shaftek.org/publications/drafts/mime-csv/rfcdiff-02-to-03.html
http://www.shaftek.org/publications/drafts/mime-csv/diff-02-to-03.txt

Yakov

From: duerst at w3.org (Martin Duerst)
Date: Thu Mar 24 00:02:04 2005
Subject: Transport Area requests new/additional Media Type review fortext/red
In-Reply-To: <[email-masked]>
References: <[email-masked]>
Message-ID: <6.0.0.20.2.20050324073019.072e20a0@localhost>

This is a general comment, but applies here, too.

It would be extremely useful if the MIME type registration contained
some indication of what the subtype name (in this case "red") stands
for. Even in the full draft I didn't find this. In the specific case
at hand, it may be better to choose a more descriptive name
(my guess is that "red" is supposed to stand for "redundant").

Also, the draft at
http://www.ietf.org/internet-drafts/draft-ietf-avt-text-red-05.txt
has serious formatting problems (undecoded quoted printable garbage).

Regards,    Martin.

At 06:11 05/03/24, Allison Mankin wrote:
 >
 >The Transport Area requests a Media Type review for the proposed
 >type text/red, intended for the IETF tree, as a Proposed Standard.
 >The specification is
 >http://www.ietf.org/internet-drafts/draft-ietf-avt-text-red-05.txt.
 >
 >This document had both a formal Applications AD Review and an active
 >IETF Last Call discussion to date.  During the latter, an approach
 >of defining a "Restrictions on Usage" field for a sub-type in
 >major type such as text that will be transported only by RTP
 >was devised.   The description of this field is found in
 >draft-freed-media-reg-02.txt, currently in IETF Last Call itself.
 >
 >As the result of this, please review the following media
 >registration for text/red instead of the one in the i-d
 >(but read other sections in the i-d as needed):
 >
 >       MIME media type name: text
 >
 >       MIME subtype name: red
 >
 >       Required parameters:
 >          rate: the RTP clock rate of the payload carried within the RTP
 >          packet.  Typically, this rate is 1000, but other rates MAY be
 >          specified.  This parameter MUST be set equal to the clock rate
 >          of the text payload format carried as the primary encoding.
 >
 >          pt: a comma-separated ordered list of RTP payload types
 >          enumerating the primary, secondary, etc., in accordance with
 >          RFC 2198.  Because comma is a special character, the list MUST
 >          be a quoted-string (enclosed in double quotes).  For static
 >          payload types, each list element is simply the type number.
 >          For dynamic payload types, each list element is a mapping of
 >          the dynamic payload type number to an embedded MIME content-
 >          type specification for the payload format corresponding to the
 >          dynamic payload type. The format of the mapping is:
 >
 >               dynamic-payload-type = content-type
 >
 >
 >          If the content-type string includes a comma, then the content-
 >          type string MUST be a quoted-string.  If the content-type
 >          string does not include a comma, it MAY still be quoted. Since
 >          it is part of the list which must itself be a quoted-string,
 >          that means the quotation marks MUST be quoted with backslash
 >          quoting as specified in RFC 2045 [4].  If the content-type
 >          string itself contains a quoted-string, then the requirement
 >          for backslash quoting is recursively applied.
 >
 >
 >         Optional parameters:
 >           ptime, maxptime (these attributes are originally
 >          defined in RFC 2327 and RFC 3267 respectively)
 >
 >
 >         Restrictions on Usage:
 >          This type is defined only for transfer for RTP.
 >          It shall not be defined for a storage format.
 >
 >         Encoding considerations:
 >          See Restrictions on Usage above; this section is
 >          included per the requirements in RFC 3555.
 >
 >       Security considerations:
 >         Refer to section 5 of RFC XXXX.
 >         (XXXX is a reference to the RFC number to be assigned to
 >         draft-ietf-avt-text-red-0t.txt)
 >
 >       Interoperability considerations: none
 >
 >       Published specification: RFC XXXX.
 >
 >       Applications which use this media type:
 >          Text streaming and conferencing tools.
 >
 >       Additional information: none
 >
 >       Person & email address to contact for further information:
 >          Paul E. Jones
 >          E-mail: [email-masked]
 >
 >       Intended usage: COMMON
 >
 >       Author:
 >         Paul E. Jones
 >         [email-masked]
 >
 >       Change Controller:
 >         AVT Working Group delegated from the IESG
 >
 >
 >
 >Please provide any comments before 6 April 2005.
 >
 >Thanks,
 >
 >Allison 

From: duerst at w3.org (Martin Duerst)
Date: Tue Jan  3 16:22:56 2006
Subject: Transport Area requests new/additional Media Type review fortext/red
In-Reply-To: <[email-masked]>
References: <[email-masked]>
Message-ID: <6.0.0.20.2.20050324073019.072e20a0@localhost>

This is a general comment, but applies here, too.

It would be extremely useful if the MIME type registration contained
some indication of what the subtype name (in this case "red") stands
for. Even in the full draft I didn't find this. In the specific case
at hand, it may be better to choose a more descriptive name
(my guess is that "red" is supposed to stand for "redundant").

Also, the draft at
http://www.ietf.org/internet-drafts/draft-ietf-avt-text-red-05.txt
has serious formatting problems (undecoded quoted printable garbage).

Regards,    Martin.

At 06:11 05/03/24, Allison Mankin wrote:
 >
 >The Transport Area requests a Media Type review for the proposed
 >type text/red, intended for the IETF tree, as a Proposed Standard.
 >The specification is
 >http://www.ietf.org/internet-drafts/draft-ietf-avt-text-red-05.txt.
 >
 >This document had both a formal Applications AD Review and an active
 >IETF Last Call discussion to date.  During the latter, an approach
 >of defining a "Restrictions on Usage" field for a sub-type in
 >major type such as text that will be transported only by RTP
 >was devised.   The description of this field is found in
 >draft-freed-media-reg-02.txt, currently in IETF Last Call itself.
 >
 >As the result of this, please review the following media
 >registration for text/red instead of the one in the i-d
 >(but read other sections in the i-d as needed):
 >
 >       MIME media type name: text
 >
 >       MIME subtype name: red
 >
 >       Required parameters:
 >          rate: the RTP clock rate of the payload carried within the RTP
 >          packet.  Typically, this rate is 1000, but other rates MAY be
 >          specified.  This parameter MUST be set equal to the clock rate
 >          of the text payload format carried as the primary encoding.
 >
 >          pt: a comma-separated ordered list of RTP payload types
 >          enumerating the primary, secondary, etc., in accordance with
 >          RFC 2198.  Because comma is a special character, the list MUST
 >          be a quoted-string (enclosed in double quotes).  For static
 >          payload types, each list element is simply the type number.
 >          For dynamic payload types, each list element is a mapping of
 >          the dynamic payload type number to an embedded MIME content-
 >          type specification for the payload format corresponding to the
 >          dynamic payload type. The format of the mapping is:
 >
 >               dynamic-payload-type = content-type
 >
 >
 >          If the content-type string includes a comma, then the content-
 >          type string MUST be a quoted-string.  If the content-type
 >          string does not include a comma, it MAY still be quoted. Since
 >          it is part of the list which must itself be a quoted-string,
 >          that means the quotation marks MUST be quoted with backslash
 >          quoting as specified in RFC 2045 [4].  If the content-type
 >          string itself contains a quoted-string, then the requirement
 >          for backslash quoting is recursively applied.
 >
 >
 >         Optional parameters:
 >           ptime, maxptime (these attributes are originally
 >          defined in RFC 2327 and RFC 3267 respectively)
 >
 >
 >         Restrictions on Usage:
 >          This type is defined only for transfer for RTP.
 >          It shall not be defined for a storage format.
 >
 >         Encoding considerations:
 >          See Restrictions on Usage above; this section is
 >          included per the requirements in RFC 3555.
 >
 >       Security considerations:
 >         Refer to section 5 of RFC XXXX.
 >         (XXXX is a reference to the RFC number to be assigned to
 >         draft-ietf-avt-text-red-0t.txt)
 >
 >       Interoperability considerations: none
 >
 >       Published specification: RFC XXXX.
 >
 >       Applications which use this media type:
 >          Text streaming and conferencing tools.
 >
 >       Additional information: none
 >
 >       Person & email address to contact for further information:
 >          Paul E. Jones
 >          E-mail: [email-masked]
 >
 >       Intended usage: COMMON
 >
 >       Author:
 >         Paul E. Jones
 >         [email-masked]
 >
 >       Change Controller:
 >         AVT Working Group delegated from the IESG
 >
 >
 >
 >Please provide any comments before 6 April 2005.
 >
 >Thanks,
 >
 >Allison 

From: mankin at psg.com (Allison Mankin)
Date: Tue Jan  3 16:22:56 2006
Subject: Transport Area requests new/additional Media Type review for text/red
Message-ID: <[email-masked]>

The Transport Area requests a Media Type review for the proposed
type text/red, intended for the IETF tree, as a Proposed Standard.
The specification is
http://www.ietf.org/internet-drafts/draft-ietf-avt-text-red-05.txt.

This document had both a formal Applications AD Review and an active
IETF Last Call discussion to date.  During the latter, an approach
of defining a "Restrictions on Usage" field for a sub-type in
major type such as text that will be transported only by RTP
was devised.   The description of this field is found in
draft-freed-media-reg-02.txt, currently in IETF Last Call itself.

As the result of this, please review the following media
registration for text/red instead of the one in the i-d
(but read other sections in the i-d as needed):

       MIME media type name: text

       MIME subtype name: red

       Required parameters:
          rate: the RTP clock rate of the payload carried within the RTP
          packet.  Typically, this rate is 1000, but other rates MAY be
          specified.  This parameter MUST be set equal to the clock rate
          of the text payload format carried as the primary encoding.

          pt: a comma-separated ordered list of RTP payload types
          enumerating the primary, secondary, etc., in accordance with
          RFC 2198.  Because comma is a special character, the list MUST
          be a quoted-string (enclosed in double quotes).  For static
          payload types, each list element is simply the type number.
          For dynamic payload types, each list element is a mapping of
          the dynamic payload type number to an embedded MIME content-
          type specification for the payload format corresponding to the
          dynamic payload type. The format of the mapping is:

               dynamic-payload-type = content-type

          If the content-type string includes a comma, then the content-
          type string MUST be a quoted-string.  If the content-type
          string does not include a comma, it MAY still be quoted. Since
          it is part of the list which must itself be a quoted-string,
          that means the quotation marks MUST be quoted with backslash
          quoting as specified in RFC 2045 [4].  If the content-type
          string itself contains a quoted-string, then the requirement
          for backslash quoting is recursively applied.

         Optional parameters: 
           ptime, maxptime (these attributes are originally
          defined in RFC 2327 and RFC 3267 respectively)

         Restrictions on Usage:
          This type is defined only for transfer for RTP.  
          It shall not be defined for a storage format.

         Encoding considerations:
          See Restrictions on Usage above; this section is
          included per the requirements in RFC 3555.
  
       Security considerations: 
         Refer to section 5 of RFC XXXX.
         (XXXX is a reference to the RFC number to be assigned to
         draft-ietf-avt-text-red-0t.txt)

       Interoperability considerations: none

       Published specification: RFC XXXX.

       Applications which use this media type:
          Text streaming and conferencing tools.

       Additional information: none

       Person & email address to contact for further information:
          Paul E. Jones
          E-mail: [email-masked]

       Intended usage: COMMON

       Author:                        
         Paul E. Jones               
         [email-masked]       

       Change Controller:
         AVT Working Group delegated from the IESG

Please provide any comments before 6 April 2005.

Thanks,

Allison

From: YakovS at solidmatrix.com (Yakov Shafranovich)
Date: Tue Jan  3 16:22:56 2006
Subject: Media Type "text/csv": new draft (-02) and Last Call
In-Reply-To: <[email-masked]>
References: <[email-masked]> <[email-masked]> <[email-masked]> <[email-masked]>
Message-ID: <[email-masked]>

Bruce Lilly wrote:
> On Wed March 23 2005 09:18, Yakov Shafranovich wrote:
> 
>>Hmm... I do think that probably every single spreadsheet program I have 
>>seen uses the optional header format so it is pretty common. Since I am 
>>trying to document a common format to go along with the MIME type, 
>>something that common deserves to be documented. As for more complex 
>>cases of headers, as you have said those can be easily accomodated 
>>within the basic format while omitting the initial header line used by 
>>spreadsheets.
> 
> How does a processor of the MIME type determine whether or not
> an optional header is in the media?
> 
> Making the header mandatory would obviate the question (generators
> can supply a line with all null fields).
> 
> A parameter indicating presence or absence of a header line (and
> possibly other format variations) could be defined. If optional,
> defaults would have to be specified in case the parameter is not
> specified.
> 

The processor doesn't know that. But then again how does the processor 
know what each field means anyway? IMHO, it seems that any processor 
that works on CSV files is already somewhat aware of the CSV format that 
they are using, i.e. what each field is, header and no header. I don't 
know whether making the header mandotary or adding an optional parameter 
would help things, it just make things more complicated than they are now.

My original intent was simply to register one single MIME type for CSV 
since multiples are used now and include a common CSV format which would 
be reasonably simple. Another alternative to make it even more simpler, 
would be to get rid of the header all together and leave it up to the 
application to figure it out as suggested by someone earlier.

Yakov

From: clyde.ingram at edl.uk.eds.com ([email-masked])
Date: Tue Jan  3 16:22:56 2006
Subject: Media Type "text/csv": new draft (-02) and Last Call
Message-ID: <[email-masked]>

Graham,

-----Original Message-----
From: Graham Klyne [mailto:[email-masked]]
Sent: Wednesday, March 23, 2005 9:55 AM
To: Yakov Shafranovich; [email-masked]
Cc: [email-masked]
Subject: Re: Media Type "text/csv": new draft (-02) and Last Call

At 01:14 23/03/05 -0500, Yakov Shafranovich wrote:

>Clyde,
>
>Thanks for pointing this out. I personally think that instead of making 
>the header record mandatory which is something that most CSV applications 
>do not have, I would rather take the comma out of the end of the record 
>and have the last field end with a CRLF instead of an optional COMMA. Do 
>you think that is a plausible solution?

No.  Some of the Excel data I process has trailing commas.  This must be 
allowed.

I also don't think it's necessary to say anything (other than maybe as a 
comment) about any special status for the first line:  such use is 
accommodated quite reasonably within the basic CSV format.

For example, having such a line when exporting Excel as CSV depends 
entirely upon how the user constructs the original spreadsheet.  Column 
headings are common, but not mandatory.  In some cases, there may be a more 
complex heading structure -- this is an application issue, not a dataset 
format issue, and as such does not belong in the dataset format
specification.

#g
--
------------

Please clarify whether the trailing commas that your Excel export generates
are there to mark the end of the last field, or to mark the start of a last
field which currently has no value.

To take a concrete example, I would expect a CSV of sibling relationships in
a mythical family to look like this, assuming the siblings are one brother
(Bart) and 2 sisters (Lisa & Maggie):

    child,sisters,brothers<CR-LF>
    Bart,Lisa & Maggie,<CR-LF>
    Lisa,Maggie,Bart<CR-LF>
    Maggie,Lisa,Bart<CR-LF>

where the trailing comma for the record of child=Bart signifies that the
"brothers" field is null, so that Bart has no brothers.  In my view this is
a logical conclusion, and in fact stripping that one trailing comma would be
an error, as that record would only have 2 fields, not 3.

Would you, however, expect the CSV file to use comma as a field-terminator,
rather than a field-separator, as follows?:

    child,sisters,brothers,<CR-LF>
    Bart,Lisa & Maggie,,<CR-LF>
    Lisa,Maggie,Bart,<CR-LF>
    Maggie,Lisa,Bart,<CR-LF>

Note that parsers that split data records on unprotected comma would detect
one field too many in this latter case.  

In a Comma SEPARATED Value file format, can you configure Excel to use comma
as a SEPARATOR between values, rather than a TERMINATOR (at the end of
values)?
 
Regarding your remarks on the header record being "an application issue, not
a dataset format issue, and as such does not belong in the dataset format
specification": XML, ASN.1, and other (application-independent) data
interchange formats,  explicitly tag individual fields so that their type is
unambiguously defined within a context.  In contrast, CSV conveys no tags
per field in a data record.  Hence, to help with application-independent
data interchange, the CSV format should convey field titles in a header
record.

Here is an example of lack of application-independence: if my application
sends yours this CSV file:

    ,Bart,Lisa & Maggie<CR-LF>
    Bart,Lisa,Maggie<CR-LF>
    Bart,Maggie,Lisa<CR-LF>
  
and your application depends on the assumption that the fields are the
sequence:

    child
    sisters
    brothers

then your application will mis-interpret the data.
But if my application precedes this with a header record, like so:

    brothers,child,sisters<CR-LF>
    ,Bart,Lisa & Maggie<CR-LF>
    Bart,Lisa,Maggie<CR-LF>
    Bart,Maggie,Lisa<CR-LF>

then your application can maintain independence from the change by my
application, because the CSV file conveys the corresponding new field
sequence (the columns "brother" and "child" have swapped).

Regards,
Clyde
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.alvestrand.no/pipermail/ietf-types/attachments/20050323/b1731075/attachment.html
>From [email-masked]  Wed Mar 23 16:37:05 2005
From: blilly at erols.com (Bruce Lilly)
Date: Tue Jan  3 16:22:56 2006
Subject: Media Type "text/csv": new draft (-02) and Last Call
In-Reply-To: <[email-masked]>
References: <[email-masked]>
	<[email-masked]>
	<[email-masked]>
Message-ID: <[email-masked]>

On Wed March 23 2005 09:18, Yakov Shafranovich wrote:

> Hmm... I do think that probably every single spreadsheet program I have 
> seen uses the optional header format so it is pretty common. Since I am 
> trying to document a common format to go along with the MIME type, 
> something that common deserves to be documented. As for more complex 
> cases of headers, as you have said those can be easily accomodated 
> within the basic format while omitting the initial header line used by 
> spreadsheets.

How does a processor of the MIME type determine whether or not
an optional header is in the media?

Making the header mandatory would obviate the question (generators
can supply a line with all null fields).

A parameter indicating presence or absence of a header line (and
possibly other format variations) could be defined. If optional,
defaults would have to be specified in case the parameter is not
specified.

From: YakovS at solidmatrix.com (Yakov Shafranovich)
Date: Tue Jan  3 16:22:56 2006
Subject: Media Type "text/csv": new draft (-02) and Last Call
In-Reply-To: <[email-masked]>
References: <[email-masked]>
Message-ID: <[email-masked]>

Graham Klyne wrote:
> 
> I feel compelled to ask the question:  should this really be a text/csv 
> type rather than application/csv?  Past comments by Ned Freed with 
> respect to text/html suggest possibly not, but similarity with 
> text/tab-separated-values suggests maybe yes.  I don't personally feel 
> strongly about this.
> 

Can you point to Ned's message?

> Section 2:
> 
> [[
>    6.  Field containing line breaks (CRLF) and commas should be enclosed
>        in double-quotes.  For example:
> 
>        "aaa","b CRLF
>        bb","ccc" CRLF
>        zzz,yyy,xxx
> ]]
> 
> Are there any CSV applications that actually quote newlines in values?  
> (I'm not sure that my own CSV handling code would accept a newline 
> within a quoted string)
> 

Some of the code I have been working with is happens to do so. I think 
it depends on which application you are using. For example I just tried 
it with OpenOffice Calc v1.1.3 and it accepts newlines within fields.

> [[
>    7.  If double-quotes are used to enclosed fields, then double-quotes
>        inside fields must be surrounded by double quotes.  For example:
> 
>        "aaa","b"""bb","ccc"
> ]]
> 
> This is NOT how Excel handles quotes, nor my own code.  Rather, when a 
> quoted string contains a double-quote character, the double quote is 
> doubled up.  Thus the example would be:
>        "aaa","b""bb","ccc"
> 

I just double checked with the applications that I am using and they 
tend to work the way you describe. I will adjust the draft.

> Section 7.1:
> [[
>    [3]  Postel, J., "Transmission Control Protocol", STD 7, RFC 793,
>         September 1981.
> ]]
> 
> I think this should be an informative reference, not normative.
> 

Thanks for catching that.

Yakov

From: YakovS at solidmatrix.com (Yakov Shafranovich)
Date: Tue Jan  3 16:22:56 2006
Subject: Media Type "text/csv": new draft (-02) and Last Call
In-Reply-To: <[email-masked]>
References: <[email-masked]> <[email-masked]> <[email-masked]>
Message-ID: <[email-masked]>

Graham Klyne wrote:
> At 01:14 23/03/05 -0500, Yakov Shafranovich wrote:
> 
>> Clyde,
>>
>> Thanks for pointing this out. I personally think that instead of 
>> making the header record mandatory which is something that most CSV 
>> applications do not have, I would rather take the comma out of the end 
>> of the record and have the last field end with a CRLF instead of an 
>> optional COMMA. Do you think that is a plausible solution?
> 
> 
> No.  Some of the Excel data I process has trailing commas.  This must be 
> allowed.
> 

The main intent of the document is focusing on the MIME type. The CSV 
common definition is an attempt at a single tight definition of the CSV 
format to go along with the MIME type. I don't think that every 
application is compliant with it but the interoperability considerations 
section does mention that implementors should be aware of divergent 
implementations.

> I also don't think it's necessary to say anything (other than maybe as a 
> comment) about any special status for the first line:  such use is 
> accommodated quite reasonably within the basic CSV format.
> 
> For example, having such a line when exporting Excel as CSV depends 
> entirely upon how the user constructs the original spreadsheet.  Column 
> headings are common, but not mandatory.  In some cases, there may be a 
> more complex heading structure -- this is an application issue, not a 
> dataset format issue, and as such does not belong in the dataset format 
> specification.
> 

Hmm... I do think that probably every single spreadsheet program I have 
seen uses the optional header format so it is pretty common. Since I am 
trying to document a common format to go along with the MIME type, 
something that common deserves to be documented. As for more complex 
cases of headers, as you have said those can be easily accomodated 
within the basic format while omitting the initial header line used by 
spreadsheets.

Yakov

From: YakovS at solidmatrix.com (Yakov Shafranovich)
Date: Wed Mar 23 15:14:01 2005
Subject: Media Type "text/csv": new draft (-02) and Last Call
In-Reply-To: <[email-masked]>
References: <[email-masked]> <[email-masked]> <[email-masked]>
Message-ID: <[email-masked]>

Chris Lilley wrote:
> 
> Is there a survey of common use of csv files? How many of them would be
> conformant?

Unfortunatly there is no single definition or survey of CSV files. My 
document is really concentrated on the MIME type with the definition 
included as an attempt of arriving at a tight common definition.

Yakov
>From [email-masked]  Wed Mar 23 15:18:15 2005
From: YakovS at solidmatrix.com (Yakov Shafranovich)
Date: Wed Mar 23 15:19:34 2005
Subject: Media Type "text/csv": new draft (-02) and Last Call
In-Reply-To: <[email-masked]>
References: <[email-masked]>
	<[email-masked]>
	<[email-masked]>
Message-ID: <[email-masked]>

Graham Klyne wrote:
> At 01:14 23/03/05 -0500, Yakov Shafranovich wrote:
> 
>> Clyde,
>>
>> Thanks for pointing this out. I personally think that instead of 
>> making the header record mandatory which is something that most CSV 
>> applications do not have, I would rather take the comma out of the end 
>> of the record and have the last field end with a CRLF instead of an 
>> optional COMMA. Do you think that is a plausible solution?
> 
> 
> No.  Some of the Excel data I process has trailing commas.  This must be 
> allowed.
> 

The main intent of the document is focusing on the MIME type. The CSV 
common definition is an attempt at a single tight definition of the CSV 
format to go along with the MIME type. I don't think that every 
application is compliant with it but the interoperability considerations 
section does mention that implementors should be aware of divergent 
implementations.

> I also don't think it's necessary to say anything (other than maybe as a 
> comment) about any special status for the first line:  such use is 
> accommodated quite reasonably within the basic CSV format.
> 
> For example, having such a line when exporting Excel as CSV depends 
> entirely upon how the user constructs the original spreadsheet.  Column 
> headings are common, but not mandatory.  In some cases, there may be a 
> more complex heading structure -- this is an application issue, not a 
> dataset format issue, and as such does not belong in the dataset format 
> specification.
> 

Hmm... I do think that probably every single spreadsheet program I have 
seen uses the optional header format so it is pretty common. Since I am 
trying to document a common format to go along with the MIME type, 
something that common deserves to be documented. As for more complex 
cases of headers, as you have said those can be easily accomodated 
within the basic format while omitting the initial header line used by 
spreadsheets.

Yakov
>From [email-masked]  Wed Mar 23 15:35:54 2005
From: YakovS at solidmatrix.com (Yakov Shafranovich)
Date: Wed Mar 23 15:37:10 2005
Subject: Media Type "text/csv": new draft (-02) and Last Call
In-Reply-To: <[email-masked]>
References: <[email-masked]>
Message-ID: <[email-masked]>

Graham Klyne wrote:
> 
> I feel compelled to ask the question:  should this really be a text/csv 
> type rather than application/csv?  Past comments by Ned Freed with 
> respect to text/html suggest possibly not, but similarity with 
> text/tab-separated-values suggests maybe yes.  I don't personally feel 
> strongly about this.
> 

Can you point to Ned's message?

> Section 2:
> 
> [[
>    6.  Field containing line breaks (CRLF) and commas should be enclosed
>        in double-quotes.  For example:
> 
>        "aaa","b CRLF
>        bb","ccc" CRLF
>        zzz,yyy,xxx
> ]]
> 
> Are there any CSV applications that actually quote newlines in values?  
> (I'm not sure that my own CSV handling code would accept a newline 
> within a quoted string)
> 

Some of the code I have been working with is happens to do so. I think 
it depends on which application you are using. For example I just tried 
it with OpenOffice Calc v1.1.3 and it accepts newlines within fields.

> [[
>    7.  If double-quotes are used to enclosed fields, then double-quotes
>        inside fields must be surrounded by double quotes.  For example:
> 
>        "aaa","b"""bb","ccc"
> ]]
> 
> This is NOT how Excel handles quotes, nor my own code.  Rather, when a 
> quoted string contains a double-quote character, the double quote is 
> doubled up.  Thus the example would be:
>        "aaa","b""bb","ccc"
> 

I just double checked with the applications that I am using and they 
tend to work the way you describe. I will adjust the draft.

> Section 7.1:
> [[
>    [3]  Postel, J., "Transmission Control Protocol", STD 7, RFC 793,
>         September 1981.
> ]]
> 
> I think this should be an informative reference, not normative.
> 

Thanks for catching that.

Yakov
>From [email-masked]  Wed Mar 23 16:19:52 2005
From: clyde.ingram at edl.uk.eds.com ([email-masked])
Date: Wed Mar 23 16:22:13 2005
Subject: Media Type "text/csv": new draft (-02) and Last Call
Message-ID: <[email-masked]>

Graham,

-----Original Message-----
From: Graham Klyne [mailto:[email-masked]]
Sent: Wednesday, March 23, 2005 9:55 AM
To: Yakov Shafranovich; [email-masked]
Cc: [email-masked]
Subject: Re: Media Type "text/csv": new draft (-02) and Last Call

At 01:14 23/03/05 -0500, Yakov Shafranovich wrote:

>Clyde,
>
>Thanks for pointing this out. I personally think that instead of making 
>the header record mandatory which is something that most CSV applications 
>do not have, I would rather take the comma out of the end of the record 
>and have the last field end with a CRLF instead of an optional COMMA. Do 
>you think that is a plausible solution?

No.  Some of the Excel data I process has trailing commas.  This must be 
allowed.

I also don't think it's necessary to say anything (other than maybe as a 
comment) about any special status for the first line:  such use is 
accommodated quite reasonably within the basic CSV format.

For example, having such a line when exporting Excel as CSV depends 
entirely upon how the user constructs the original spreadsheet.  Column 
headings are common, but not mandatory.  In some cases, there may be a more 
complex heading structure -- this is an application issue, not a dataset 
format issue, and as such does not belong in the dataset format
specification.

#g
--
------------

Please clarify whether the trailing commas that your Excel export generates
are there to mark the end of the last field, or to mark the start of a last
field which currently has no value.

To take a concrete example, I would expect a CSV of sibling relationships in
a mythical family to look like this, assuming the siblings are one brother
(Bart) and 2 sisters (Lisa & Maggie):

    child,sisters,brothers<CR-LF>
    Bart,Lisa & Maggie,<CR-LF>
    Lisa,Maggie,Bart<CR-LF>
    Maggie,Lisa,Bart<CR-LF>

where the trailing comma for the record of child=Bart signifies that the
"brothers" field is null, so that Bart has no brothers.  In my view this is
a logical conclusion, and in fact stripping that one trailing comma would be
an error, as that record would only have 2 fields, not 3.

Would you, however, expect the CSV file to use comma as a field-terminator,
rather than a field-separator, as follows?:

    child,sisters,brothers,<CR-LF>
    Bart,Lisa & Maggie,,<CR-LF>
    Lisa,Maggie,Bart,<CR-LF>
    Maggie,Lisa,Bart,<CR-LF>

Note that parsers that split data records on unprotected comma would detect
one field too many in this latter case.  

In a Comma SEPARATED Value file format, can you configure Excel to use comma
as a SEPARATOR between values, rather than a TERMINATOR (at the end of
values)?
 
Regarding your remarks on the header record being "an application issue, not
a dataset format issue, and as such does not belong in the dataset format
specification": XML, ASN.1, and other (application-independent) data
interchange formats,  explicitly tag individual fields so that their type is
unambiguously defined within a context.  In contrast, CSV conveys no tags
per field in a data record.  Hence, to help with application-independent
data interchange, the CSV format should convey field titles in a header
record.

Here is an example of lack of application-independence: if my application
sends yours this CSV file:

    ,Bart,Lisa & Maggie<CR-LF>
    Bart,Lisa,Maggie<CR-LF>
    Bart,Maggie,Lisa<CR-LF>
  
and your application depends on the assumption that the fields are the
sequence:

    child
    sisters
    brothers

then your application will mis-interpret the data.
But if my application precedes this with a header record, like so:

    brothers,child,sisters<CR-LF>
    ,Bart,Lisa & Maggie<CR-LF>
    Bart,Lisa,Maggie<CR-LF>
    Bart,Maggie,Lisa<CR-LF>

then your application can maintain independence from the change by my
application, because the CSV file conveys the corresponding new field
sequence (the columns "brother" and "child" have swapped).

Regards,
Clyde
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://eikenes.alvestrand.no/pipermail/ietf-types/attachments/20050323/b1731075/attachment.html
>From [email-masked]  Wed Mar 23 16:37:05 2005
From: blilly at erols.com (Bruce Lilly)
Date: Wed Mar 23 16:37:31 2005
Subject: Media Type "text/csv": new draft (-02) and Last Call
In-Reply-To: <[email-masked]>
References: <[email-masked]>
	<[email-masked]>
	<[email-masked]>
Message-ID: <[email-masked]>

On Wed March 23 2005 09:18, Yakov Shafranovich wrote:

> Hmm... I do think that probably every single spreadsheet program I have 
> seen uses the optional header format so it is pretty common. Since I am 
> trying to document a common format to go along with the MIME type, 
> something that common deserves to be documented. As for more complex 
> cases of headers, as you have said those can be easily accomodated 
> within the basic format while omitting the initial header line used by 
> spreadsheets.

How does a processor of the MIME type determine whether or not
an optional header is in the media?

Making the header mandatory would obviate the question (generators
can supply a line with all null fields).

A parameter indicating presence or absence of a header line (and
possibly other format variations) could be defined. If optional,
defaults would have to be specified in case the parameter is not
specified.
>From [email-masked]  Wed Mar 23 19:04:59 2005
From: YakovS at solidmatrix.com (Yakov Shafranovich)
Date: Wed Mar 23 19:05:35 2005
Subject: Media Type "text/csv": new draft (-02) and Last Call
In-Reply-To: <[email-masked]>
References: <[email-masked]>
	<[email-masked]>
	<[email-masked]>
	<[email-masked]>
Message-ID: <[email-masked]>

Bruce Lilly wrote:
> On Wed March 23 2005 09:18, Yakov Shafranovich wrote:
> 
>>Hmm... I do think that probably every single spreadsheet program I have 
>>seen uses the optional header format so it is pretty common. Since I am 
>>trying to document a common format to go along with the MIME type, 
>>something that common deserves to be documented. As for more complex 
>>cases of headers, as you have said those can be easily accomodated 
>>within the basic format while omitting the initial header line used by 
>>spreadsheets.
> 
> How does a processor of the MIME type determine whether or not
> an optional header is in the media?
> 
> Making the header mandatory would obviate the question (generators
> can supply a line with all null fields).
> 
> A parameter indicating presence or absence of a header line (and
> possibly other format variations) could be defined. If optional,
> defaults would have to be specified in case the parameter is not
> specified.
> 

The processor doesn't know that. But then again how does the processor 
know what each field means anyway? IMHO, it seems that any processor 
that works on CSV files is already somewhat aware of the CSV format that 
they are using, i.e. what each field is, header and no header. I don't 
know whether making the header mandotary or adding an optional parameter 
would help things, it just make things more complicated than they are now.

My original intent was simply to register one single MIME type for CSV 
since multiples are used now and include a common CSV format which would 
be reasonably simple. Another alternative to make it even more simpler, 
would be to get rid of the header all together and leave it up to the 
application to figure it out as suggested by someone earlier.

Yakov
>From [email-masked]  Wed Mar 23 22:11:51 2005
From: mankin at psg.com (Allison Mankin)
Date: Wed Mar 23 22:12:01 2005
Subject: Transport Area requests new/additional Media Type review for
	text/red
Message-ID: <[email-masked]>

The Transport Area requests a Media Type review for the proposed
type text/red, intended for the IETF tree, as a Proposed Standard.
The specification is
http://www.ietf.org/internet-drafts/draft-ietf-avt-text-red-05.txt.

This document had both a formal Applications AD Review and an active
IETF Last Call discussion to date.  During the latter, an approach
of defining a "Restrictions on Usage" field for a sub-type in
major type such as text that will be transported only by RTP
was devised.   The description of this field is found in
draft-freed-media-reg-02.txt, currently in IETF Last Call itself.

As the result of this, please review the following media
registration for text/red instead of the one in the i-d
(but read other sections in the i-d as needed):

       MIME media type name: text

       MIME subtype name: red

       Required parameters:
          rate: the RTP clock rate of the payload carried within the RTP
          packet.  Typically, this rate is 1000, but other rates MAY be
          specified.  This parameter MUST be set equal to the clock rate
          of the text payload format carried as the primary encoding.

          pt: a comma-separated ordered list of RTP payload types
          enumerating the primary, secondary, etc., in accordance with
          RFC 2198.  Because comma is a special character, the list MUST
          be a quoted-string (enclosed in double quotes).  For static
          payload types, each list element is simply the type number.
          For dynamic payload types, each list element is a mapping of
          the dynamic payload type number to an embedded MIME content-
          type specification for the payload format corresponding to the
          dynamic payload type. The format of the mapping is:

               dynamic-payload-type = content-type

          If the content-type string includes a comma, then the content-
          type string MUST be a quoted-string.  If the content-type
          string does not include a comma, it MAY still be quoted. Since
          it is part of the list which must itself be a quoted-string,
          that means the quotation marks MUST be quoted with backslash
          quoting as specified in RFC 2045 [4].  If the content-type
          string itself contains a quoted-string, then the requirement
          for backslash quoting is recursively applied.

         Optional parameters: 
           ptime, maxptime (these attributes are originally
          defined in RFC 2327 and RFC 3267 respectively)

         Restrictions on Usage:
          This type is defined only for transfer for RTP.  
          It shall not be defined for a storage format.

         Encoding considerations:
          See Restrictions on Usage above; this section is
          included per the requirements in RFC 3555.
  
       Security considerations: 
         Refer to section 5 of RFC XXXX.
         (XXXX is a reference to the RFC number to be assigned to
         draft-ietf-avt-text-red-0t.txt)

       Interoperability considerations: none

       Published specification: RFC XXXX.

       Applications which use this media type:
          Text streaming and conferencing tools.

       Additional information: none

       Person & email address to contact for further information:
          Paul E. Jones
          E-mail: [email-masked]

       Intended usage: COMMON

       Author:                        
         Paul E. Jones               
         [email-masked]       

       Change Controller:
         AVT Working Group delegated from the IESG

Please provide any comments before 6 April 2005.

Thanks,

Allison

From: YakovS at solidmatrix.com (Yakov Shafranovich)
Date: Tue Jan  3 16:22:56 2006
Subject: Media Type "text/csv": new draft (-02) and Last Call
In-Reply-To: <[email-masked]>
References: <[email-masked]> <[email-masked]> <[email-masked]>
Message-ID: <[email-masked]>

Chris Lilley wrote:
> 
> Is there a survey of common use of csv files? How many of them would be
> conformant?

Unfortunatly there is no single definition or survey of CSV files. My 
document is really concentrated on the MIME type with the definition 
included as an attempt of arriving at a tight common definition.

Yakov

From: chris at w3.org (Chris Lilley)
Date: Wed Mar 23 12:08:14 2005
Subject: Media Type "text/csv": new draft (-02) and Last Call
In-Reply-To: <[email-masked]>
References: <[email-masked]> <[email-masked]>
Message-ID: <[email-masked]>

On Wednesday, March 23, 2005, 7:14:24 AM, Yakov wrote:

YS> Clyde,

YS> Thanks for pointing this out. I personally think that instead of making
YS> the header record mandatory which is something that most CSV 
YS> applications do not have, I would rather take the comma out of the end
YS> of the record and have the last field end with a CRLF instead of an 
YS> optional COMMA. Do you think that is a plausible solution?

Its a much better solution, because it allows missing values in the last
field to be unambiguously signalled.

23,45,,67,91
clearly has a missing value in the third field but if

23,45,63,67,91
23,45,63,67,91,

are equivalent then there is an ambiguity

Is there a survey of common use of csv files? How many of them would be
conformant?
YS> Yakov

YS> [email-masked] wrote:
>> In section "2.Definition of the CSV format", items 3 & 4 state:
>> 
>> 3. There maybe an optional header line appearing as the first line of
>> the file with the same format as normal record lines. This header will
>> contain names corresponding to the fields in the file and will usually
>> contain the same number of fields as the records in the rest of the 
>> file. For example:
>> 
>> field_name,field_name,field_name CRLF
>> aaa,bbb,ccc CRLF
>> zzz,yyy,xxx CRLF
>> 4. Within the header and each record there may be one or more fields,
>> delimited by commas. The last field in the record may or may not be 
>> followed by a comma. For example:
>> 
>> aaa,bbb,ccc
>> 
>> Why would you permit the last field in the record to be followed by a
>> comma?
>> If a CSV record comprises:
>> 
>> aaa,,ccc,ddd,,CRLF
>> 
>> does it have 6 fields or 5?
>> If comma is a field separator only, there are 6 fields:
>> 
>> 1. aaa
>> 2. <null>
>> 3. ccc
>> 4. ddd
>> 5. <null>
>> 6. <null>
>> 
>> But if the comma is also a mandatory terminator for the last field 
>> (effectively the record separator becomes comma-CRLF), then there are 5
>> fields.
>> 
>> In my view, permitting the last field to end with comma leads to 
>> ambiguity, and prevents an application from checking that an exact 
>> number of fields is present.  The only way to guarantee the exact number
>> of fields is then to count the fields in the header.  But then your item
>> 3 allows the header record to be omitted.
>> 
>> Would it not be safer to make the header record mandatory?
>> 
>> Thank-you and regards,
>> Clyde Ingram
>> 

-- 
 Chris Lilley                    mailto:[email-masked]
 Chair, W3C SVG Working Group
 W3C Graphics Activity Lead

From: chris at w3.org (Chris Lilley)
Date: Tue Jan  3 16:22:56 2006
Subject: Media Type "text/csv": new draft (-02) and Last Call
In-Reply-To: <[email-masked]>
References: <[email-masked]> <[email-masked]>
Message-ID: <[email-masked]>

On Wednesday, March 23, 2005, 7:14:24 AM, Yakov wrote:

YS> Clyde,

YS> Thanks for pointing this out. I personally think that instead of making
YS> the header record mandatory which is something that most CSV 
YS> applications do not have, I would rather take the comma out of the end
YS> of the record and have the last field end with a CRLF instead of an 
YS> optional COMMA. Do you think that is a plausible solution?

Its a much better solution, because it allows missing values in the last
field to be unambiguously signalled.

23,45,,67,91
clearly has a missing value in the third field but if

23,45,63,67,91
23,45,63,67,91,

are equivalent then there is an ambiguity

Is there a survey of common use of csv files? How many of them would be
conformant?
YS> Yakov

YS> [email-masked] wrote:
>> In section "2.Definition of the CSV format", items 3 & 4 state:
>> 
>> 3. There maybe an optional header line appearing as the first line of
>> the file with the same format as normal record lines. This header will
>> contain names corresponding to the fields in the file and will usually
>> contain the same number of fields as the records in the rest of the 
>> file. For example:
>> 
>> field_name,field_name,field_name CRLF
>> aaa,bbb,ccc CRLF
>> zzz,yyy,xxx CRLF
>> 4. Within the header and each record there may be one or more fields,
>> delimited by commas. The last field in the record may or may not be 
>> followed by a comma. For example:
>> 
>> aaa,bbb,ccc
>> 
>> Why would you permit the last field in the record to be followed by a
>> comma?
>> If a CSV record comprises:
>> 
>> aaa,,ccc,ddd,,CRLF
>> 
>> does it have 6 fields or 5?
>> If comma is a field separator only, there are 6 fields:
>> 
>> 1. aaa
>> 2. <null>
>> 3. ccc
>> 4. ddd
>> 5. <null>
>> 6. <null>
>> 
>> But if the comma is also a mandatory terminator for the last field 
>> (effectively the record separator becomes comma-CRLF), then there are 5
>> fields.
>> 
>> In my view, permitting the last field to end with comma leads to 
>> ambiguity, and prevents an application from checking that an exact 
>> number of fields is present.  The only way to guarantee the exact number
>> of fields is then to count the fields in the header.  But then your item
>> 3 allows the header record to be omitted.
>> 
>> Would it not be safer to make the header record mandatory?
>> 
>> Thank-you and regards,
>> Clyde Ingram
>> 

-- 
 Chris Lilley                    mailto:[email-masked]
 Chair, W3C SVG Working Group
 W3C Graphics Activity Lead

From: GK-lists at ninebynine.org (Graham Klyne)
Date: Thu Mar 24 02:55:00 2005
Subject: Media Type "text/csv": new draft (-02) and Last Call
In-Reply-To: <[email-masked]>
References: <[email-masked]> <[email-masked]>
Message-ID: <[email-masked]>

At 01:14 23/03/05 -0500, Yakov Shafranovich wrote:

>Clyde,
>
>Thanks for pointing this out. I personally think that instead of making 
>the header record mandatory which is something that most CSV applications 
>do not have, I would rather take the comma out of the end of the record 
>and have the last field end with a CRLF instead of an optional COMMA. Do 
>you think that is a plausible solution?

No.  Some of the Excel data I process has trailing commas.  This must be 
allowed.

I also don't think it's necessary to say anything (other than maybe as a 
comment) about any special status for the first line:  such use is 
accommodated quite reasonably within the basic CSV format.

For example, having such a line when exporting Excel as CSV depends 
entirely upon how the user constructs the original spreadsheet.  Column 
headings are common, but not mandatory.  In some cases, there may be a more 
complex heading structure -- this is an application issue, not a dataset 
format issue, and as such does not belong in the dataset format specification.

#g
--

>Yakov
>
>[email-masked] wrote:
>>In section "2.Definition of the CSV format", items 3 & 4 state:
>>3. There maybe an optional header line appearing as the first line of the 
>>file with the same format as normal record lines. This header will 
>>contain names corresponding to the fields in the file and will usually 
>>contain the same number of fields as the records in the rest of the file. 
>>For example:
>>field_name,field_name,field_name CRLF
>>aaa,bbb,ccc CRLF
>>zzz,yyy,xxx CRLF
>>4. Within the header and each record there may be one or more fields, 
>>delimited by commas. The last field in the record may or may not be 
>>followed by a comma. For example:
>>aaa,bbb,ccc
>>Why would you permit the last field in the record to be followed by a comma?
>>If a CSV record comprises:
>>aaa,,ccc,ddd,,CRLF
>>does it have 6 fields or 5?
>>If comma is a field separator only, there are 6 fields:
>>1. aaa
>>2. <null>
>>3. ccc
>>4. ddd
>>5. <null>
>>6. <null>
>>But if the comma is also a mandatory terminator for the last field 
>>(effectively the record separator becomes comma-CRLF), then there are 5 fields.
>>In my view, permitting the last field to end with comma leads to 
>>ambiguity, and prevents an application from checking that an exact number 
>>of fields is present.  The only way to guarantee the exact number of 
>>fields is then to count the fields in the header.  But then your item 3 
>>allows the header record to be omitted.
>>Would it not be safer to make the header record mandatory?
>>Thank-you and regards,
>>Clyde Ingram

------------
Graham Klyne
For email:
http://www.ninebynine.org/#Contact

From: GK-lists at ninebynine.org (Graham Klyne)
Date: Tue Jan  3 16:22:57 2006
Subject: Media Type "text/csv": new draft (-02) and Last Call
In-Reply-To: <[email-masked]>
References: <[email-masked]> <[email-masked]>
Message-ID: <[email-masked]>

At 01:14 23/03/05 -0500, Yakov Shafranovich wrote:

>Clyde,
>
>Thanks for pointing this out. I personally think that instead of making 
>the header record mandatory which is something that most CSV applications 
>do not have, I would rather take the comma out of the end of the record 
>and have the last field end with a CRLF instead of an optional COMMA. Do 
>you think that is a plausible solution?

No.  Some of the Excel data I process has trailing commas.  This must be 
allowed.

I also don't think it's necessary to say anything (other than maybe as a 
comment) about any special status for the first line:  such use is 
accommodated quite reasonably within the basic CSV format.

For example, having such a line when exporting Excel as CSV depends 
entirely upon how the user constructs the original spreadsheet.  Column 
headings are common, but not mandatory.  In some cases, there may be a more 
complex heading structure -- this is an application issue, not a dataset 
format issue, and as such does not belong in the dataset format specification.

#g
--

>Yakov
>
>[email-masked] wrote:
>>In section "2.Definition of the CSV format", items 3 & 4 state:
>>3. There maybe an optional header line appearing as the first line of the 
>>file with the same format as normal record lines. This header will 
>>contain names corresponding to the fields in the file and will usually 
>>contain the same number of fields as the records in the rest of the file. 
>>For example:
>>field_name,field_name,field_name CRLF
>>aaa,bbb,ccc CRLF
>>zzz,yyy,xxx CRLF
>>4. Within the header and each record there may be one or more fields, 
>>delimited by commas. The last field in the record may or may not be 
>>followed by a comma. For example:
>>aaa,bbb,ccc
>>Why would you permit the last field in the record to be followed by a comma?
>>If a CSV record comprises:
>>aaa,,ccc,ddd,,CRLF
>>does it have 6 fields or 5?
>>If comma is a field separator only, there are 6 fields:
>>1. aaa
>>2. <null>
>>3. ccc
>>4. ddd
>>5. <null>
>>6. <null>
>>But if the comma is also a mandatory terminator for the last field 
>>(effectively the record separator becomes comma-CRLF), then there are 5 fields.
>>In my view, permitting the last field to end with comma leads to 
>>ambiguity, and prevents an application from checking that an exact number 
>>of fields is present.  The only way to guarantee the exact number of 
>>fields is then to count the fields in the header.  But then your item 3 
>>allows the header record to be omitted.
>>Would it not be safer to make the header record mandatory?
>>Thank-you and regards,
>>Clyde Ingram

------------
Graham Klyne
For email:
http://www.ninebynine.org/#Contact

From: GK-lists at ninebynine.org (Graham Klyne)
Date: Thu Mar 24 02:54:59 2005
Subject: Media Type "text/csv": new draft (-02) and Last Call
In-Reply-To: <[email-masked]>
Message-ID: <[email-masked]>

At 11:15 22/02/05 -0500, Yakov Shafranovich wrote:

>I posted a new draft (-02) with minor ABNF corrections. The new draft can 
>be found in the IETF repository:
>
>http://www.ietf.org/internet-drafts/draft-shafranovich-mime-csv-02.txt

Reviewing this version...

(I'm currently using this format to process genomic data from MS Excel 
spreadsheets, so some comments are based on my own code.)

I feel compelled to ask the question:  should this really be a text/csv 
type rather than application/csv?  Past comments by Ned Freed with respect 
to text/html suggest possibly not, but similarity with 
text/tab-separated-values suggests maybe yes.  I don't personally feel 
strongly about this.

Section 2:

[[
    6.  Field containing line breaks (CRLF) and commas should be enclosed
        in double-quotes.  For example:

        "aaa","b CRLF
        bb","ccc" CRLF
        zzz,yyy,xxx
]]

Are there any CSV applications that actually quote newlines in 
values?  (I'm not sure that my own CSV handling code would accept a newline 
within a quoted string)

[[
    7.  If double-quotes are used to enclosed fields, then double-quotes
        inside fields must be surrounded by double quotes.  For example:

        "aaa","b"""bb","ccc"
]]

This is NOT how Excel handles quotes, nor my own code.  Rather, when a 
quoted string contains a double-quote character, the double quote is 
doubled up.  Thus the example would be:
        "aaa","b""bb","ccc"

Section 7.1:
[[
    [3]  Postel, J., "Transmission Control Protocol", STD 7, RFC 793,
         September 1981.
]]

I think this should be an informative reference, not normative.

#g

------------
Graham Klyne
For email:
http://www.ninebynine.org/#Contact

From: GK-lists at ninebynine.org (Graham Klyne)
Date: Tue Jan  3 16:22:57 2006
Subject: Media Type "text/csv": new draft (-02) and Last Call
In-Reply-To: <[email-masked]>
Message-ID: <[email-masked]>

At 11:15 22/02/05 -0500, Yakov Shafranovich wrote:

>I posted a new draft (-02) with minor ABNF corrections. The new draft can 
>be found in the IETF repository:
>
>http://www.ietf.org/internet-drafts/draft-shafranovich-mime-csv-02.txt

Reviewing this version...

(I'm currently using this format to process genomic data from MS Excel 
spreadsheets, so some comments are based on my own code.)

I feel compelled to ask the question:  should this really be a text/csv 
type rather than application/csv?  Past comments by Ned Freed with respect 
to text/html suggest possibly not, but similarity with 
text/tab-separated-values suggests maybe yes.  I don't personally feel 
strongly about this.

Section 2:

[[
    6.  Field containing line breaks (CRLF) and commas should be enclosed
        in double-quotes.  For example:

        "aaa","b CRLF
        bb","ccc" CRLF
        zzz,yyy,xxx
]]

Are there any CSV applications that actually quote newlines in 
values?  (I'm not sure that my own CSV handling code would accept a newline 
within a quoted string)

[[
    7.  If double-quotes are used to enclosed fields, then double-quotes
        inside fields must be surrounded by double quotes.  For example:

        "aaa","b"""bb","ccc"
]]

This is NOT how Excel handles quotes, nor my own code.  Rather, when a 
quoted string contains a double-quote character, the double quote is 
doubled up.  Thus the example would be:
        "aaa","b""bb","ccc"

Section 7.1:
[[
    [3]  Postel, J., "Transmission Control Protocol", STD 7, RFC 793,
         September 1981.
]]

I think this should be an informative reference, not normative.

#g

------------
Graham Klyne
For email:
http://www.ninebynine.org/#Contact

From: blilly at erols.com (Bruce Lilly)
Date: Tue Mar 15 18:24:47 2005
Subject: Fwd: I-D ACTION:draft-lilly-text-troff-02.txt
Message-ID: <[email-masked]>

----------  Forwarded Message excerpt ----------

Subject: I-D ACTION:draft-lilly-text-troff-02.txt
Date: Tue March 15 2005 10:06
From: [email-masked]
To: [email-masked]

A New Internet-Draft is available from the on-line Internet-Drafts directories.

	Title		: Media subtype registration for media type text/troff
	Author(s)	: B. Lilly
	Filename	: draft-lilly-text-troff-02.txt
	Pages		: 14
	Date		: 2005-3-11
	
A text media subtype for tagging content consisting of juxtaposed
   text and formatting directives as used by the troff series of
   programs and for conveying information about the intended processing
   steps necessary to produce formatted output is described.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-lilly-text-troff-02.txt

To remove yourself from the I-D Announcement list, send a message to 
[email-masked] with the word unsubscribe in the body of the message.  
You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce 
to change your subscription settings.

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

-------------------------------------------------------

This is the third version of the text/troff registration (there
was a second version announced shortly before IETF 62). A change
history is located in Appendix B of the draft; security/privacy
have been more fully addressed as a result of earlier comments.
If there are no significant additional substantive comments in
the next two weeks, I plan to ask the Applications Area Directors
to shepherd this through the IETF/IESG process as an Informational
RFC.

HTML and PostScript/PDF versions formatted for ANSI A (a.k.a.
"letter") and ISO A4 paper can be found at
http://users.erols.com/blilly/Internet-Drafts/
-------------- next part --------------
Skipped content of type message/external-body-------------- next part --------------
Skipped content of type message/external-body-------------- next part --------------
_______________________________________________
I-D-Announce mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/i-d-announce
>From [email-masked]  Tue Mar 22 17:51:36 2005
From: clyde.ingram at edl.uk.eds.com ([email-masked])
Date: Tue Mar 22 17:53:06 2005
Subject: Media Type "text/csv": new draft (-02) and Last Call
Message-ID: <[email-masked]>

In section "2.Definition of the CSV format", items 3 & 4 state:

3. There maybe an optional header line appearing as the first line of the
file with the same format as normal record lines. This header will contain
names corresponding to the fields in the file and will usually contain the
same number of fields as the records in the rest of the file. For example: 

field_name,field_name,field_name CRLF
aaa,bbb,ccc CRLF
zzz,yyy,xxx CRLF 
4. Within the header and each record there may be one or more fields,
delimited by commas. The last field in the record may or may not be followed
by a comma. For example: 

aaa,bbb,ccc 

Why would you permit the last field in the record to be followed by a comma?
If a CSV record comprises:

aaa,,ccc,ddd,,CRLF

does it have 6 fields or 5?
If comma is a field separator only, there are 6 fields:

1. aaa
2. <null>
3. ccc
4. ddd
5. <null>
6. <null>

But if the comma is also a mandatory terminator for the last field
(effectively the record separator becomes comma-CRLF), then there are 5
fields.

In my view, permitting the last field to end with comma leads to ambiguity,
and prevents an application from checking that an exact number of fields is
present.  The only way to guarantee the exact number of fields is then to
count the fields in the header.  But then your item 3 allows the header
record to be omitted.

Would it not be safer to make the header record mandatory?

Thank-you and regards,
Clyde Ingram

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://eikenes.alvestrand.no/pipermail/ietf-types/attachments/20050322/4e13e2df/attachment.html
>From [email-masked]  Wed Mar 23 07:14:24 2005
From: YakovS at solidmatrix.com (Yakov Shafranovich)
Date: Wed Mar 23 07:15:54 2005
Subject: Media Type "text/csv": new draft (-02) and Last Call
In-Reply-To: <[email-masked]>
References: <[email-masked]>
Message-ID: <[email-masked]>

Clyde,

Thanks for pointing this out. I personally think that instead of making 
the header record mandatory which is something that most CSV 
applications do not have, I would rather take the comma out of the end 
of the record and have the last field end with a CRLF instead of an 
optional COMMA. Do you think that is a plausible solution?

Yakov

[email-masked] wrote:
> In section "2.Definition of the CSV format", items 3 & 4 state:
> 
> 3. There maybe an optional header line appearing as the first line of 
> the file with the same format as normal record lines. This header will 
> contain names corresponding to the fields in the file and will usually 
> contain the same number of fields as the records in the rest of the 
> file. For example:
> 
> field_name,field_name,field_name CRLF
> aaa,bbb,ccc CRLF
> zzz,yyy,xxx CRLF
> 4. Within the header and each record there may be one or more fields, 
> delimited by commas. The last field in the record may or may not be 
> followed by a comma. For example:
> 
> aaa,bbb,ccc
> 
> Why would you permit the last field in the record to be followed by a 
> comma?
> If a CSV record comprises:
> 
> aaa,,ccc,ddd,,CRLF
> 
> does it have 6 fields or 5?
> If comma is a field separator only, there are 6 fields:
> 
> 1. aaa
> 2. <null>
> 3. ccc
> 4. ddd
> 5. <null>
> 6. <null>
> 
> But if the comma is also a mandatory terminator for the last field 
> (effectively the record separator becomes comma-CRLF), then there are 5 
> fields.
> 
> In my view, permitting the last field to end with comma leads to 
> ambiguity, and prevents an application from checking that an exact 
> number of fields is present.  The only way to guarantee the exact number 
> of fields is then to count the fields in the header.  But then your item 
> 3 allows the header record to be omitted.
> 
> Would it not be safer to make the header record mandatory?
> 
> Thank-you and regards,
> Clyde Ingram
> 

From: blilly at erols.com (Bruce Lilly)
Date: Tue Jan  3 16:22:56 2006
Subject: Fwd: I-D ACTION:draft-lilly-text-troff-02.txt
Message-ID: <[email-masked]>

----------  Forwarded Message excerpt ----------

Subject: I-D ACTION:draft-lilly-text-troff-02.txt
Date: Tue March 15 2005 10:06
From: [email-masked]
To: [email-masked]

A New Internet-Draft is available from the on-line Internet-Drafts directories.

	Title		: Media subtype registration for media type text/troff
	Author(s)	: B. Lilly
	Filename	: draft-lilly-text-troff-02.txt
	Pages		: 14
	Date		: 2005-3-11
	
A text media subtype for tagging content consisting of juxtaposed
   text and formatting directives as used by the troff series of
   programs and for conveying information about the intended processing
   steps necessary to produce formatted output is described.

A URL for this Internet-Draft is:
http://www.ietf.org/internet-drafts/draft-lilly-text-troff-02.txt

To remove yourself from the I-D Announcement list, send a message to 
[email-masked] with the word unsubscribe in the body of the message.  
You can also visit https://www1.ietf.org/mailman/listinfo/I-D-announce 
to change your subscription settings.

Below is the data which will enable a MIME compliant mail reader
implementation to automatically retrieve the ASCII version of the
Internet-Draft.

-------------------------------------------------------

This is the third version of the text/troff registration (there
was a second version announced shortly before IETF 62). A change
history is located in Appendix B of the draft; security/privacy
have been more fully addressed as a result of earlier comments.
If there are no significant additional substantive comments in
the next two weeks, I plan to ask the Applications Area Directors
to shepherd this through the IETF/IESG process as an Informational
RFC.

HTML and PostScript/PDF versions formatted for ANSI A (a.k.a.
"letter") and ISO A4 paper can be found at
http://users.erols.com/blilly/Internet-Drafts/
-------------- next part --------------
Skipped content of type message/external-body-------------- next part --------------
Skipped content of type message/external-body-------------- next part --------------
_______________________________________________
I-D-Announce mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/i-d-announce
>From [email-masked]  Tue Mar 22 17:51:36 2005
From: clyde.ingram at edl.uk.eds.com ([email-masked])
Date: Tue Jan  3 16:22:56 2006
Subject: Media Type "text/csv": new draft (-02) and Last Call
Message-ID: <[email-masked]>

In section "2.Definition of the CSV format", items 3 & 4 state:

3. There maybe an optional header line appearing as the first line of the
file with the same format as normal record lines. This header will contain
names corresponding to the fields in the file and will usually contain the
same number of fields as the records in the rest of the file. For example: 

field_name,field_name,field_name CRLF
aaa,bbb,ccc CRLF
zzz,yyy,xxx CRLF 
4. Within the header and each record there may be one or more fields,
delimited by commas. The last field in the record may or may not be followed
by a comma. For example: 

aaa,bbb,ccc 

Why would you permit the last field in the record to be followed by a comma?
If a CSV record comprises:

aaa,,ccc,ddd,,CRLF

does it have 6 fields or 5?
If comma is a field separator only, there are 6 fields:

1. aaa
2. <null>
3. ccc
4. ddd
5. <null>
6. <null>

But if the comma is also a mandatory terminator for the last field
(effectively the record separator becomes comma-CRLF), then there are 5
fields.

In my view, permitting the last field to end with comma leads to ambiguity,
and prevents an application from checking that an exact number of fields is
present.  The only way to guarantee the exact number of fields is then to
count the fields in the header.  But then your item 3 allows the header
record to be omitted.

Would it not be safer to make the header record mandatory?

Thank-you and regards,
Clyde Ingram

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://www.alvestrand.no/pipermail/ietf-types/attachments/20050322/4e13e2df/attachment.html
>From [email-masked]  Wed Mar 23 07:14:24 2005
From: YakovS at solidmatrix.com (Yakov Shafranovich)
Date: Tue Jan  3 16:22:56 2006
Subject: Media Type "text/csv": new draft (-02) and Last Call
In-Reply-To: <[email-masked]>
References: <[email-masked]>
Message-ID: <[email-masked]>

Clyde,

Thanks for pointing this out. I personally think that instead of making 
the header record mandatory which is something that most CSV 
applications do not have, I would rather take the comma out of the end 
of the record and have the last field end with a CRLF instead of an 
optional COMMA. Do you think that is a plausible solution?

Yakov

[email-masked] wrote:
> In section "2.Definition of the CSV format", items 3 & 4 state:
> 
> 3. There maybe an optional header line appearing as the first line of 
> the file with the same format as normal record lines. This header will 
> contain names corresponding to the fields in the file and will usually 
> contain the same number of fields as the records in the rest of the 
> file. For example:
> 
> field_name,field_name,field_name CRLF
> aaa,bbb,ccc CRLF
> zzz,yyy,xxx CRLF
> 4. Within the header and each record there may be one or more fields, 
> delimited by commas. The last field in the record may or may not be 
> followed by a comma. For example:
> 
> aaa,bbb,ccc
> 
> Why would you permit the last field in the record to be followed by a 
> comma?
> If a CSV record comprises:
> 
> aaa,,ccc,ddd,,CRLF
> 
> does it have 6 fields or 5?
> If comma is a field separator only, there are 6 fields:
> 
> 1. aaa
> 2. <null>
> 3. ccc
> 4. ddd
> 5. <null>
> 6. <null>
> 
> But if the comma is also a mandatory terminator for the last field 
> (effectively the record separator becomes comma-CRLF), then there are 5 
> fields.
> 
> In my view, permitting the last field to end with comma leads to 
> ambiguity, and prevents an application from checking that an exact 
> number of fields is present.  The only way to guarantee the exact number 
> of fields is then to count the fields in the header.  But then your item 
> 3 allows the header record to be omitted.
> 
> Would it not be safer to make the header record mandatory?
> 
> Thank-you and regards,
> Clyde Ingram
> 

From: mankin at psg.com (Allison Mankin)
Date: Fri Mar 11 17:49:58 2005
Subject: Transport Area requests Media Type review for audio/amr-wb+
Message-ID: <[email-masked]>

The Transport Area requests a Media Type review for the proposed
new type  w, intended for the IETF tree, as a Proposed Standard, 
and specified in 
http://www.ietf.org/internet-drafts/draft-ietf-avt-rtp-amrwbplus-06.txt
(Section 7.1).

Please provide any comments as soon as possible.

Thanks,

Allison

To: [email-masked]
Subject:  Transport Area requests Media Type review for 
Cc: [email-masked], [email-masked]
Fcc: +outbox
Reply-To: [email-masked]
--------

The Transport Area requests a Media Type review for the proposed
new type  w, intended for the IETF tree, as a Proposed Standard, 
and specified in 
http://www.ietf.org/internet-drafts/draft-ietf-uncomp-video-06.txt
(Section  ).

We know of the following needed changes:

1.
Author will be separated from Change Controller,
and Change Controller will be modified to
"AVT Working Group delegated from the IESG"

Please provide any comments as soon as possible.

Thanks,

Allison

From: mankin at psg.com (Allison Mankin)
Date: Tue Jan  3 16:22:56 2006
Subject: Transport Area requests Media Type review for audio/amr-wb+
Message-ID: <[email-masked]>

The Transport Area requests a Media Type review for the proposed
new type  w, intended for the IETF tree, as a Proposed Standard, 
and specified in 
http://www.ietf.org/internet-drafts/draft-ietf-avt-rtp-amrwbplus-06.txt
(Section 7.1).

Please provide any comments as soon as possible.

Thanks,

Allison

To: [email-masked]
Subject:  Transport Area requests Media Type review for 
Cc: [email-masked], [email-masked]
Fcc: +outbox
Reply-To: [email-masked]
--------

The Transport Area requests a Media Type review for the proposed
new type  w, intended for the IETF tree, as a Proposed Standard, 
and specified in 
http://www.ietf.org/internet-drafts/draft-ietf-uncomp-video-06.txt
(Section  ).

We know of the following needed changes:

1.
Author will be separated from Change Controller,
and Change Controller will be modified to
"AVT Working Group delegated from the IESG"

Please provide any comments as soon as possible.

Thanks,

Allison

From: mankin at psg.com (Allison Mankin)
Date: Tue Jan  3 16:22:56 2006
Subject: Transport Area requests Media Type review for audio/VMR-WB
Message-ID: <[email-masked]>

The Transport Area requests a Media Type review for the proposed
new type  audio/VMR-WB, intended for the IETF tree, as a Proposed Standard, 
and specified in 
http://www.ietf.org/internet-drafts/draft-ietf-avt-rtp-vmr-wb-10.txt
(Section  9.1).

Please provide any comments as soon as possible.

Thanks,

Allison

From: blilly at erols.com (Bruce Lilly)
Date: Tue Jan  3 16:22:56 2006
Subject: Media type versioning (was: Re: Scripting Media Types)
In-Reply-To: <[email-masked]>
References: <[email-masked]>
Message-ID: <[email-masked]>

On Thu March 10 2005 03:26, Larry Masinter wrote:

> the specific proposal of using
> MIME type _parameters_ is a bad design choice for conveying
> information as slippery as _version_.

Please define "slippery" in this context, then please explain
precisely how your pet preference (Content-Features) is a more
suitable choice in that respect, including the specific registered
feature tag that you propose using.

> >> Maybe; if transfer encoding is applied, that would entail
> >> decoding and opening the media content to determine version
> >> information.  If the media type is specified via
> >> message/external-body or a similar mechanism, additional
> >> steps are necessary to obtain the media in order to
> >> determine version information.  If a particular instance
> >> of content is large, that may result in a substantial waste
> >> of resources if it turns out to have an incompatible version.
> 
> I don't think those are problems; aren't these forwarding bodies
> (that are looking at MIME types and parameters) also unwrapping
> transfer encodings and fetching external bodies? 

No (i.e. not necessarily).
 
> You're optimizing what I assume is mainly a failure case,
> anyway: you got sent something you can't read.

Bad assumption. One might have been sent one or more pointers
to external data.  One would like to be able to determine
ahead of time whether any of them can be read *before*
retrieving huge amounts of data, and that may depend on version
information.
 
> Using MIME parameters isn't the only place to put auxiliary
> information. If you need auxiliary information, put it somewhere
> else -- such as in a content-features header.

Version information is not a feature.  Your suggestion requires
registration of a feature tag (RFC 2506), waiting for that tag
to be recognized (i.e. replacement of deployed implementations
of feature tag parsers), generating an additional field (viz.
Content-Features) in addition to Content-Type, hoping that that
additional field isn't lost or mangled in transit, and extreme
optimism regarding whether a receiving implementation will pay
any attention at all to a Content-Features field, much less be
able to interpret that tag *and* its associated value.  That
rigamarole is certainly not as light-weight as simply using a
version parameter in the Content-Type field.

N.B. there is nothing resembling "version" registered as a
media feature tag
http://www.iana.org/assignments/media-feature-tags

> I meant "negotiation" in a more general way: describing content
> in a way that subsequent processors can make decisions about
> whether or how to process without opening the content itself.
> In this case, I recommend putting this auxiliary data somewhere
> where it won't just confuse and pollute the processing that
> is necessary for the content-type itself.
> 
> "Content-features" is useful in cases where there is no back-channel.

But not for version information.  Content-Type is similarly
useful (including for version information) in the absence of
a negotiation mechanism.

> Hardly. Pulling out redundant information and sticking it
> on the wrapper to facilitate processing is an optimization,

One might only have the wrapper (e.g. message/external-body),
in which case there is no redundancy, and provision for
appropriately dealing with the content consistent with the
Internet Architecture (RFC 1958, specifically section 3.1);
assuming that all recipients will have unlimited (electrical)
power, infinite communications bandwidth, gibibytes of local
storage, and every conceivable version of every type of
decoder is not consistent with that architecture.  It is not
an optimization; it is information which is essential to the
operator of a low-power, limited bandwidth, small-memory
device to be able to determine whether or not it is advisable
to download some content of a specific media type.

> >> o negotiation via content-features may be a viable mechanism
> >>   where negotiation is possible, but it is not always
> >>   possible (and some means of indicating version must be
> >>   available to the content negotiation mechanism)
> 
> Wrong. "Content-Features" can be used to carry version information
> as well as the parameters of "Content-Type". There is no
> situational difference.

See above re. no registered feature tag for version.
  
> > > that leaves parameters associated with the media type as an
> > > efficient mechanism for indicating version information.
> 
> I disagree -- it's extraneous information, unnecessary and often
> lost in processing pipelines.  

It's not extraneous or unnecessary (often essential).  And of
course Content-Features fields are *always* preserved and
considered, right?
 
> We haven't really talked much about the terrible deployment experience
> and prospects for *any* MIME parameters. They usually get lost;
> most of the operating systems that support MIME type mappings to
> files don't support parameters (Windows file associations, MacOS,
> mime.types, etc.).  

And that differs from Content-Features ... how?  Why should we
base design decisions on abysmally bad implementations of things
(e.g. OSes) unrelated to what is being specified (media types)?
 
> We could write standards about castles in the air all day, and
> I suppose it would be "free" in some sense of the word, but it
> is wrong, and detracts from the value of the standard. Don't
> specify things that either don't work or are undeployable. Otherwise,
> what's the point?

Ooh, that sounds like the sort of remarks that could come back
to haunt a fellow :-).

From: blilly at erols.com (Bruce Lilly)
Date: Thu Mar 10 18:51:37 2005
Subject: Media type versioning (was: Re: Scripting Media Types)
In-Reply-To: <[email-masked]>
References: <[email-masked]>
Message-ID: <[email-masked]>

On Wed March 9 2005 20:06, Eric Burger wrote:

> "Hey - this is an ECMAscript4 application: is that OK?"  Joe Random freaks
> out, because he thought he was getting a movie, not a cup of coffee.

ECMAscript has nothing to do with coffee.  Concern would be an
appropriate response, just as with any other case where unexpected
executable content is involved.
>From [email-masked]  Thu Mar 10 19:53:47 2005
From: blilly at erols.com (Bruce Lilly)
Date: Thu Mar 10 19:54:15 2005
Subject: Media type versioning (was: Re: Scripting Media Types)
In-Reply-To: <[email-masked]>
References: <[email-masked]>
Message-ID: <[email-masked]>

On Thu March 10 2005 03:26, Larry Masinter wrote:

> the specific proposal of using
> MIME type _parameters_ is a bad design choice for conveying
> information as slippery as _version_.

Please define "slippery" in this context, then please explain
precisely how your pet preference (Content-Features) is a more
suitable choice in that respect, including the specific registered
feature tag that you propose using.

> >> Maybe; if transfer encoding is applied, that would entail
> >> decoding and opening the media content to determine version
> >> information.  If the media type is specified via
> >> message/external-body or a similar mechanism, additional
> >> steps are necessary to obtain the media in order to
> >> determine version information.  If a particular instance
> >> of content is large, that may result in a substantial waste
> >> of resources if it turns out to have an incompatible version.
> 
> I don't think those are problems; aren't these forwarding bodies
> (that are looking at MIME types and parameters) also unwrapping
> transfer encodings and fetching external bodies? 

No (i.e. not necessarily).
 
> You're optimizing what I assume is mainly a failure case,
> anyway: you got sent something you can't read.

Bad assumption. One might have been sent one or more pointers
to external data.  One would like to be able to determine
ahead of time whether any of them can be read *before*
retrieving huge amounts of data, and that may depend on version
information.
 
> Using MIME parameters isn't the only place to put auxiliary
> information. If you need auxiliary information, put it somewhere
> else -- such as in a content-features header.

Version information is not a feature.  Your suggestion requires
registration of a feature tag (RFC 2506), waiting for that tag
to be recognized (i.e. replacement of deployed implementations
of feature tag parsers), generating an additional field (viz.
Content-Features) in addition to Content-Type, hoping that that
additional field isn't lost or mangled in transit, and extreme
optimism regarding whether a receiving implementation will pay
any attention at all to a Content-Features field, much less be
able to interpret that tag *and* its associated value.  That
rigamarole is certainly not as light-weight as simply using a
version parameter in the Content-Type field.

N.B. there is nothing resembling "version" registered as a
media feature tag
http://www.iana.org/assignments/media-feature-tags

> I meant "negotiation" in a more general way: describing content
> in a way that subsequent processors can make decisions about
> whether or how to process without opening the content itself.
> In this case, I recommend putting this auxiliary data somewhere
> where it won't just confuse and pollute the processing that
> is necessary for the content-type itself.
> 
> "Content-features" is useful in cases where there is no back-channel.

But not for version information.  Content-Type is similarly
useful (including for version information) in the absence of
a negotiation mechanism.

> Hardly. Pulling out redundant information and sticking it
> on the wrapper to facilitate processing is an optimization,

One might only have the wrapper (e.g. message/external-body),
in which case there is no redundancy, and provision for
appropriately dealing with the content consistent with the
Internet Architecture (RFC 1958, specifically section 3.1);
assuming that all recipients will have unlimited (electrical)
power, infinite communications bandwidth, gibibytes of local
storage, and every conceivable version of every type of
decoder is not consistent with that architecture.  It is not
an optimization; it is information which is essential to the
operator of a low-power, limited bandwidth, small-memory
device to be able to determine whether or not it is advisable
to download some content of a specific media type.

> >> o negotiation via content-features may be a viable mechanism
> >>   where negotiation is possible, but it is not always
> >>   possible (and some means of indicating version must be
> >>   available to the content negotiation mechanism)
> 
> Wrong. "Content-Features" can be used to carry version information
> as well as the parameters of "Content-Type". There is no
> situational difference.

See above re. no registered feature tag for version.
  
> > > that leaves parameters associated with the media type as an
> > > efficient mechanism for indicating version information.
> 
> I disagree -- it's extraneous information, unnecessary and often
> lost in processing pipelines.  

It's not extraneous or unnecessary (often essential).  And of
course Content-Features fields are *always* preserved and
considered, right?
 
> We haven't really talked much about the terrible deployment experience
> and prospects for *any* MIME parameters. They usually get lost;
> most of the operating systems that support MIME type mappings to
> files don't support parameters (Windows file associations, MacOS,
> mime.types, etc.).  

And that differs from Content-Features ... how?  Why should we
base design decisions on abysmally bad implementations of things
(e.g. OSes) unrelated to what is being specified (media types)?
 
> We could write standards about castles in the air all day, and
> I suppose it would be "free" in some sense of the word, but it
> is wrong, and detracts from the value of the standard. Don't
> specify things that either don't work or are undeployable. Otherwise,
> what's the point?

Ooh, that sounds like the sort of remarks that could come back
to haunt a fellow :-).
>From [email-masked]  Fri Mar 11 17:49:46 2005
From: mankin at psg.com (Allison Mankin)
Date: Fri Mar 11 17:49:50 2005
Subject: Transport Area requests Media Type review for audio/VMR-WB
Message-ID: <[email-masked]>

The Transport Area requests a Media Type review for the proposed
new type  audio/VMR-WB, intended for the IETF tree, as a Proposed Standard, 
and specified in 
http://www.ietf.org/internet-drafts/draft-ietf-avt-rtp-vmr-wb-10.txt
(Section  9.1).

Please provide any comments as soon as possible.

Thanks,

Allison

From: blilly at erols.com (Bruce Lilly)
Date: Tue Jan  3 16:22:56 2006
Subject: Media type versioning (was: Re: Scripting Media Types)
In-Reply-To: <[email-masked]>
References: <[email-masked]>
Message-ID: <[email-masked]>

On Wed March 9 2005 20:06, Eric Burger wrote:

> "Hey - this is an ECMAscript4 application: is that OK?"  Joe Random freaks
> out, because he thought he was getting a movie, not a cup of coffee.

ECMAscript has nothing to do with coffee.  Concern would be an
appropriate response, just as with any other case where unexpected
executable content is involved.

From: LMM at acm.org (Larry Masinter)
Date: Tue Jan  3 16:22:56 2006
Subject: Media type versioning (was: Re: Scripting Media Types)
In-Reply-To: <[email-masked]>
Message-ID: <[email-masked]>

>>> My opinion is that content negotiation for versioning capability
>>> using MIME type parameters is unworkable.
>> Negotiation isn't always possible; sometimes it's a one-way
>> street.

I was using "negotiation" in more general sense. If the
are no choices to be made, then there's no point in sending
any additional information. And my point isn't that doing
content negotiation or, in general, making choices about content
is impossible -- rather that the specific proposal of using
MIME type _parameters_ is a bad design choice for conveying
information as slippery as _version_.

>>> The use of the MIME 
>>> type is to describe the payload sufficiently so that you know
>>> HOW to process it, but the content-type label cannot (and thus
>>> should not) be extended in an attempt to use it to determine
>>> WHETHER a given processor knows enough to be able to process it.
>> We're in agreement that the media type/subtype tag should not
>> be used for versioning (that would result in an unnecessary
>> proliferation of tags).
> Is this really so bad?  There are all of 93 application/foo's 
> registered.

The original point was about whether it is possible
to decide whether a receiver is capable of interpreting
something, in general, just based on the MIME type.

But registering and using a new MIME type for a new version
is often just the right thing:

Unless the old MIME type was defined carefully with sufficient
attention to extensibility so that all old processors will
behave gracefully when presented with new content, registering
a new type may be the only choice.

>>> So whether version information should or should not be in
>>> a media type parameter depends pretty much on whether there
>>> is an embedded, easy-to-find version indicator in the data
>>> itself; if there isn't, and processors need to know the version
>>> to choose between different processing methods, then the
>>> version parameter should be mandatory. There is no strong
>>> use case for an optional version parameter, or in general
>>> for duplicating, in MIME parameters, information that is
>>> readily obtained from the content itself.
 
>> Maybe; if transfer encoding is applied, that would entail
>> decoding and opening the media content to determine version
>> information.  If the media type is specified via
>> message/external-body or a similar mechanism, additional
>> steps are necessary to obtain the media in order to
>> determine version information.  If a particular instance
>> of content is large, that may result in a substantial waste
>> of resources if it turns out to have an incompatible version.

I don't think those are problems; aren't these forwarding bodies
(that are looking at MIME types and parameters) also unwrapping
transfer encodings and fetching external bodies? 

You're optimizing what I assume is mainly a failure case,
anyway: you got sent something you can't read. In some cases,
you're just deciding _which_ processor to send it to, so fetching
the whole body isn't an extra overhead anyway.
 
> I don't like the idea of embedded data being required to determine if the
> entity can accept the message.  By that point in time, it is too late.  If
> you need a data element to determine, at the protocol level, you can
process
> a message, then you need that data element at the protocol level.  E.g., a
> MIME parameter rather than an XML tag in a body part.

Using MIME parameters isn't the only place to put auxiliary
information. If you need auxiliary information, put it somewhere
else -- such as in a content-features header. Don't overloading
everything into the same header.

>>> If you want to do content negotiation, then consider
>>> using media features and media feature negotiation; with
>>> media features you can negotiate not only version information,
>>> but other parameters that might also be necessary to know
>>> in order to determine interpretability, e.g., availability
>>> of compression modes, codecs, fonts, color capabilities,
>>> buffer size limitations, etc.
 
>> That's fine for protocols that support negotiation, but
>> not all protocols do so (HTTP does, RFC 3297 layered on
>> top of messaging can, but FTP does not).

FTP in most cases is done without MIME anyway -- the receiver
guesses the MIME type based on the file extension and sniffing
of information about the host operating system.

I meant "negotiation" in a more general way: describing content
in a way that subsequent processors can make decisions about
whether or how to process without opening the content itself.
In this case, I recommend putting this auxiliary data somewhere
where it won't just confuse and pollute the processing that
is necessary for the content-type itself.

"Content-features" is useful in cases where there is no back-channel.
You can ignore it if you want, but if you're in a situation where further
processing of some content-types is expensive (expensive
local processor, or forwarding agent to further processor)
and you have some information about the next step's capabilities,
you could try to match the next step's capabilities against
the content-features. 

> > Summary:
> > o media content might contain version information, but reliance
> >   on that can result in wasted resources
> 
> And a layer violation and asking for hacked stacks.

Hardly. Pulling out redundant information and sticking it
on the wrapper to facilitate processing is an optimization,
but it is also a "layer violation". Sometimes we tolerate
layer violations if they're important for optimizing performance.
 
>> o negotiation via content-features may be a viable mechanism
>>   where negotiation is possible, but it is not always
>>   possible (and some means of indicating version must be
>>   available to the content negotiation mechanism)

Wrong. "Content-Features" can be used to carry version information
as well as the parameters of "Content-Type". There is no
situational difference.
 
> > o proliferation of type/subtypes tags for the sole purpose
> >   of versioning is undesirable
> 
> I'm not sold on this one.  If a new version of a media type is not
> backwards-compatible, I would offer that it is a different media type.  As
> noted above, in 8 years of MIME types, we have all of 93 
> application/foo registered types.

New subtypes are often necessary for new versions, although
not always "desirable".
 
> > that leaves parameters associated with the media type as an
> > efficient mechanism for indicating version information.

I disagree -- it's extraneous information, unnecessary and often
lost in processing pipelines.  

We haven't really talked much about the terrible deployment experience
and prospects for *any* MIME parameters. They usually get lost;
most of the operating systems that support MIME type mappings to
files don't support parameters (Windows file associations, MacOS,
mime.types, etc.).  

> After saying that this is not the only method, I also would offer that one
> is ALWAYS free, in the definition of the MIME type, list a parameter and
> describe normative behavior for processing that parameter.

We could write standards about castles in the air all day, and
I suppose it would be "free" in some sense of the word, but it
is wrong, and detracts from the value of the standard. Don't
specify things that either don't work or are undeployable. Otherwise,
what's the point?

Larry
-- 
http://larry.masinter.net

From: eburger at brooktrout.com (Eric Burger)
Date: Thu Mar 10 02:15:46 2005
Subject: Media type versioning (was: Re: Scripting Media Types)
Message-ID: <[email-masked]>

Oooh -- I really don't like this one.

True, there may be situations (specific MIME types) where this makes sense.
However, if my UA *knows* how to handle different versions, why would I ever
present a selection to the user?

Scenario:
Joe Random goes to a porn web site.  The web site has a Java applet that
displays the movies.  On hitting the web site, a dialog box pops up saying,
"Hey - this is an ECMAscript4 application: is that OK?"  Joe Random freaks
out, because he thought he was getting a movie, not a cup of coffee.

> -----Original Message-----
> From: [email-masked]
> [mailto:[email-masked]]On Behalf Of Bruce Lilly
> Sent: Friday, February 11, 2005 12:21 AM
> To: [email-masked]
> Cc: Bjoern Hoehrmann
> Subject: Re: Media type versioning (was: Re: Scripting Media Types)
> 
> 
> On Wed February 9 2005 16:46, Bjoern Hoehrmann wrote:
> 
> > At least in the context of HTTP, use of such parameters where media
> > types provide them is not common in my experience, using them would
> > typically require access to the server configuration and users that
> > know about the parameters and how to configure the server 
> accordingly;
> > RFC 2854 notes that this did not work in practise. The 
> situation might
> > be different when this would be relevant for the proposed types, but
> > that seems unlikely...
> 
> 2854 only mentions one parameter (charset) and it seems to encourage
> its use; I see no mention of problems with use of (as opposed to
> failure to use) that parameter.
> 
> > Could you make a suggestion for the draft that would address your
> > concern? Would it be sufficient to refer to the version parameter
> > as optional parameter with an unconstrained lexical space for its
> > value and with the semantics that implementations must consider any
> > use of the parameter to indicate that the content is unsupported?
> 
> One approach would be to recommend or require display to the user
> and confirmation before executing the content.  Since we're dealing
> with potentially dangerous executable content, user confirmation
> should be required anyway; providing some version information to
> the user with a request for confirmation provides the user with
> information which may be useful in making a decision. Structured
> version information could go further; it might provide applications
> with the ability to determine that execution is not possible (due
> to lack of availability of some resource, or due to some
> incompatibility); that could replace a request for confirmation by
> an informative message (possibly localized) which would explain the
> problem to the user (as opposed to simply displaying unstructured
> "human-readable" versioning content, which might in fact be unreadable
> to some humans due to charset and/or language issues).
> 
> As to what specific sort of structured versioning information
> might be appropriate, that really depends on how the (scripting)
> language versions and related compatibility issues are structured.
> I can, however, offer an observation about what hasn't worked well,
> and how to avoid problems which have plagued other versioning
> mechanisms.
> 
> MIME itself provides for versioning via a MIME-Version header field.
> The field body is defined as two dot-separated integers.  However,
> there are several problems:
> 1. the semantics are undefined; aside from the initial version
>    of 1.0, we can't tell how a hypothetical version 1.1 would be
>    partially compatible, or a hypothetical version 2.0, etc.
> 2. the syntax is silent about leading zeroes in the integer
>    parts. That's a recipe for interoperability problems.
> 
>From [email-masked]  Thu Mar 10 02:06:51 2005
From: eburger at brooktrout.com (Eric Burger)
Date: Thu Mar 10 02:15:54 2005
Subject: Media type versioning (was: Re: Scripting Media Types)
Message-ID: <[email-masked]>

> From: Bruce Lilly
> Sent: Thursday, February 10, 2005 11:30 PM
> On Thu February 10 2005 15:31, Larry Masinter wrote:
> 
> > My opinion is that content negotiation for versioning capability
> > using MIME type parameters is unworkable.
> 
> Negotiation isn't always possible; sometimes it's a one-way
> street.
> 
> > The use of the MIME 
> > type is to describe the payload sufficiently so that you know
> > HOW to process it, but the content-type label cannot (and thus
> > should not) be extended in an attempt to use it to determine
> > WHETHER a given processor knows enough to be able to process it.
> 
> We're in agreement that the media type/subtype tag should not
> be used for versioning (that would result in an unnecessary
> proliferation of tags).

Is this really so bad?  There are all of 93 application/foo's registered.

> > So whether version information should or should not be in
> > a media type parameter depends pretty much on whether there
> > is an embedded, easy-to-find version indicator in the data
> > itself; if there isn't, and processors need to know the version
> > to choose between different processing methods, then the
> > version parameter should be mandatory. There is no strong
> > use case for an optional version parameter, or in general
> > for duplicating, in MIME parameters, information that is
> > readily obtained from the content itself.
> 
> Maybe; if transfer encoding is applied, that would entail
> decoding and opening the media content to determine version
> information.  If the media type is specified via
> message/external-body or a similar mechanism, additional
> steps are necessary to obtain the media in order to
> determine version information.  If a particular instance
> of content is large, that may result in a substantial waste
> of resources if it turns out to have an incompatible version.

I don't like the idea of embedded data being required to determine if the
entity can accept the message.  By that point in time, it is too late.  If
you need a data element to determine, at the protocol level, you can process
a message, then you need that data element at the protocol level.  E.g., a
MIME parameter rather than an XML tag in a body part.

> > If you want to do content negotiation, then consider
> > using media features and media feature negotiation; with
> > media features you can negotiate not only version information,
> > but other parameters that might also be necessary to know
> > in order to determine interpretability, e.g., availability
> > of compression modes, codecs, fonts, color capabilities,
> > buffer size limitations, etc.
> 
> That's fine for protocols that support negotiation, but
> not all protocols do so (HTTP does, RFC 3297 layered on
> top of messaging can, but FTP does not).

Well, if you need negotiation, use a protocol that offers negotiation :)

> Summary:
> o media content might contain version information, but reliance
>   on that can result in wasted resources

And a layer violation and asking for hacked stacks.

> o negotiation via content-features may be a viable mechanism
>   where negotiation is possible, but it is not always
>   possible (and some means of indicating version must be
>   available to the content negotiation mechanism)

Agreed.

> o proliferation of type/subtypes tags for the sole purpose
>   of versioning is undesirable

I'm not sold on this one.  If a new version of a media type is not
backwards-compatible, I would offer that it is a different media type.  As
noted above, in 8 years of MIME types, we have all of 93 application/foo
registered types.

> that leaves parameters associated with the media type as an
> efficient mechanism for indicating version information.

After saying that this is not the only method, I also would offer that one
is ALWAYS free, in the definition of the MIME type, list a parameter and
describe normative behavior for processing that parameter.  That could be
something like "version".  That said, I am really uneasy in saying that one
MUST use a version tag.  Some of the thread above suggested that one could
generically use the combination of MIME type and "version" as an indicator
of MIME type.  For example, if the "version" is "bigger" than what I know, I
know to generically barf on the message.  Besides the issues that Ned
pointed out (V1, V2, Vn vs. 1.2.3, 1.2.4, m.n.o vs. ...), some MIME types,
like the PostScript example, still work on "older" processors, while others
are completely different between versions.

The result is the semantics and normative behavior for receivers for the
"version" parameter is MIME type specific, and thus NOT subject to generic
behavior specification.
>From [email-masked]  Thu Mar 10 09:26:50 2005
From: LMM at acm.org (Larry Masinter)
Date: Thu Mar 10 09:28:30 2005
Subject: Media type versioning (was: Re: Scripting Media Types)
In-Reply-To: <[email-masked]>
Message-ID: <[email-masked]>

>>> My opinion is that content negotiation for versioning capability
>>> using MIME type parameters is unworkable.
>> Negotiation isn't always possible; sometimes it's a one-way
>> street.

I was using "negotiation" in more general sense. If the
are no choices to be made, then there's no point in sending
any additional information. And my point isn't that doing
content negotiation or, in general, making choices about content
is impossible -- rather that the specific proposal of using
MIME type _parameters_ is a bad design choice for conveying
information as slippery as _version_.

>>> The use of the MIME 
>>> type is to describe the payload sufficiently so that you know
>>> HOW to process it, but the content-type label cannot (and thus
>>> should not) be extended in an attempt to use it to determine
>>> WHETHER a given processor knows enough to be able to process it.
>> We're in agreement that the media type/subtype tag should not
>> be used for versioning (that would result in an unnecessary
>> proliferation of tags).
> Is this really so bad?  There are all of 93 application/foo's 
> registered.

The original point was about whether it is possible
to decide whether a receiver is capable of interpreting
something, in general, just based on the MIME type.

But registering and using a new MIME type for a new version
is often just the right thing:

Unless the old MIME type was defined carefully with sufficient
attention to extensibility so that all old processors will
behave gracefully when presented with new content, registering
a new type may be the only choice.

>>> So whether version information should or should not be in
>>> a media type parameter depends pretty much on whether there
>>> is an embedded, easy-to-find version indicator in the data
>>> itself; if there isn't, and processors need to know the version
>>> to choose between different processing methods, then the
>>> version parameter should be mandatory. There is no strong
>>> use case for an optional version parameter, or in general
>>> for duplicating, in MIME parameters, information that is
>>> readily obtained from the content itself.
 
>> Maybe; if transfer encoding is applied, that would entail
>> decoding and opening the media content to determine version
>> information.  If the media type is specified via
>> message/external-body or a similar mechanism, additional
>> steps are necessary to obtain the media in order to
>> determine version information.  If a particular instance
>> of content is large, that may result in a substantial waste
>> of resources if it turns out to have an incompatible version.

I don't think those are problems; aren't these forwarding bodies
(that are looking at MIME types and parameters) also unwrapping
transfer encodings and fetching external bodies? 

You're optimizing what I assume is mainly a failure case,
anyway: you got sent something you can't read. In some cases,
you're just deciding _which_ processor to send it to, so fetching
the whole body isn't an extra overhead anyway.
 
> I don't like the idea of embedded data being required to determine if the
> entity can accept the message.  By that point in time, it is too late.  If
> you need a data element to determine, at the protocol level, you can
process
> a message, then you need that data element at the protocol level.  E.g., a
> MIME parameter rather than an XML tag in a body part.

Using MIME parameters isn't the only place to put auxiliary
information. If you need auxiliary information, put it somewhere
else -- such as in a content-features header. Don't overloading
everything into the same header.

>>> If you want to do content negotiation, then consider
>>> using media features and media feature negotiation; with
>>> media features you can negotiate not only version information,
>>> but other parameters that might also be necessary to know
>>> in order to determine interpretability, e.g., availability
>>> of compression modes, codecs, fonts, color capabilities,
>>> buffer size limitations, etc.
 
>> That's fine for protocols that support negotiation, but
>> not all protocols do so (HTTP does, RFC 3297 layered on
>> top of messaging can, but FTP does not).

FTP in most cases is done without MIME anyway -- the receiver
guesses the MIME type based on the file extension and sniffing
of information about the host operating system.

I meant "negotiation" in a more general way: describing content
in a way that subsequent processors can make decisions about
whether or how to process without opening the content itself.
In this case, I recommend putting this auxiliary data somewhere
where it won't just confuse and pollute the processing that
is necessary for the content-type itself.

"Content-features" is useful in cases where there is no back-channel.
You can ignore it if you want, but if you're in a situation where further
processing of some content-types is expensive (expensive
local processor, or forwarding agent to further processor)
and you have some information about the next step's capabilities,
you could try to match the next step's capabilities against
the content-features. 

> > Summary:
> > o media content might contain version information, but reliance
> >   on that can result in wasted resources
> 
> And a layer violation and asking for hacked stacks.

Hardly. Pulling out redundant information and sticking it
on the wrapper to facilitate processing is an optimization,
but it is also a "layer violation". Sometimes we tolerate
layer violations if they're important for optimizing performance.
 
>> o negotiation via content-features may be a viable mechanism
>>   where negotiation is possible, but it is not always
>>   possible (and some means of indicating version must be
>>   available to the content negotiation mechanism)

Wrong. "Content-Features" can be used to carry version information
as well as the parameters of "Content-Type". There is no
situational difference.
 
> > o proliferation of type/subtypes tags for the sole purpose
> >   of versioning is undesirable
> 
> I'm not sold on this one.  If a new version of a media type is not
> backwards-compatible, I would offer that it is a different media type.  As
> noted above, in 8 years of MIME types, we have all of 93 
> application/foo registered types.

New subtypes are often necessary for new versions, although
not always "desirable".
 
> > that leaves parameters associated with the media type as an
> > efficient mechanism for indicating version information.

I disagree -- it's extraneous information, unnecessary and often
lost in processing pipelines.  

We haven't really talked much about the terrible deployment experience
and prospects for *any* MIME parameters. They usually get lost;
most of the operating systems that support MIME type mappings to
files don't support parameters (Windows file associations, MacOS,
mime.types, etc.).  

> After saying that this is not the only method, I also would offer that one
> is ALWAYS free, in the definition of the MIME type, list a parameter and
> describe normative behavior for processing that parameter.

We could write standards about castles in the air all day, and
I suppose it would be "free" in some sense of the word, but it
is wrong, and detracts from the value of the standard. Don't
specify things that either don't work or are undeployable. Otherwise,
what's the point?

Larry
-- 
http://larry.masinter.net

From: eburger at brooktrout.com (Eric Burger)
Date: Tue Jan  3 16:22:56 2006
Subject: Media type versioning (was: Re: Scripting Media Types)
Message-ID: <[email-masked]>

Oooh -- I really don't like this one.

True, there may be situations (specific MIME types) where this makes sense.
However, if my UA *knows* how to handle different versions, why would I ever
present a selection to the user?

Scenario:
Joe Random goes to a porn web site.  The web site has a Java applet that
displays the movies.  On hitting the web site, a dialog box pops up saying,
"Hey - this is an ECMAscript4 application: is that OK?"  Joe Random freaks
out, because he thought he was getting a movie, not a cup of coffee.

> -----Original Message-----
> From: [email-masked]
> [mailto:[email-masked]]On Behalf Of Bruce Lilly
> Sent: Friday, February 11, 2005 12:21 AM
> To: [email-masked]
> Cc: Bjoern Hoehrmann
> Subject: Re: Media type versioning (was: Re: Scripting Media Types)
> 
> 
> On Wed February 9 2005 16:46, Bjoern Hoehrmann wrote:
> 
> > At least in the context of HTTP, use of such parameters where media
> > types provide them is not common in my experience, using them would
> > typically require access to the server configuration and users that
> > know about the parameters and how to configure the server 
> accordingly;
> > RFC 2854 notes that this did not work in practise. The 
> situation might
> > be different when this would be relevant for the proposed types, but
> > that seems unlikely...
> 
> 2854 only mentions one parameter (charset) and it seems to encourage
> its use; I see no mention of problems with use of (as opposed to
> failure to use) that parameter.
> 
> > Could you make a suggestion for the draft that would address your
> > concern? Would it be sufficient to refer to the version parameter
> > as optional parameter with an unconstrained lexical space for its
> > value and with the semantics that implementations must consider any
> > use of the parameter to indicate that the content is unsupported?
> 
> One approach would be to recommend or require display to the user
> and confirmation before executing the content.  Since we're dealing
> with potentially dangerous executable content, user confirmation
> should be required anyway; providing some version information to
> the user with a request for confirmation provides the user with
> information which may be useful in making a decision. Structured
> version information could go further; it might provide applications
> with the ability to determine that execution is not possible (due
> to lack of availability of some resource, or due to some
> incompatibility); that could replace a request for confirmation by
> an informative message (possibly localized) which would explain the
> problem to the user (as opposed to simply displaying unstructured
> "human-readable" versioning content, which might in fact be unreadable
> to some humans due to charset and/or language issues).
> 
> As to what specific sort of structured versioning information
> might be appropriate, that really depends on how the (scripting)
> language versions and related compatibility issues are structured.
> I can, however, offer an observation about what hasn't worked well,
> and how to avoid problems which have plagued other versioning
> mechanisms.
> 
> MIME itself provides for versioning via a MIME-Version header field.
> The field body is defined as two dot-separated integers.  However,
> there are several problems:
> 1. the semantics are undefined; aside from the initial version
>    of 1.0, we can't tell how a hypothetical version 1.1 would be
>    partially compatible, or a hypothetical version 2.0, etc.
> 2. the syntax is silent about leading zeroes in the integer
>    parts. That's a recipe for interoperability problems.
> 

From: eburger at brooktrout.com (Eric Burger)
Date: Tue Jan  3 16:22:56 2006
Subject: Media type versioning (was: Re: Scripting Media Types)
Message-ID: <[email-masked]>

> From: Bruce Lilly
> Sent: Thursday, February 10, 2005 11:30 PM
> On Thu February 10 2005 15:31, Larry Masinter wrote:
> 
> > My opinion is that content negotiation for versioning capability
> > using MIME type parameters is unworkable.
> 
> Negotiation isn't always possible; sometimes it's a one-way
> street.
> 
> > The use of the MIME 
> > type is to describe the payload sufficiently so that you know
> > HOW to process it, but the content-type label cannot (and thus
> > should not) be extended in an attempt to use it to determine
> > WHETHER a given processor knows enough to be able to process it.
> 
> We're in agreement that the media type/subtype tag should not
> be used for versioning (that would result in an unnecessary
> proliferation of tags).

Is this really so bad?  There are all of 93 application/foo's registered.

> > So whether version information should or should not be in
> > a media type parameter depends pretty much on whether there
> > is an embedded, easy-to-find version indicator in the data
> > itself; if there isn't, and processors need to know the version
> > to choose between different processing methods, then the
> > version parameter should be mandatory. There is no strong
> > use case for an optional version parameter, or in general
> > for duplicating, in MIME parameters, information that is
> > readily obtained from the content itself.
> 
> Maybe; if transfer encoding is applied, that would entail
> decoding and opening the media content to determine version
> information.  If the media type is specified via
> message/external-body or a similar mechanism, additional
> steps are necessary to obtain the media in order to
> determine version information.  If a particular instance
> of content is large, that may result in a substantial waste
> of resources if it turns out to have an incompatible version.

I don't like the idea of embedded data being required to determine if the
entity can accept the message.  By that point in time, it is too late.  If
you need a data element to determine, at the protocol level, you can process
a message, then you need that data element at the protocol level.  E.g., a
MIME parameter rather than an XML tag in a body part.

> > If you want to do content negotiation, then consider
> > using media features and media feature negotiation; with
> > media features you can negotiate not only version information,
> > but other parameters that might also be necessary to know
> > in order to determine interpretability, e.g., availability
> > of compression modes, codecs, fonts, color capabilities,
> > buffer size limitations, etc.
> 
> That's fine for protocols that support negotiation, but
> not all protocols do so (HTTP does, RFC 3297 layered on
> top of messaging can, but FTP does not).

Well, if you need negotiation, use a protocol that offers negotiation :)

> Summary:
> o media content might contain version information, but reliance
>   on that can result in wasted resources

And a layer violation and asking for hacked stacks.

> o negotiation via content-features may be a viable mechanism
>   where negotiation is possible, but it is not always
>   possible (and some means of indicating version must be
>   available to the content negotiation mechanism)

Agreed.

> o proliferation of type/subtypes tags for the sole purpose
>   of versioning is undesirable

I'm not sold on this one.  If a new version of a media type is not
backwards-compatible, I would offer that it is a different media type.  As
noted above, in 8 years of MIME types, we have all of 93 application/foo
registered types.

> that leaves parameters associated with the media type as an
> efficient mechanism for indicating version information.

After saying that this is not the only method, I also would offer that one
is ALWAYS free, in the definition of the MIME type, list a parameter and
describe normative behavior for processing that parameter.  That could be
something like "version".  That said, I am really uneasy in saying that one
MUST use a version tag.  Some of the thread above suggested that one could
generically use the combination of MIME type and "version" as an indicator
of MIME type.  For example, if the "version" is "bigger" than what I know, I
know to generically barf on the message.  Besides the issues that Ned
pointed out (V1, V2, Vn vs. 1.2.3, 1.2.4, m.n.o vs. ...), some MIME types,
like the PostScript example, still work on "older" processors, while others
are completely different between versions.

The result is the semantics and normative behavior for receivers for the
"version" parameter is MIME type specific, and thus NOT subject to generic
behavior specification.