Subject: NON-ASCII agenda file for IETF 64.

From: Michael Lee <[email-masked]>
Date: Tue, 11 Oct 2005 19:31:29 -0400

Dear IETF WG and BOF chairs:

I was looking at the uploaded agendas and I noticed that one WG, dccp, 
in the Transport area submitted a non-ascii text file.  One of my Web 
browsers (Mozilla, LINUX) couldn't properly convert the non-ascii 
characters to plain english text.

If you have submitted your agenda in any format other than plain ascii, 
then I suggest that you please resubmit it in plain ascii text.

Thanks,
Michael Lee,
The IETF Secretariat.

---

From: Lars Eggert <[email-masked]>
Date: Wed, 12 Oct 2005 14:04:59 +0200

Hi,

On Oct 12, 2005, at 1:31, Michael Lee wrote:
> I was looking at the uploaded agendas and I noticed that one WG,  
> dccp, in the Transport area submitted a non-ascii text file.  One  
> of my Web browsers (Mozilla, LINUX) couldn't properly convert the  
> non-ascii characters to plain english text.

sorry - I thought it was plain ASCII, but apparently there were some  
control characters in the beginning that messed things up.

I've uploaded a new version, but it hasn't replaced the one at http:// 
tools.ietf.org/wg/dccp/agenda yet.

Lars

PS: The upload script should check this :-)
--
Lars Eggert                                     NEC Network Laboratories

---

From: Henrik Levkowetz <[email-masked]>
Date: Wed, 12 Oct 2005 17:58:13 +0200

Hi Lars, Michael,

> control characters in the beginning that messed things up.

Actually, the file was UTF-16 (Big-Endian).  That, not the initial
BOM (Byte-Order-Mark) was probably what messed up some browsers.

> tools.ietf.org/wg/dccp/agenda yet.

That will happen within at most 1 hour after the upload (and has happened
now).

> PS: The upload script should check this :-)

That would be nice.  Mike - you could for instance run "file -Li" on
the file; yesterday that gave :

  00:30 /www/tools.ietf.org/agenda/64
  henrik@shiraz $ file -Li dccp.txt
  dccp.txt:       text/plain; charset=utf-16
  
  00:30 /www/tools.ietf.org/agenda/64
  henrik@shiraz $ 

while now the result is:

  17:52 /www/tools.ietf.org/agenda/64
  henrik@shiraz $ file -Li dccp.txt 
  dccp.txt: text/plain; charset=us-ascii
  
  17:52 /www/tools.ietf.org/agenda/64
  henrik@shiraz $ 

You may want to accept latin-1 / iso-8859-1 / iso-8859-15
in addition to us-ascii, but utf-16 is probably a bit over
the top...

	Henrik

---

From: Henrik Levkowetz <[email-masked]>
Date: Thu, 13 Oct 2005 09:23:58 +0200

Hi Martin,

    These are files that people type in on their own, and
they are uploaded without any additional information as to
mime type and character set

Your approach may be ideologically sound, but I think it's
pragmatically problematic in this case.

This tool should handle correctly what people will normally
hand it when they want to get an agenda out.  If we can't
agree on iso-8859-1 and iso-8859-15 we have to fall back to
us-ascii only.  The reason I mentioned iso-8851-1 is because
it is the default encoding for http (RFC 2616) and thus have
some chances of being handled correctly even if no encoding
is specified.

If we want to take this from a simple upload tool to a change
which involves a whole chain of tools, by moving from simple
.txt files to files with also include mime-type and charset
information, then we can do anything we want, and I'd not
hesitate to suggest that also utf-16 should be accepted.

But I don't think that is the situation we have here.  

	Henrik

on 2005-10-13 03:41 Martin Duerst said the following:
> At 00:58 05/10/13, Henrik Levkowetz wrote:
> 
> The IETF definitely should definitely also accept UTF-8.
> See RFC 2277 and RFC 3629 (STD 63!). On the other hand,
> we should discourage latin-1 / iso-8859-1 / iso-8859-15.
> 
> The other thing that should be done is to make sure that
> the files are served with the right information. The easiest
> way to do this is to only use UTF-8 (because it's very
> easy to check if a file is UTF-8, whereas other encodings
> are very difficult to check automatically).
> 
> Regards,    Martin. 
> 
>

---

From: Carsten Bormann <[email-masked]>
Date: Thu, 13 Oct 2005 09:38:33 +0200

The web is moving to UTF-8.

We should be, too.

(And UTF-8 is ASCII-compatible as long as only characters from the  
ASCII character set are used, so there is no visible change for ASCII  
uploaders.)

Gruesse, Carsten

---

From: Martin Duerst <[email-masked]>
Date: Thu, 13 Oct 2005 10:41:09 +0900

Regards,    Martin.

---

From: Henrik Levkowetz <[email-masked]>
Date: Thu, 13 Oct 2005 22:16:49 +0200

Hi Carsten,

> The web is moving to UTF-8.

Ahh, but that's a question of the encoding with which the agenda is
served from the web server.

Here we are concerned with the question of what encoding your (and other
people's) editor is producing.

Does the editor you use to write agendas output UTF-8 by default?

> ASCII character set are used, so there is no visible change for ASCII  

That is true.  But that also means that running "file -i" on the file
will report us-ascii, not utf-8 ...

Regards,

	Henrik

---

From: Brian E Carpenter <[email-masked]>
Date: Fri, 14 Oct 2005 16:01:57 +0200

The IETF's official language is English and
our archival format is ASCII.

I think applying these rules to agendas
is normal. This isn't an anti-I18N statement;
it's just our local convention.

     Brian

---

From: Thierry Ernst <[email-masked]>
Date: Sat, 15 Oct 2005 16:34:05 +0900

Yes, may be, however, let me take the opportunity ;-) to say that I'm in
favor of relieving ASCII as the requirement - in 2005, this is odd. HTML
would be nice (particularly for drafts, and this wouldn't be hard to do
from drafts produced in .xml format).

Thierry.

 On Fri, 14 Oct 2005 16:01:57 +0200

---

From: Martin Duerst <[email-masked]>
Date: Tue, 25 Oct 2005 15:30:11 +0900

[I'm fine with keeping things to us-ascii, I'll
just note a few details below.]

At 16:23 05/10/13, Henrik Levkowetz wrote:
 >hand it when they want to get an agenda out.

Well, if we told them that they can hand in UTF-8, some would.
If we in addition ran a check to see whether it's really
UTF-8 or not (which is easy to do for UTF-8, in contrast
to most other encodings), we could make pretty sure that
they wouldn't send in anything else.

 >If we can't

ISO-8859-1 being the default encoding for http is nothing but
a legend. It looks that way because in many parts of Europe,
people have ISO-8859-1 set as a default in their browsers.
But in other parts of the world, especially in Asia, that's
different.

 >hesitate to suggest that also utf-16 should be accepted.

Well, I was proposing an intermediate solution: Accept
US-ASCII and UTF-8, which don't need labeling; run a test
for UTF-8 (which includes US-ASCII), which is just a
replacement for the test for US-ASCII that I guess is
in place now (or should be in place). The other part
is to make sure that the agendas (or whatever) are
served as UTF-8.

 >But I don't think that is the situation we have here.

This all sounds eminently practical to me, despite
the fact that it might be, as you put it,
'ideologically sound'. Or did I miss something?

Regards,   Martin.

 >Content-Type: application/pgp-signature; name="signature.asc"
 >Content-Description: OpenPGP digital signature
 >Content-Disposition: attachment; filename="signature.asc"
 >
 >-----BEGIN PGP SIGNATURE-----
 >Version: GnuPG v1.4.1 (GNU/Linux)
 >
 >iD8DBQFDTguXeVhrtTJkXCMRAuV6AJ0Z+g835dK4u1reEBaKucXIR41lqQCghmUH
 >vP5PvWn529x3zQPYGKYo+K4=
 >=eVrO
 >-----END PGP SIGNATURE-----

---

From: Carsten Bormann <[email-masked]>
Date: Tue, 25 Oct 2005 13:59:51 +0200

On Oct 25 2005, at 08:30 Uhr, Martin Duerst wrote:

> a legend.

My copy of RFC2616 must be different from yours.

Section 3.7.1:

    When no explicit charset
    parameter is provided by the sender, media subtypes of the "text"
    type are defined to have a default charset value of "ISO-8859-1"  
when
    received via HTTP. Data in character sets other than "ISO-8859-1" or
    its subsets MUST be labeled with an appropriate charset value.

I agree that this is not a particularly good part of the  
specification, but it's still the standard.

Gruesse, Carsten

---

From: Harald Tveit Alvestrand <[email-masked]>
Date: Tue, 25 Oct 2005 06:40:43 -0700

> My copy of RFC2616 must be different from yours.

I think "wishful thinking" is a more accuate description than "legend".....