Subject: clarification: "escaped"

From: Bill de hÓra <[email-masked]>
Date: Tue, 25 Jul 2006 22:46:38 +0100

The RFC says that the content should be 'escaped' for type 'text/html' 
in 3.1.1.2, but dosn't define what that is. Is it as defined in XML:

http://www.w3.org/TR/REC-xml/#dt-escape

?

cheers
Bill

---

From: "Robert Sayre" <[email-masked]>
Date: Tue, 25 Jul 2006 18:05:47 -0400

On 7/25/06, Bill de hÓra <[email-masked]> wrote:
> in 3.1.1.2, but dosn't define what that is.

IIRC, WG discussion touched on this point, and the WG decided the
definition wasn't important, given the example. Is there a problem
you're hoping to clear up?

-- 

Robert Sayre

"I would have written a shorter letter, but I did not have the time."

---

From: Bill de hÓra <[email-masked]>
Date: Wed, 26 Jul 2006 00:06:46 +0100

> you're hoping to clear up?

It came up on django irc. I'd assumed for whatever reason that escaping 
was limited to the usual XML suspects, but when asked about html content 
I knew I didn't know for sure, especially wrt HTML character entities. 
And I didn't know whether Atom code could get away with escaping < and &.

cheers
Bill

---

From: "Robert Sayre" <[email-masked]>
Date: Tue, 25 Jul 2006 19:28:44 -0400

> And I didn't know whether Atom code could get away with escaping < and &.

I'm not certain I understand the issue, but if the question concerns
what happens when an Atom processor encounters a document with no
declared entities and contains a title like this:

<atom:title type="html">&lt;b>&nbsp;hmm&lt;b></atom:title>

that is an XML fatal error, no doubt, as the ampersand before "nbsp"
must be escaped. Concretely, Mozilla will give you a DOM with a
non-breaking space if you write this:

<atom:title type="html">&lt;b>&amp;nbsp;hmm&lt;b></atom:title>

-- 

Robert Sayre

---

From: "A. Pagaltzis" <[email-masked]>
Date: Wed, 26 Jul 2006 01:51:40 +0200

* Robert Sayre <[email-masked]> [2006-07-26 01:45]:
> that is an XML fatal error, no doubt, as the ampersand before
> "nbsp" must be escaped.

But he did say “escaping < and &”, so it would be. I’m not sure
what Bill’s question even is.

Regards,
-- 
Aristotle Pagaltzis // <http://plasmasturm.org/>

---

From: Bill de hÓra <[email-masked]>
Date: Wed, 26 Jul 2006 10:19:35 +0100

> what Bill’s question even is.

What do I escape, so I know what to unescape?

cheers
Bill

---

From: Antone Roundy <[email-masked]>
Date: Wed, 26 Jul 2006 08:32:38 -0600

> What do I escape, so I know what to unescape?

The point is that after your XML parser has unescaped the content of  
the element, it should be suitable for handling as HTML.  Escape  
whatever you have to ensure that the consumer gets HTML from their  
XML parser.  Converting & to &amp; and < to &lt; is sufficient  
(assuming that you've started with HTML--if you've started with plain  
text, then you need to double escape, but in that case, you should be  
using type="text" anyway to save yourself the trouble).  You could  
also convert > to &gt;, " to &quot;, ' to &apos; and any other  
characters to numeric character references.  Or you put the whole  
thing in a CDATA block.

---

From: "A. Pagaltzis" <[email-masked]>
Date: Wed, 26 Jul 2006 17:34:20 +0200

* Antone Roundy <[email-masked]> [2006-07-26 16:45]:
> Or you put the whole thing in a CDATA block.

Which is the easiest option, so long as you remember the edge
case of having to turn any `]]>` sequences in the input into
`]]>]]&gt;<![CDATA[`.

Regards,

---

From: "James Holderness" <[email-masked]>
Date: Wed, 26 Jul 2006 16:31:03 +0100

Antone Roundy wrote:
> Converting & to &amp; and < to &lt; is sufficient

People keep missing this so I'm going to point it out one more time: there 
are certain rare circumstances when a right angle bracket (>) MUST be 
escaped so if you're just doing ampersands and left angle brackets that 
WON'T always be sufficient. To be safe it's best to always encode all three.

As for CDATA sections, it's worth noting that you wouldn't have been able to 
syndicate this message thread if you always escaped everything with CDATA.

Regards
James