Subject: Language Negotiation

From: James M Snell <[email-masked]>
Date: Wed, 26 Jul 2006 12:33:55 -0700

Quick question regarding language negotiation with feeds...

By way of example, IBM's internal blogging infrastructure supports
bloggers in every locale IBM does business around the world, meaning
that there are posts in many different languages (japanese, french,
german, chinese, etc).  We have a dashboard/planet view that lists all
of the most recent posts across the entire system.  The content of each
post is presented in it's original language, but the metadata in the
feed is always in English, so we end up with things like...

  <!-- abbreviated example -->
  <feed xml:lang="en">
    <title>Dashboard</title>
    <entry>
      <content xml:lang="en">...</content>
    </entry>
    <entry>
      <content xml:lang="fr">...</content>
    </entry>
    <entry>
      <content xml:lang="jp">...</content>
    </entry>
  </feed>

So, given this, what (c|sh)ould be the expected behavior if a client
includes an Accept-Language: fr header, for instance, when GET'ing the
dashboard feed?

Now imagine that we start to apply machine translation to entries, so
that we can say, give me all entries, but translate them to French, or
English, etc.  Would that be best done using conneg or separate URIs?

- James

---

From: David Powell <[email-masked]>
Date: Thu, 27 Jul 2006 01:36:02 +0100

> English, etc.  Would that be best done using conneg or separate URIs?

I'd go for seperate URLs. Server-driven conneg (SDN) has its uses, but
for a protocol like atom (-syntax), the benefits are very limited, and
the risks and costs are high. With client-driven negotiation (CDN),
the client and server both have a better understanding of what they
are asking for and what is available - there is less risk of anything
going wrong.

The only advantage of SDN, is that it selects what it thinks is the
best feed without any user interaction - but that is something that
only needs to be done once anyway.

With SDN, you'll need to employ the Vary header, which can adversely affect
caching.

Hosted services (eg Bloglines) won't work unless they store multiple
seperate lists of entries and feed data for each set of request
headers that the request varies over; this seems very optimistic.

If you want to use a user's language preference to provide them with a
localized feed, I'd do it at the HTML level - use accept headers to
provide the correct autodiscovery link, or even better, to sort the
list of multiple links in a suitable order.

What are you going to do about ids BTW? I'd probably mint new ids for
the translated entries and feeds, and employ some sort of
link/extension if you need to be able to associate them.

-- 
Dave

---

From: James M Snell <[email-masked]>
Date: Wed, 26 Jul 2006 19:40:58 -0700

David Powell wrote:
> [snip]

This is perhaps the single most important reason for using CDN, at least
for now.

> list of multiple links in a suitable order.

And within feed documents in the form of language-qualified alternate
links (e.g., <link rel="self" hreflang="fr" href="..." />, <link
rel="self" hreflang="de" href="..." />, etc)

> link/extension if you need to be able to associate them.

I'd also lean towards minting new ids for translated resources.

- James

---

From: "Thomas Broyer" <[email-masked]>
Date: Thu, 27 Jul 2006 08:58:49 +0200

> rel="self" hreflang="de" href="..." />, etc)

You rather meant <link rel="alternate" hreflang="fr"
type="application/atom+xml" href="..." /> <link rel="alternate"
hreflang="de" type="application/atom+xml" href="..." />, weren't you?
;-)

From RFC4287:
   1.  The value "alternate" signifies that the IRI in the value of the
       href attribute identifies an alternate version of the resource
       described by the containing element.
[…]
   3.  The value "self" signifies that the IRI in the value of the href
       attribute identifies a resource equivalent to the containing
       element.

Here, you're linking to alternate versions of the resource (in
alternate languages), not to an "equivalent" resource (it's not
equivalent, because it's in another language).

…using xml:base rather than <link rel="self"/> would have been so much cleaner…

…or maybe at least rather a <link rel="live" /> or <link
rel="subscribable" />, because the rationale of rel="self" were "if an
aggregator is given a copy of a feed without information about its
original IRI, how can it find which URI to subscribe to?"…

-- 
Thomas Broyer

---

From: James Aylett <[email-masked]>
Date: Thu, 27 Jul 2006 08:25:01 +0100

> I'd also lean towards minting new ids for translated resources.

This took me quite a while to think through, but in the end I
agree. Translations of a resource will often have slightly different
contents in terms of the semantics of what is said, so I'd give them
different ids.

On the other hand, a way of subscribing to one feed that sometimes
publishes in both English and (say) French, and only reading one
translation of each would be handy. That's probably rel='alternate'
within the atom:entry, though.

James

-- 
/--------------------------------------------------------------------------\
  James Aylett                                                  xapian.org
  [email-masked]                               uncertaintydivision.org

---

From: "Sylvain Hellegouarch" <[email-masked]>
Date: Thu, 27 Jul 2006 09:13:57 +0100 (BST)

>
> 2006/7/27, Sylvain Hellegouarch:
>>
>> True. If you buy a book in English and then the translation of that book
>> in a different language you will end up with two books having each their
>> own ISBN.
>
> Well, actually, once a book is published, if you later update it,
> you'll have to use a new ISBN, so that's probably not a good analogy…
>

Indeed but wasn't I only talking about different language version of a
book? I haven't talked about updating a book ;)

- Sylvain

---

From: "Sylvain Hellegouarch" <[email-masked]>
Date: Thu, 27 Jul 2006 08:34:46 +0100 (BST)

---

From: "Thomas Broyer" <[email-masked]>
Date: Thu, 27 Jul 2006 10:04:40 +0200

-- 
Thomas Broyer

---

From: Eric Scheid <[email-masked]>
Date: Thu, 27 Jul 2006 18:43:07 +1000

>> contents in terms of the semantics of what is said, so I'd give them

what would happen if you used conneg on the @rel='self' link (to the
<entry/> document), asking for a different language?

e.

---

From: "Thomas Broyer" <[email-masked]>
Date: Thu, 27 Jul 2006 11:42:09 +0200

> <entry/> document), asking for a different language?

You mean, sending an Accept-Language request-header?

406 Not Acceptable or return the entry even if it does not match the
"accepted languages".
http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.7

-- 
Thomas Broyer

---

From: Eric Scheid <[email-masked]>
Date: Thu, 27 Jul 2006 22:49:15 +1000

> http://www.w3.org/Protocols/rfc2616/rfc2616-sec10.html#sec10.4.7

so, do not return the requested language alternate?

e.

---

From: "Thomas Broyer" <[email-masked]>
Date: Thu, 27 Jul 2006 16:40:42 +0200

> so, do not return the requested language alternate?

Hmm, sorry, I think I didn't correctly understood you first question...
...at least I don't understand the second...

-- 
Thomas Broyer

---

From: James M Snell <[email-masked]>
Date: Thu, 27 Jul 2006 07:41:51 -0700

> hreflang="de" type="application/atom+xml" href="..." />, weren't you?

Doh! Yeah. For some reason I've been transposing these two alot this
week.  Thx.

- James