Subject: Recap - call for preferences/discussion - Re: New Version Notification for draft-divilly-atom-hierarchy-00

From: Al Brown <[email-masked]>
Date: Tue, 2 Jun 2009 10:58:51 -0700

This is to recap a discussion that happened on the cmis atom wg call this
morning and see if the group beyond Nikunj, Julian and I have any
preference for the options below.  If this group does not have a preference
for any of the options, the representation of a tree structure will be
removed from the hierarchy I-D.  This is deemed the best course of action
currently.

Due to the reasons already discussed by Julian and myself on this list,
CMIS will leverage option #2 if it is included in the I-D but probably not
option #1.  #1 does not ideally address CMIS' use case of how to represent
a tree of atom entries in a single document but rather provide a generic
pre-fetching/inlining mechanism.

The options so far discussed are:

1. Including the resource itself inside <link>
>>>  <atom:entry>
>>>    <atom:link rel="down"
>>>      href="/finance/feeds/default/portfolios/1/positions">
>>>      <atom:feed>
>>>        <atom:link rel="self"
>>>         href="/finance/feeds/default/portfolios/1/positions"/>
>>>         ...
>>>      </atom:feed>
>>>    </atom:link>
>>>    ...
>>>  </atom:entry>

2. Using an extension element in atom:entry to contain 0..n atom:entry 's
(could also include feed's though CMIS most likely would not leverage that)
<atom:entry>
      <ah:children>
            <atom:entry>...</atom:entry>
      </ah:children>
</atom:entry>

If there is not a consensus, then the representation of a tree structure
will be removed from the I-D by Nikunj in the next draft.

Thanks,
-Al

Al Brown
Emerging Standards and Industry Frameworks
CMIS: https://w3.tap.ibm.com/w3ki07/display/ECMCMIS/Home
Industry Frameworks: https://w3.tap.ibm.com/w3ki07/display/ECMIF/Home

Office 714 327 3453
Mobile 714 263 6441
Email  [email-masked]
CONFIDENTIAL NOTICE: The contents of this message, including any
attachments, are confidential and are intended solely for the use of the
person or entity to whom the message was addressed. If you are not the
intended recipient of this message, please be advised that any
dissemination, distribution, or use of the contents of this message is
strictly prohibited. If you received this message in error, please notify
the sender. Please also permanently delete all copies of the original
message and any attached documentation.

  From:       "Nikunj R. Mehta" <[email-masked]>                                                                       
                                                                                                                                
  To:         Julian Reschke <[email-masked]>                                                                            
                                                                                                                                
  Cc:         Atom-Syntax Syntax <[email-masked]>                                                                          
                                                                                                                                
  Date:       05/28/2009 12:32 PM                                                                                               
                                                                                                                                
  Subject:    Re: New Version Notification for draft-divilly-atom-hierarchy-00                                                  

There is precedence to the proposal I have submitted. GData uses
feedLink [1] and entryLink [2] mechanisms in Google's own namespace to
include in-line representations of entries and feeds. Notice how these
elements have both @rel and @href, critical to the functioning of
Atom's own link mechanisms.

The atom-hierarchy-ID is merely trying to standardize such behavior
and not invent new behavior. Moreover, the proposed syntax is not
merely syntactically legal, it is also semantically valuable. There
are three benefits to this approach:

1. The context of the inlined feed or entry is immediately clear from
the @rel value
2. The server can choose to inline content as per requirements and
configuration
3. The format supports inlining of any number of entries - a single or
multiple. It also expresses the absence of any entries, when this is
required.

Which of these advantages can you claim for directly embedding an
entry inside another?

Nikunj
http://o-micron.blogspot.com

On May 27, 2009, at 5:12 AM, Julian Reschke wrote:

>
> Nikunj R. Mehta wrote:
>> On May 25, 2009, at 8:42 AM, Julian Reschke wrote:
>>> with respect to <
http://tools.ietf.org/html/draft-divilly-atom-hierarchy-00#section-3
>>> >, "Inline Representation of Hierarchical Resources"...
>>>
>>> It seems to be that using atom:link as a container elements
>>> stretches its semantics of "reference from an entry or feed to a
>>> Web resource" too much.
>> I don't agree with that. A previous discussion on this mailing list
>> more than a year ago [1] has already concluded that it is perfectly
>> legal to do so. Additionally, the hierarchy-ID approach appears
>> similar to the
>
> I do not agree with that conclusion, but nevertheless, just because
> something is syntactically legal doesn't make it a good choice.
>
> atom:link is defined to as:
>
>  "The "atom:link" element defines a reference from an entry or feed
> to a Web resource."
>
> Inlining the content of referenced web resource is IMHO contrary to
> that definition.
>
>> approach taken by the RFC4287 with atom:content usage models, where
>> several options about inline and out-of-line content delivery are
>> available.
>
> But atom:content has been explicitly designed that way:
>
>  "The "atom:content" element either contains or links to the content
> of the entry." -- <
http://greenbytes.de/tech/webdav/rfc4287.html#rfc.section.4.1.3
> while atom:link has not.

I respectfully disagree although you are entitled to your opinion.
Previous discussion on this mailing list [3], which included several
people who were part of the early crowd of atom-syntax, has made it
clear that one should not go by any prejudice about the lack of
specification of a certain previously undefined mark up usage.

>
>
> So I'll stick with my comment that atom:entry is better suited for
> that purpose, being defined as:
>
>   "The "atom:entry" element represents an individual entry, acting
> as a container for metadata and data associated with the entry." -- <
http://greenbytes.de/tech/webdav/rfc4287.html#rfc.section.4.1.2
> >
>
>
>>> As Al pointed out on the CMIS mailing list (<
http://lists.oasis-open.org/archives/cmis/200905/msg00186.html
>>> >), it may be better to use a container element inside atom:entry.
>>>
>>> So, the example in <
http://tools.ietf.org/html/draft-divilly-atom-hierarchy-00#section-3.2.3
>>> >:
>>>
>>>  <atom:entry>
>>>    <atom:link rel="down"
>>>      href="/finance/feeds/default/portfolios/1/positions">
>>>      <atom:feed>
>>>        <atom:link rel="self"
>>>         href="/finance/feeds/default/portfolios/1/positions"/>
>>>         ...
>>>      </atom:feed>
>>>    </atom:link>
>>>    ...
>>>  </atom:entry>
>>>
>>>
> > ...
>
> BTW: the format above is currently forbidden by the RNC (because of
> the use of the atom namespace for the contained element); I know
> that part of the spec is informative, but still...
>

Again, you are misleading the reader to assume that RNC schema is
normative. There is a very good reason for it to not be normative and
the spec editors were clear about them.

Nikunj

[1] http://code.google.com/apis/gdata/elements.html#gdFeedLink
[2] http://code.google.com/apis/gdata/elements.html#gdEntryLink
[3] http://www.imc.org/atom-syntax/mail-archive/msg20456.html

---

From: Peter Keane <[email-masked]>
Date: Tue, 2 Jun 2009 15:35:06 -0500

On Tue, Jun 2, 2009 at 12:58 PM, Al Brown <[email-masked]> wrote:
> morning and see if the group beyond Nikunj, Julian and I have any preference
> for the options below. If this group does not have a preference for any of
> the options, the representation of a tree structure will be removed from the
> hierarchy I-D. This is deemed the best course of action currently.
>
> Due to the reasons already discussed by Julian and myself on this list, CMIS
> will leverage option #2 if it is included in the I-D but probably not option
> #1. #1 does not ideally address CMIS' use case of how to represent a tree of
> atom entries in a single document but rather provide a generic

Hi All-

This option makes little sense to me in that it creates and represents
a completely new type of "set" of atom entries that is in fact NOT a
feed (more akin to a "bag" I suppose)?  What relationship, if any do
these children have to one another?

I assume this is to represent a directory-tree like structure?  If,
so, I believe there are much better ways that do not break the the
essential nature of Atom as documents and lists of documents.  To
quote Roy F. from  a recent thread:

"...Atom categories are analogous to folders in CMIS.
It would make more sense to replace most of the CMIS extensions
by using the Atom category mechanism to hold its filing relations.

....Roy"   [http://www.imc.org/atom-protocol/mail-archive/msg11379.html]

I other words, atom entries live in feeds that may bear no
relationship to physical place on the filesystem.  Those places are
described in atom:category in entries, e.g.:

<entry>
  <category term="/home"/>
</entry>

<entry>
  <category term="/home/peter"/>
</entry>

<entry>
  <category term="/home/peter/docs/>
</entry>

By the way, I have no problem w/ #1 above for inlining a feed.

--peter

> "Nikunj R. Mehta" ---05/28/2009 12:32:53 PM---There is precedence to the
> proposal I have submitted. GData uses
>
>
> From:
> "Nikunj R. Mehta" <[email-masked]>
> To:
> Date:
> 05/28/2009 12:32 PM
> Subject:
> Re: New Version Notification for draft-divilly-atom-hierarchy-00
> ________________________________
>
>
>
>>>> with respect to
>>>> <http://tools.ietf.org/html/draft-divilly-atom-hierarchy-00#section-3
>> of the entry." --
>> <http://greenbytes.de/tech/webdav/rfc4287.html#rfc.section.4.1.3
>> >
>>
>> while atom:link has not.
>
>> as a container for metadata and data associated with the entry." --
>> <http://greenbytes.de/tech/webdav/rfc4287.html#rfc.section.4.1.2
>> >
>>
>>
>>>> As Al pointed out on the CMIS mailing list
>>>> (<http://lists.oasis-open.org/archives/cmis/200905/msg00186.html
>>>> So, the example in
>>>> <http://tools.ietf.org/html/draft-divilly-atom-hierarchy-00#section-3.2.3
>>>> >:
>>>>
>>>>  <atom:entry>
>>>>    <atom:link rel="down"

---

From: "Nikunj R. Mehta" <[email-masked]>
Date: Tue, 2 Jun 2009 13:53:46 -0700

Just to be clear, I have no interest in option #2. In case CMIS ends  
up using that mechanism it would be their choice and in their namespace.

I have provided several reasons why #2 is not a great idea [1] and  
also realize that standardizing option #1 means adding new semantics  
to Atom, i.e., revising Atom compatibly. It is very likely that such a  
thing would be long drawn out but I am in favor of taking that  
approach for the long run. I will continue to edit atompub-hierarchy I- 
D [2] along with Colm Divilly and include the in-lining mechanisms  
there.

Nikunj

[1] CMIS XV: Perils of embedding entry inside entry
[2] http://www.ietf.org/internet-drafts/draft-divilly-atompub-hierarchy-00.txt

On Jun 2, 2009, at 10:58 AM, Al Brown wrote:

> This is to recap a discussion that happened on the cmis atom wg call  
> this morning and see if the group beyond Nikunj, Julian and I have  
> any preference for the options below. If this group does not have a  
> preference for any of the options, the representation of a tree  
> structure will be removed from the hierarchy I-D. This is deemed the  
> best course of action currently.
>
> Due to the reasons already discussed by Julian and myself on this  
> list, CMIS will leverage option #2 if it is included in the I-D but  
> probably not option #1. #1 does not ideally address CMIS' use case  
> of how to represent a tree of atom entries in a single document but  
> rather provide a generic pre-fetching/inlining mechanism.
>
> 2. Using an extension element in atom:entry to contain 0..n  
> atom:entry 's (could also include feed's though CMIS most likely  
> would not leverage that)
> <atom:entry>
> <ah:children>
> If there is not a consensus, then the representation of a tree  
> structure will be removed from the I-D by Nikunj in the next draft.
>
> Thanks,
> -Al
>
> Al Brown
> attachments, are confidential and are intended solely for the use of  
> the person or entity to whom the message was addressed. If you are  
> not the intended recipient of this message, please be advised that  
> any dissemination, distribution, or use of the contents of this  
> message is strictly prohibited. If you received this message in  
> error, please notify the sender. Please also permanently delete all  
> copies of the original message and any attached documentation.
>
> "Nikunj R. Mehta" ---05/28/2009 12:32:53 PM---There is  
> precedence to the proposal I have submitted. GData uses
>
> From:	
> >>> with respect to <http://tools.ietf.org/html/draft-divilly-atom-hierarchy-00#section-3
> > of the entry." -- <http://greenbytes.de/tech/webdav/rfc4287.html#rfc.section.4.1.3
> > >
> >
> > while atom:link has not.
>
> > as a container for metadata and data associated with the entry."  
> -- <http://greenbytes.de/tech/webdav/rfc4287.html#rfc.section.4.1.2
> > >
> >
> >
> >>> As Al pointed out on the CMIS mailing list (<http://lists.oasis-open.org/archives/cmis/200905/msg00186.html
> >>> So, the example in <http://tools.ietf.org/html/draft-divilly-atom-hierarchy-00#section-3.2.3
> >>> >:
> >>>
> >>>  <atom:entry>
> >>>    <atom:link rel="down"

---

From: Al Brown <[email-masked]>
Date: Tue, 2 Jun 2009 14:29:38 -0700

This thread was to see if the group had a preference beyond what was voiced
so far on this list.  This is to influence how CMIS will express a
hierarchy in an atom document.

CMIS exposes folders as: a collection (so items can be added), a feed (it's
direct children), and an atom entry (metadata).  CMIS also needs to expose
a tree if any of those direct descendents are also folders.  On the
choices, since atom expresses metadata (entry) and a set of documents
(feed), how should we proceed to expose a set of sets of documents?

If there is not a consensus on this list, CMIS will do a CMIS-specific
extension and expressing a hierarchy will be removed from the I-D.  This is
what the cmis wg with Nikunj agreed upon this morning.  This is probably
the best course of action at this time.

-Al

Al Brown

Office 714 327 3453
Mobile 714 263 6441

  From:       Peter Keane <[email-masked]>                                                                              
                                                                                                                                
  To:         Al Brown/Costa Mesa/IBM@IBMUS                                                                                     
                                                                                                                                
  Cc:         "Nikunj R. Mehta" <[email-masked]>, Atom-Syntax Syntax <[email-masked]>, Julian Reschke             
              <[email-masked]>, [email-masked], Sam Ruby/Raleigh/IBM@IBMUS, James M                      
              Snell/Fresno/IBM@IBMUS                                                                                            
                                                                                                                                
  Date:       06/02/2009 01:35 PM                                                                                               
                                                                                                                                
  Subject:    Re: Recap - call for preferences/discussion - Re: New Version     Notification for                                
              draft-divilly-atom-hierarchy-00                                                                                   

> morning and see if the group beyond Nikunj, Julian and I have any
preference
> Due to the reasons already discussed by Julian and myself on this list,
CMIS

Hi All-

<entry>
  <category term="/home"/>
</entry>

<entry>

<entry>

--peter

>>>> <

---

From: Al Brown <[email-masked]>
Date: Tue, 2 Jun 2009 14:32:14 -0700

>....Roy"   [http://www.imc.org/atom-protocol/mail-archive/msg11379.html]
Categories are not a good fit for CMIS folders, or another way, a lot will
have to be added to categories for CMIS to use them to represent as
folders.  CMIS requires the following functionality for folders:
1. Typed: Folders have metadata
2. Constrained: Folders can constrain membership to certain CMIS types and
also have run-time restrictions
3. Must be able to retrieve a portion of the folder tree (only things this
folder contains including other folders, to n-levels, or complete) and
include a) only folders, b) folders and objects, c) only objects
4. Easy to add/remove resources (documents) from a folder
5. Clients must be able to create folders of a type with metadata
6. Performance - systems can have 10's of millions of folders; it is
important that clients only work with portions of the folder tree that are
relevant.
7. Query - folders and what they contain must be able to be exposed via
search
8. Also allow vendors that support categorization to expose that
functionality.

All of these capabilities are common across ECM systems.  CMIS is a
specification that is leveraging Atom to expose these capabilities.  Based
on these requirements, categories would have to be extended significantly
and IMHO not as a good a fit as atom:entry.

I'd be happy to buy Roy coffee and walk him through CMIS and it's use of
Atom, APP and HTTP if he's got the time.

-Al

Al Brown

Office 714 327 3453
Mobile 714 263 6441

                                                                                                                                

                                                                                                                                

preference
CMIS

Hi All-

<entry>
  <category term="/home"/>
</entry>

<entry>

<entry>

--peter

---

From: "Nikunj R. Mehta" <[email-masked]>
Date: Tue, 2 Jun 2009 15:03:46 -0700

On Jun 2, 2009, at 10:58 AM, Al Brown wrote:
> #1 does not ideally address CMIS' use case of how to represent a  
> tree of atom entries in a single document but rather provide a  
> generic pre-fetching/inlining mechanism.

This appears rather cavalier to me. But you would not be the first  
person to have committed this mistake.

On Jun 2, 2009, at 2:29 PM, Al Brown wrote:
> how should we proceed to expose a set of sets of documents?

If you research the atom-syntax ML as well blog postts related to  
hierarchy in Atom, you would see Dare Obasanjo griping about AtomPub's  
lack of support for "hierarchical data inlining"  [1] and Microsoft's  
justification for Web3S [2], which spawned a discussion about whether  
Atom could deal with recursive structures such as a hierarchy.  
Microsoft subsequently rolled back their plans and instead switched to  
using Atom as is [3]. Joe Gregorio eloquently refuted claims that Atom  
syntax is inappropriate for hierarchies [4] by, essentially,  
advocating its hypertext mechanisms, which is exactly what the atom- 
hierarchy I-D is proposing [5].

Quoting James Snell [6] about how he dealt with hierarchy in IBM Lotus  
Connections

In reality, it turns out that hierarchy in APP is actually quite  
simple to achieve. The IBM Lotus Connections product includes a  
component called “Activities” which has a very hierarchical data  
structure. Every user has a set of collections, each of which has a  
set of entries, each of which can have any number of comments that can  
be nested to any depth. When I first designed the Atom Publishing  
Protocol API for activities, I created a top level collection of  
Activities. Every entry in this collection represents an Activity.  
Every activity is also a collection. I post entries and comments to  
the activity collection using the standardized Atom Threading  
Extension to provide the additional hierarchy. An Atom Entry can  
represent pretty much anything, including a collection of Atom entries.

At IBM we’ve look at pretty much all of these issues and found simple  
approaches to resolving them that fit perfectly well within the bounds  
of the Atom specs. We’re using APP for a lot of different things, not  
all of which fall neatly into the original use cases APP was intended  
to cover. Yes, we had to give certain aspects some careful thought but  
we generally could not find any showstoppers.

It would be unfortunate if you have not received this advice from  
James directly. It would have spared all of us quite some grief.

If you are right, perhaps the atom-syntax group, Microsoft, IBM, or  
Google would have included your approach in their APIs. I can only  
guess, having hung around here for the last 3 years, that in-lining an  
entire hierarchy was not a good hypertext-based protocol design  
approach, just like HTML does not inline all its CSS, images, JS, and  
embedded objects.

Perhaps all this evidence will make you see why the approach proposed  
in atom-hierarchy I-D is suitable for what you are trying to achieve.  
But perhaps you are right in what you are doing. We'll leave it for  
the future to decide.

Nikunj

[1] http://www.25hoursaday.com/weblog/2007/06/09/WhyGDataAPPFailsAsAGeneralPurposeEditingProtocolForTheWeb.aspx
[2] http://www.25hoursaday.com/weblog/CommentView.aspx?guid=020611C4-F53C-4DF0-8DB0-9A050DD6CD9C
[3] http://www.25hoursaday.com/weblog/2008/02/28/WindowsLivePlatformNewsMicrosoftStandardizesOnAtomPubForWebServicesAndOtherStories.aspx
[4] http://bitworking.org/news/197/In-which-we-narrowly-save-Dare-from-inventing-his-own-publishing-protocol
[5] http://www.ietf.org/internet-drafts/draft-divilly-atom-hierarchy-00.txt
[6] http://www.snellspace.com/wp/?p=681

---

From: Bill de hOra <[email-masked]>
Date: Tue, 02 Jun 2009 23:17:09 +0100

Al Brown wrote:
> Categories are not a good fit for CMIS folders, or another way, a lot 
> will have to be added to categories for CMIS to use them to represent as 
> 2. Constrained: Folders can constrain membership to certain CMIS types 
> and also have run-time restrictions
> 3. Must be able to retrieve a portion of the folder tree (only things 
> this folder contains including other folders, to n-levels, or complete) 
> and include a) only folders, b) folders and objects, c) only objects
> important that clients only work with portions of the folder tree that 
> are relevant.
> All of these capabilities are common across ECM systems. 

I think it's important to understand the containment semantics before 
making a syntax decision, specifically whether

- a child folder/file can exist without a parent folder.

- a folder/file can be referenced across folders.

- children can have ACLs (and so do/don't require read-ahead for viewing)

Bill

CMIS is a
> specification that is leveraging Atom to expose these capabilities. 
> Based on these requirements, categories would have to be extended 
> significantly and IMHO not as a good a fit as atom:entry.
> 
> strictly prohibited. If you received this message in error, please 
> notify the sender. Please also permanently delete all copies of the 
> original message and any attached documentation.
> 
> Inactive hide details for Peter Keane ---06/02/2009 01:35:23 PM---On 
> Tue, Jun 2, 2009 at 12:58 PM, Al Brown <[email-masked] Keane 
> ---06/02/2009 01:35:23 PM---On Tue, Jun 2, 2009 at 12:58 PM, Al Brown 
> <[email-masked]> wrote:
> 
> 
> From:	
> Peter Keane <[email-masked]>
> 
> To:	
> Cc:	
> "Nikunj R. Mehta" <[email-masked]>, Atom-Syntax Syntax 
> <[email-masked]>, Julian Reschke <[email-masked]>, 
> [email-masked], Sam Ruby/Raleigh/IBM@IBMUS, James M 
> Snell/Fresno/IBM@IBMUS
> 
> Date:	
> 06/02/2009 01:35 PM
> 
> Subject:	
> Re: Recap - call for preferences/discussion - Re: New Version 
> Notification for draft-divilly-atom-hierarchy-00
> 
> ------------------------------------------------------------------------
> 
> 
> 
> list, CMIS

---

From: Al Brown <[email-masked]>
Date: Tue, 2 Jun 2009 16:27:29 -0700

>making a syntax decision, specifically whether
Good questions.  I've answered in the context of CMIS.

>- a child folder/file can exist without a parent folder.
CMIS supports repositories where a document (file) may be in 0..n folders,
1..n, 0..1, or 1 folder.  Typically though, it will be 0..n or 1.  Folders
in CMIS must be in 1 with the exception of the root folder.

>- a folder/file can be referenced across folders.
See above.  Documents (file) can be in more than one folder

>- children can have ACLs (and so do/don't require read-ahead for viewing)
All objects in CMIS (documents, folders, relationships) may have ACLs. They
may also be other security mechanisms in the underlying repository
including security propagation such as by folder inheritance.

-Al

Al Brown

Office 714 327 3453
Mobile 714 263 6441

  From:       Bill de hOra <[email-masked]>                                                                                    

  Cc:         Peter Keane <[email-masked]>, Atom-Syntax Syntax <[email-masked]>, James M Snell/Fresno/IBM@IBMUS,   
              Julian Reschke <[email-masked]>, "Nikunj R. Mehta" <[email-masked]>,                              
              [email-masked], [email-masked], Sam Ruby/Raleigh/IBM@IBMUS                                     
                                                                                                                                
  Date:       06/02/2009 03:18 PM                                                                                               
                                                                                                                                
  Subject:    Re: CMIS Folders & categories                                                                                     

Bill

CMIS is a
> Notification for draft-divilly-atom-hierarchy-00
this
>  > If there is not a consensus, then the representation of a tree
structure
>  > attachments, are confidential and are intended solely for the use of
the
>  > strictly prohibited. If you received this message in error, please
notify
>  > Atom-Syntax Syntax <[email-masked]>
>  > Date:
>  > 05/28/2009 12:32 PM
>  > Subject:

---

From: Al Brown <[email-masked]>
Date: Tue, 2 Jun 2009 17:19:09 -0700

The main reason we need trees in this protocol is the ability to retrieve
the tree in 1 round-trip.  If we use N round-trips, entries and feeds are
sufficient per James' info below.  Almost all ECM systems support this
capability to retrieve a tree and it is important to the TC to represent
it.  Folder navigation is a large part of the API set clients use.  All the
vendors have learned the hard way that it is very important to have folder
navigation perform well. Returning a tree was the common optimization most
vendors provided. If it was an option to not expose, the cmis tc, would not
have gone through the effort to date.

Also, AFAIK, nobody has tried to support the ECM use-cases with Atom and
APP before CMIS.  CMIS is the first.  All the advice we have gotten, from
all the sources, we have adopted and worked through.  Why wouldn't CMIS
want to be a good atom citizen?

In the future, please keep the tone non-combative and focused on technical
aspects. I'd like to think the atom community as welcoming and supportive.

Best, -Al

Al Brown

Office 714 327 3453
Mobile 714 263 6441

                                                                                                                                
                                                                                                                                
  Cc:         Peter Keane <[email-masked]>, Atom-Syntax Syntax <[email-masked]>, James M Snell/Fresno/IBM@IBMUS,   
              Julian Reschke <[email-masked]>, [email-masked], [email-masked], Sam                    
              Ruby/Raleigh/IBM@IBMUS                                                                                            
                                                                                                                                
  Date:       06/02/2009 03:31 PM                                                                                               
                                                                                                                                
  Subject:    Re: Recap - call for preferences/discussion - Re: New Version Notification for draft-divilly-atom-hierarchy-00    

On Jun 2, 2009, at 10:58 AM, Al Brown wrote:
      #1 does not ideally address CMIS' use case of how to represent a tree
      of atom entries in a single document but rather provide a generic
      pre-fetching/inlining mechanism.

This appears rather cavalier to me. But you would not be the first person
to have committed this mistake.

If you research the atom-syntax ML as well blog postts related to hierarchy
in Atom, you would see Dare Obasanjo griping about AtomPub's lack of
support for "hierarchical data inlining"  [1] and Microsoft's justification
for Web3S [2], which spawned a discussion about whether Atom could deal
with recursive structures such as a hierarchy. Microsoft subsequently
rolled back their plans and instead switched to using Atom as is [3]. Joe
Gregorio eloquently refuted claims that Atom syntax is inappropriate for
hierarchies [4] by, essentially, advocating its hypertext mechanisms, which
is exactly what the atom-hierarchy I-D is proposing [5].

     represent pretty much anything, including a collection of Atom
     entries.

It would be unfortunate if you have not received this advice from James
directly. It would have spared all of us quite some grief.

If you are right, perhaps the atom-syntax group, Microsoft, IBM, or Google
would have included your approach in their APIs. I can only guess, having
hung around here for the last 3 years, that in-lining an entire hierarchy
was not a good hypertext-based protocol design approach, just like HTML
does not inline all its CSS, images, JS, and embedded objects.

Perhaps all this evidence will make you see why the approach proposed in
atom-hierarchy I-D is suitable for what you are trying to achieve. But
perhaps you are right in what you are doing. We'll leave it for the future
to decide.

Nikunj

[1]
http://www.25hoursaday.com/weblog/2007/06/09/WhyGDataAPPFailsAsAGeneralPurposeEditingProtocolForTheWeb.aspx
[2]
http://www.25hoursaday.com/weblog/CommentView.aspx?guid=020611C4-F53C-4DF0-8DB0-9A050DD6CD9C

[3]
http://www.25hoursaday.com/weblog/2008/02/28/WindowsLivePlatformNewsMicrosoftStandardizesOnAtomPubForWebServicesAndOtherStories.aspx
[4]
http://bitworking.org/news/197/In-which-we-narrowly-save-Dare-from-inventing-his-own-publishing-protocol
[5] http://www.ietf.org/internet-drafts/draft-divilly-atom-hierarchy-00.txt
[6] http://www.snellspace.com/wp/?p=681

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 3 Jun 2009 11:28:05 +1000

> something is syntactically legal doesn't make it a good choice.

+1 - the clearest way to communicate what's going on here is to use a  
new child element.

Assuming that the contents of the link element are inlined content are  
adding an extension without explicitly identifying it; this may  
conflict with future uses. There isn't a way for an Atom processor to  
inspect a link element and know that the content is inlined; they have  
to guess that this specification is in effect, therefore the link  
content is the inlined content.  This isn't good practice.

--
Mark Nottingham     http://www.mnot.net/

---

From: Peter Keane <[email-masked]>
Date: Tue, 2 Jun 2009 20:31:29 -0500

Hi Al-

[apologies if threading on this discussion is messed up -- I seem to
have it in 3 thread]

I'd contend that atom:category is exactly the right mechanism for the
functionality you've listed (I'll explain below).  The same and/or
related issues have come up again and again in regards to Atom, and
I'd suggest that there is a real need for a few "practices" (I've
learned not to prefix that w/ "best" :)) that might facilite these
uses.

First, atom:category can serve as a typing mechanism for atom entires
& documents.  That typing can take many forms, and the
atom:category@scheme give you a namespace within which to create
types.   This from RFC 5023, 8.3.6: "The server MAY reject attempts to
create or store members whose categories are not present in its
categories list." I take to mean that categories can offer another
means (aside from app:accept) to restrict the type of documents that
can be added to a collection.

Second, standard querying mechanisms are a must (we don't yet have one
[1]) but they are essential to provide links to virtual feeds.  I see
no reason why the category querying mechanism in GData [2] wouldn't
work just fine.

On Tue, Jun 2, 2009 at 4:32 PM, Al Brown <[email-masked]> wrote:
> have to be added to categories for CMIS to use them to represent as folders.
> CMIS requires the following functionality for folders:

> 1. Typed: Folders have metadata

Yes, folders should be represented by atom:entries.  Use atom:category
w/ @term=folder and a scheme that identifies  the filetypes namespace
i.e., @scheme="http://.../cmis.org/category/filetypes

in addition, every atom entry for a file (directory OR regular file)
has an atom:category that describes its virtual "place" in the
hierarchy.  Per your message re cardinality, this category is optional
(ie., NOT in a directory) and can have one or more instances (covering
1...n). e.g.:
<entry>
  <atom:category scheme="http://cmis.org/category/filepath"
term="/home/keane/docs"/>
  <atom:category scheme="http://cmis.org/category/filepath"
term="/public/docs"/>
</entry>

> also have run-time restrictions

Every entry of that is a "folder" type also has an
atom:link@rel=service, the href of which points to the service doc
defining the atompub collection for this folder.  I.e., to add a file
to this folder, post an atom:entry to the collection defined in that
service doc.  Posted entires will either be required to have or will
be assign a category indicating that they are in this directory.
Moving files around (or creating these "symlinks" to other location is
a matter of GET/EDIT/PUT of the atom:entry representing the file or
directory (and adding/deleting atom:categories)

> include a) only folders, b) folders and objects, c) only objects

This needs to be based on a query syntax.  I could see a query of
search?category=[@scheme=filetype]/home/pkeane/** getting me all docs
recursively, or search?category=[@scheme=filetype]/home/pkeane/*
getting me one level deep, etc.  (obviously, a well-defined query
syntax would need to be worked out.

> 4. Easy to add/remove resources (documents) from a folder

Since every folder is represented by an atom:entry that is "backed" by
a collection, collecton semantics apply. (find the atom:entry
representing the file and http DELETE the edit url.

> 5. Clients must be able to create folders of a type with metadata

Use atom:category for typing. collections can define the categories possible.

> important that clients only work with portions of the folder tree that are

Again, an appropriately sophisticated query syntax should allow this
w/o too much trouble.

> 7. Query - folders and what they contain must be able to be exposed via

Query again...

> 8. Also allow vendors that support categorization to expose that

Atom:category is extensible using @scheme "namespaces"

[1] http://dret.typepad.com/dretblog/2009/04/feeds-as-query-result-serializations.html
[2] http://code.google.com/apis/gdata/docs/2.0/reference.html#QueryRequests

--peter keane

> specification that is leveraging Atom to expose these capabilities. Based on
> these requirements, categories would have to be extended significantly and
> IMHO not as a good a fit as atom:entry.
>

[..snipped...]

---

From: Peter Keane <[email-masked]>
Date: Tue, 2 Jun 2009 22:31:58 -0500

On Tue, Jun 2, 2009 at 8:28 PM, Mark Nottingham <[email-masked]> wrote:
> +1 - the clearest way to communicate what's going on here is to use a new
> child element.
>
> adding an extension without explicitly identifying it; this may conflict
> with future uses. There isn't a way for an Atom processor to inspect a link
> element and know that the content is inlined; they have to guess that this
> specification is in effect, therefore the link content is the inlined
> content.  This isn't good practice.
>

This strikes me as a sound reasoning against using atom:link content
for embedded feeds. Since the idea was partly an attempt to
standardize a common use (Google's feedLink), I'm wondering why not
simply use Google's extension (Google folks have stated many times
they'd be happy to see their protocol extensions used).  Certainly,
Yahoo's media extensions have caught on widely.

--peter keane

---

From: Mark Nottingham <[email-masked]>
Date: Wed, 3 Jun 2009 13:37:18 +1000

I'm a little wary of just using a proprietary extension -- no matter  
how well-intentioned the authors -- because change control can become  
an issue down the road. I think the media extensions are a pretty  
relevant example here.

Cheers,

On 03/06/2009, at 1:31 PM, Peter Keane wrote:

>> conflict

---

From: Peter Keane <[email-masked]>
Date: Tue, 2 Jun 2009 22:51:44 -0500

On Tue, Jun 2, 2009 at 10:37 PM, Mark Nottingham <[email-masked]> wrote:
> I'm a little wary of just using a proprietary extension -- no matter how
> well-intentioned the authors -- because change control can become an issue
> down the road. I think the media extensions are a pretty relevant example
> here.
>

Agreed -- Yahoo's stewardship of MediaRSS has been a bit rocky.  The
Google extensions would need to go through a standards process, I'd
think.  See [1] [2]

[1] http://twitter.com/dewitt/status/1781073886
[2] http://twitter.com/dewitt/status/1781081467

--peter

---

From: "Nikunj R. Mehta" <[email-masked]>
Date: Wed, 3 Jun 2009 08:44:46 -0700

On Jun 2, 2009, at 6:28 PM, Mark Nottingham wrote:

> +1 - the clearest way to communicate what's going on here is to use  
> a new child element.
>

Did you read the I-D being discussed? If not, I would encourage you to  
do so. The I-D is specifying a meaning for an atom:entry or atom:feed  
found inside atom:link. If this is not how Atom can be extended, then  
pray tell me how.

> Assuming that the contents of the link element are inlined content  
> are adding an extension without explicitly identifying it; this may  
> conflict with future uses.

Our proposal is /the/ future use, so I don't see how it can conflict  
with future uses. It is our intention to promote an extension of Atom.  
By submitting the I-D to the IETF and by bringing this discussion to  
atom-syntax, we have made the intention quite clear, don't you agree?

It is a different story if Atom cannot be extended as we wish. May be  
it would be useful if you or others who claim that our approach is  
wrong can explain what is the process for extending Atom. Is it  
creating a brand new working group?

> There isn't a way for an Atom processor to inspect a link element  
> and know that the content is inlined; they have to guess that this  
> specification is in effect, therefore the link content is the  
> inlined content.

The I-D is proposing a backwards compatible extension of Atom. An old  
Atom processor would just ignore that in-lined content. Only new Atom  
processors would therefore be able to process the in-lined content.

> This isn't good practice.

Why so?

---

From: Julian Reschke <[email-masked]>
Date: Wed, 03 Jun 2009 18:15:03 +0200

> atom-syntax, we have made the intention quite clear, don't you agree?

I think the point is that this is not an extension point for general 
use; thus if it is to be used it would need to be done by a spec that's 
on the standards track, and updates RFC4287. For that you'll likely need 
a WG to reach the consensus that *this* is the way to go.

> It is a different story if Atom cannot be extended as we wish. May be it 
> would be useful if you or others who claim that our approach is wrong 
> can explain what is the process for extending Atom. Is it creating a 
> brand new working group?

The Atom format has extension points that allow distributed 
extensibility, but the content of atom:link isn't one of them, as far as 
I can tell.

BR, Julian

---

From: "Nikunj R. Mehta" <[email-masked]>
Date: Wed, 3 Jun 2009 09:30:23 -0700

On Jun 2, 2009, at 5:19 PM, Al Brown wrote:
> The main reason we need trees in this protocol is the ability to  
> retrieve the tree in 1 round-trip.
>
The atom-hierarchy I-D meets this requirement as demonstrated in the I- 
D text.
> Why wouldn't CMIS want to be a good atom citizen?
>
The burden of proof is not on me. And besides, we have been through  
this with a big software company before that was trying to upstage  
AtomPub by simply spreading mis-truths.

> In the future, please keep the tone non-combative and focused on  
> technical aspects. I'd like to think the atom community as welcoming  
> and supportive.
>
I'd also like to think of the CMIS TC and its spec editors as open to  
honest discussions based on technical merits. It doesn't help to  
systematically mislead the community as has been the case. Let me cite  
some instances.

On May 27, 2009, at 5:59 AM, Al Brown wrote:

> I have an atom:entry in a hierarchy.  It has down and down-tree.   
> The document also wants to display a heirarchy.
>
> If I use link to show the hierarchy, I will have duplicate resources  
> in both links - one level in down and one to n levels in down-tree.
>
> If the hierarchy is outside link and in atom:entry, then there is no  
> duplication.

I pointed out how this is a fatally flawed conclusion [1].

On May 27, 2009, at 8:36 AM, Al Brown wrote:

> I, personally, think there is an alternative  that does not have the  
> semantic conflict Julian mentioned nor the performance / caching  
> issues with prefetching the link and including.

There are none. A client can use this information as an optimization,  
not as the exact representation, just like atom:entry inside atom:feed.

> generic pre-fetching/inlining mechanism.

I refuted this claim above. The use case is simple enough to  be  
addressed in many different ways, including atom-hierarchy-ID.

On Jun 2, 2009, at 5:19 PM, Al Brown wrote:
> Also, AFAIK, nobody has tried to support the ECM use-cases with Atom  
> and APP before CMIS. CMIS is the first. All the advice we have  
> gotten, from all the sources, we have adopted and worked through.
>
You seem to be uninterested in investigating other's experiences in  
dealing with hierarchy and in-lining. Please read up on Astoria or  
ADO.NET Data Services [2]. I quote from the referenced article

Using AtomPub constructs and extensibility mechanisms to enable  
Astoria features:
Inline expansion of links (“GET a given entry and all the entries  
related through this named link”, how we represent a request and the  
answer to such a request in Atom?).
The exact mechanism for in-lining expanded links is exactly as  
specified in atom-hierarchy-ID [3].

Nikunj

[1] http://www.imc.org/atom-syntax/mail-archive/msg21041.html
[2] http://carnage4life.spaces.live.com/Blog/cns!616444EE7A34F417!2324.entry
[3] http://blogs.msdn.com/astoriateam/archive/2008/02/18/related-entries-and-feeds-links-and-link-expansion.aspx

---

From: "Nikunj R. Mehta" <[email-masked]>
Date: Wed, 3 Jun 2009 09:37:45 -0700

On Jun 3, 2009, at 9:15 AM, Julian Reschke wrote:

>>> are adding an extension without explicitly identifying it; this  
>>> may conflict with future uses.
>> Our proposal is /the/ future use, so I don't see how it can  
>> conflict with future uses. It is our intention to promote an  
>> extension of Atom. By submitting the I-D to the IETF and by  
>> bringing this discussion to atom-syntax, we have made the intention  
>> quite clear, don't you agree?
>> ...
>
> use; thus if it is to be used it would need to be done by a spec  
> that's on the standards track, and updates RFC4287. For that you'll  
> likely need a WG to reach the consensus that *this* is the way to go.

The IETF AD for Applications has already suggested that she would be  
supportive of a new WG to look at this issue. Frankly, IETF needs to  
look at the issue of hierarchical representation in Atom. Frankly,  
there is a lot of experience in dealing with hierarchy in Atom/ 
AtomPub. It would be futile to think otherwise. As examples, take a  
look at Google's GData APIs and Microsoft's ADO.NET use of Atom/AtomPub.

I was dissuaded from submitting the atompub-hierarchy I-D along  
standards track, but it looks like the right way to proceed. Of  
course, that also means getting together a new WG. Would you be  
willing to help form such a WG?

>> It is a different story if Atom cannot be extended as we wish. May  
>> be it would be useful if you or others who claim that our approach  
>> is wrong can explain what is the process for extending Atom. Is it  
> extensibility, but the content of atom:link isn't one of them, as  
> far as I can tell.

It was never the intention of atom-hierarchy I-D to perform  
independent extension of Atom. Therefore, I don't see this as a problem.

Nikunj

---

From: "Nikunj R. Mehta" <[email-masked]>
Date: Wed, 3 Jun 2009 09:43:06 -0700

On Jun 2, 2009, at 6:28 PM, Mark Nottingham wrote:

> conflict with future uses. There isn't a way for an Atom processor  
> to inspect a link element and know that the content is inlined; they  
> have to guess that this specification is in effect, therefore the  
> link content is the inlined content.  This isn't good practice.

Just FYI, Joe Gregorio and by implication Google supports directly  
embedding atom:content inside atom:link. See the last comment on [1].  
I don't know what we mean by practice here, but that is exactly what  
is going on in lots of places.

Nikunj

[1] http://blogs.msdn.com/astoriateam/archive/2008/02/18/related-entries-and-feeds-links-and-link-expansion.aspx#8573352

---

From: Julian Reschke <[email-masked]>
Date: Wed, 03 Jun 2009 18:58:34 +0200

Nikunj R. Mehta wrote:
> standards track, but it looks like the right way to proceed. Of course, 
> that also means getting together a new WG. Would you be willing to help 
> form such a WG?
> ...

I would certainly join it and contribute, but I'd probably not be able 
to help with *forming* it (which, in the IETF, is a quite expensive -- 
time-wise -- process).

BR, Julian

---

From: Al Brown <[email-masked]>
Date: Wed, 3 Jun 2009 10:10:05 -0700

honest discussions based on technical merits. It doesn't help to
systematically mislead the >community as has been the case.
Please coordinate with Ryan and the rest of the Oracle standards team on
CMIS.  Knowing them, they will do quite well with your feedback and make
sure the TC acts on it.  They are a very talented and responsive team.

Best, -Al
Al Brown

Office 714 327 3453
Mobile 714 263 6441

                                                                                                                                
                                                                                                                                
  Cc:         Atom-Syntax Syntax <[email-masked]>, James M Snell/Fresno/IBM@IBMUS, Julian Reschke <[email-masked]>, 
              [email-masked], Peter Keane <[email-masked]>, Sam Ruby/Raleigh/IBM@IBMUS, Ryan Mcveigh                 
              <[email-masked]>                                                                                         
                                                                                                                                
  Date:       06/03/2009 09:51 AM                                                                                               

The atom-hierarchy I-D meets this requirement as demonstrated in the I-D
text.

The burden of proof is not on me. And besides, we have been through this
with a big software company before that was trying to upstage AtomPub by
simply spreading mis-truths.

I'd also like to think of the CMIS TC and its spec editors as open to
honest discussions based on technical merits. It doesn't help to
systematically mislead the community as has been the case. Let me cite some
instances.

      I have an atom:entry in a hierarchy.  It has down and down-tree.  The
      document also wants to display a heirarchy.

There are none. A client can use this information as an optimization, not
as the exact representation, just like atom:entry inside atom:feed.

I refuted this claim above. The use case is simple enough to  be addressed
in many different ways, including atom-hierarchy-ID.

You seem to be uninterested in investigating other's experiences in dealing
with hierarchy and in-lining. Please read up on Astoria or ADO.NET Data
Services [2]. I quote from the referenced article

The exact mechanism for in-lining expanded links is exactly as specified in
atom-hierarchy-ID [3].

Nikunj

[1] http://www.imc.org/atom-syntax/mail-archive/msg21041.html
[2]
http://carnage4life.spaces.live.com/Blog/cns!616444EE7A34F417!2324.entry
[3]
http://blogs.msdn.com/astoriateam/archive/2008/02/18/related-entries-and-feeds-links-and-link-expansion.aspx

---

From: Pablo Castro <[email-masked]>
Date: Wed, 3 Jun 2009 15:30:27 -0700

Sorry for coming late to the thread, somebody forwarded me this and I thought I'd add a couple of comments from the Astoria (ADO.NET Data Services) side.

We have a similar need in Astoria to include inline content. This is not for hierarchies, but more in general because the Astoria data model consists basically of entities (mapped to entries) and associations (mapped to links to entries or links to feeds depending on the cardinality of the association-end). We needed to allow clients to request a given entry and pre-fetch related entries (this is mostly a round-trip optimization to help with latency, but it also results in a couple of extra features in Astoria).

The link that Nikunj included below describes the reasoning in more detail:
http://blogs.msdn.com/astoriateam/archive/2008/02/18/related-entries-and-feeds-links-and-link-expansion.aspx

We interpreted the Atom spec as saying that while the spec itself didn't define any meaning for content inside the link element, it didn't prohibit either. In order to avoid future conflicts with Atom elements inside link, we ended up putting an <inline> element in our own namespace immediately under link, and then an Atom entry or feed in it. If the link points to something that hasn't been created yet or that the user can't see due to security reasons, then we still include the inline element, but it's empty. That way a client processor can know that the link is expanded but there is no actual resource at the other end of it.

Expanding the entry/feed inside the link element means that if we have more than one expanded link we don't need to add any indication of what entry extension element corresponds to what link, which we would need if we included the inlined content as a peer of the link element instead of as a child.

It's also easy for client parsers. We parse the link as usual (extract url and such) and then if we see an inner element inline in our namespace then we know the link was inlined. 

There is of course the question of the risk of pulling down a giant graph because of a client asking to expand too much. The most common form of this issue is expanding long feeds inline inside another entry. We actually use the usual Atom paging constructs even in inlined feeds, so the server is free to include a few entries and then a "next" link where the client can get more. This allows for a good balance between low-latency first fetch to get and display data and progressive retrieval of more data as needed.

We had a discussion about the topic of inline expansion some time ago in this list also:
http://www.imc.org/atom-syntax/mail-archive/msg20444.html

-pablo

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of Nikunj R. Mehta
Sent: Wednesday, June 03, 2009 9:43 AM
To: Mark Nottingham
Cc: Julian Reschke; Atom-Syntax Syntax
Subject: Re: New Version Notification for draft-divilly-atom-hierarchy-00

Nikunj

---

From: Mark Nottingham <[email-masked]>
Date: Thu, 4 Jun 2009 11:32:07 +1000

As Julian pointed out, you'll need an IETF WG to update or obsolete  
the Atom syntax specification to make this legitimate. The bar is much  
lower for a "traditional" extension.

None of this is your fault; the extension model in Atom isn't terribly  
well-specified.

Regardless of how you serialise the hierarchy, at some point the  
document you're going to end up with will not be very useful to a  
generic Atom processor (as deployed today), because the majority of  
the information is in extensions. When that happens, it's worth  
considering minting a new media type to identify the document, so that  
people can differentiate them.

This also gives an opportunity to put in fixes that would otherwise be  
backwards-incompatible, and in general tighten things up. IMO, there  
are a number of things that needs to be cleaned up in Atom, especially  
for the non-blog use case.

Just food for thought,

On 04/06/2009, at 2:43 AM, Nikunj R. Mehta wrote:

>>> I do not agree with that conclusion, but nevertheless, just  
>>> because something is syntactically legal doesn't make it a good  
>>> choice.
>>
>> to inspect a link element and know that the content is inlined;  
>> they have to guess that this specification is in effect, therefore  
>> the link content is the inlined content.  This isn't good practice.
>
> embedding atom:content inside atom:link. See the last comment on  
> [1]. I don't know what we mean by practice here, but that is exactly  
> what is going on in lots of places.
>
> Nikunj

---

From: Jan Algermissen <[email-masked]>
Date: Wed, 03 Jun 2009 22:57:08 -0400

On Jun 3, 2009, at 9:32 PM, Mark Nottingham wrote:
>
> This also gives an opportunity to put in fixes that would otherwise  
> be backwards-incompatible, and in general tighten things up. IMO,  
> there are a number of things that needs to be cleaned up in Atom,  
> especially for the non-blog use case.
>
> Just food for thought,

What do you think needs to be cleaned up? We could use this as an  
informal list of items that coud at some point form the basis for a  
new media type effort.

I have used Atom in 2 1/2 non blogging projects and felt at times that  
some things are 'not right' but not to a point that I could nail them  
down. Here is a very rough sketch of issues I had (these are AtomPub  
issues actually):

- underspecification of the <workspace> element. It is grouping  
collections but what is the significance of such groups?
- missing built-in indexing/querying forms in <collection> element
- missing support for 'typing' collections so clients can discover  
them based on capability (a start was James' features draft)
- missing native support for hierarchies
- missing native support for POE functionality (though having this  
orthogonal to AtomPub is propably preferable)
- missing native support for processing status/proression information  
when returning a 202 Accepted

Also, just food for thought.

Jan

>>> +1 - the clearest way to communicate what's going on here is to  
>>> use a new child element.
>>>
>>> may conflict with future uses. There isn't a way for an Atom  
>>> processor to inspect a link element and know that the content is  
>>> inlined; they have to guess that this specification is in effect,  
>>> therefore the link content is the inlined content.  This isn't  
>>> good practice.
>>
>> [1]. I don't know what we mean by practice here, but that is  
>> exactly what is going on in lots of places.
>>
>> Nikunj

---

From: Al Brown <[email-masked]>
Date: Wed, 3 Jun 2009 20:19:34 -0700

Interesting.  There's a solution that seems reasonable and is similar to
what google did with feedLink[1] and you still get generic inlining.  This
also uses the traditional extension mechanism and does not, I believe, have
the issues raised by Mark, Julian or I.  This also could be done in an I-D
without requiring a new WG.

Proposal:
1. New element, let's call it <ah:inline>
2. It supports the following attributes: rel (same as link rel; required);
href (optional); etag (optional)
3. inline rel has the same semantics and constraints as link rel
4. if href is not specified, then it corresponds to the link with the same
relation.  If href is specified, it corresponds to the link with the same
relation and href.
5. ah:inline supports any markup - atom, app, or other
6. ah:inline can be used in atom:entry or atom:feed and can have a
cardinality 0..n

In the example you posted, it would be:

<atom:entry>
...
>   <link rel="related" type="application/atom+xml;type=entry"
title="Venue" href="Events(456)/Venue">
>   </link>
>	...
>     <ah:inline rel="related" href="Events(456)/Venue">
>       <entry m:type="EventsSample.Venue">
>         <id>http://localhost:81/EventsSample/Venues(789)</id>
>         <title type="text"></title>
>         <updated>2008-02-17T03:01:18Z</updated>
>         <author>
>           <name />
>         </author>
>         <link rel="edit" title="Venue" href="Venues(789)" />
>         <link rel="related" type="application/atom+xml;type=entry"
title="SalesContact" href="Venues(789)/SalesContact" />
>         <content type="application/xml">
>           <d:VenueID m:type="Int32">789</d:VenueID>
>           <d:Name>The Cool Place</d:Name>
>           <d:Description>Great place for parties!</d:Description>
>           <d:Capacity m:type="Int32">1500</d:Capacity>
>           <d:Type>Nightclub</d:Type>
>         </content>
>       </entry>
>     </ah:inline>
</atom:entry>

CMIS would then use it this way:
<atom:entry>
      ...
      <link rel="down" href="/res1/down" type="application/atom
+xml;type=feed" />
      <link rel="down-tree" href="/res1/downtree" type="application/atom
+xml;type=feed" />
      ...
      <ah:inline rel="down">
            <atom:feed>
                  <atom:entry >...</atom:entry>
                  ...
            </atom:feed>
      </ah:inline>
</atom:entry>

Al Brown

Office 714 327 3453
Mobile 714 263 6441

  From:       Pablo Castro <[email-masked]>                                                                         
                                                                                                                                
  To:         "Nikunj R. Mehta" <[email-masked]>, Mark Nottingham <[email-masked]>                                      
                                                                                                                                
  Cc:         Julian Reschke <[email-masked]>, Atom-Syntax Syntax <[email-masked]>                                  
                                                                                                                                
  Date:       06/03/2009 03:55 PM                                                                                               
                                                                                                                                
  Subject:    RE: New Version Notification for draft-divilly-atom-hierarchy-00                                                  

Sorry for coming late to the thread, somebody forwarded me this and I
thought I'd add a couple of comments from the Astoria (ADO.NET Data
Services) side.

We have a similar need in Astoria to include inline content. This is not
for hierarchies, but more in general because the Astoria data model
consists basically of entities (mapped to entries) and associations (mapped
to links to entries or links to feeds depending on the cardinality of the
association-end). We needed to allow clients to request a given entry and
pre-fetch related entries (this is mostly a round-trip optimization to help
with latency, but it also results in a couple of extra features in
Astoria).

We interpreted the Atom spec as saying that while the spec itself didn't
define any meaning for content inside the link element, it didn't prohibit
either. In order to avoid future conflicts with Atom elements inside link,
we ended up putting an <inline> element in our own namespace immediately
under link, and then an Atom entry or feed in it. If the link points to
something that hasn't been created yet or that the user can't see due to
security reasons, then we still include the inline element, but it's empty.
That way a client processor can know that the link is expanded but there is
no actual resource at the other end of it.

Expanding the entry/feed inside the link element means that if we have more
than one expanded link we don't need to add any indication of what entry
extension element corresponds to what link, which we would need if we
included the inlined content as a peer of the link element instead of as a
child.

It's also easy for client parsers. We parse the link as usual (extract url
and such) and then if we see an inner element inline in our namespace then
we know the link was inlined.

There is of course the question of the risk of pulling down a giant graph
because of a client asking to expand too much. The most common form of this
issue is expanding long feeds inline inside another entry. We actually use
the usual Atom paging constructs even in inlined feeds, so the server is
free to include a few entries and then a "next" link where the client can
get more. This allows for a good balance between low-latency first fetch to
get and display data and progressive retrieval of more data as needed.

We had a discussion about the topic of inline expansion some time ago in
this list also:
http://www.imc.org/atom-syntax/mail-archive/msg20444.html

-pablo

-----Original Message-----
From: [email-masked] [mailto:[email-masked]
] On Behalf Of Nikunj R. Mehta
Sent: Wednesday, June 03, 2009 9:43 AM
To: Mark Nottingham
Cc: Julian Reschke; Atom-Syntax Syntax
Subject: Re: New Version Notification for draft-divilly-atom-hierarchy-00

Nikunj

[1]
http://blogs.msdn.com/astoriateam/archive/2008/02/18/related-entries-and-feeds-links-and-link-expansion.aspx#8573352

---

From: Kyle Marvin <[email-masked]>
Date: Thu, 4 Jun 2009 04:29:13 -0700

FWIW, we have been looking at revising our strategy for inlining entity
references and have reached largely the same conclusion that you have
reached.   The ideal model is not some new extension type (like
gd:entryLink/gd:feedLink) but rather a model that lets you nest the content
as an extension directly within a link element.   This way, clients who are
unaware of the inlining extensions can still process it as a regular
reference (via the link) and follow it, albeit less efficiently than if they
just used the inlined content.    I agree with comments on this thread that
if data is inlined inside of link, it is better if it's done inside of an
extension that clearly marks the content as being the inlined data and
avoids collisions with any any future link extensions.
I'm not as enthusiastic about the idea of defining some inlining syntax that
is a peer of the atom:link as was suggested earlier (just put an entry
inside an entry) or later (ah:inline) on this thread.   You then lose or
must duplicate the context (the "rel") that tells you the relationship of
the contained entity to the containing resource.  You may even lose the
ability to know how to deference the inlined resource independently;  there
are potential use cases where you might inline an XML entity that doesn't
have the equivalent of a 'self' link.    It also means you have to
redundantly specify the content type of the inlined data to inform
processors when this information is may already available via
atom:link/@type.

What seems apparent from this thread is that multiple existing large-scale
implementors leveraging the Atom Syntax are seeing a need for this and seem
to be converging on a roughly similar solution.   As such, there's probably
significant benefit on proposing an extension that makes this unambiguous
and consistent across implementations so Atom clients can rely on it.   We'd
certainly be supportive of such an extension but ideally it would be a
standalone draft for link inlining that's not directly tied to the hierarchy
use case that started this thread (but could certainly be leveraged by it)

As Pablo notes, an inline extension can be a big latency win on first-fetch
use cases where there's a set of related resources and you don't want to do
multiple serialized round trips to get them all, but it also nicely
preserves the ability to allow subsequent interactions with the inlined
entities as independent HTTP resources using a model that Atom clients
already understand (link following).

Cheers!

-- Kyle Marvin
   (Google GData Team)

On Wed, Jun 3, 2009 at 3:30 PM, Pablo Castro <[email-masked]>wrote:

> for hierarchies, but more in general because the Astoria data model consists
> basically of entities (mapped to entries) and associations (mapped to links
> to entries or links to feeds depending on the cardinality of the
> with latency, but it also results in a couple of extra features in Astoria).
>
> From: [email-masked] [mailto:
> [email-masked]] On Behalf Of Nikunj R. Mehta

---

From: "Nikunj R. Mehta" <[email-masked]>
Date: Thu, 4 Jun 2009 09:15:31 -0700

On Jun 3, 2009, at 3:30 PM, Pablo Castro wrote:

>
> Sorry for coming late to the thread, somebody forwarded me this and  
> I thought I'd add a couple of comments from the Astoria (ADO.NET  
> Data Services) side.

Thanks for contributing to this thread. We have had little activity on  
atom-syntax lately, but this topic did stir pretty strong reactions on  
all sides. That can only indicate a strong desire to solve this  
problem and arrive at a consensus so as to avoid fragmentation - I see  
it as a good omen for a new RFC.

>
>
> We have a similar need in Astoria to include inline content. This is  
> not for hierarchies, but more in general because the Astoria data  
> model consists basically of entities (mapped to entries) and  
> associations (mapped to links to entries or links to feeds depending  
> on the cardinality of the association-end). We needed to allow  
> clients to request a given entry and pre-fetch related entries (this  
> is mostly a round-trip optimization to help with latency, but it  
> also results in a couple of extra features in Astoria).
>
> We interpreted the Atom spec as saying that while the spec itself  
> didn't define any meaning for content inside the link element, it  
> didn't prohibit either. In order to avoid future conflicts with Atom  
> elements inside link, we ended up putting an <inline> element in our  
> own namespace immediately under link, and then an Atom entry or feed  
> in it. If the link points to something that hasn't been created yet  
> or that the user can't see due to security reasons, then we still  
> include the inline element, but it's empty. That way a client  
> processor can know that the link is expanded but there is no actual  
> resource at the other end of it.
>
> Expanding the entry/feed inside the link element means that if we  
> have more than one expanded link we don't need to add any indication  
> of what entry extension element corresponds to what link, which we  
> would need if we included the inlined content as a peer of the link  
> element instead of as a child.
>
> It's also easy for client parsers. We parse the link as usual  
> (extract url and such) and then if we see an inner element inline in  
> our namespace then we know the link was inlined.
>
> There is of course the question of the risk of pulling down a giant  
> graph because of a client asking to expand too much. The most common  
> form of this issue is expanding long feeds inline inside another  
> entry. We actually use the usual Atom paging constructs even in  
> inlined feeds, so the server is free to include a few entries and  
> then a "next" link where the client can get more. This allows for a  
> good balance between low-latency first fetch to get and display data  
> and progressive retrieval of more data as needed.
>

---

From: "Nikunj R. Mehta" <[email-masked]>
Date: Thu, 4 Jun 2009 08:56:46 -0700

On Jun 3, 2009, at 6:32 PM, Mark Nottingham wrote:

> None of this is your fault; the extension model in Atom isn't  
> terribly well-specified.
>

I don't know of anyone else complaining about this. I am not sure this  
is the general perception. Can you provide some evidence to back up  
this claim?

> considering minting a new media type to identify the document, so  
> that people can differentiate them.

I don't see any reason to mint a new media type for dealing with  
extensions such as the one I am proposing in atom-hierarchy-ID. Can  
you explain what will break and in what way we are creating an  
incompatible extension of Atom to force us out of application/atom+xml?

---

From: "Nikunj R. Mehta" <[email-masked]>
Date: Thu, 4 Jun 2009 09:15:19 -0700

On Jun 4, 2009, at 4:29 AM, Kyle Marvin wrote:

> FWIW, we have been looking at revising our strategy for inlining  
> entity references and have reached largely the same conclusion that  
> you have reached.   The ideal model is not some new extension type  
> (like gd:entryLink/gd:feedLink) but rather a model that lets you  
> nest the content as an extension directly within a link element.

We have generic clients that consume the GData feeds and the use of  
gd:feedLink and gd:entryLink does make it difficult to process  
inlining since they are in a proprietary namespace. So a standard for  
in-lining will be definitely help, especially if it is blessed by IETF.

> This way, clients who are unaware of the inlining extensions can  
> still process it as a regular reference (via the link) and follow  
> it, albeit less efficiently than if they just used the inlined  
> content.    I agree with comments on this thread that if data is  
> inlined inside of link, it is better if it's done inside of an  
> extension that clearly marks the content as being the inlined data  
> and avoids collisions with any any future link extensions.

I had considered this approach [1] and am open to a specialized  
extension such as ah:inline. I would, however, strongly refute the  
claim that the atom:link element cannot be used for any inlining of  
content that a few on this mailing list are advocating.

>
> I'm not as enthusiastic about the idea of defining some inlining  
> syntax that is a peer of the atom:link as was suggested earlier  
> (just put an entry inside an entry) or later (ah:inline) on this  
> thread.   You then lose or must duplicate the context (the "rel")  
> that tells you the relationship of the contained entity to the  
> containing resource.  You may even lose the ability to know how to  
> deference the inlined resource independently;  there are potential  
> use cases where you might inline an XML entity that doesn't have the  
> equivalent of a 'self' link.    It also means you have to  

These are all problems I have pointed out [2] earlier in the thread.  
So I agree with your conclusions.

>
> What seems apparent from this thread is that multiple existing large- 
> scale implementors leveraging the Atom Syntax are seeing a need for  
> this and seem to be converging on a roughly similar solution.   As  
> such, there's probably significant benefit on proposing an extension  
> that makes this unambiguous and consistent across implementations so  
> Atom clients can rely on it.   We'd certainly be supportive of such  
> an extension but ideally it would be a standalone draft for link  
> inlining that's not directly tied to the hierarchy use case that  
> started this thread (but could certainly be leveraged by it)

It seems unwise to consider two separate I-Ds - one for specifying new  
link @rel values and another to deal with in-lining. Will it buy us  
anything in the IETF process? Would you be interested in seeing this  
get on the experimental track so that when it is time to consider a  
revision of Atom the experience accumulated by then can be used to  
propose something along standards track?

>
> As Pablo notes, an inline extension can be a big latency win on  
> first-fetch use cases where there's a set of related resources and  
> you don't want to do multiple serialized round trips to get them  
> all, but it also nicely preserves the ability to allow subsequent  
> interactions with the inlined entities as independent HTTP resources  
> using a model that Atom clients already understand (link following).
>
> Cheers!
>
> -- Kyle Marvin
>    (Google GData Team)
>
> On Wed, Jun 3, 2009 at 3:30 PM, Pablo Castro <[email-masked] 
> > wrote:
>
[1] http://o-micron.blogspot.com/2009/05/atom-multiple-links-with-same-rel-value.html
[2] http://o-micron.blogspot.com/2009/05/cmis-xvi-perils-of-embedding-entry.html

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 5 Jun 2009 09:14:24 +1000

On 05/06/2009, at 1:56 AM, Nikunj R. Mehta wrote:

> I don't know of anyone else complaining about this. I am not sure  
> this is the general perception. Can you provide some evidence to  
> back up this claim?

*shrug* It's just an opinion. However, Atom bucks the trend of well- 
designed IETF protocols with its "everything is extensible" approach;  
by not designating *how* to extend each possible point, and how that  
extension is managed, it leaves extension open to interpretation (as  
we see here).

I think this might be because at the time Atom was written, the  
extensibility of XML was considered a good thing, and the unspoken  
reasoning was "the more extensibility, the better." In practice, I  
don't think this is working; there's been lots of discussion along  
these lines on-list (e.g., whether to use attribute extensibility,  
whether to use entry extensions or atom:content).

> incompatible extension of Atom to force us out of application/atom 

I didn't say it would become incompatible; I said that at some point,  
you extend a document so much that it's no longer useful or meaningful  
to a generic processor. At that point, it's worth *considering* (note  
the emphasis, please) identifying the new format with a new media  
type, to make it distinct from the old one.

---

From: "Nikunj R. Mehta" <[email-masked]>
Date: Thu, 4 Jun 2009 16:38:33 -0700

On Jun 4, 2009, at 4:14 PM, Mark Nottingham wrote:

> designed IETF protocols with its "everything is extensible"  
> approach; by not designating *how* to extend each possible point,  
> and how that extension is managed, it leaves extension open to  
> interpretation (as we see here).
>

Hmm... while Atom's current extensibility points are sufficient for  
what we are attempting to do, I interpret your points as more general  
issues with XML, rather than specifically with Atom. I also interpret  
broken extensibility along the lines of XSD rather than having too  
many options. However, I can now see your perspective as well.

>>> generic Atom processor (as deployed today), because the majority  
>>> of the information is in extensions. When that happens, it's worth  
> I didn't say it would become incompatible; I said that at some  
> point, you extend a document so much that it's no longer useful or  
> meaningful to a generic processor. At that point, it's worth  
> *considering* (note the emphasis, please) identifying the new format  
> with a new media type, to make it distinct from the old one.

IMHO, anyone would be reluctant to consider a new media type for XML  
feeds if the kind of reaction about the ownership and meaning of text/ 
html is any indication. We should exhaust all options before resorting  
to minting a new media type. And the kind of extension we are  
suggesting seems legal, prevalent, and useful. Also, it is primarily  
additional semantics that are compatible with existing semantics. So I  
was a little surprised to see you projecting as far out as you did.  
But I have noted your emphasis and, hopefully, communicated my  
concerns too.

Regards,
Nikunj

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 5 Jun 2009 09:50:51 +1000

On 05/06/2009, at 9:38 AM, Nikunj R. Mehta wrote:

> what we are attempting to do, I interpret your points as more  
> general issues with XML, rather than specifically with Atom. I also  
> interpret broken extensibility along the lines of XSD rather than  
> having too many options. However, I can now see your perspective as  
> well.

XML is just a metamodel and syntax for expressing a language; it's up  
to the language to define how to extend itself...

>>>> of the information is in extensions. When that happens, it's  
>>>> worth considering minting a new media type to identify the  
>>>> document, so that people can differentiate them.
>>>
>>> extensions such as the one I am proposing in atom-hierarchy-ID.  
>>> Can you explain what will break and in what way we are creating an  
>> *considering* (note the emphasis, please) identifying the new  
>> format with a new media type, to make it distinct from the old one.
>
> feeds if the kind of reaction about the ownership and meaning of  
> text/html is any indication. We should exhaust all options before  
> resorting to minting a new media type. And the kind of extension we  
> are suggesting seems legal, prevalent, and useful. Also, it is  
> primarily additional semantics that are compatible with existing  
> semantics. So I was a little surprised to see you projecting as far  
> out as you did. But I have noted your emphasis and, hopefully,  
> communicated my concerns too.

I think HTML is a different case; you can extend it with new elements  
that have no meaning in HTML, but it can still be functional HTML, in  
particular because you can embed scripts to interpret those new  
elements. Atom doesn't have this option, and is more structured, as  
well as more intended for machine interpretation in the cases that  
people are talking about here. Distinct media types are useful, and we  
shouldn't be afraid of them, except perhaps where they cause legacy  
problems (as with HTML).

My understanding was that the utility of this extension was being able  
to express graphs of entries in a single document; that's why it  
sounded so different to me. (as opposed to just adorning a feed with  
additional metadata, which is the typical use case for extensions in  
Atom). Also, as I think I said earlier, there are a lot of things that  
could be improved in Atom (especially for the machine-oriented cases)  
if we had a chance to do that.

Anyway, sounds like we understand each other.

Cheers,

---

From: Peter Keane <[email-masked]>
Date: Thu, 4 Jun 2009 23:10:05 -0500

On Thu, Jun 4, 2009 at 6:50 PM, Mark Nottingham<[email-masked]> wrote:
>>>> is the general perception. Can you provide some evidence to back up this
>>>> claim?
>>>
>>> *shrug* It's just an opinion. However, Atom bucks the trend of
>>> well-designed IETF protocols with its "everything is extensible" approach;
>>> extension is managed, it leaves extension open to interpretation (as we see
>>> here).
>>>
>>> extensibility of XML was considered a good thing, and the unspoken reasoning
>>> was "the more extensibility, the better." In practice, I don't think this is
>>> working; there's been lots of discussion along these lines on-list (e.g.,
>>> whether to use attribute extensibility, whether to use entry extensions or
>>> atom:content).
>>
>> Hmm... while Atom's current extensibility points are sufficient for what
>> we are attempting to do, I interpret your points as more general issues with
>> XML, rather than specifically with Atom. I also interpret broken
>> extensibility along the lines of XSD rather than having too many options.
>> However, I can now see your perspective as well.
>
> XML is just a metamodel and syntax for expressing a language; it's up to the
> language to define how to extend itself...
>
>
>>>>> document you're going to end up with will not be very useful to a generic
>>>>> Atom processor (as deployed today), because the majority of the information
>>>>> is in extensions. When that happens, it's worth considering minting a new
>>>>> media type to identify the document, so that people can differentiate them.
>>>>
>>>> extensions such as the one I am proposing in atom-hierarchy-ID. Can you
>>>> explain what will break and in what way we are creating an incompatible
>>>> extension of Atom to force us out of application/atom+xml?
>>>
>>>
>>> I didn't say it would become incompatible; I said that at some point, you
>>> extend a document so much that it's no longer useful or meaningful to a
>>> generic processor. At that point, it's worth *considering* (note the
>>> emphasis, please) identifying the new format with a new media type, to make
>>> it distinct from the old one.
>>
>> IMHO, anyone would be reluctant to consider a new media type for XML feeds
>> if the kind of reaction about the ownership and meaning of text/html is any
>> indication. We should exhaust all options before resorting to minting a new
>> media type. And the kind of extension we are suggesting seems legal,
>> prevalent, and useful. Also, it is primarily additional semantics that are
>> compatible with existing semantics. So I was a little surprised to see you
>> projecting as far out as you did. But I have noted your emphasis and,
>> hopefully, communicated my concerns too.
>
>
> I think HTML is a different case; you can extend it with new elements that
> have no meaning in HTML, but it can still be functional HTML, in particular
> because you can embed scripts to interpret those new elements. Atom doesn't
> have this option, and is more structured, as well as more intended for
> machine interpretation in the cases that people are talking about here.
> Distinct media types are useful, and we shouldn't be afraid of them, except
> perhaps where they cause legacy problems (as with HTML).
>
> My understanding was that the utility of this extension was being able to
> express graphs of entries in a single document; that's why it sounded so
> different to me. (as opposed to just adorning a feed with additional
> metadata, which is the typical use case for extensions in Atom). Also, as I
> think I said earlier, there are a lot of things that could be improved in
> Atom (especially for the machine-oriented cases) if we had a chance to do
> that.
>

I hope I'm not too far off w/ this, but it seems to me there are a
couple distinct concerns in this thread: First is a need for what is
essentially multiple atom:content elements in an entry.  atom:link
seems like a logical home for that content (given that we can assign
an appropriate @rel).  The fact that it may or may not contain a
hierarchical data structure is immaterial to the semantics of the
atom:entry itself. Second, there is a need to express hiearchical
relationship between entries and entries, entries and feeds, and feeds
and feeds.  Link semantics fit the bill quite well and the proposed
@rel values in the hierarchy ID cover the need.

I'd agree with Nikunj that this is not so dramatic a change that new
media types need to be considered.  It feels more like a tweak to me,
and I do not see much difficulty at all in using my current tools for
parsing or producing such atom documents.

My previous championing of Google's feedLink is moot since it does not
appear to be an easier "sell" than atom:link content (my main motive),
nor as useful per comments from others on this thread.

--peter keane

---

From: Mark Nottingham <[email-masked]>
Date: Fri, 5 Jun 2009 15:05:48 +1000

On 05/06/2009, at 2:10 PM, Peter Keane wrote:

> an appropriate @rel).

Well, the most straightforward way to fix that would be to fix Atom  
and allow multiple atom:content elements in an entry...

> parsing or producing such atom documents.

Ah, maybe that's where the misunderstanding lies. It's not that you  
can't parse it with an Atom library, with appropriate code to handle  
the extensions; that's obviously possible.

My concern is that a *generic* Atom processor (e.g., a feed reader) or  
more importantly a generic MIME dispatcher (e.g., a browser or an OS)  
wouldn't be able to do much that's useful with such a feed, because --  
as I understand the use cases -- most of the information will be in  
extensions that the generic processor won't be able to take advantage  
of, and because the generic MIME dispatcher won't know to send this  
document to a handler that can do something with it.

This is the role media types play on the Web (and elsewhere, of  
course)...

Cheers,

---

From: Kyle Marvin <[email-masked]>
Date: Fri, 5 Jun 2009 12:32:47 -0500

>> seems like a logical home for that content (given that we can assign
Peter, I'm not sure I see the need like this;  you want more than multiple
content items.  What you have here is multiple resources (some of which
might be metadata + content, i.e. an atom entry, or might not even be Atom
at all) with some type of relationship between them *and* the desire to
retrieve these multiple related resources using a single GET.   A key
difference between 'content' and a 'resource' is that the latter is
independently dereferencable whereas the same isn't necessarily true for any
atom:content types other than out-of-line-content.   A link has the nice
property of having a rel that describes how it is related to what it is
contained within (i.e. why it might be referenced), a type attribute that
informs how to process it (either when referenced or potentially inline),
and a href that says where to deference it (i.e. how to reference it
independently even if it is inline).   All nice and useful properties within
the composite resource that allow it to be used in a decoupled fashion after
the initial GET of the composite resource.

>>
> Well, the most straightforward way to fix that would be to fix Atom and
> allow multiple atom:content elements in an entry...
>
I think this is valid but an orthogonal concern.  The key point is that you
want to nest a resource that itself has a MIME type and can be processed.
 Whether these are new or existing MIME types depends on the use cases and
what is the most appropriate MIME type for that relationship.

>
> Ah, maybe that's where the misunderstanding lies. It's not that you can't
> parse it with an Atom library, with appropriate code to handle the
> extensions; that's obviously possible.
>
> My concern is that a *generic* Atom processor (e.g., a feed reader) or more
> importantly a generic MIME dispatcher (e.g., a browser or an OS) wouldn't be
> able to do much that's useful with such a feed, because -- as I understand
> the use cases -- most of the information will be in extensions that the
> generic processor won't be able to take advantage of, and because the
> generic MIME dispatcher won't know to send this document to a handler that
> can do something with it.

It might surprise you but I'd agree that using MIME types more would be
goodness.  It's frequently the case (including in GData) that the lines
between what is metadata and what is content are being blurred since both
are found in Atom extensions.   This fact does make life harder for Atom
processors, since they have to rely on sideband signals (a 'kind' or a
'type') that inform them about what extensions to expect and what they mean
and there's not much standardization happening in this space.

>
> This is the role media types play on the Web (and elsewhere, of course)...

I don't know, though, that this is a problem the Atom WG (or a derivative
effort) can solve.  In most cases, those MIME types for nested content are
domain-specific representations that are best standardized by (or at least
generally agreed upon) by groups working in that domain.    I don't see a
proliferation of vendor-specific media types as being much better than the
status quo other than that they'd let you write vendor-specific MIME
processors;  better structured from an Atom perspective but not really
pushing the Web forward in the right direction from a collaborative
perspective.

Let me know if you think I'm missing something!

-- Kyle

---

From: Kyle Marvin <[email-masked]>
Date: Fri, 5 Jun 2009 12:51:48 -0500

On Thu, Jun 4, 2009 at 11:15 AM, Nikunj R. Mehta <[email-masked]>wrote:

> as an extension directly within a link element.
>
>
> gd:feedLink and gd:entryLink does make it difficult to process inlining
> since they are in a proprietary namespace. So a standard for in-lining will
> be definitely help, especially if it is blessed by IETF.
>
> This way, clients who are unaware of the inlining extensions can still
> process it as a regular reference (via the link) and follow it, albeit less
> efficiently than if they just used the inlined content.    I agree with
> comments on this thread that if data is inlined inside of link, it is better
> if it's done inside of an extension that clearly marks the content as being
> the inlined data and avoids collisions with any any future link extensions.
>
>
>
> I had considered this approach [1] and am open to a specialized extension
> such as ah:inline. I would, however, strongly refute the claim that the
> atom:link element cannot be used for any inlining of content that a few on
> this mailing list are advocating.
>
>
> I'm not as enthusiastic about the idea of defining some inlining syntax
> that is a peer of the atom:link as was suggested earlier (just put an entry
> These are all problems I have pointed out [2] earlier in the thread. So I
> agree with your conclusions.
>
>
> It seems unwise to consider two separate I-Ds - one for specifying new link
> @rel values and another to deal with in-lining. Will it buy us anything in
> the IETF process?
>

It would get us a clear model for doing inlining that's valid beyond the
specific use case of hierarchy which would be more useful to us since we
don't have a specific need for the hierarchy I-D;  as someone else stated
earlier, we have many types of relations where inlining would be beneficial
so the specifics of the spec is too narrow to support what we need.

> Would you be interested in seeing this get on the experimental track so
> that when it is time to consider a revision of Atom the experience
> accumulated by then can be used to propose something along standards track?
>

I'm not opposed but if this is considered to be prior art that will drive
how inlining should be done more generally, then I think the net effect
(read as "the hoops you will have to jump through to get consensus") are
going to be the same whether the inlining functionality is joined or split
off so why not get the general solution's benefit for the effort.

It could be there are aspects of this decision I'm missing.   I understand
that what I'm asking for is more work for you (or someone) and I'm
sympathetic to that.

>> Expanding the entry/feed inside the link element means that if we have
>> more than one expanded link we don't need to add any indication of what
>> entry extension element corresponds to what link, which we would need if we
> http://o-micron.blogspot.com/2009/05/atom-multiple-links-with-same-rel-value.html
> [2]
> http://o-micron.blogspot.com/2009/05/cmis-xvi-perils-of-embedding-entry.html
>

---

From: "Nikunj R. Mehta" <[email-masked]>
Date: Fri, 5 Jun 2009 11:00:54 -0700

Hi Kyle,

I am fine with splitting the I-D along those lines, although I have  
just published a revision that separates the two into independent  
sections but keeps it in one I-D. Just as an author, I am interested  
first in better understanding the process by which we can land RFCs  
for the two parts. Please advise if you can suggest how we move  
forward from here.

For now, I hope you would be interested in reviewing the new I-D  
revision. I am sure there is a lot of existing implementation  
experience to apply and look forward to your and others' contributions.

Nikunj

On Jun 5, 2009, at 10:51 AM, Kyle Marvin wrote:

> It seems unwise to consider two separate I-Ds - one for specifying  
> new link @rel values and another to deal with in-lining. Will it buy  
> us anything in the IETF process?
>
> It would get us a clear model for doing inlining that's valid beyond  
> the specific use case of hierarchy which would be more useful to us  
> since we don't have a specific need for the hierarchy I-D;  as  
> someone else stated earlier, we have many types of relations where  
> inlining would be beneficial so the specifics of the spec is too  
> narrow to support what we need.
>
> Would you be interested in seeing this get on the experimental track  
> so that when it is time to consider a revision of Atom the  
> experience accumulated by then can be used to propose something  
> along standards track?
>
> I'm not opposed but if this is considered to be prior art that will  
> drive how inlining should be done more generally, then I think the  
> net effect (read as "the hoops you will have to jump through to get  
> consensus") are going to be the same whether the inlining  
> functionality is joined or split off so why not get the general  
> solution's benefit for the effort.
>
> It could be there are aspects of this decision I'm missing.   I  
> understand that what I'm asking for is more work for you (or  
> someone) and I'm sympathetic to that.

---

From: Mark Nottingham <[email-masked]>
Date: Sat, 6 Jun 2009 12:47:46 +1000

Just checking -- you do realise I'm suggesting it's worth using a new  
media type for the atom document that contains these extensions, not a  
new media type in it? Your response was worded in such a way that I'm  
not sure...

Cheers,

On 06/06/2009, at 3:32 AM, Kyle Marvin wrote:

> Peter, I'm not sure I see the need like this;  you want more than  
> multiple content items.  What you have here is multiple resources  
> (some of which might be metadata + content, i.e. an atom entry, or  
> might not even be Atom at all) with some type of relationship  
> between them *and* the desire to retrieve these multiple related  
> resources using a single GET.   A key difference between 'content'  
> and a 'resource' is that the latter is independently dereferencable  
> whereas the same isn't necessarily true for any atom:content types  
> other than out-of-line-content.   A link has the nice property of  
> having a rel that describes how it is related to what it is  
> contained within (i.e. why it might be referenced), a type attribute  
> that informs how to process it (either when referenced or  
> potentially inline), and a href that says where to deference it  
> (i.e. how to reference it independently even if it is inline).   All  
> nice and useful properties within the composite resource that allow  
> it to be used in a decoupled fashion after the initial GET of the  
> composite resource.
>
>
>
> I think this is valid but an orthogonal concern.  The key point is  
> that you want to nest a resource that itself has a MIME type and can  
> be processed.  Whether these are new or existing MIME types depends  
> on the use cases and what is the most appropriate MIME type for that  
> relationship.
>
>
> My concern is that a *generic* Atom processor (e.g., a feed reader)  
> or more importantly a generic MIME dispatcher (e.g., a browser or an  
> OS) wouldn't be able to do much that's useful with such a feed,  
> because -- as I understand the use cases -- most of the information  
> will be in extensions that the generic processor won't be able to  
> take advantage of, and because the generic MIME dispatcher won't  
> know to send this document to a handler that can do something with it.
>
> It might surprise you but I'd agree that using MIME types more would  
> be goodness.  It's frequently the case (including in GData) that the  
> lines between what is metadata and what is content are being blurred  
> since both are found in Atom extensions.   This fact does make life  
> harder for Atom processors, since they have to rely on sideband  
> signals (a 'kind' or a 'type') that inform them about what  
> extensions to expect and what they mean and there's not much  
> standardization happening in this space.
>
>
>
> I don't know, though, that this is a problem the Atom WG (or a  
> derivative effort) can solve.  In most cases, those MIME types for  
> nested content are domain-specific representations that are best  
> standardized by (or at least generally agreed upon) by groups  
> working in that domain.    I don't see a proliferation of vendor- 
> specific media types as being much better than the status quo other  
> than that they'd let you write vendor-specific MIME processors;   
> better structured from an Atom perspective but not really pushing  
> the Web forward in the right direction from a collaborative  
> perspective.
>

---

From: Kyle Marvin <[email-masked]>
Date: Fri, 5 Jun 2009 23:48:49 -0500

On Fri, Jun 5, 2009 at 9:47 PM, Mark Nottingham <[email-masked]> wrote:

> Just checking -- you do realise I'm suggesting it's worth using a new media
> type for the atom document that contains these extensions, not a new media
> type in it? Your response was worded in such a way that I'm not sure...

Yes, I understand.   I think there are issues with media type under-use at
both levels (document and content).   There are definitely cases where a new
MIME type on the Atom document would be helpful to processors and others
where more judicious usage of specialized MIME types for atom:content would
be helpful.   Let me see if I can give you an example and see if you agree
or disagree.   Take the case of a feed that describes a calendar and
contains entries that describe events in the calendar.   If this had a MIME
type of something like "application/calendar+atom+xml", then it would might
inform a client processing it that there are other possible processing
options besides regular syndication, like importing to a local calendar.
Such a MIME type might explain the set of metadata extensions specific to
calendars that are expected (ex timezone info) and possibly even put
constraints or expectations on the MIME type of atom:content found within
the entries.

Of course, the "application/calendar+atom+xml" is fictitious because there's
no model for saying that something is an atom feed of a specific type in the
same way that "application/atom+xml" says that it's an XML document with
Atom syntax constraints.   Rooted in this might be Nikunj's concern but
there's probably more (see next paragraph).   Without a model for defining
Atom media subtypes, you'd have to type it as something that is going to
cause a generic Atom processor to ignore it even though it could still be
processed in useful (albeit less vertical way) to let me know when
data/events are available or have changed via a feed reader interface.

I don't know, though, that anything in the hierarchy proposal justifies a
new MIME type and given that you say it might be worth *considering* I'm not
sure you see it that way either.   There are definitely cases where you're
just adding extensions that are informative and additional metadata and I
don't think these alone define a new data format that needs a media type.
 Media RSS is possibly another example.   Consider these the Atom Syntax
equivalent of a ""mixin".  If such extensions are standardized and/or get
enough de facto usage lots of Atom processors are likely to handle them just
fine.   So the current proposal is perhaps not the best poster child for
using media types more but I definitely think it's something that the Atom
community should consider to better enable client use cases in addition to
reader-based syndication.

All this being said, I think the right venue for creating more specialized
Atom media types is in communities that would benefit from their existence
and the collaboration or processing models that would be enabled by them.
 Just minting new MIME types because you've extended Atom is not all that
helpful in the bigger scheme of things, imo.

HTH,

-- Kyle

>> I think this is valid but an orthogonal concern.  The key point is that
>> you want to nest a resource that itself has a MIME type and can be
>> processed.  Whether these are new or existing MIME types depends on the use
>> cases and what is the most appropriate MIME type for that relationship.
>>
>>
>> wouldn't be able to do much that's useful with such a feed, because -- as I
>> understand the use cases -- most of the information will be in extensions
>> that the generic processor won't be able to take advantage of, and because
>> the generic MIME dispatcher won't know to send this document to a handler
>> that can do something with it.
>>

---

From: James M Snell <[email-masked]>
Date: Mon, 08 Jun 2009 12:41:07 -0700

A new workgroup chartered to focus on specific extensions to the Atom 
format would be valuable and welcome. There are several drafts, such as 
the Bidi extensions, that I would really like to see finalized and 
standardized.  I would gladly participate.

- James

Nikunj R. Mehta wrote:
>>> with future uses. It is our intention to promote an extension of 
>>> Atom. By submitting the I-D to the IETF and by bringing this 
>>> discussion to atom-syntax, we have made the intention quite clear, 
>>> don't you agree?
>>> ...
>>
> there is a lot of experience in dealing with hierarchy in 
> Atom/AtomPub. It would be futile to think otherwise. As examples, take 
> a look at Google's GData APIs and Microsoft's ADO.NET use of 
> Atom/AtomPub.
>
>> extensibility, but the content of atom:link isn't one of them, as far 
>> as I can tell.
>

---

From: "Nikunj R. Mehta" <[email-masked]>
Date: Mon, 8 Jun 2009 13:44:11 -0700

On Jun 5, 2009, at 9:48 PM, Kyle Marvin wrote:

> Just checking -- you do realise I'm suggesting it's worth using a  
> new media type for the atom document that contains these extensions,  
> not a new media type in it? Your response was worded in such a way  
> that I'm not sure...
>
> Yes, I understand.   I think there are issues with media type under- 
> use at both levels (document and content).   There are definitely  
> cases where a new MIME type on the Atom document would be helpful to  
> processors and others where more judicious usage of specialized MIME  
> types for atom:content would be helpful.   Let me see if I can give  
> you an example and see if you agree or disagree.   Take the case of  
> a feed that describes a calendar and contains entries that describe  
> events in the calendar.   If this had a MIME type of something like  
> "application/calendar+atom+xml", then it would might inform a client  
> processing it that there are other possible processing options  
> besides regular syndication, like importing to a local calendar.    
> Such a MIME type might explain the set of metadata extensions  
> specific to calendars that are expected (ex timezone info) and  
> possibly even put constraints or expectations on the MIME type of  
> atom:content found within the entries.
>
> Of course, the "application/calendar+atom+xml" is fictitious because  
> there's no model for saying that something is an atom feed of a  
> specific type in the same way that "application/atom+xml" says that  
> it's an XML document with Atom syntax constraints.   Rooted in this  
> might be Nikunj's concern but there's probably more (see next  
> paragraph).   Without a model for defining Atom media subtypes,  
> you'd have to type it as something that is going to cause a generic  
> Atom processor to ignore it even though it could still be processed  
> in useful (albeit less vertical way) to let me know when data/events  
> are available or have changed via a feed reader interface.

I find Kyle's commentary sympathetic to my previous concern.

>
> I don't know, though, that anything in the hierarchy proposal  
> justifies a new MIME type and given that you say it might be worth  
> *considering* I'm not sure you see it that way either.   There are  
> definitely cases where you're just adding extensions that are  
> informative and additional metadata and I don't think these alone  
> define a new data format that needs a media type.  Media RSS is  
> possibly another example.   Consider these the Atom Syntax  
> equivalent of a ""mixin".  If such extensions are standardized and/ 
> or get enough de facto usage lots of Atom processors are likely to  
> handle them just fine.   So the current proposal is perhaps not the  
> best poster child for using media types more but I definitely think  
> it's something that the Atom community should consider to better  
> enable client use cases in addition to reader-based syndication.
>
> All this being said, I think the right venue for creating more  
> specialized Atom media types is in communities that would benefit  
> from their existence and the collaboration or processing models that  
> would be enabled by them.    Just minting new MIME types because  
> you've extended Atom is not all that helpful in the bigger scheme of  
> things, imo.
>

In reviewing an industry specification for content management [1], I  
came across a situation where additional constraints were placed on a  
valid/acceptable Atom entry and it just made me think whether a  
generic client can be warned about this through the app:accept kind of  
signaling. I recommended that the specification use a media type  
parameter for Atom that cautions about the server's expectation about  
acceptable content. Since a number of users of Atom feeds beyond  
blogging and news frequently require additional markup in Atom  
documents, there is a latent interoperability problem with and lack of  
adequate metadata about such interactions. Perhaps, a reasoned  
approach could be devised to deal with growing use of Atom documents  
beyond blogs.

[1] http://www.oasis-open.org/committees/download.php/32668/Draft%2062a.zip

> -- Kyle-

---

From: Kyle Marvin <[email-masked]>
Date: Wed, 24 Jun 2009 12:18:19 -0500

On Mon, Jun 8, 2009 at 2:41 PM, James M Snell <[email-masked]> wrote:

> format would be valuable and welcome. There are several drafts, such as the
> Bidi extensions, that I would really like to see finalized and standardized.
>  I would gladly participate.

Sorry to be slow to reply (I've been OOTO), but I'd be supportive and
willing to participate  as well.

-- Kyle

>>>>> with future uses.
>>>> with future uses. It is our intention to promote an extension of Atom. By
>>>> submitting the I-D to the IETF and by bringing this discussion to
>>> I think the point is that this is not an extension point for general use;
>>> thus if it is to be used it would need to be done by a spec that's on the
>>> standards track, and updates RFC4287. For that you'll likely need a WG to
>>> reach the consensus that *this* is the way to go.
>>>
>>
>> supportive of a new WG to look at this issue. Frankly, IETF needs to look at
>> the issue of hierarchical representation in Atom. Frankly, there is a lot of
>> experience in dealing with hierarchy in Atom/AtomPub. It would be futile to
>> think otherwise. As examples, take a look at Google's GData APIs and
>> Microsoft's ADO.NET use of Atom/AtomPub.
>>
>> I was dissuaded from submitting the atompub-hierarchy I-D along standards
>> track, but it looks like the right way to proceed. Of course, that also
>> means getting together a new WG. Would you be willing to help form such a
>> WG?
>>
>>>> would be useful if you or others who claim that our approach is wrong can
>>>> explain what is the process for extending Atom. Is it creating a brand new
>>>> working group?
>>>>
>>>
>>> extensibility, but the content of atom:link isn't one of them, as far as I
>>> can tell.
>>>
>>
>> It was never the intention of atom-hierarchy I-D to perform independent
>> extension of Atom. Therefore, I don't see this as a problem.
>>
>> Nikunj