Subject: New Pace: PaceAggregationInSeparateSpec

From: James Snell <[email-masked]>
Date: Thu, 3 Feb 2005 22:07:54 -0800

Figured I would formalize what I've been evangelizing the past couple of days.  

http://www.intertwingly.net/wiki/pie/PaceAggregationInSeparateSpec

== Abstract ==

Defer the definition of solutions for aggregated feeds to a separate
Internet-Draft that is not a part of the Atom core syntax
specification.

== Status ==

Proposed (by [JamesSnell])

== Rationale ==

Aggregated feeds, while important, are currently not supported by the
existing RSS mechanisms and are relatively rare in comparison to their
single feed cousins.  Given the guidelines for proposals set forth in
this Wiki, this alone would justify moving the aggregation stuff off
to a separate document, at least for now.

* The 80/20 rule: If a feature will only be used by a small number of
people, and will create extra work and headaches for everyone else, it
probably doesn't belong in the core spec.
* Pick stuff that's already been proven to work and be interoperable,
and writing it down in a clean, clear way
* Keep it simple: The simplest thing that can possibly work tends to
be preferred over more complex solutions.

I absolutely acknowledge that there are a subset of folks for whom
aggregated feeds are very important.  But this is a subset.  Let that
subset document their ideas in a separate Internet-Draft; let them
implement those ideas and build momentum for them; then let us later
come back later and discuss the merits of merging those ideas into the
core.

== Proposal ==

(see abstract)

== Impacts ==

Defers PageAggregationDocument and PageFeedRecursive to a separate
Internet-Draft

* Lets Atom 1.0 get out the door faster.  
* Lets folks gain valuable implementation experience before committing
to major changes to the Atom core spec to support what is currently an
edge case
* Keeps the Atom core simple

== Notes ==

-- 
- James Snell
  http://www.snellspace.com
  [email-masked]

---

From: Antone Roundy <[email-masked]>
Date: Thu, 3 Feb 2005 23:39:51 -0700

> http://www.intertwingly.net/wiki/pie/PaceAggregationInSeparateSpec
The only reason why I'm not in favor of this (in fact, it occurred to 
me a little before I saw this message) is that leaving things as they 
are and deferring deciding how to handle aggregation would irreversibly 
enshrine HeadInEntry into the format, which all of the current 
organizational proposals are trying to replace.  Deciding to 
defer==deciding to do it HeadInEntry style.

---

From: Eric Scheid <[email-masked]>
Date: Fri, 04 Feb 2005 18:08:10 +1100

> Internet-Draft that is not a part of the Atom core syntax

+1

---

From: Robert Sayre <[email-masked]>
Date: Fri, 04 Feb 2005 02:14:14 -0500

Antone Roundy wrote:
> 
> leaving things as they are 
> and deferring deciding how to handle aggregation would irreversibly 
> organizational proposals are trying to replace.

That's right. Besides, HeadInEntry is trivial to do as an extension, so 
there's no reason to leave it in.

Robert Sayre

---

From: James Snell <[email-masked]>
Date: Thu, 3 Feb 2005 23:48:04 -0800

> there's no reason to leave it in.

I agree with this so long as there is a core mechanism that allows a
standalone entry to identify the feed to which it belongs.  That
mechanism does not have to be atom:head, but it does need to be part
of the core.

---

From: Robert Sayre <[email-masked]>
Date: Fri, 04 Feb 2005 02:51:28 -0500

> mechanism does not have to be atom:head, but it does need to be part

Why?

Robert Sayre

---

From: James Snell <[email-masked]>
Date: Thu, 3 Feb 2005 22:56:03 -0800

> defer==deciding to do it HeadInEntry style.

I don't agree.  HeadInEntry can be debated seprately on its own merits
(or lack thereof).

For Atom Entry Documents, there MUST be a way of providing some set of
Feed metadata identifying a feed to which this entry belongs or is
related.  HeadInEntry accomplishes that goal, but is not the only way
to achieve it.

If HeadInEntry's primary motivator is to act as a support for
aggregated feeds, then it to is affected by the
PaceAggregationInSeparateSpec proposal and should likely also be
removed from the core and deferred so long as the requirement stated
above is still met (somehow)

For example, it would be possible to achieve the linking requirement
for entry documents using the link element rather than head, which
could allow us to remove HeadInEntry.

<Entry>
   <link rel="feed" href="..." />
</Entry>

This, obviously, is not the only solution, but the point remains the
same.  If HeadInEntry doesn't make sense,
PaceAggregationInSeparateSpec does nothing to require that it stay in
the core spec.

---

From: James Snell <[email-masked]>
Date: Fri, 4 Feb 2005 00:05:20 -0800

> > mechanism does not have to be atom:head, but it does need to be part

As soon as I posted I knew you were going to come back and ask that :-)

As defined by the spec, entries exist either independently or within
the context of a feed.  The state of being associated with a feed is a
part of the core metadata about an entry given this definition.  In
standalone entries, absent HeadInEntry, there is no mechanism of
expressing whether or not the entry is associated with a feed.  The
only assumption that can be made in that case, is that a standalone
entry is not associated with any feed.  As examples, the
Atom-XMPP-Notify and Atom-HTTP-Notify proposals both illustrate cases
where this assumption is not acceptable.  Granted, however, both of
these proposals could just as easily use some shared non-core
extension element to identify the parent feed.

Bottom line: In my opinion, the parent feed is just as core to the
entries metadata as is the date it was updated or any of the other
core elements.  It *could* be defined as an extension, but I feel it
is better handled in the core.

---

From: Henry Story <[email-masked]>
Date: Fri, 4 Feb 2005 09:24:24 +0100

> is better handled in the core.

I have heard interesting arguments "It's all about the Entries, stupid!"
that made the opposite assessment: namely that the entries are what is
important, and that what feed an Entry is part of, is a accident of 
life.

The idea there is that Entries are the stand alone entities. They can
be made to be part of any feed whatsoever. A feed in this 
conceptualization,
is a little like a search engine result listing where the pages they 
refer
to are like entries (notice that search engine results pages are just a
type of web page too). Which feed your entry appears in will depend very
much on the type of query the user of the search engine made.

Of course if an entry has a tag such as "origin" (which used to be on 
the
table) then the entry it points to would be part of the metadata of the
entry and so be a legitimate way of creating special selection of 
entries.

Henry Story

---

From: Eric Scheid <[email-masked]>
Date: Fri, 04 Feb 2005 20:56:51 +1100

> there's no reason to leave it in.

+1 to HeadInEntry being removed from the 1.0 spec.

e.

---

From: Eric Scheid <[email-masked]>
Date: Fri, 04 Feb 2005 20:59:50 +1100

> mechanism does not have to be atom:head, but it does need to be part

<link rel="feed" type="application/atom+xml" .../>

hmmm ... it could even be:

<link rel="feed" type="application/atom+xml" title="My Blog" .../>
<link rel="feed" type="application/atom+xml" title="category foo feed".../>
<link rel="feed" type="application/atom+xml" title="my blog, with
comments".../>

e.

---

From: James Snell <[email-masked]>
Date: Fri, 4 Feb 2005 00:41:44 -0800

> much on the type of query the user of the search engine made.

OK, I can buy that, as long as that is indeed how things are supposed
to be defined.  Note, however, that the Atom syntax spec focuses on
feeds as resources and that entries just happen to be contained in
feeds. e.g. from Section 1: "Atom is an XML-based document format
which describes lists of related information known as "feeds".  Feeds
are composed of a number of items, known as "entries", each with an
extensible set of attached metadata."  If the assertion that the
entries are the standalone entities is correct, then the text in the
spec needs to be changed to reflect that assertion.

Granting the assumption that the entries are the standalone entities
and feeds are merely just a convenient container for entries, the
identification of containing parent-feed becomes less important.  The
need to identify where the entity came from, however, obviously still
remains.. although not necessarily as a core requirement.

Back to the original point about HeadInEntry:  HeadInEntry is not
required to achieve the origin/container identification, regardless of
whether or not origin/container identification should be part of the
core.  If there are no other reasons why HeadInEntry should remain in
the core, then by all means, rip that sucker out of there. 
PaceAggregationInSeparateSpec by no means locks it in place.

---

From: Robert Sayre <[email-masked]>
Date: Fri, 04 Feb 2005 05:12:38 -0500

Bob Wyman wrote:
> Robert Sayre wrote:
> 
>>HeadInEntry is trivial to do as an extension, so there's no 
>>reason to leave it in.
> 
> 	There are a number of excellent reasons to leave it in!

I totally disagree! But you can keep it! (see other mail :)

> 	1. Just about the only functional "advantage" that Atom has over RSS
> is that HeadInEntry provides core support for aggregated feeds. If you're
> unwilling to innovate in even this one tiny spot, why the heck bother with
> this effort at all? Perhaps we should all join Dare over in RSS-land... With
> Microsoft being so anti-Atom, we need *some* argument for expending the
> effort and political capital needed to support this format!

We'll do a SOAP protocol and they can call it "Atom Indigo". Don't you 
think that sounds approproately cool but also banal? Besides, if you 
move to RSS-land, Dare will have to move somewhere else (XAML-land?) and 
call people morons from there.

> 	We decided to support HeadInEntry. It doesn't make sense to back off
> now. Deferring HeadInEntry to a non-core extension essentially kills it and
> ensures that Atom use will provide virtually no advantage to anyone who is
> building aggregated feeds.

Oh, that's not true. No one will call you names when you build 
extensions, and there will always be titles, ids, and dates--like other 
formats that work.

Robert Sayre

---

From: Antone Roundy <[email-masked]>
Date: Fri, 4 Feb 2005 08:59:36 -0700

On Friday, February 4, 2005, at 01:29  AM, Bob Wyman wrote:
> 	4. We've been talking about HeadInEntry ever since I proposed it
> back in June at the Atom Community meeting. On numerous occasions, the 
> group
> as a whole has rejected the various "feed of feed" proposals as overly
> complex and unnecessary. It is a bit bizarre to see all this being 
> rehashed
> in a rush as we drive towards a Last Call.

It seems to me that the main reason for the objections to "feed of 
feeds" was that it seemed to open the way to unlimited recursion.  I 
too am a little surprised to see proposals for recursive feeds of 
feeds--that's what sparked the idea of the Aggregation Document you've 
taken such a disliking to.

> 	6. All of the alternatives that have been proposed require drastic
> and unlikely modifications to the design and architecture of readers. 
> (i.e.
> stuff like adding an aggregation document!!) Such proposals are not 
> viable
> alternatives no matter how sweet one might find their abstractions...

I suspect that the large volume of proposed text that 
PaceAggregationDocument lifted from PaceFeedRecursive made it look a 
lot more drastic than it really is.  In both of those proposals, the 
proposed text was largely identical to what's in the current 
draft--it's just been moved around and one or two little changes made.  
However, there is a point to the argument that we should minimize how 
much mucking around with do with things right now, so I've posted 
PaceAggregationDocument2, which only proposes the changes necessary to 
create an Aggregation Document.

As for HeadInEntry, the best argument I see for it if we adopt a 
containment-based aggregation mechanism is that Entry Documents may 
need to be able to refer to their feeds for context.  I can think of 
three ways to handle that:

1) A link to their feed
2) Use a one-entry Feed Document instead of an Entry Document
3) Allow HeadInEntry in Entry Documents only

---

From: Paul Hoffman <[email-masked]>
Date: Fri, 4 Feb 2005 09:32:52 -0800

>mechanism does not have to be atom:head, but it does need to be part

What if an entry is part of many feeds? What if someone takes an 
entry and puts it into a new feed?

It seems like the WG has coalesced on the idea that entries can stand 
alone, and can also be combined into one or (many) more feeds. Any 
pointer from an entry to a feed would be almost instantly 
out-of-date, wouldn't it?

The outward pointer or pointer mechanism doesn't feel like part of 
the core of describing entries and how to create a feed of entries.

--Paul Hoffman, Director
--Internet Mail Consortium

---

From: "Bob Wyman" <[email-masked]>
Date: Fri, 4 Feb 2005 03:29:30 -0500

1. Just about the only functional "advantage" that Atom has over RSS
is that HeadInEntry provides core support for aggregated feeds. If you're
unwilling to innovate in even this one tiny spot, why the heck bother with
this effort at all? Perhaps we should all join Dare over in RSS-land... With
Microsoft being so anti-Atom, we need *some* argument for expending the
effort and political capital needed to support this format!
	2. The Internet Draft "Atom over XMPP" requires HeadInEntry in order
to provide attribution for entries.
	3. Entry Documents will often need HeadInEntry to identify their
origin.
	4. We've been talking about HeadInEntry ever since I proposed it
back in June at the Atom Community meeting. On numerous occasions, the group
as a whole has rejected the various "feed of feed" proposals as overly
complex and unnecessary. It is a bit bizarre to see all this being rehashed
in a rush as we drive towards a Last Call. 
	5. If HeadInEntry is not in the core, it will not be widely
implemented by clients. Given the speed with which the IETF approves RFC's
it is also likely that it would be a *long* time before an extension RFC
could get approved -- yet we need HeadInEntry TODAY! The reduced penetration
of HeadInEntry that would come from it being defined in a future extension
makes it largely useless and not worth the effort to even try to get a
formal extension. What you'll be doing is forcing people who are in the
aggregation business to define private extensions to address these issues
and to do so outside the Atom process. We have work to do and customers to
serve. We can't just put our businesses on hold for years...
	6. All of the alternatives that have been proposed require drastic
and unlikely modifications to the design and architecture of readers. (i.e.
stuff like adding an aggregation document!!) Such proposals are not viable
alternatives no matter how sweet one might find their abstractions...
	7. We at PubSub have had "running code" since July that demonstrates
the utility of HeadInEntry (i.e. our "ps:source-feed"). None of the other
proposals that have been made are supported or "proven" by running-code.
There is no field experience in their use. I have also not seen any
alternatives proposed by anyone who actually appears to be in the feed
aggregation business...

		bob wyman

---

From: Henry Story <[email-masked]>
Date: Sat, 5 Feb 2005 13:49:58 +0100

I don't have that much of an opinion now on the head in entry and 
various
other proposals.

But I do find your comment that moving something off to an extension
essentially kills it to be a very important remark. This is clearly
to say that Atom has not yet dealt with the extension part of the 
charter.
And even though many people seem to willing to create fill in language
for that part of the spec to make it seem like this part has been
addressed, your on the ground initial reaction is the correct one:
there is no well defined extension mechanism.

So perhaps what we could do in the next weeks is fill in the work
I started in my proposal AtomAsRDF, that would allow Atom to be
seen as an RDF/XML document, though one constrained by an Relax-NG 
syntax.
This will require a week or two of serious group effort, but then 
extensions
will fall out from the RDF side of things.

In the mean time I am fully behind every movement to remove non core
elements from Atom.  Lets have a clean spec well defined spec.

When enough people are left outside of the core, then I am sure talk
of extensions will finally get to be serious.

Henry Story

---

From: "Roger B." <[email-masked]>
Date: Sat, 5 Feb 2005 10:06:27 -0600

> there is no well defined extension mechanism.

Henry: I suspect that Bob's reaction would have been the same, no
matter how well-defined the extension mechanism. Anything outside the
core will have spotty (at best) support in aggregators and publishing
tools.

--
Roger Benningfield
JournURL

---

From: Henry Story <[email-masked]>
Date: Sat, 5 Feb 2005 17:25:08 +0100

On 5 Feb 2005, at 13:49, Henry Story wrote:
> extensions will fall out from the RDF side of things.

I was just looking at the RSS1.1 [1] spec again and noticed that this
is exactly what they have done. They have:
   - a Relax NG syntax to constrain the syntax
   - an OWL ontology to describe the model

I think this is a very good example.
I am not completely happy with their model, but this just seems
so easy and so clean.

Please have a look at it.

Henry

[1] http://inamidst.com/rss1.1/

---

From: "Bob Wyman" <[email-masked]>
Date: Sat, 5 Feb 2005 14:18:20 -0500

Roger Benningfield wrote:
> matter how well-defined the extension mechanism. Anything outside
> the core will have spotty (at best) support in aggregators and 
> publishing tools.
	You are absolutely correct. While I think I feel just as strongly as
Henry does that there is a need for a well defined and robust extension
mechanism, I also know from past experience that implementation of most
extensions is vastly less probable then implementation of core features.
	My feeling that HeadInEntry or Feeder should be in the Atom core is
completely independent of my feelings concerning the utility of and
desirability of a robust extension mechanism for Atom. Frankly, I'm a bit
miffed by the idea that Henry, or anyone else, could have attempted to
"highjack" my concern for what is core and suggest that this implies how I
feel about some other mechanism. The cause of extensibility is not enhanced
by such underhanded rhetorical trickery.

		bob wyman

---

From: Henry Story <[email-masked]>
Date: Sat, 5 Feb 2005 20:44:53 +0100

> extensions is vastly less probable then implementation of core 

Good.

> by such underhanded rhetorical trickery.

Yes, but perhaps if everyone's favorite feature were possibly to be
left to an extension, then people would be thinking more carefully
about getting that very important part right. This is a little
like the thought behind Rawl's "Theory of Justice". If you set out
to build a good political system, you should try to think of it from
the point of view of someone who does not know what position he will
have in that society. It helps focus the mind on the plight of the poor.

Similarly here, everybody is so concerned to get their bit in the core
spec (everyone wants a constitution where thinking they will start of
as billionairs) that they don't care to make the extension mechanism
clear and well defined.

> 		bob wyman

Well to tell the truth I don't understand what the fuss about the head
element in the feed is (which is not quite to say that I am fully behind
the head_in_entry). I think the head element is just a special entry.

Henry Story