Subject: Advertising not-yet-existent files... (was Re: Comments on atompub-format-08 (Modified by Tim Bray))

From: "Bob Wyman" <[email-masked]>
Date: Mon, 25 Apr 2005 00:57:32 -0400

John Cowan wrote in response to Graham:
>> The idea is that wherever possible content is internal. Also, why are 
>> you proposing advertising the existence of files that don't exist?
> It decouples the process of creating content from creating feeds.
> If I know I am going to have a chunk of content here, then I can
> pick a dereferenceable URI for it and not worry about what the content
> is until it's time for both to be published.  As long as the content
> exists when the feed is published, all is well.
	A slightly different case, but instructive: At PubSub, we have a
situation where we regularly publish data in Atom feeds even though the
"alternate" pages aren't yet available on the network.
	This happens constantly with our Earthquake feeds[1]. While each
entry we publish has most of the relevant information on an earthquake, each
entry also points to pages on the USGS site. The USGS pages provide
augmented information concerning the earthquake event in HTML. The problem
is that we regularly publish entries in our feeds at least 15 seconds before
and sometimes a few minutes before the corresponding pages are available on
the USGS site. Their content management system is a bit slower then ours
is... What we've had to do is get the USGS folk to change their "page not
found" message so that it says: "The webpage you are looking for has either
moved or may still be in preparation if it describes a recent earthquake."
You can see this message at: 
http://earthquake.usgs.gov/recenteqsUS/Quakes/nosuchpage.htm
	Most polling-based readers will not, of course, ever come across
this timing problem since their polling frequencies are vastly larger then
the 15 second normal publication delay. However, users of push-based Atom
readers (like the "Atom over XMPP" based Sidebars[2] that we supply to
PubSub users) will often see and click on a new Atom entry from PubSub
within two to three seconds of our having published the entry. (Yes, this
makes this an edge case; however, as push-based consumption of Atom
increases, it will become more common.)
	Admittedly, publishing real-time, volatile data is not yet a common
usage of Atom and push-based Atom consumers are still not common. However, I
think we're going to see more of this in the future. Certainly, we at PubSub
expect to generate more such feeds and we're certainly going to try to get
more people using push-protocols. The difficulty is that delaying publishing
until all the alternates or referenced URIs are known to exist imposes
unacceptable constraints on the system. With data like Earthquake or Tsunami
notices, we should *not* be delaying the publishing of messages simply to
conform to the specification...

	bob wyman

[1] http://www.pubsub.com/earthquakes.php
[2] http://www.pubsub.com/downloads.php