Subject: Re: One reason we have duplicates entries is that we have duplicate feeds...

From: Lenny Domnitser <[email-masked]>
Date: Mon, 18 Apr 2005 21:17:21 -0400

Regarding the potential for a DoS attack with stealing somebody else's
GUIDs, no special hacks [1] are needed. Any decent aggregator would
trust IDs with a grain of salt.

* If the IDs of two entries from different feeds are the same **and
the data is the same** show it in only one feed.
* If the IDs of two entries from different feeds are the same and the
data is different, this can be one of two things: a DoS attack or an
update to the entry. In either case, show the updated version without
wiping out the old one.
* If two entries with the same ID are from the same feed, this can't
be a DoS, so only show the new one, unless the user wants it
otherwise.

[1]: http://www.imc.org/atom-syntax/mail-archive/msg14094.html

---

From: "Bob Wyman" <[email-masked]>
Date: Mon, 18 Apr 2005 23:52:59 -0400

> update to the entry. In either case, show the updated version without
	What if the two entries are both found in the same feed but have
different source feeds identified in their atom:source elements? These two
entries are clearly *from* different feeds, yet they might be found in a
single feed.
	The problem here is, of course, that 4.1.1 of the specification
says: "atom:feed elements MUST NOT contain atom:entry elements with
identical atom:id values." Thus, one of the two entries should not have been
written to the feed. Which of the two entries should NOT have been written?
Note: There is no good answer here. You might have an attack or you might
just have a "secondary" copied entry that is being inserted. In any case,
there isn't enough info to know which to accept and which to reject. So...
	I believe that the prohibition against multiple entries with
identical IDs should be relaxed to say:
	"atom:feed elements MUST NOT contain atom:entry elements with
identical atom:id values unless each such entry contains an atom:source
element which is different from that of any of the other entries with which
the entry shares identical atom:id values. Two or more atom:source elements
MUST only be considered different if they each contain an atom:link element
with a rel value of "self" which identifies different feed URIs."
	Yes, I know that language is very tortured... In any case, you can't
build a synthetic feed generator which is robust against attack unless the
proposed relaxation is supported. ID's must be feed specific where feed is
defined as the source feed -- not the feed in which the entry is found.

	bob wyman

---

From: "Bob Wyman" <[email-masked]>
Date: Mon, 18 Apr 2005 23:51:42 -0400

bob wyman

---

From: Bill de hÓra <[email-masked]>
Date: Tue, 19 Apr 2005 09:13:17 +0100

> entries are clearly *from* different feeds, yet they might be found in a

Not clearly; one or more of them might be misattributed.

> identical IDs should be relaxed to say: [...]

We discussed this and the consensus said that the prohibition should 
stand (iirc I was pro multiple IDs in a feed). As it happens I think 
error handling will be contextual in reality - I really doubt client 
aggregators will drop an entire planet* Atom feed because two entries 
have the same ID.

> defined as the source feed -- not the feed in which the entry is found.

I'm going to think about it some more but atm I'm not sure what you're 
proposing helps against DOS.

cheers
Bill

---

From: Henry Story <[email-masked]>
Date: Tue, 19 Apr 2005 18:12:46 +0200

I don't see where the consensus was by the way. And I never saw any 
vote on the
issue. Like most issues one gets the impression that Atom is run on 
some pretense
democracy. Some people have made up their minds for god knows what 
reason, and others
are just here to follow. The more we chatter on this group the more we 
lend credence
to the notion that there was some consulting done on the issues. But 
any carefully
crafted arguments put in favor of any issue are ignored. No wonder any 
attempt to
be precise about issues gets ignored here, as that may just reveal the 
corruption
behind the scenes.

As far as I can remember not one good reason was put forward as to why 
there should
be a restriction of one entry with the same id per feed. In fact I was 
nearly convinced
that the consensus was to drop the restriction.

So I call for a real open vote on the issue.

Henry Story

---

From: Bill de hÓra <[email-masked]>
Date: Tue, 19 Apr 2005 17:27:36 +0100

> So I call for a real open vote on the issue.

You don't need to call for a vote, just ask the chairs/editors who keep 
track of such matters, about the particular specification.

If you can point out to me my recollection of the consensus on that 
issue is incorrect, then do so. Beyond that, dark mutterings about 
corruption carry little weight with me.

cheers
Bill

---

From: Henry Story <[email-masked]>
Date: Tue, 19 Apr 2005 18:53:41 +0200

On 19 Apr 2005, at 18:27, Bill de hÓra wrote:
> You don't need to call for a vote, just ask the chairs/editors who 
> keep track of such matters, about the particular specification.

Well we need some objective way to tell what the consensus is. Going 
from your or my recollection does not seem to help, since by reading 
this list we end up with completely
different points of view on where the consensus is.

> corruption carry little weight with me.

We never had a vote so what can I point to? I could point to some 
people putting forward some
arguments and then others putting others forward. The reason we have 
arguments is in order
to convince someone, so how do I know what the consensus is at the end 
of the argument? People
may have changed their position throughout the argument, especially if 
there are highly intelligent people putting very good reasons forward.

On the issue of not allowing more than one entry per feed with the same 
id, I heard some
very bizaare reasons, such as there being a major problem with web 
architecture that nobody has
heard of. But is this a reason that has convinced anyone?

---

From: Robert Sayre <[email-masked]>
Date: Tue, 19 Apr 2005 12:58:41 -0400

Henry Story wrote:
> vote on the issue. Like most issues one gets the impression that Atom
>  is run on some pretense democracy.

There is no voting in the IETF. Please read the beginners' documentation 
found on ietf.org. Here's a good one:
http://www.ietf.org/tao.html

> Some people have made up their minds for god knows what reason, and
> others are just here to follow. The more we chatter on this group the
> more we lend credence to the notion that there was some consulting
> done on the issues. But any carefully crafted arguments put in favor
> of any issue are ignored. No wonder any attempt to be precise about
> issues gets ignored here, as that may just reveal the corruption
> behind the scenes.

This statement is incredibly inappropriate and offensive. You may have a 
valid technical point somewhere, but mudslinging is not going to help 
your case.

BTW, Graham's suggestion sounds reasonable.

Robert Sayre

---

From: Paul Hoffman <[email-masked]>
Date: Tue, 19 Apr 2005 10:20:46 -0700

At 6:12 PM +0200 4/19/05, Henry Story wrote:
>I don't see where the consensus was by the way.

Correct. There was no consensus to remove the current wording.

>  And I never saw any vote on the
>issue.

Correct: there was no voting, there was a consensus call. I'm not 
sure how you could have missed it: PaceRepeatIdInDocument was 
discussed *heavily* in the past few months. The conclusion was:

PaceRepeatIdInDocument
Lots of discussion, more -1's than +1's.
DISPOSITION: No consensus, close it.  But now we have a problem, in 
that this removed ambiguity in one direction, just closing it leaves 
the ambiguity.  So the only logical conclusion is that the WG is 
directing the editors to put language in that explicitly forbids 
entries with duplicate <atom:id> in an <atom:feed>.

Earlier, PaceMinimalEntryVersioning was dropped because of an even 
split of views.

>  Like most issues one gets the impression that Atom is run on some pretense
>democracy.

Democracy != consensus. (I don't know if there is a math symbol for 
"emphatically does not equal".)

>  Some people have made up their minds for god knows what reason, and others
>are just here to follow.

...and others have open minds and try to follow debates.

>  The more we chatter on this group the more we lend credence
>crafted arguments put in favor of any issue are ignored.

-1. Disagreeing != ignoring.

>As far as I can remember not one good reason was put forward as to 
>why there should
>be a restriction of one entry with the same id per feed. In fact I 
>was nearly convinced

See the discussion summary above and see the archives.

>So I call for a real open vote on the issue.

If you are unhappy with the consensus process and want to move to a 
voting process, you need to take the Atom work to a venue other than 
the IETF. The same is true if you want WG participants to be the ones 
calling for consensus instead of the WG chairs. If what you meant was 
"I want to move Atom out of the IETF so I can call a vote", please 
change the subject line of the thread.

If what you wanted was for the chairs to again call for consensus on 
the already-closed Pace, please say so explicitly, possibly without 
using the phrase "corruption behind the scenes" in your request.

--Paul Hoffman, Director
--Internet Mail Consortium

---

From: Graham <[email-masked]>
Date: Tue, 19 Apr 2005 17:30:12 +0100

I'm in favour of allowing duplicate ids when the source-id is different 
to simplify creating merged feeds, which would allow the client to 
figure out what to do. Under any other circumstance, definitely not.

Graham

---

From: Henry Story <[email-masked]>
Date: Tue, 19 Apr 2005 19:59:16 +0200

Ok. I am sorry. I thought I had made a really good case for a simple 
argument to allow
multiple entries with the same id in a feed, and thought it had in fact 
made it into the spec.
I then discovered that it still had not.

I cleazrly just have no idea how one goes around convincing this group 
of anything, so I'd better not participate anymore.

On 19 Apr 2005, at 19:20, Paul Hoffman wrote:
> Correct: there was no voting, there was a consensus call. I'm not sure 
> how you could have missed it: PaceRepeatIdInDocument was discussed 
> *heavily* in the past few months. The conclusion was:
>
> PaceRepeatIdInDocument

Yes. I remember the above point very well. And I was very surprised by 
it. After all
here is a decision being made to close a pace and take a decision with 
regard to it that is
not explicitly stated in that Pace without then putting it back forward 
to the group.

I in fact did question that decision and this went on in a long thread 
PaceRepeatIdInDocumentSolution and so on. I suppose the result of that 
argument
was not clear either.

Again I suppose I just don't understand how to make an argument that 
can convince
the group.

Henry Story

---

From: Henry Story <[email-masked]>
Date: Tue, 19 Apr 2005 20:56:15 +0200

Just to end on a positive note, I'll +1 this suggestion by Graham.

Henry

On 19 Apr 2005, at 18:30, Graham wrote:
> I'm in favour of allowing duplicate ids when the source-id is 
> different to simplify creating merged feeds, which would allow the 
> client to figure out what to do. Under any other circumstance, 
> definitely not.
>
> Graham
>

---

From: "Bob Wyman" <[email-masked]>
Date: Tue, 19 Apr 2005 14:59:20 -0400

> proposing helps against DOS.
	My proposal says that things are considered the same only if found
in the same feed. This is, I believe, the only way to prevent someone from
maliciously erasing someone else's entries with a blocking entry having the
same ID as the entry under attack. Admittedly, the approach will lead to a
number of false negatives in comparisons if people do not consistently
publish "self" links when they build synthetic feeds. The only way I can see
to improve on this situation is to support the "rel='superset'" link that I
discussed earlier. If the superset relationship can, in fact, be expressed,
the rule would change to:

atom:feed elements MUST NOT contain atom:entry elements with identical
atom:id values unless each such entry contains an atom:source element which
is different from that of any of the other entries with which the entry
shares identical atom:id values. 
Two atom:source elements MUST only be considered different if they each
contain an atom:link element with a rel value of "self" which identifies
different feed URIs and neither of the feeds identified is known to be a
'superset' of the other feed. A superset feed is identified in a subset feed
by a link element with rel value of "superset". Synthetic feed generators
and aggregators MAY use other sources of information, public or private, to
determine the relationships between feeds.
If two atom:feed elements have identical atom:id values yet only one of them
has an atom:source element with a rel value of "self", that element which
contains the atom:source element with rel value of "self" SHOULD be
preferred over the other.

	bob wyman

---

From: Antone Roundy <[email-masked]>
Date: Tue, 19 Apr 2005 13:42:50 -0600

> preferred over the other.

This illustrates the superiority of "self" links over atom:id in 
identifying a feed.  If one were to attempt to use atom:id the way Bob 
is proposing to use a "self" link above to protect again DOS attacks, 
it wouldn't help (or could be easily circumvented), because there's no 
way to resolve disputes between feeds claiming the same atom:id.  But 
it's easy to verify whether a feed is really the owner of it's "self" 
link.

There would still be an attack vector, where one would use the atom:id 
of an entry that had left the legitimate feed's sliding window.  Anyone 
subscribed directly to both would be protected if their feed, reader 
were to ignore all entries claiming to be from other feeds to which 
they are subscribed.  But those receiving the legitimate entry via an 
aggregator that didn't do that or wasn't subscribed to both sources 
might still have trouble resolving conflicts.

Of course, a feed may move over time, so atom:feed/atom:id is useful, 
but really only as a hint that feeds found at two locations might be 
the same.

...which reminds me of PaceFeedIdOrSelf, PaceFeedIdOrAlternate and 
PaceCoConstraintsAreBad.  Out of curiosity, are these actually going to 
be considered at some point in the process, or is it just too late?

---

From: "Bob Wyman" <[email-masked]>
Date: Tue, 19 Apr 2005 17:41:37 -0400

> I'm in favour of allowing duplicate ids when the source-id is different
	+1
	This is essentially what I was trying to get at with the horribly
tortured language that I recently proposed. Clearly, I support Graham's
suggestion and hope that someone with better skill at the language can
figure out how to write this into the spec.
	Note: At PubSub, we expect to ensure that every entry we publish
once we move to Atom V1.0 will contain an atom:source with a "self" link.

		bob wyman

-----Original Message-----
From: [email-masked] [mailto:[email-masked]]
On Behalf Of Graham
Sent: Tuesday, April 19, 2005 12:30 PM
To: Henry Story
Cc: atom-syntax Syntax'
Subject: Re: call for a vote - was: One reason we have duplicates entries is
that we have duplicate feeds...

Graham