Subject: Re: [NNTP] Article Reinstatement

From: Sabahattin Gucukoglu <[email-masked]>
Date: Sun, 12 Jun 2011 04:57:40 +0100

I think we really need an effort to roll all these errata into 3977bis.

On 14 May 2011, at 20:37, Julien ÉLIE wrote:
>>> The low watermark can not decrease.
>> 
>> Even when the client disconnects?
> 
> Yes, the low water mark SHOULD NOT decrease.
> 
> Section 6.1.1.2:
> 
>   Except when the group is empty and all three numbers are zero,
>   whenever a subsequent GROUP command for the same newsgroup is issued,
>   either by the same client or a different client, the reported low
>   water mark in the response MUST be no less than that in any previous
>   response for that newsgroup in this session, and it SHOULD be no less
>   than that in any previous response for that newsgroup ever sent to
>   any client.  Any failure to meet the latter condition SHOULD be
>   transient only.  The client may make use of the low water mark to
>   remove all remembered information about articles with lower numbers,
>   as these will never recur.
> 
>>> It is not an RFC-compliant behaviour. min could not have been set
>>> to 2.

Yes, (as Clive correctly points out) very sensible.  But how do we distinguish, inside the server, an article that *might* be reinstated and one that just expired?

>> So what does a server do when the lowest-numbered article is pending
>> reinstatement and the client requests it?
> 
> The news server just answers the article does not exist:
> 
>   First form (message-id specified)
>     430                   No article with that message-id
> 
>   Second form (article number specified)
>     423                   No article with that number
> 
>   Third form (current article number used)
>     423                   No article with that number
> 
> Note that reinstatement can be done on other articles than the lowest-numbered article.

Is there any language which specifically allows the use of these codes for reporting an invalid article number because of a (possible) reinstatement?  From here it looks like just an implementation detail.  I'd otherwise assume that the server simply pretends the article isn't valid even though it might be, in the future.  There is a semantic difference that the server hides.

>> Looking back at my original message, I see that I made an assumption
>> that the high and low watermarks were always valid articles, but I
>> can see now that the RFC has us covered in the special case of
>> reinstatement.  Good. :-)
> 
> And also the fact that articles can expire and no longer be present in the news spool.  The low water mark does not necessarily point to a still retrievable article at the time an ARTICLE command is sent.

That's fine; such an article really would be no longer existent.

> news.test has 5 messages, min=1 max=5 count=5 1. remove #1 min=1
>>> max=5 count=4 2. reinstate #1 and remove #3 min=1 max=5 count=4
>> 
>> By which we understand that an invalid article response will result
>> from requesting a removed article, yes?
> 
> Not "invalid", but "inexistent".

Yes, OK, this needs to be rolled into bis.

> So a client that comes between action 1 and action 2 will never see
>>> #1; it won't ask for #1 afterwards because the high water mark is
>>> 5.  The same way as a client coming after action 2 will never see
>>> #3. But is it really a problem?
>> 
>> Taking into account the use of article reinstatement, probably not in
>> many cases, but you can imagine that if you are a news puller or
>> offline reader, you want to go forward in the article stream and not
>> worry about what came before.  Any circumstance that results in
>> article deletion should be permanent, from the client's perspective.
>> But yes, it's just me, picking nits. :-)
> 
> It is up to the client to decide how to implement its retrieval of articles.  If it assumes the article stream always go forward (which happens nearly all the time), then it will miss reinstatements.  That's all.  The client just prefers rapidity to exhaustivity (sending a LISTGROUP every time, or every n times, it connects to a news group).

Yes, indeed.  This seems to be how I've come away with this.  I used to do such exhaustive checks, but noting that reinstated articles were removed and then the same numbers used on the reinstatements, I decided that the client would benefit from the speed, if we assume the reader is acting on behalf of the user at the time the articles are retrieved, rather than read (for caching).  This doesn't always work, but it does work 99% of the time.  Especially since many servers just don't do the reinstatement thing at all.

Cheers,
Sabahattin

---

From: Julien ÉLIE <[email-masked]>
Date: Sun, 12 Jun 2011 11:12:30 +0200

Hi Sabahattin,

> I think we really need an effort to roll all these errata into

… and move the NNTP protocol to Draft Standard at the same time.

> But how do we
> distinguish, inside the server, an article that *might* be reinstated
> and one that just expired?

I think it is implementation-specific, and therefore not to be dealt 
with in an RFC.

>>> So what does a server do when the lowest-numbered article is
>>> pending reinstatement and the client requests it?
>>
> Is there any language which specifically allows the use of these
> codes for reporting an invalid article number because of a (possible)
> reinstatement?

The article does not exist.  The language is to respond 430/423/420.
(Please note that I wrote twice 423 in my previous mail; it was 423+420.)

Maybe you were asking for a new code telling that the article does not 
exist but may be reinstated?
It could be something to look at in RFC3977bis.  However, clients do not 
currently expect this possible new response code to existing commands 
like ARTICLE.

> This doesn't always
> work, but it does work 99% of the time.  Especially since many
> servers just don't do the reinstatement thing at all.

Though I do not have figures, I would tend to think that it is far more 
than 99%.  "Many servers don't do the reinstatement thing" might also be 
an euphemism :)

-- 
Julien ÉLIE

« Un voyage de mille lieues commence toujours par un premier pas. »
   (Lao Zi)

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Sun, 12 Jun 2011 13:05:14 +0100

> I think we really need an effort to roll all these errata into 3977bis.

I'm not clear that there is an error here.

> Yes, (as Clive correctly points out) very sensible.  But how do we distinguish, inside the server, an article that *might* be reinstated and one that just expired?

That's up to the implementer.

>> The news server just answers the article does not exist:

Exactly.

> Is there any language which specifically allows the use of these codes for reporting an invalid article number because of a (possible) reinstatement?

Until and unless the article is reinstated, it does not exist. Full stop.

Reinstatement is either:
(1) a rare event (e.g. because of an erroneous cancel)
(2) a side-effect of running a server farm that isn't perfectly in sync.

> That's fine; such an article really would be no longer existent.

Until and unless an article has been reinstated, it does not exist.

> Yes, OK, this needs to be rolled into bis.

What needs to be rolled in? 6.2.1.2 makes it perfectly clear that article
numbers are invalid once the article is removed, and then become valid
again if it is reinstated.

> This doesn't always work, but it does work 99% of the time.  Especially since many servers just don't do the reinstatement thing at all.

"Those who do not learn from history are doomed to repeat it."

The reinstatement wording was carefully worked out after experience with
real users and real server farms that weren't in perfect sync. You might
examine demon.service from 1995 to 1999 for some examples of upset users
(no, I can't give exact cites from this long ago).

-- 
Clive D.W. Feather          | If you lie to the compiler,
Email: [email-masked]     | it will get its revenge.
Web: http://www.davros.org  |   - Henry Spencer
Mobile: +44 7973 377646