Subject: Re: Re: [NNTP] [2501] Reader commands in transit servers and vice-versa

From: "Clive D.W. Feather" <[email-masked]>
Date: Thu, 2 Dec 2004 10:38:18 +0000

Okay, new text for 3.3. This replaces the bulleted list and the paragraph
either side of it; the rest of 3.3 is unchanged so far. This closes issue
2501 (subject to any comments on this wording).

    The specific description of each command will describe it as a
    "transit command", a "reader command", or a "general command". An
    NNTP server may be "transit-only", "reader-only", or "general-use".

    * A general-use server MUST fully implement all the commands in this
      specification.
    * All servers MUST recognise all the commands in this specification
      (and so MUST NOT return a 500 response code for them).
    * All servers MUST fully implement all general commands.
    * A transit-only server MUST fully implement all transit commands,
      but does not need to implement reader commands.
    * A reader-only server MUST fully implement all reader commands,
      but does not need to implement transit commands.

    For this purpose, "commands in this specification" includes the
    commands in any extensions that the server supports (except where
    the command is optional) but not those in Section 8 where the server
    does not support the relevant extension. 

    Where a transit-only or reader-only server does not implement a reader
    or transit command, it MUST always generate a 502 generic response
    code.  Otherwise the command must be fully implemented as specified; 
    a server MUST NOT only partially implement any of the commands in this
    specification. 

    Note: some commands have variants that require other commands to be
    used first. If the former command is implemented but the latter is not,
    the former MUST still generate the relevant specific response code and
    not the 502 generic code. For example, if ARTICLE is implemented but
    GROUP is not, the correct response to "ARTICLE 1234" remains 412.

-- 
Clive D.W. Feather  | Work:  <[email-masked]>   | Tel:    +44 20 8495 6138
Internet Expert     | Home:  <[email-masked]>  | Fax:    +44 870 051 9937
Demon Internet      | WWW: http://www.davros.org | Mobile: +44 7973 377646
Thus plc            |                            |

---

From: Richard Clayton <[email-masked]>
Date: Thu, 2 Dec 2004 11:20:26 +0000

In message <[email-masked]>, Clive D.W.
Feather <[email-masked]> writes

>      but does not need to implement transit commands.

Calling it "reader only" when you'd expect POST to work (so that end-
users with normal sorts of existing software can expect to interoperate
with it usefully) is rather obscure :(

Surely you can't mean for people to have lots of "general-use" systems
since who would wish to implement all of the gubbins around IHAVE (and
other transit type chatter) just so they can provide POST

ie: the categories need to be something like "transit", "customer" and
"general-use", with appropriate command categorisation following from
that!

- -- 
richard                                              Richard Clayton

They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety.         Benjamin Franklin

---

From: Russ Allbery <[email-masked]>
Date: Thu, 02 Dec 2004 10:26:35 -0800

Richard Clayton <[email-masked]> writes:
> Clive D.W. Feather <[email-masked]> writes

> "general-use", with appropriate command categorisation following from

Could you rephrase that?  I honestly have no idea what you just said or
what you're driving at.  If someone didn't want to implement IHAVE, they'd
provide a reader-only server, not a general-use server.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

---

From: Ken Murchison <[email-masked]>
Date: Thu, 02 Dec 2004 13:41:31 -0500

> provide a reader-only server, not a general-use server.

I think Richard is arguing that POST is a transit command, not a reader 
command, because its a "push" of data rather than a "pull" of data and 
therefore the term "reader-only" excludes POST.  If my understanding of 
his statements are correct, I disagree with this assessment.  The POST 
command is used by a "reading agent" to inject a news article and IHAVE 
is used by a "transit server" to propogate articles.

-- 
Kenneth Murchison     Oceana Matrix Ltd.
Software Engineer     21 Princeton Place
716-662-8973 x26      Orchard Park, NY 14127
--PGP Public Key--    http://www.oceana.com/~ken/ksm.pgp

---

From: Richard Clayton <[email-masked]>
Date: Fri, 3 Dec 2004 12:36:20 +0000

In message <[email-masked]>, Russ Allbery
<[email-masked]> writes
>Could you rephrase that?

I suspect that a "reader only" configuration will allow POST (but may
not allow IHAVE).

This is confusing because POST is "writing" ... so I think its a dumb
name (based on old concepts) to put into a new spec :(

>I honestly have no idea what you just said or

A new point (which I hope I can express clearly): "end-user" systems (as
opposed to transit servers run by ISPs) usually use POST, but sometimes
use IHAVE...  making servers fully implement everything in order to
accommodate legacy users of IHAVE doesn't seem likely -- so I expect one
will see "reader only" systems with extensions to cope with IHAVE.

- -- 
richard @ highwayman . com                       "Nothing seems the same
                          Still you never see the change from day to day
                                And no-one notices the customs slip away"

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Fri, 3 Dec 2004 14:00:07 +0000

> with it usefully) is rather obscure :(

POST is a "reader command".

> other transit type chatter) just so they can provide POST

This isn't required anywhere.

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Fri, 3 Dec 2004 14:06:12 +0000

> name (based on old concepts) to put into a new spec :(

So you think that we shouldn't use the term "reader", but don't actually
think we should change the specification other than by changing this word?

Other opinions? I don't like "customer", but can we find a perhaps-less
confusing name?

> will see "reader only" systems with extensions to cope with IHAVE.

At present IHAVE is the *only* transit command, so that would actually be a
general-purpose server.

In case we ever have more transit commands in the core specification, do we
want a separate capability for IHAVE? Or do we see this (IHAVE but not
other transit commands) as an unlikely enough case to ignore?

---

From: Ken Murchison <[email-masked]>
Date: Fri, 03 Dec 2004 13:49:57 -0500

> Other opinions? I don't like "customer", but can we find a perhaps-less

I really don't see a problem with READER and TRANSIT, since the are just 
protocol tokens which only the client software (and developer) ever sees.

I suppose I could live with something like USER" and TRANSIT or USER and 
PEER or USER_MODE and PEER_MODE, but I really don't see the point. 
Everyone already knows that POST is a reader (user-agent) command and 
READER gives us symmetry with MODE READER.

> other transit commands) as an unlikely enough case to ignore?

This seems like a bridge we only need to cross if we ever come to it. 
In order for a new transit command to be added to the core spec, the new 
document, with presumably a new VERSION 3 token, could handle this.

---

From: Russ Allbery <[email-masked]>
Date: Fri, 03 Dec 2004 10:51:48 -0800

> PEER or USER_MODE and PEER_MODE, but I really don't see the
> point. Everyone already knows that POST is a reader (user-agent) command
> and READER gives us symmetry with MODE READER.

These are also the terms already used by NNTP software implementors.

> This seems like a bridge we only need to cross if we ever come to it. In
> order for a new transit command to be added to the core spec, the new

Agreed; any other transit commands short of a base protocol revision would
be extensions and would therefore come with their own capability labels.
So this is a problem that we can safely defer.

---

From: "Andrew - Supernews" <[email-masked]>
Date: Sat, 04 Dec 2004 15:51:26 +0000

I can't help thinking that way too much verbiage and complexity is
being spent on this now, especially given the desire to relegate
MODE READER to the dustbin of history.

I'm going to propose the following alternative approach (pretty much
the same in semantics, just described differently):

NNTP can be considered as two closely related protocols, which we'll
call NNRP and NNSP. NNRP is as defined in the base spec, except that
MODE READER is explicitly a no-op. NNSP is exactly identical to NNRP
except that all of the base commands, other than QUIT, IHAVE(?) and
CAPABILITIES, are optional (with a usage note saying that they are
usually not implemented except for a small handful).

Our alternative transit-only port (433) is already registered as
"nnsp". Port 433 connections are assumed to be NNSP. For port 119,
CAPABILITIES (if supported) tells you which.

On an NNRP connection, MODE READER must succeed and do nothing. On an
NNSP connection, MODE READER may be unrecognized or fail, but if it
succeeds, the result is an NNRP connection _in exactly the state it
would be in after the initial banner_.

>From the point of view of the specification, this could all be
documented as follows, without getting into contortions regarding
"mode-switching" and its effects on other commands:

 - an introductory paragraph defining the difference, mentioning
   port 433, etc

 - description of the relevent CAPABILITIES output

 - description of MODE READER.

I don't think there's any need at all to get into an extended
discussion of transit vs. reader servers. All that matters is to point
out that newsfeeding is done using what amounts to a very tiny subset
of the protocol, and under what circumstances MODE READER is needed
(for legacy servers and for new servers based on CAPABILITY output).
Going into too much detail is silly for something like this.

-- 
Andrew, Supernews
http://www.supernews.com

---

From: "Charles Lindsey" <[email-masked]>
Date: Sat, 4 Dec 2004 19:26:40 GMT

>READER gives us symmetry with MODE READER.

I think I would prefer USER and PEER. Those are the terms that the world
out there uses.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: "Charles Lindsey" <[email-masked]>
Date: Sat, 4 Dec 2004 19:23:41 GMT

>At present IHAVE is the *only* transit command, so that would actually be a

Not necessarily, because the semantics of IHAVE is typically different in
reader mode than when in transit mode. Not different from a pure NNTP POV,
but different in what happens behinds the scenes, e.g. it might behave
more like an injecting agent, performing extra checks, inserting
NNTP-Posting-Host (or Injection-Info) etc. For example, uni-berlin behaves
like that.

>other transit commands) as an unlikely enough case to ignore?

Yes, when IHAVE is available in reader mode, I would like to see a
capability to tell me so.

---

From: Russ Allbery <[email-masked]>
Date: Mon, 06 Dec 2004 10:11:12 -0800

Charles Lindsey <[email-masked]> writes:
> "Clive D.W. Feather" <[email-masked]> writes:

>> In case we ever have more transit commands in the core specification,
>> do we want a separate capability for IHAVE? Or do we see this (IHAVE
>> but not other transit commands) as an unlikely enough case to ignore?

> capability to tell me so.

That's what the TRANSIT capability does.

Given that IHAVE is at present the *only* transit command, I'm wondering
if it might be more future-proof to just call the TRANSIT capability IHAVE
instead.  What do people feel about that?

If we did that, we could also potentially take Andrew's simplification
proposal, at least to some degree, and reduce matters to three cases
(reader commands supported, reader commands supported after MODE READER,
reader commands not supported).  But regardless of whether we went that
route, I think renaming the capability may be a good idea.

---

From: Ken Murchison <[email-masked]>
Date: Mon, 06 Dec 2004 16:23:40 -0500

> route, I think renaming the capability may be a good idea.

Having digested this weekend's posts, here's my current thinking.

-  We have a set of mandatory "core" commands which *all* servers MUST 
implement.  The set of "core" commands is defined by the VERSION 
capability and as of VERSION 2, these are:  CAPABILITIES, QUIT, HEAD, 
STAT, LIST ACTIVE

-  All other commands are optional and MUST be advertised by an 
appropriate capability.  All optional commands (or set of commands) are 
orthorgonal and may be advertised independent of each other.

-  We have a set of optional "reader" commands which, if implemented, 
MUST be implemented in its entirety.  The set of "reader" commands is 
also defined by the VERSION capability and is advertised by the READER 
capability.  The VERSION 2 "reader" commands currently are: GROUP, LAST, 
NEXT, ARTICLE, BODY, DATE, HELP, NEWGROUPS, NEWNEWS, LIST NEWSGROUPS

(Note: a case might be made that both DATE and HELP should be "core" 
commands, and that NEWNEWS should be split from the "reader" commands)

- The other optional capabilities currently are:

MODE_READER (MODE READER required)
POST (might be an argument to READER)
IHAVE
LIST (framework for ACTIVE.TIMES, DISTRIB.PATS, etc)
LISTGROUP
OVER (OVER + LIST OVERVIEW.FMT)
HDR (HDR + LIST HEADERS)

I've included the last 3 in this list because given the fact that the 
line between capabilities and extensions has been blurred, I don't think 
it makes any sense to define "extensions" within the base document.  I'm 
currently thinking that an "extension" is any command (or set of 
commands) defined outside of the base document.  The base document 
should just define mandatory and optional commands, the framework for 
optional LIST variants, and the framework for extensions.

---

From: "Andrew - Supernews" <[email-masked]>
Date: Tue, 07 Dec 2004 00:53:37 +0000

>>>>> "Ken" == Ken Murchison <[email-masked]> writes:

 Ken> Having digested this weekend's posts, here's my current
 Ken> thinking.

 Ken> - We have a set of mandatory "core" commands which *all* servers
 Ken> MUST implement.  The set of "core" commands is defined by the
 Ken> VERSION capability and as of VERSION 2, these are: CAPABILITIES,
 Ken> QUIT, HEAD, STAT, LIST ACTIVE

You can make a good argument that even in that list, HEAD, STAT and
LIST ACTIVE don't belong. In fact a majority of transit-only
implementations don't implement at least one of those.

-- 
Andrew, Supernews
http://www.supernews.com

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 7 Dec 2004 14:15:43 GMT

>> instead.  What do people feel about that?

Yes, I think I might prefer that. There might be other transit commands
some day.

Essentially, I would like to be able to distinguish the case of
"IHAVE plus READER" from the case of "IHAVE without READER". The first is
basically a reader system (e.g. NNRP) with a (possibly differently
implemented) IHAVE feature added, and the second is definitely a genuine
transit engine.

>capability and as of VERSION 2, these are:  CAPABILITIES, QUIT, HEAD, 

I am dubious about HEAD there. And even LIST ACTIVE (don't some pure
transit engines manage without an Active file?).

>commands, and that NEWNEWS should be split from the "reader" commands)

Yes, NEWNEWS might be optional (some systems turn it off), but DATE should
be paired with NEWNEWS.

>optional LIST variants, and the framework for extensions.

Please leave OVER and HDR as "extensions". It is just too much
restructuring of the document to change that now.

---

From: Ken Murchison <[email-masked]>
Date: Tue, 07 Dec 2004 12:43:09 -0500

> implementations don't implement at least one of those.

This list is based on the fact that they are all listed as "general" 
commands in the current pre4 draft.  I make no claims as to their 
correctness, but I recall discussion that they are used by some feeding 
clients/servers.

---

From: Russ Allbery <[email-masked]>
Date: Tue, 07 Dec 2004 10:10:30 -0800

> restructuring of the document to change that now.

It's not a restructuring; it's just a change in wording.  We can leave the
document structured exactly as it is now and still do this.

---

From: Russ Allbery <[email-masked]>
Date: Tue, 07 Dec 2004 10:11:45 -0800

> correctness, but I recall discussion that they are used by some feeding

HEAD and STAT are pretty much only for debugging purposes; they don't
actually have a protocol purpose between transit servers.  LIST ACTIVE is
used sometimes to build feed patterns, but Andrew's correct that some
transit servers have no concept of it.

Not really sure what to do about this.

---

From: Ken Murchison <[email-masked]>
Date: Tue, 07 Dec 2004 13:43:55 -0500

Russ Allbery wrote:

> actually have a protocol purpose between transit servers.

So, I guess in my current line of thinking, these would be "reader" 
commands (only *guaranteed* to be implemented when READER is 
advertised), and if some sites decide to implement them for transit 
servers, they would be unadvertised.

We *could* add some protocol cruft to allow:

[C] CAPABILITIES
[S] VERSION 2
[S] IHAVE
[S] STAT MSGID
[S] HEAD MSGID
[S] LIST ACTIVE
[S] .

OR

[C] CAPABILITIES
[S] VERSION 2
[S] TRANSIT DEBUG
[S] .

But that seems like too much effort just to advertise a debugging config.

> LIST ACTIVE is

Hopefully this will be superceded by a "real" feed control extension.