Subject: [NNTP] Snapshot 6

From: "Clive D.W. Feather" <[email-masked]>
Date: Thu, 6 Jan 2005 12:36:15 +0000

Okay, snapshot 6 is available in the usual place:

<http://www.davros.org/nntp-texts/draft25.pre-6.txt>
<http://www.davros.org/nntp-texts/draft25.pre-6.html>

This one completes the structural changes; unless there are any silly errors
or things people want adjusting, it's probably ready to go out as the next
official draft. Russ: I suggest that I take apply comments between now and
the morning (my time) of Monday 10th and then send it to the I-D editor.

Summary of changes from snapshot 5:

* No outstanding issues.
* Definition of "keyword" moved from 3.1 to 2.
* Statement in 3.1 that a server MUST implement all commands removed.
* 502 = "wrong mode" made illegal.
* 3.6 moved to 3.3.
* Paragraph moved to 3.3.1 from CAPABILITIES.
* TRANSIT capability renamed as IHAVE.
* Assorted changes in 3.3.2 (was 3.6.2), including new capability labels.
* Comment modifiers removed.
* Minor wording change in 3.3.3 (was 3.6.3) in respect of -MODE-READER
  (also renamed from -MODE_READER).
* Term "standard extension" deleted.
* Changes to the requirements for extensions in 3.3.4 (was 3.6.4).
* Changes to the cross-references in the table in 3.3.5 (was 3.6.5).
* New section 3.4 replaces old 3.3:
    3.4  Mandatory and Optional Commands
      3.4.1  Reading and Transit Servers
      3.4.2  Mode switching
* 3.4 and 3.5 renumbered as 3.5 and 3.6.
* Minor wording changes in 3.5 (was 3.4).
* <> put round formal syntactic entities in several places.
* Command definitions lose the "reader-only" concept but gain either
  "is mandatory" or "indicating capability" lines. Similar changes to
  the table in 7.6.2.
* Assorted changes to CAPABILITIES examples.
* MODE READER completely rewritten.
* LISTGROUP moved to between GROUP and LAST and description simplified to
  refer to GROUP (which has a minor consequential wording change).
* Paragraph moved from 7.5 ("Time") to DATE.
* Section 8 reorganised and renamed:
  8.  Article field access commands
    8.1  Article metadata
      8.1.1  The :bytes metadata item
      8.1.2  The :lines metadata item
    8.2  Database consistency
    8.3  OVER
    8.4  LIST OVERVIEW.FMT
    8.5  HDR
    8.6  LIST HEADERS
  (Since LISTGROUP has moved, this eliminates the concept of "standard
  extension"). Text from the old 8.3 and 8.4 distributed to appropriate
  places, and consequential changes made to the command descriptions.
* 8.3 to 8.6 more consistent about using "fields" rather than "headers
  and metadata".
* Text in OVER distinguishing "Lines" from ":lines" expanded.
* Section 9 (Deprecated commands) removed and subsequent sections
  revert to the numbering used up to snapshot 3.
* New section 9.8 "Extensions and Validation" describes how an extension
  author should provide syntax that seamlessly merges in; minor changes
  elsewhere in section 9 (was 10) to ease this.
* Wording change in the acknowledgement of DRUMS and RFC 1869.
* Fix spelling of Ken's name.
* New appendix B "Summary of Commands".
* Old appendix B renamed as C.
* Old appendix C "Formal specification of the standard extensions" deleted.

-- 
Clive D.W. Feather  | Work:  <[email-masked]>   | Tel:    +44 20 8495 6138
Internet Expert     | Home:  <[email-masked]>  | Fax:    +44 870 051 9937
Demon Internet      | WWW: http://www.davros.org | Mobile: +44 7973 377646
Thus plc            |                            |

---

From: Ken Murchison <[email-masked]>
Date: Thu, 06 Jan 2005 13:09:11 -0500

> <http://www.davros.org/nntp-texts/draft25.pre-6.html>

Comments based mostly on the changes since pre-5.

3.2.2:

Since LISTGROUP, OVER and HDR have become optional capabilities rather 
than extensions (which I'm fine with), I'd like to see some kind of 
uniformity with the way that optional reader commands are advertised. 
We have LISTGROUP, OVER and HDR advertised with their own capabilities, 
yet POST is an argument to READER.  Obviously OVER needs to be a 
separate capability because it has its own argument (unless we tweak 
syntax), but LISTGROUP and HDR could simply be additional arguments to 
READER.

Similarly, if we change the IHAVE capability back to TRANSIT, then the 
STREAMING extension could simply be a STREAMING argument to TRANSIT 
rather than its own capability.

However, for simplicity and consistency it probably makes more sense to 
just make POST its own capability.  Having said all that, I won't argue 
strenuously for any changes.

3.3.5:

- The meaning of IHAVE should probably just say "IHAVE command 
available" rather than "Transit commands available".

- The meaning of SASL should probably say "Supported SASL mechanisms" 
rather than "SASL capabilities".

3.4.2:

Objection to "MUST advertise the READER capability with the -MODE-READER 
modifier" noted in another thread.

5.2.3:

Second example advertises LIST HEADERS but not HDR, as I feared might 
happen in real implementations.  I'm going to renew my concern over 
having two capabilities for a single command (HDR & LIST HEADERS, READER 
& LIST ACTIVE, etc), but I'll defer to the consensus.

6.3.1.1:

Should the indicating capability be "READER POST" instead of just "POST"?

9:

Not really a problem with the base doc, but with the STREAMING doc. 
Section 9 states that <command-continuation> follows a server 
<response>.  We use <command-continuation> with the TAKETHIS command, 
but this is clearly incorrect because we don't get a server response 
before we send the article.  I'm guessing that the syntax for TAKETHIS 
should probably be something along the lines of:

takethis-command = "TAKETHIS" WS message-id EOL encoded-article

However, since <command-line> includes EOL, the above would be broken 
because we'd be ending with  "." CRLF *(SP / TAB) CRLF

The closest we can get with the current syntax elements is:

takethis-command = "TAKETHIS" WS message-id EOL content-lines "."

but this still allows *(SP / TAB) after the "." which isn't correct.

Thoughts?  Perhaps the easiest thing to do is to adjust the sequence 
listed in 9.

-- 
Kenneth Murchison     Oceana Matrix Ltd.
Software Engineer     21 Princeton Place
716-662-8973 x26      Orchard Park, NY 14127
--PGP Public Key--    http://www.oceana.com/~ken/ksm.pgp

---

From: Russ Allbery <[email-masked]>
Date: Thu, 06 Jan 2005 15:31:14 -0800

Clive D W Feather <[email-masked]> writes:

> This one completes the structural changes; unless there are any silly
> errors or things people want adjusting, it's probably ready to go out as
> the next official draft. Russ: I suggest that I take apply comments
> between now and the morning (my time) of Monday 10th and then send it to
> the I-D editor.

Works for me.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

---

From: Russ Allbery <[email-masked]>
Date: Thu, 06 Jan 2005 15:34:47 -0800

> uniformity with the way that optional reader commands are advertised. We
> have LISTGROUP, OVER and HDR advertised with their own capabilities, yet
> POST is an argument to READER.  Obviously OVER needs to be a separate
> capability because it has its own argument (unless we tweak syntax), but
> LISTGROUP and HDR could simply be additional arguments to READER.

> STREAMING extension could simply be a STREAMING argument to TRANSIT rather
> than its own capability.

> strenuously for any changes.

Hm, just making POST its own capability is very tempting, I agree.  I like
that better than doing what you describe in the first two paragraphs
above.  (My general feeling is that arguments to capabilities should be
used sparingly and avoided where possible.  Note that the total amount of
data returned by the server is exactly the same whether it's a separate
capability or an argument to another capability.  *grin*)

---

From: Ken Murchison <[email-masked]>
Date: Thu, 06 Jan 2005 18:40:27 -0500

Russ Allbery wrote:
> above.

Agreed.  I was thinking (typing) out loud and realized at the end that 
making POST its own capability was the path of least resistance.

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Fri, 7 Jan 2005 14:01:14 +0000

Russ Allbery said:
> used sparingly and avoided where possible.

To answer both of you at once, the difference between the approaches is
that making POST a separate capability allows it to be advertised by a
transit server, while making POST an argument only allows it to be
advertised by those servers that fully support the READER group.

[A server can implement anything, but it can't put it into CAPABILITIES
output unless it follows the rules.]

So the arguments approach allows us to put one level of dependency into
place without special wording that can be misunderstood.

On that basis, I'd argue that POST and LISTGROUP should both be arguments
of READER. OVER and HDR should be separate capabilities because they *can*
be used by message-ID (and note that we still have HEAD in the mandatory
commands).

[If we wanted to make OVER an argument of READER, we can simply say that
either one argument OVER or two arguments OVER MSGID, in that order,
appear in the argument list. There's no rule that says that arguments have
to be independent and reorderable; it's just how things have turned out so
far.]

STREAMING is a reasonable replacement for IHAVE, so it should also be
separate.

> Note that the total amount of

Not so. READER and POST as two separate capabilities is one octet more than
READER POST as a single capability. *bigger grin*

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Fri, 7 Jan 2005 14:32:24 +0000

Addressed elsewhere.

> available" rather than "Transit commands available".

Oops. Missed that one.

> rather than "SASL capabilities".

Okay.

> modifier" noted in another thread.

Will address there.

> happen in real implementations.

Bad editing on my part; sorry. I've now checked all examples using
CAPABILITIES, and LIST always includes entries implied by other
capabilities.

> I'm going to renew my concern over 

Technically there is a difference. A server could provide LIST OVERVIEW.FMT
without OVER, or LIST NEWSGROUPS without the reader commands. Unless we
categorically want to forbid it, which has problems of its own, I think we
just live with that.

More generally, I've been converted to the "show all LIST variants in the
capabilities line" view.

> Should the indicating capability be "READER POST" instead of just "POST"?

You mean "READER"? I was thinking in terms of "POST is part of the READER
group of commands", but I suppose "READER with argument POST" is actually
the best way to put it.

> before we send the article.

Ah, I didn't think of this case. My bad.

I've changed the base document to read:

In 9:
   * the client sends an instance of <command-line>;
+  * if the client is one that immediately streams data [1], it sends
+    an instance of <command-datastream>;
   * the server sends an instance of <response>;
[...]
+  [1] There are no commands in this specification that immediately
+  stream data, but this non-terminal is defined for the convenience of
+  extensions.

In 9.2:

+  command-datastream = UNDEFINED
+    ; not used, provided as a hook for extensions

In 9.8:

   command
      for each new command other than a variant of the LIST command -
      the syntax of each command MUST be compatible with the definition
      of <X-command>;
+  command-datastream
+     for each new command that immediately streams data;
   command-continuation 
      for each new command that sends further material after the initial
[...]
+  the non-terminal <UNDEFINED> is deliberately not defined.

> I'm guessing that the syntax for TAKETHIS 

No, it would become:

    command =/ takethis-command
    command-datastream =/ takethis-datastream

    takethis-command = "TAKETHIS" WS message-id
    takethis-datastreadm = encoded-article

---

From: Ken Murchison <[email-masked]>
Date: Fri, 07 Jan 2005 09:46:21 -0500

> advertised by those servers that fully support the READER group.

Good point.

> of READER.

Agreed.

> OVER and HDR should be separate capabilities because they *can*

Would a transit server really support either of these?

> to be independent and reorderable; it's just how things have turned out so

True, but having two arguments tied together makes me uncomfortable.  If 
we were to make OVER part of reader, I'd rather see something like OVER 
and OVER-MSGID or OVER-RANGE and OVER-MSGID.

Like I said earlier, I'm not going to make a big stink about this, but I 
think making LISTGROUP an argument to READER and leaving HDR and OVER as 
separate capabilities is a good compromise (for all of the reasons Clive 
mentioned).

---

From: Ken Murchison <[email-masked]>
Date: Fri, 07 Jan 2005 09:52:18 -0500

Works for me.

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Fri, 7 Jan 2005 14:59:26 +0000

>>   takethis-datastreadm = encoded-article

Minus the spurious 'd', that is.

Do you want a copy of my bugfixed version of abnfvalidate?

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Fri, 7 Jan 2005 15:06:36 +0000

> Would a transit server really support either of these?

Pass, but I could make an argument for it if I tried.

> separate capabilities is a good compromise (for all of the reasons Clive 

Works for me. Done.

---

From: "Charles Lindsey" <[email-masked]>
Date: Fri, 7 Jan 2005 14:52:30 GMT

>rather than its own capability.

Yes, it seems more logical for the capability to be TRANSIT, because
extensions might offer more transit commands (TAKETHIS, for example).

Moreover, you MAY provide the IHAVE command even when in READER mode, but
does this make you a transit/general server (i.e. do you advertise the
TRANSIT/IHAVE capability)? I would have thought not (certainly not
TRANSIT) because the exact semantics of IHAVE may be slightly different
when in READER mode, and you wouldn't expect TAKETHIS suddenly to appear
as well.

>available" rather than "Transit commands available".

Indeed.

>& LIST ACTIVE, etc), but I'll defer to the consensus.

Yes. If we no longer have OVER and HDR as extensions, then we have lost
the ability to say "If you provide this extension, then you MUST provide
BOTH of the following commands". I don't think we ever intended to lose
the ability to say that.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: Russ Allbery <[email-masked]>
Date: Fri, 07 Jan 2005 09:58:26 -0800

>>> OVER and HDR should be separate capabilities because they *can* be
>>> used by message-ID (and note that we still have HEAD in the mandatory
>>> commands).

> Pass, but I could make an argument for it if I tried.

I doubt it, but I also don't want to deal with having different arguments
for the two different OVER styles.

>> Like I said earlier, I'm not going to make a big stink about this, but
>> I think making LISTGROUP an argument to READER and leaving HDR and OVER
>> as separate capabilities is a good compromise (for all of the reasons
>> Clive mentioned).

> Works for me. Done.

Sounds good here.

---

From: Russ Allbery <[email-masked]>
Date: Fri, 07 Jan 2005 10:02:06 -0800

> Moreover, you MAY provide the IHAVE command even when in READER mode,
> but does this make you a transit/general server (i.e. do you advertise
> the TRANSIT/IHAVE capability)? I would have thought not (certainly not

The way I look at this is that really it would be ideal to have every
command have its own capability.  We don't do that for reader commands
because there are too many of them, so we use the READER bundle with some
options instead.  But for transit, there's only one command, so we should
do things in the ideal way.

(It's worth noting that even a mode switching server may provide IHAVE in
reader mode -- but it may be a much less efficient IHAVE than what that
server wants high-traffic peers to use.  I'm not sure how to best capture
this in the language.  I'm not sure that the current MODE READER language
that focuses on whether one wants to use IHAVE is the right concept.
Blah.)

>> having two capabilities for a single command (HDR & LIST HEADERS,
>> READER & LIST ACTIVE, etc), but I'll defer to the consensus.

> BOTH of the following commands". I don't think we ever intended to lose

You mean OVER and HDR as capabilities, not extensions, right?

---

From: Ken Murchison <[email-masked]>
Date: Fri, 07 Jan 2005 15:24:41 -0500

> <http://www.davros.org/nntp-texts/draft25.pre-6.html>

3.3.3:

I believe I have discovered a problem with the capability modifier 
mechanism.  If my server supports AUTHINFO, how do I advertise that 
AUTHINFO USER and AUTHINFO SASL PLAIN require TLS, but all other SASL 
mechanisms can be used without TLS?

We can't/shouldn't have the same capability advertised twice, e.g.:

-483 AUTHINFO USER
AUTHINFO SASL
-483 SASL PLAIN
SASL CRAM-MD5 DIGEST-MD5

Do we allow modifiers to be interspersed in arguments, e.g.:

AUTHINFO SASL -483 USER
SASL CRAM-MD5 DIGEST-MD5 -483 PLAIN

Granted, in this case a -483 isn't necessary to tell the client that it 
needs TLS before AUTHINFO USER or AUTHINFO SASL PLAIN, since it can 
infer this by the presence of STARTTLS and the absence of AUTHINFO USER 
and SASL PLAIN.
-- 
Kenneth Murchison     Oceana Matrix Ltd.
Software Engineer     21 Princeton Place
716-662-8973 x26      Orchard Park, NY 14127
--PGP Public Key--    http://www.oceana.com/~ken/ksm.pgp

---

From: Ken Murchison <[email-masked]>
Date: Fri, 07 Jan 2005 15:29:38 -0500

>>>as separate capabilities is a good compromise (for all of the reasons

Since we're all in agreement on this, we could revisit allowing NEWNEWS 
to be an optional argument to READER.  I only bring this up because 
AFAIK INN still allows NEWNEWS to be optional.

---

From: Russ Allbery <[email-masked]>
Date: Fri, 07 Jan 2005 12:59:33 -0800

> AFAIK INN still allows NEWNEWS to be optional.

Yeah, I keep thinking about that.  That may be a good idea.

---

From: [email-masked] (Peter Robinson)
Date: Sun, 9 Jan 2005 21:58:15 +0000

Clive D.W. Feather <[email-masked]> wrote:

> Okay, snapshot 6 is available in the usual place

I've not looked at this draft for a while, and I must say I'm very
enthusiastic about the recent work.  A few comments though...

| 3.3.4 Extensions
[...]
| An extension is a package of associated facilities, often but not
| always including one or new commands.

'one or more commands'?
'one or more new commands'?

| 3.3.5 Initial IANA register

The initial registry includes an entry for the generic modifier, -label.
That, combined with 3.3.3, attempts to define a modifier for *every*
capability label, whether it makes sense or not.  [And 'use of the
facility with this capability label' doesn't make sense e.g. for a
capability that doesn't add new commands.]

[in 3.3.3:]
| -label
|   the capability can be enabled by use of the facility with this
|   capability label; in particular, -MODE-READER indicates that the
|   server is a mode-switching server and the capability can be enabled
|   by use of the MODE READER command.

If -label is left in the initial registry as it is, surely that mandates
the server returning all sorts of oddness, and requires a client to make
some kind of sense of it.  E.g.:

VERSION 2
-AUTHINFO READER       [Should be -480 READER.]
-READER OVER           [Which READER command must client send first?]
-HDR XNEWCOMMAND       [Must the client send HDR or LIST HEADERS?]
-IMPLEMENTATION READER [!?]
etc.

I'd take -label out of the initial registry, but keep the wording from
3.3.3 defining what it should mean in general terms, as long as it's
made clear that it's not actually defining any modifiers on it own.

Also, there should probably also be an explicit -MODE-READER entry in
the list in 3.3.3 since it says

| The following such modifiers are defined by this specification:

but includes only -label and not -MODE-READER.

| 5.1.1 [Initial Connection] Usage

Shouldn't this say that it MUST NOT be pipelined?

What was the consensus on whether to make POST, OVER and HDR arguments
to READER or separate capabilities in their own right?  FWIW, I'd rather
make them each independent capabilites, and also require the server to
include the corresponding arguments with the LIST capability as
appropriate.

Peter

---

From: [email-masked] (Peter Robinson)
Date: Sun, 9 Jan 2005 21:59:09 +0000

Ken Murchison <[email-masked]> wrote:

> 3.3.3:
> 
> If my server supports AUTHINFO, how do I advertise that 

Hmm.  It's true that 5.2 CAPABILITIES currently says

| The server MUST NOT list the same capability twice in the response.

but it violates that in its own examples by listing READER twice:

|  [C] CAPABILITIES 
|  [S] 101 Capability list: 
|  [S] VERSION 2 
|  [S] READER 
|  [S] -480 READER POST 
|  [S] LIST ACTIVE NEWSGROUPS 
|  [S] AUTHINFO SASL 
|  [S] SASL GSSAPI 
|  [S] .

First of all, that's one good argument for having POST as a separate
capability in its own right, but IMHO it would still be good to allow
this kind of functionality in the spec.  Can we relax the restriction on
having repeated capabilities to allow multiple copies as long as they
all have different modifiers and/or arguments?  Can it be defined
unambiguously?

> SASL CRAM-MD5 DIGEST-MD5 -483 PLAIN

That's another way, though I think I'd prefer doing it with repeated
capability lines.

[snip]

Peter

---

From: "Charles Lindsey" <[email-masked]>
Date: Mon, 10 Jan 2005 13:00:14 GMT

>You mean OVER and HDR as capabilities, not extensions, right?

I mean that if the OVER command is merely an "optional" command, and the
LIST OVERVIEW.FMT is also an "optional" command (or an "optional"
parameter to LIST, which comes to the same thing), then there is nothing
to prevent a server offerring either, neither, or both (and at least one
of those 4 possibilities is a nonsense).

In the days when OVER was an extension, then those commands were never
"optional" (they were obligatory if you implemented the extension).

I just want to be sure that the nonsense case(s) does not arise. I am not
sufficiently au fait with the finer details of capabilities and extensions
to be sure that they cannot arise. So please can somebody ensure that it
is covered somehow.

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Tue, 11 Jan 2005 09:56:17 +0000

> We can't/shouldn't have the same capability advertised twice, e.g.:

Actually, that's how I intended things to be done, and there are examples
saying so. Wording changed to:

    The server MUST NOT list the same capability twice in the response
    without modifiers or with the same set of modifiers.

> SASL CRAM-MD5 DIGEST-MD5 -483 PLAIN

That would require us to put a restriction on the form of arguments. I'd
rather not do that.

> infer this by the presence of STARTTLS and the absence of AUTHINFO USER 

No, because:
(a) the server might offer XENCRYPT rather than STARTLS;
(b) even after a privacy layer is in effect, there's no requirement that
    the server will offer AUTHINFO USER or SASL PLAIN.

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Tue, 11 Jan 2005 09:59:14 +0000

Ken Murchison said:
> to be an optional argument to READER.

We made the policy decision to keep NEWNEWS mandatory (as it is in 977)
years ago. I'd rather not reopen this argument.

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Tue, 11 Jan 2005 10:11:39 +0000

Charles Lindsey said:
> I just want to be sure that the nonsense case(s) does not arise.

This is not possible.

Under the old regime, nothing stopped a server creating an XOVER extension
that did OVER but not LIST OVERVIEW.FMT. Or an XOVERFMT extension that did
LIST OVERVIEW.FMT but not OVER. However, if it advertised the OVER
extension in LIST EXTENSIONS, then it had to provide both commands.

Under the new regime, the same applies mutans mutandis.

Here's the actual situation:

  If server provides               then it

  * Neither                        MUST NOT advertise OVER capability
                                   MUST NOT advertise OVERVIEW.FMT ... [1]
  * Only LIST OVERVIEW.FMT         MUST NOT advertise OVER capability
                                   MUST advertise OVERVIEW.FMT argument
  * Only OVER                      MUST NOT advertise OVER capability
                                   MUST NOT advertise OVERVIEW.FMT argument
  * Both                           SHOULD [2] advertise OVER capability
                                   MUST advertise OVERVIEW.FMT argument

  If server advertises             then it

  * OVERVIEW.FMT argument          MUST provide LIST OVERVIEW.FMT

  * OVER capability                MUST provide both commands
                                   MUST advertise OVERVIEW.FMT argument.

  If server does not advertise     then it

  * OVERVIEW.FMT argument          MUST NOT provide LIST OVERVIEW.FMT
                                   MUST NOT advertise OVER capability

  * OVER capability                MAY provide either command
                                   SHOULD NOT provide both commands

[1] ... argument to the LIST capability (omitted to keep text on one line).

[2] "The server MUST ensure that the capability list accurately reflects
the capabilities (including extensions) currently available." but this
wording was written in the context of including and excluding a capability
as the extension becomes available or not during a session. Later on we
say "If the label is not included, the server MAY support none or some of
the commands, but SHOULD NOT support all of them."

Okay?

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Tue, 11 Jan 2005 10:12:41 +0000

> BOTH of the following commands". I don't think we ever intended to lose

We never had it. See other message.

We had the ability to say "if you advertise this extension, then you MUST
provide all of it". We still have that ability.

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Tue, 11 Jan 2005 10:42:12 +0000

> 'one or more new commands'?

The latter. Good catch.

> The initial registry includes an entry for the generic modifier, -label.

That's right. Registration of a capability label automatically defines the
modifier as well.

> capability that doesn't add new commands.]

It might do. For example, a "named article" extension could add "name"
to the syntax for article-ref and range-ref, but not add any new commands.
Okay, I can't see why some other feature might be conditional on using
named commands, but there's no reason to rule it out a priori.

> some kind of sense of it.

It *permits* all sorts of oddness, but the client is ALWAYS allowed to
ignore it.

> -AUTHINFO READER       [Should be -480 READER.]

Actually there is a subtle difference: -480 is more inclusive than -AUTHINFO.

> -READER OVER           [Which READER command must client send first?]

Any of them?

> -HDR XNEWCOMMAND       [Must the client send HDR or LIST HEADERS?]

See the specification of HDR.

> -IMPLEMENTATION READER [!?]

Hmm.

> made clear that it's not actually defining any modifiers on it own.

You've convinced me that there's a problem, but I'm not convinced you've
provided the right fix. In particular, I don't want to have to have
everything registered twice.

Okay, the text has been changed to:

    -label   the capability can be enabled by use of the facility with
             this capability label; the server MUST NOT use this modifier
             except as specified by the definition of the latter
             capability.

and the -MODE-READER wording taken to a separate paragraph.

Similar wording change made to the 401 response code, which has the same
issue.

> but includes only -label and not -MODE-READER.

MODE-READER *is* a label.

> Shouldn't this say that it MUST NOT be pipelined?

I suppose it wouldn't hurt.

> to READER or separate capabilities in their own right?

It was decided to make POST and LISTGROUP arguments to READER, but leave
OVER and HDR separate.

---

From: Ken Murchison <[email-masked]>
Date: Tue, 11 Jan 2005 12:35:58 -0500

>     the server will offer AUTHINFO USER or SASL PLAIN.

Technically you're correct, but I find it highly unlikely that a server 
would advertise AUTHINFO and a security layer such as STARTTLS or 
XENCRYPT, but *not* implement AUTHINFO USER or SASL PLAIN under a 
security layer.

---

From: Ken Murchison <[email-masked]>
Date: Tue, 11 Jan 2005 12:43:53 -0500

>                                    SHOULD NOT provide both commands

Isn't providing LIST OVERVIEW.FMT or LIST HEADERS without OVER or HDR 
(respectively) nonsense?  I really don't see how the former are useful 
without the latter.

This goes back to my opposition to having OVERVIEW.FMT and HEADERS as 
arguments to the LIST capability.  By having these arguments, we open 
ourselves up to this kind of nonsense.  I believe that OVER and LIST 
OVERVIEW.FMT should be tied together via the OVER capability (you can't 
implement one without the other), and similarly for HDR.  In fact you 
almost state as much in pre-6 because the indicating capability for LIST 
OVERVIEW.FMT is OVER.

---

From: Russ Allbery <[email-masked]>
Date: Tue, 11 Jan 2005 10:51:18 -0800

> XENCRYPT, but *not* implement AUTHINFO USER or SASL PLAIN under a

Hm, I can actually see a very paranoid Kerberos site simply not allowing
SASL PLAIN no matter what encryption layer is negotiated, requiring all
clients to instead use GSSAPI so that their password never leaves their
system.  I certainly wouldn't expect this to be a common configuration,
but if the site is, say, using Kerberos with smart cards exclusively as
their authentication mechanism, it could happen.

I'm still unhappy with the modifiers for a different reason, though,
namely that it doesn't seem to me like it adds that much useful
information to a client unless we make them mandatory (which I think we're
agreed on not doing).  If something like:

    AUTHINFO SASL
    -480 READER POST

is there, then the client knows it *might* be able to post after
authenticating, but still doesn't know if it actually will be able to post
afterwards since it's going to depend on the credentials presented.  If
this line isn't there and the server just says:

    AUTHINFO SASL
    READER LISTGROUP

then the client doesn't know for certain that authentication *won't* help,
since the server may just not be using modifiers.  So it feels like the
modifiers are both insufficiently precise to allow the client to make
complete decisions as to whether or not to try something, and at the same
time not sufficient to help the client decide *not* to try something.

It seems like the -- modifiers are the only ones that are really explicit
about telling the client that it doesn't need to try things, if they're
present.

I'm not really sure how I'd use these capabilities as a client even if I
wanted to.  Clive, could you possibly sketch out how you anticipate a
client using those capabilities to make decisions?

It still seems like a lot of complexity for little gain to me.

---

From: [email-masked] (Peter Robinson)
Date: Wed, 12 Jan 2005 00:10:22 +0000

>     without modifiers or with the same set of modifiers.

OK, I like that functionality, but it does mean that the following two
capability responses say different things about the core READER commands
(e.g. ARTICLE):

[S] -480 READER POST
[S] READER
[S] .

[S] -480 READER POST
[S] .

Is everyone happy with that?  Is it sufficiently clear/obvious that the
-480 applies only to POST in the first example?

And what does this:

[S] -480 READER
[S] -483 READER
[S] .

mean?  Is it one or the other, or is it both required to activate
READER?

Peter

---

From: [email-masked] (Peter Robinson)
Date: Wed, 12 Jan 2005 00:10:22 +0000

> named commands, but there's no reason to rule it out a priori.

What I'm saying is that, for each capability, whether it defines a
-label capability modifier, and how to activate it, should be explicitly
stated in the specification for that capability.  And that the wording
about the -label wildcard should be for guidance on how such modifiers
should be defined, and should not define any capability modifiers in
itself.

> Actually there is a subtle difference: -480 is more inclusive than -AUTHINFO.

So it does.  I had assumed -AUTHINFO was a bug not a feature.  Are you
expecting clients that do auth via AUTHINFO alone to understand the
-AUTHINFO modifier as commonly as they would -480 from reading this
specification?  And that it's perfectly normal for a server to use
-AUTHINFO not -480?  But I suppose that's a job for the authinfo
document, not this one.

> > -READER OVER           [Which READER command must client send first?]

If so, I'd say this needs to be stated in the specification for the
READER capability.

> See the specification of HDR.

?

Do you mean the single sentence from 3.3.2 defining the commands
provided by the HDR capability, or the definition of the HDR command in
8.5, or somewhere else?  I don't know this document as well as you, so I
may be missing it, but I don't see the answer to that question anywhere.

[...]

> everything registered twice.

I didn't imagine that you actually wanted a -label modifier for
essentially every capability!

>              except as specified by the definition of the latter

I'm happy with that text.  Does that mean that -MODE-READER is now the
only capability-derived modifier defined by this document?

> and the -MODE-READER wording taken to a separate paragraph.

Excellent.

[...]

> I suppose it wouldn't hurt.

To clarify what I meant:  the document says elsewhere in the text that
it MUST NOT be pipelined, but in that case it's supposed to say so in
the Usage section for the 'command'.

> It was decided to make POST and LISTGROUP arguments to READER, but leave

OK.

Peter

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Wed, 12 Jan 2005 05:24:05 +0000

[...]
> should be defined, and should not define any capability modifiers in

I did end up agreeing with you, with the minor detail that I didn't want to
force the registry to duplicate lots of stuff.

[...]
> So it does.  I had assumed -AUTHINFO was a bug not a feature.

It was.

While I really hope Ken doesn't do this - and will object if he did - in
principle the AUTHINFO document *could* say "-AUTHINFO means this
capability is only available after AUTHINFO authentication, and not after
any other kind of authentication". But I really don't expect anyone to want
to do it that way.

[...]

My next few responses were getting ironic; it was meant to indicate that I
understood your point.

In particular:
> I don't know this document as well as you, so I

is because the document *doesn't* answer it.

> essentially every capability!

Well, I did, but only as a placeholder for when it's useful.

> only capability-derived modifier defined by this document?

Correct.

>> I suppose it wouldn't hurt.

and, to be clear, I've made the change.

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Wed, 12 Jan 2005 05:25:58 +0000

> capability responses say different things about the core READER commands

Right.

> -480 applies only to POST in the first example?

A client is allowed to ignore the -480 line, so anyone writing a client
should be clear that READER is available in the first instance whether or
not you parse the modifier or not.

> mean?  Is it one or the other, or is it both required to activate

Either, because:

  [S] -483 -480 READER

means "privacy, then authentication" required.

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Wed, 12 Jan 2005 05:31:57 +0000

Russ Allbery said:
> agreed on not doing).

Yes, we're agreed on not doing it. At least, not in this round of things;
the requirements on clients are simply not to barf on them.

> If something like:
> afterwards since it's going to depend on the credentials presented.

Right, but it also says "if you want to post, you need to authenticate
first" (and "SASL authentication is available"). In particular, if the
client doesn't have credentials to present, it knows it can't post.

> If
> since the server may just not be using modifiers.

True. Which means the client has to fall back to "suck it and see"; the
whole point of capabilities was to get rid of that, surely?

I gree that, so long as they're not mandatory, clients have to fall back to
heuristics. But I see this as something that will encourage the use of
modifiers; by your logic, I could claim that there's no point in having
CAPABILITIES at all because the server can always implement a command
without listing it in the output.

> client using those capabilities to make decisions?

I'll try, but I don't have time this minute.

---

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 12 Jan 2005 11:18:43 GMT

>(respectively) nonsense?  I really don't see how the former are useful 

Well I suppose a server that implemented only the old XOVER might still
want to provide LIST OVERVIEW.FMT.

>ourselves up to this kind of nonsense.

No, I like them as arguments to the LIST capability, but it would be nicer
if it could be arranged that this nonsense was forbidden by some other
means. Though it will not be the end of the world if it isn't (I can't see
any server in Real Life actually doing such things).

---

From: Ken Murchison <[email-masked]>
Date: Wed, 12 Jan 2005 09:56:28 -0500

> any server in Real Life actually doing such things).

We already do, its the OVER capability.  OVER mandates that LIST 
OVERVIEW.FMT is available, the separate LIST OVERVIEW.FMT capability is 
redundant and potentially confusing.

I really don't see why its useful.  I assume that a client would only 
use LIST OVERVIEW.FMT if it intends to use OVER.  If it intends to use 
OVER, it will be looking for the OVER capability, since it, and only it, 
dictates whether OVER is available.  All the client can glean from the 
presence of LIST OVERVIEW.FMT is that LIST OVERVIEW.FMT is available 
(which the OVER capability already tell us), it tells us nothing about OVER.

If a LIST variant is useful by itself, then I can see why we would 
advertise it separately, but if it is implicitly tied to other 
functionality which has its own capability, then its makes no sense to 
me whatsoever.  In my experience, allowing for ambiguity in a protocol 
is a *bad* thing.

---

From: Ken Murchison <[email-masked]>
Date: Wed, 12 Jan 2005 09:58:08 -0500

> any other kind of authentication". But I really don't expect anyone to want

I don't have any plans to discuss capability modifiers in the extension 
docs.

---

From: Russ Allbery <[email-masked]>
Date: Wed, 12 Jan 2005 11:24:35 -0800

> use LIST OVERVIEW.FMT if it intends to use OVER.

Well, in the case of LIST OVERVIEW.FMT, the client may be intending to use
XOVER, as Charles points out.  That being said, it's hard to construct an
example of where that would imply that advertising LIST OVERVIEW.FMT but
not OVER would be something a server would do.

> me whatsoever.  In my experience, allowing for ambiguity in a protocol

The problem is that we have two different things with which we can be
consistent:  we can make a simple rule across the board that all list
varients are advertised via the LIST capability, or we can have one
capability per linked set of commands.  Either way we go, we break the
other consistency principle.

---

From: Ken Murchison <[email-masked]>
Date: Wed, 12 Jan 2005 14:41:26 -0500

> not OVER would be something a server would do.

But if a client is still using XOVER, it probably isn't going to use 
CAPABILITIES anyways, and advertising LIST OVERVIEW.FMT is pointless.

If the client does use CAPABILITIES and still uses XOVER rather than 
OVER, than I don't think we really care about this client (since it 
should be trivial to make an XOVER client an OVER client).

> other consistency principle.

My simple rule would be that all list variants not tied to other 
capabilities are advertised via the LIST capability.

---

From: [email-masked] (Peter Robinson)
Date: Wed, 12 Jan 2005 23:07:40 +0000

> My next few responses were getting ironic; it was meant to indicate that I

Sorry, that obviously went way over my head!

> > Does that mean that -MODE-READER is now the

Excellent.

Peter

---

From: [email-masked] (Peter Robinson)
Date: Wed, 12 Jan 2005 23:07:41 +0000

> > [S] -480 READER POST
> > [S] READER
> > [S] .
> > 
> > [S] -480 READER POST
> > [S] .
> 
> > Is it sufficiently clear/obvious that the

OK, so one rule is that a line without a modifier always trumps a line
with.  I'm thinking about how to parse the whole lot, including modifers
in the general case.

> means "privacy, then authentication" required.

So the rule is that conflicting modifiers (or lack of modifiers) are
always combined with 'or'.  That's good enough to be unambiguous in
every case.  Any chance of putting that idea in the document somehow?

Peter

---

From: "Charles Lindsey" <[email-masked]>
Date: Thu, 13 Jan 2005 11:19:34 GMT

>other consistency principle.

Indeed. I prefer the first, Ken prefers the second. If nobody else chips
in, you get to toss the coin :-) .

---

From: [email-masked] (Peter Robinson)
Date: Thu, 13 Jan 2005 22:51:34 +0000

Charles Lindsey wrote:

> Indeed. I prefer the first, Ken prefers the second.

I prefer the first (for all LIST variants to be arguments to the LIST
capability) more from a sense of neatness than anything else.  As long
as the document is clear one way or the other, I don't see this as a
huge issue.

Peter

---

From: [email-masked] (Kai Henningsen)
Date: 15 Jan 2005 12:04:00 +0200

> I just want to be sure that the nonsense case(s) does not arise. I am not

Why? What interoperability problem would arise from "the nonsense  
case(s)"?

If none, then it's not our job to prevent them.

MfG Kai

---

From: Russ Allbery <[email-masked]>
Date: Mon, 17 Jan 2005 13:18:25 -0800

>> If this line isn't there and the server just says:
>> 
>>     AUTHINFO SASL
>>     READER LISTGROUP
>> 
>> then the client doesn't know for certain that authentication *won't*
>> help, since the server may just not be using modifiers.

> whole point of capabilities was to get rid of that, surely?

My point is that you can't get rid of it completely, so I'm not sure how
much getting rid of it in a few more cases really helps.  You still don't
know whether you can post after authentication; you just know that you
*might* be able to.  How many cases are there where you can't actually
post even after authentication but authentication is listed as a
capability?  In my experience, that's an extremely unusual configuration.

I'm weighing that against the additional complexity, with questions like
what it means when you're listing the same capability multiple times with
different modifiers and the like.

---

From: "Charles Lindsey" <[email-masked]>
Date: Mon, 17 Jan 2005 20:25:20 GMT

>Why? What interoperability problem would arise from "the nonsense  

The meaning of LIST OVERVIEW.FMT is not well defined (indeed not defined
at all) if the OVER capability is not there (modulo strange quirks arising
from the old XOVER).

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Tue, 18 Jan 2005 06:48:17 +0000

Russ Allbery said:
> much getting rid of it in a few more cases really helps.

If we have a mechanism for making things clear to the client then,
hopefully, over time it will get used. Yes, there's a certain amount of
bootstrap problem (clients won't use it until servers do, and vice versa),
but that's the case with many new features.

> You still don't
> *might* be able to.

I don't think that's a fair comparison. The idea of

    -480 READER POST

is to say "you need to authenticate to post, you can't post without it". If
the client doesn't have good enough credentials, that's a different matter
entirely.

> How many cases are there where you can't actually

How about a server carrying private read-only groups? Authenticating is
necessary to get at those groups, but that doesn't mean posting will be
available.

But, in any case, we should NOT be building such assumptions into a
specification, especially if they're not documented. You appear to be
saying (and please correct me if I'm wrong) that our document should
contain wording to the effect of:

    If the READER capability is advertised without POST, and the
    AUTHINFO capability or some other authentication capability is
    also advertised, then the client MAY assume that posting will become
    available after authentication.

Is that really something to build into the core protocol?

> different modifiers and the like.

I'm weighing against the hidden assumptions.

A capability line beginning "-480" basically means: "if you authenticate
yourself properly, this capability line will reappear with the -480
stripped (and, if relevant, merged with any other line with the same
label)". Ditto -483 and -label.

---

From: Russ Allbery <[email-masked]>
Date: Sat, 29 Jan 2005 15:19:10 -0800

>> My point is that you can't get rid of it completely, so I'm not sure
>> how much getting rid of it in a few more cases really helps.

> hopefully, over time it will get used.

You're missing the point.  Even if this protocol is used extensively and
reliably by the server, it *still* doesn't get rid of "try it and see"
because there's no way for this protocol to express what will happen after
STARTTLS or AUTHINFO commands are used.  The results are going to vary
depending on the strength of encryption, the authentication credentials,
and other issues.  The client is *still* going to have to try it and see
in may cases; they just may have a few cases where it's no longer
ambiguous.

In other words, all this additional complexity doesn't actually solve the
problem for the client except in a few edge cases.  Is that worth it?

>> You still don't know whether you can post after authentication; you
>> just know that you *might* be able to.

> is to say "you need to authenticate to post, you can't post without
> it". If the client doesn't have good enough credentials, that's a
> different matter entirely.

Why would this be a different matter entirely?  It's precisely my point!
Even given the above, the client has no more reliable information than if
the capabilities were just:

    READER
    AUTHINFO SASL

In both cases, the client knows that authentication is available and
posting is not.  In both cases, posting may or may not be available after
authentication.  In both cases, the client has to just try authentication
and see.  We're giving the client exactly the same amount of information
either way.

>> How many cases are there where you can't actually post even after
>> authentication but authentication is listed as a capability?  In my
>> experience, that's an extremely unusual configuration.

> necessary to get at those groups, but that doesn't mean posting will be

And, as I said, this is an extremely unusual configuration.

>     AUTHINFO capability or some other authentication capability is also
>     advertised, then the client MAY assume that posting will become

No, that is absolutely NOT what I am saying.  What I'm saying is that
listing READER without POST and listing AUTHINFO gives the client exactly
the same information in practice as listing -480 READER POST, but without
introducing all this additional complexity into the CAPABILITIES syntax.
And by that, I mean that the client reaction to both of those situations
has to be the same; the addition of capability modifiers doesn't allow the
client to make different, more accurate decisions.

The only case in which it allows the client to make different, more
accurate decisions is in the case where posting is unavailable and
authentication cannot possibly change the client's ability to post, which
as I stated above is an unusual situation and not one that, IMO, is seen
frequently enough to justify introducing new syntax to deal with it.