Subject: [NNTP] LISTGROUP

From: Russ Allbery <[email-masked]>
Date: Tue, 22 Mar 2005 16:05:10 -0800

Mark Crispin pointed out a couple of things about LISTGROUP that I think
are worth considering.

First, the command is pretty widely implemented, and it's not obvious
anything useful is gained by making it optional.  If we're going to make
NEWNEWS mandatory, LISTGROUP is a lot easier to implement and more widely
supported right now, and that would get rid of one of our capabilities
that we have to keep track of.

Second, LISTGROUP would be much more useful for some purposes if it took a
range as an optional second argument.  It's pretty trivial to specify that
argument:  it would be identical to the ranges that OVER and HDR take, and
LISTGROUP would function exactly as before but would limit the article
numbers returned to articles within that range.  If there are no articles
in the group within that range, an empty multiline response would be
returned but everything else about LISTGROUP would remain unchanged.

(The reason for this is that, with a client that's tracked read articles
via the normal .newsrc method and knows that, say, articles 1-4000 have
been read, entering a group with "LISTGROUP example.test 4001-" is a
fairly useful thing to do.)

Looking over both of these ideas, the only problem I had with either is
that we're very late in the process to make these sorts of changes.  I
couldn't see any objections at all other than that one, and the latter
extension is so minor, so straightforward, and so easy to describe that
I'd feel silly writing a separate document to specify an extension, adding
a capability modifier for it, etc.  The boilerplate on an I-D would be
bigger than the specification.

How do people feel about looking at these changes at this point?  (Making
LISTGROUP mandatory and adding an optional range argument as described
above.)

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

---

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 23 Mar 2005 11:21:47 GMT

>returned but everything else about LISTGROUP would remain unchanged.

I think we are a bit late for changing whether LISTGROUP is optional or
not, but I like the idea of a range parameter. Is that currently implemented
anywhere to anyone's knowledge?

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: Russ Allbery <[email-masked]>
Date: Wed, 23 Mar 2005 11:10:35 -0800

Charles Lindsey <[email-masked]> writes:

> not, but I like the idea of a range parameter. Is that currently
> implemented anywhere to anyone's knowledge?

Not so far as I know, but I'll probably implement it in INN by the end of
the day.

---

From: Ken Murchison <[email-masked]>
Date: Wed, 23 Mar 2005 14:51:29 -0500

> that we have to keep track of.

If we want to make it mandatory, the VERSION 2 or READER capabilities 
can certainly encompass that.

> returned but everything else about LISTGROUP would remain unchanged.

This is fine by me especially if we make LISTGROUP mandatory.

If we don't make it mandatory and leave it as a separate capability, 
perhaps we add an optional RANGE parameter to the LISTGROUP capability 
to advertise this.

-- 
Kenneth Murchison     Oceana Matrix Ltd.
Software Engineer     21 Princeton Place
716-662-8973 x26      Orchard Park, NY 14127
--PGP Public Key--    http://www.oceana.com/~ken/ksm.pgp

---

From: "Andrew - Supernews" <[email-masked]>
Date: Wed, 23 Mar 2005 20:00:05 +0000

>>>>> "Russ" == Russ Allbery <[email-masked]> writes:

 >> I think we are a bit late for changing whether LISTGROUP is
 >> optional or not, but I like the idea of a range parameter. Is that
 >> currently implemented anywhere to anyone's knowledge?

 Russ> Not so far as I know, but I'll probably implement it in INN by
 Russ> the end of the day.

LISTGROUP can already take an optional group parameter - what
combinations do you think should be allowed? And how do you cope with
the fact that "9-11" might actually be used as a group name?

-- 
Andrew, Supernews
http://www.supernews.com

---

From: Ken Murchison <[email-masked]>
Date: Wed, 23 Mar 2005 15:13:09 -0500

> the fact that "9-11" might actually be used as a group name?

Forgive my ignorance and/or naivety, but unless the client is actually 
trying to list articles in a toplevel group (do toplevel groups even 
contain articles?) wouldn't the group parameter always contain at least 
one hierarchy separator (".")?  If so, "." clearly is not a valid 
character for use in a range, so we *should* be able to differtiate 
between "LISTGROUP <group>" and "LISTGROUP <range>".

Alternatively, we can stipulate that the range parameter can only be 
used if a group is also specified:

LISTGROUP [group [range]]

---

From: Russ Allbery <[email-masked]>
Date: Wed, 23 Mar 2005 12:41:55 -0800

> the fact that "9-11" might actually be used as a group name?

I would require the newsgroup name if one is going to give the range
parameter.  Any other syntax would be a nightmare for backward
compatibility, unless I'm missing some beautiful solution.

---

From: "Andrew - Supernews" <[email-masked]>
Date: Wed, 23 Mar 2005 21:33:51 +0000

>>>>> "Ken" == Ken Murchison <[email-masked]> writes:

 Ken> Forgive my ignorance and/or naivety, but unless the client is
 Ken> actually trying to list articles in a toplevel group (do
 Ken> toplevel groups even contain articles?) wouldn't the group
 Ken> parameter always contain at least one hierarchy separator (".")?

One sometimes sees a flat group namespace (no '.' anywhere) used in
non-Usenet applications, usually with only a small number of groups.
One can't rely on '.' in a group name for disambiguation.

 Ken> Alternatively, we can stipulate that the range parameter can
 Ken> only be used if a group is also specified:

 Ken> LISTGROUP [group [range]]

This (as Russ also said) seems simplest from a compatibility
perspective. But it's logically broken in that you don't know what
the valid range is until you actually enter the group.

-- 
Andrew, Supernews
http://www.supernews.com

---

From: Russ Allbery <[email-masked]>
Date: Wed, 23 Mar 2005 13:51:57 -0800

> perspective. But it's logically broken in that you don't know what the
> valid range is until you actually enter the group.

Why is that logically broken?  You know what group you're dealing with in
any case, so it's easy to add the group to the command even if you're
already in the group.  (And I don't think it's the case that you don't
know the range before entering the group; the most useful range on group
entry would be NNN-, where NNN was the last message read by the client in
that group plus one.)

On the other hand, maybe this conversation implies that just adding this
feature isn't the best idea.

(I'm a lot more concerned with the merits adding a new feature at this
point than I am with making LISTGROUP mandatory, actually.)

---

From: Ken Murchison <[email-masked]>
Date: Wed, 23 Mar 2005 17:06:42 -0500

> point than I am with making LISTGROUP mandatory, actually.)

Which is why I suggested that we could leave LISTGROUP as optional and 
the range feature optional as well.  So we make "LISTGROUP" its own 
capability with an optional "RANGE" argument.

---

From: Russ Allbery <[email-masked]>
Date: Wed, 23 Mar 2005 14:21:20 -0800

> capability with an optional "RANGE" argument.

Yeah, we can do that, and it probably won't hurt anything.  Mark was
talking about the merits of just requiring a minimum feature set, and I'm
sympathetic towards that.  I'm just not sure it's a good idea to make that
change at this point.

---

From: "Ruud H.G. van Tol" <[email-masked]>
Date: Thu, 24 Mar 2005 00:07:57 +0100

Russ Allbery schreef:

> LISTGROUP [group [range[]]

Maybe the <current group> could be referred to by a single dot.

-- 
Grtz, Ruud

---

From: Matthias Andree <[email-masked]>
Date: Thu, 24 Mar 2005 01:54:04 +0100

> anywhere to anyone's knowledge?

Will the current article number (pointer) be set to the first article in
the group, as GROUP demands, or will it be set to the first article in
the range? I'm inclined to suggest the latter.

After that question is answered, the implementation will take fifteen
minutes in leafnode-2 as proof-of-concept and some minutes to document
it.

Note that the new syntax should contain the same prefix as RFC-2980 laid
out, so if a range will be given, the group parameter must be made
mandatory.

I suggest that the standard DO NOT define a special "current group"
pointer, as forking code into regular and special case increases
implementation effort and doubles the testing effort. Keep it simple.
If the newsreader cannot remember the most recently selected newsgroup,
it's broken, the standard should not take an interest in this SOEP(*).

BTW, another round of discussion about such features doesn't make a
difference. What are a few weeks versus two decades? NNTPv2 needs to
last another 19 years... so why not do it right? :-)

(*) SOEP = some-one else's problem

-- 
Matthias Andree

---

From: Russ Allbery <[email-masked]>
Date: Wed, 23 Mar 2005 17:22:14 -0800

> the range? I'm inclined to suggest the latter.

After thinking about this for a moment, I have no strong opinion, except
to say that if we want it to be range-dependent, we need to hammer out
exactly what implication that has if there are no articles in the
specified range.

> minutes in leafnode-2 as proof-of-concept and some minutes to document

Yeah, that's about how I felt about INN, and the reason why I brought it
up.  If it were any harder than that, I'd want to wait and write it up as
a separate proposal, but this is pretty trivial to do.

> it's broken, the standard should not take an interest in this SOEP(*).

Agreed.

> last another 19 years... so why not do it right? :-)

Man, I hope it won't be another 19 years.  I don't intend to disappear
after we publish; I expect to take a short break and then start looking at
gathering information about what's implemented to prepare for advancing to
draft standard status.

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Thu, 24 Mar 2005 08:36:38 +0000

> exactly what implication that has if there are no articles in the

And other questions, such as what are the effects of NEXT and LAST if they
would take the current article number out of the range?

I can live with the following:

* A range can only be provided if the group name is provided.
* The range acts as a filter on the returned results of the LISTGROUP
  command, but does not affect the server state.

In other words, it's a way to reduce the size of the response by discarding
unwanted material, but that's all.

I am strongly against anything that tries to push the envelope beyond that.

>> it's broken, the standard should not take an interest in this SOEP(*).

Actually, I'm not so sure. Modularity of code means it's sometimes awkward
to keep track of this stuff, and having an enquiry facility can be useful.
Note how you *can* find the current article number by going STAT with no
arguments.

Furthermore, LISTGROUP already needs to know the currently selected group
in case it's invoked with no arguments. This is a way to get the group name
but, of course, you then get the entire group listing dumped at you.

Can I suggest that GROUP with no arguments should be permitted? This would
improve the commonality between the two commands (to all intents and
purposes, LISTGROUP is already GROUP with more output).

-- 
Clive D.W. Feather  | Work:  <[email-masked]>   | Tel:    +44 20 8495 6138
Internet Expert     | Home:  <[email-masked]>  | Fax:    +44 870 051 9937
Demon Internet      | WWW: http://www.davros.org | Mobile: +44 7973 377646
Thus plc            |                            |

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Thu, 24 Mar 2005 08:40:15 +0000

Russ Allbery said:
> NEWNEWS mandatory,

EXCUSE ME!

NEWNEWS has always been mandatory. RFC 977 section 3 paragraph 3:

    Every command described in this section must be implemented by all
    NNTP servers.

But anyway:

> LISTGROUP is a lot easier to implement and more widely

I don't have a significant problem with doing this. Is it a significant
load on servers?

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Thu, 24 Mar 2005 08:46:39 +0000

Ken Murchison said:
> one hierarchy separator (".")?

No. Traditionally group names without a dot were local to a site; I believe
that B News and C News automatically excluded them from feeds.

For the record, our current syntax defines a newsgroup name as being a
wildmat item with no wildcards. That is, it is a string of any of the 88
printable ASCII characters except * , ? [ \ and ].

Thus "current group" would be best specified as * or ? (the former is more
consistent with wildmat meaning; the latter is more obvious to humans).

> LISTGROUP [group [range]]

I prefer this.

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Thu, 24 Mar 2005 08:49:48 +0000

> capability with an optional "RANGE" argument.

Yuk.

We made LISTGROUP an argument of READER because it makes little sense to
implement LISTGROUP (or GROUP, for that matter) without ARTICLE.

This range thing is small, but it's either useful to lots of people - and
so should be mandatory - or isn't - in which case drop it. Remember what we
write in our own document:

    Experience with many protocols has shown that:

        Protocols with few options tend towards ubiquity, whilst
        protocols with many options tend towards obscurity.

---

From: Russ Allbery <[email-masked]>
Date: Thu, 24 Mar 2005 01:22:37 -0800

Clive D W Feather <[email-masked]> writes:

> Can I suggest that GROUP with no arguments should be permitted? This
> would improve the commonality between the two commands (to all intents
> and purposes, LISTGROUP is already GROUP with more output).

Hurm.  I can see your point, but this also feels like a path down a
slippery slope to me.  Maybe what this is telling us is that adding
features to LISTGROUP is leading into more complexity than we should be
dealing with at this stage.

---

From: Russ Allbery <[email-masked]>
Date: Thu, 24 Mar 2005 01:24:24 -0800

>> anything useful is gained by making it optional.  If we're going to
>> make NEWNEWS mandatory,

>     Every command described in this section must be implemented by all

Yes, but as we're all painfully aware, the connection between RFC 977 and
reality is tenuous at best.

>> LISTGROUP is a lot easier to implement and more widely supported right
>> now, and that would get rid of one of our capabilities that we have to
>> keep track of.

> I don't have a significant problem with doing this. Is it a significant

For INN, it's a traversal of the overview records for the group, so there
is some load but it's not anything I'd worry tremendously about.  For
older servers, it's basically a readdir plus a sort.  I would expect that
most current servers will do something vaguely equivalent to traversing
the group overview database.

---

From: [email-masked] (Kai Henningsen)
Date: 24 Mar 2005 10:58:00 +0200

> capability with an optional "RANGE" argument.

Sounds like far too many knobs for far too little benefit to me.

MfG Kai

---

From: Matthias Andree <[email-masked]>
Date: Thu, 24 Mar 2005 11:25:23 +0100

>     Every command described in this section must be implemented by all

There are servers that don't implement this command (leafnode 1.X
versions) or allow to disable it (INN), or implement it partially
(leafnode 2.X assumes the argument is a newsgroup name, not a pattern).

While such behavior is not allowed by the standard, it has never been
cause for complaints about leafnode in the years I maintain it, and no
pertinent bug is open in Debian Linux's bug tracker either.

It seems to me that NEWNEWS is not widely used by clients.

> I don't have a significant problem with doing this. Is it a significant

If the server keeps OVER data cached (leafnode and INN do) or keeps a
"traditional spool" format, LISTGROUP is cheap. In the latter case, just
listing the file names does the job, in the former case, the cached OVER
data can be used.

-- 
Matthias Andree

---

From: Matthias Andree <[email-masked]>
Date: Thu, 24 Mar 2005 11:49:31 +0100

> exactly what implication that has if there are no articles in the

I'd suggest to invalidate the "current article number" as would be the
case for an empty group.

Setting the "current article number" to the first article in the range
is more obvious and seems to be less astonishing than to the frist
article in the group (possibly outside the range).

successful "LISTGROUP example.group 4-7" w/ setting the current article
to 2 is ugly.

> gathering information about what's implemented to prepare for advancing to

I'll probably have to look into NNTPv2 and adjust leafnode-2 to
implement it. Lack of spare time has prevented that so far, and
leafnode-2 is a spare time project.

> would take the current article number out of the range?

I'd say that range does not constrain NEXT and LAST, the client will be
given an article number it can check.

> * A range can only be provided if the group name is provided.

I think we agree here to keep the syntax simple, although using an
extensible syntax like this (ABNF) could generally work without
ambiguities (brackets not being allowed in newsgroup names), too.

listgroup-command = "LISTGROUP" [WS newsgroup-name] [ "[" "RANGE=" range "]" ]

examples would be

LISTGROUP
LISTGROUP examp.le
LISTGROUP [RANGE=1234-]
LISTGROUP examp.le [RANGE=1234-3333]

Obviously, missing newsgroup name would mean "currently selected
newsgroup", missing range would mean "full range" and missing upper
article number in the range would mean "until highest present article
number in the group".

RANGE= could then later be extended to form some "parameter = value"
syntax - but this complicates the parser and may not be desirable at
this time.

> unwanted material, but that's all.

I'd want to set the current article number to the lowest number in the
range, if any, or invalid, otherwise, for reasons given above,

> purposes, LISTGROUP is already GROUP with more output).

Fine with me.

> dealing with at this stage.

I'm fine with postponing this now that I see how complex this is really
getting.

-- 
Matthias Andree

---

From: "Charles Lindsey" <[email-masked]>
Date: Thu, 24 Mar 2005 15:42:39 GMT

>the valid range is until you actually enter the group.

The number you give will usully be one more than the last article you read
from the same group yesterday.

However, there is one niggle. Suppose the group has become empty since
yesterday (a mass expiry/cancellation/whatever). It is now legitimate
(though deprecated) to reset the low water mark to zero and start off from
there (well, that is one interpretation of section 6.1.1.2, though the
wording does seem a little contradictory).

So, is there a problem there, and is there also a similar problem in other
commands with a range parameter? Of have I just misunderstood section
6.1.1.2?

---

From: Russ Allbery <[email-masked]>
Date: Thu, 24 Mar 2005 10:50:47 -0800

Charles Lindsey <[email-masked]> writes:

> (though deprecated) to reset the low water mark to zero and start off
> from there (well, that is one interpretation of section 6.1.1.2, though
> the wording does seem a little contradictory).

Er, no, it most certainly is not legitimate to do that.  It is legitimate
to return all zeroes in response to the GROUP command, but when another
article comes in, it has to be numbered higher than any previous article.

6.1.1.2 only governs the syntax of the return from GROUP, not the internal
article number.  For the internal article number, please see section 6, in
particular:

   [...]
   Another key is composed of the newsgroup name and the article number
   within that newsgroup.  That key MUST be unique to a particular
   server (there will be only one article with that number within a
   particular newsgroup), but is not required to be globally unique.
   [...]

   The server MUST ensure that article numbers are issued in order of
   arrival timestamp; that is, articles arriving later MUST have higher
   numbers than those that arrive earlier.  The server SHOULD allocate
   the next sequential unused number to each new article.

There are no exceptions for transiently empty groups.

---

From: "Charles Lindsey" <[email-masked]>
Date: Fri, 25 Mar 2005 12:07:49 GMT

>printable ASCII characters except * , ? [ \ and ].

Actually no. It is those ASCII characters you mention plus UTF8-non-ascii.

Usefor does not currently permit that UTF8-non-ascii as the moment, but it
might in its future I18N extension, and actually, this NNTP requirement
would rather constrain Usefor to either put newsgroup-names in UTF-8 or
else to encode them in ASCII somehow.

OTOH, as you say elsewhere in these threads, NNTP still says you
SHOULD/MUST accept octets outside of the UTF8-non-ascii range, so a
Chinese cooperating subnet might still get away with Big8, or whatever
their code is called.

That would indeed be a Bad Thing, but since when did that stop the Chinese
from doing what they wanted :-( .

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Fri, 25 Mar 2005 12:59:18 +0000

> Actually no. It is those ASCII characters you mention plus UTF8-non-ascii.

Oops: you're quite right.

> Chinese cooperating subnet might still get away with Big8, or whatever

Not in a newsgroup name.

The formal syntax uses special non-terminals S-CHAR, S-NONTAB, and S-TEXT
that have two separate definitions: one that MUST be accepted, and one that
SHOULD be generated.
* Header content in articles (when unfolded) is S-CHAR.
* Header contents in HDR/OVER responses is S-NONTAB.
* Newsgroup description is S-TEXT.
Apart from article bodies and the HELP output, the entire remaining syntax
is UTF-8 based.

For the record:

                  MUST accept                 SHOULD generate
    S-CHAR        %x21-FF                     any UTF-8 from U+0021 upwards
    S-NONTAB      any except TAB              any UTF-8 except TAB
    S-TEXT        any but not beginning       any UTF-8, but beginning
                  with TAB or SP              with U+0021 or above

(in all the above, "any" excludes NUL, CR, and LF).

---

From: "Charles Lindsey" <[email-masked]>
Date: Mon, 28 Mar 2005 10:50:06 GMT

>(in all the above, "any" excludes NUL, CR, and LF).

Now I am even more confused, because one has to define carefully when
"accept" applies and when "generate" applies.

Suppose "!@#$" represents some sequence of octets not allowed within
UTF8-non-ascii, and suppose it is used in headers by some Chinese system
that insists on using Big-8 (or whatever the Chinese charset is called) in
defiance of all the format standards.

So an article is POSTed containing the header "Subject: !@#$". That is a
"MUST accept", so the server accepts it. What does the server then do with
it? That is not really our business, but having accepted it we should not
be surprised if it stores it and/or attempts to relay it to other sites.

So the server has stored it, and now some other client tries to READ it.
Are you saying that your "SHOULD generate" is violated if the article is
now sent, including that "!@#$", in response to the READ. Likewise, is that
"SHOULD generate" violated if the server becomes a client and says IHAVE
that article to another server, and then sends it as-is (in which case
it is a "MUST accept" for the other site).

For sure, things get a little more interesting if this header gets
involved in the response to an OVER or HDR command. But even there, if the
server naively includes that "!@#$" in the proper place within the
response, is that "SHOULD generate" violated?

In fact, I think it is clear that all existing implementations will simply
include that "!@#$" in all the relevant places, simply because it it too
much hassle and a waste of resources to try and detect these obscure
happenings (and observe that the "!@#$" is perfectly acceptable in bodies
given a suitable Content-Type header).

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Mon, 28 Mar 2005 15:02:47 +0100

> "accept" applies and when "generate" applies.

The above was my paraphrasing. The actual text reads:

3.6:
   The content of a header SHOULD be in UTF-8.  However, if a server
   receives an article from elsewhere that uses octets in the range 128
   to 255 in some other manner, it MAY pass it to a client without
   modification.  Therefore clients MUST be prepared to receive such
   headers and also data derived from them (e.g.  in the responses from
   the OVER (Section 8.3) command) and MUST NOT assume that they are
   always UTF-8.  How the client will then process those headers,
   including identifying the encoding used, is outside the scope of this
   document.

We don't say anything specific in the description of LIST NEWSGROUPS.

Finally, the formal syntax says:

   The following non-terminals require special consideration.  They
   represent situations where material SHOULD be restricted to UTF-8,
   but implementations MUST be able to cope with other character
   encodings.  Therefore there are two sets of definitions for them.

   Implementations MUST accept any content that meets this syntax:

     S-CHAR   = %x21-FF
     S-NONTAB = CTRL / SP / S-CHAR
     S-TEXT   = (CTRL / S-CHAR) *B-CHAR

   Implementations SHOULD only generate content that meets this syntax:

     S-CHAR   = P-CHAR
     S-NONTAB = U-NONTAB
     S-TEXT   = U-TEXT

Clearly this needs to be made clearer.

> be surprised if it stores it and/or attempts to relay it to other sites.

The intent is that that behaviour is conforming.

> it is a "MUST accept" for the other site).

No to both.

> happenings

Exactly.

I think the text in 3.6 is approximately correct, but it mixes up roles.
I will change it:

-  The content of a header SHOULD be in UTF-8.  However, if a server
+  The content of a header SHOULD be in UTF-8.  However, if an implementation
   receives an article from elsewhere that uses octets in the range 128
-  to 255 in some other manner, it MAY pass it to a client without
+  to 255 in some other manner, it MAY pass it to a client or server without
-  modification.  Therefore clients MUST be prepared to receive such
+  modification.  Therefore implementations MUST be prepared to receive such
   headers and also data derived from them (e.g.  in the responses from
   the OVER (Section 8.3) command) and MUST NOT assume that they are
-  always UTF-8.  How the client will then process those headers,
+  always UTF-8.  Any external processing of those headers,
   including identifying the encoding used, is outside the scope of this
   document.

I've added to LIST NEWSGROUPS:

   The description SHOULD be in UTF-8.  However, servers sometimes
   obtain the information from an external source which has used a
   different encoding (one that uses octets in the range 128 to 255
   in some other manner).  In this case they MAY pass it on unchanged
   and clients MUST be prepared to receive such descriptions.

Finally, I've changed the formal syntax to:

   Implementations MUST accept any content that meets this syntax:

     S-CHAR   = %x21-FF
     S-NONTAB = CTRL / SP / S-CHAR
     S-TEXT   = (CTRL / S-CHAR) *B-CHAR

   and MAY pass such content on unaltered.

   When generating new content or re-encoding existing content,
   implementations SHOULD conform to this syntax:

     S-CHAR   = P-CHAR
     S-NONTAB = U-NONTAB
     S-TEXT   = U-TEXT

I could easily be convinced that that SHOULD should be a MUST.

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 29 Mar 2005 11:01:40 GMT

Indeed. My problem was that the ABNF and the text with it did not fulfil
that promise. But you have now fixed it below.

>   including identifying the encoding used, is outside the scope of this

OK, so you have removed some possibly artificial distinctins between the
roles of serves and clients.

>   and clients MUST be prepared to receive such descriptions.

I think "sometimes obtain" is a little too strong, implying that was (or
might become) an accepted practice. (It might, but I hope it won't.)

So I would suggest rather "However, the information available to a server
might have arisen from an external source which had used a different
encoding (one that uses octets in the range 128 to 255 in some other
manner).  In this case they MAY pass it on unchanged.

And I have removed that "clients MUST", because it is not clear just what
that is requiring them to do. I think that anywhere a "MAY" occurs in a
standard there is an automatic implication that systems must cope with it
somehow (i.e. not too dis-gracefully).

In practice, it this occurs at all, it will be within cooperating subnets
where the clients will know what to do. Clients outside of the subnet will
display garbage.

>   and MAY pass such content on unaltered.

Yes, that seems to fix the problem.

>I could easily be convinced that that SHOULD should be a MUST.

Maybe. I think the point is that NNTP implementations never "generate" the
content we are talking about. Agreed if re-encoding ever takes place (and
when Usefor finally tackles I18N it may or may not call for re-encoding
somewhere in the protocols).

---

From: [email-masked] (Peter Robinson)
Date: Wed, 30 Mar 2005 00:15:05 +0100

> that we have to keep track of.

Was a consensus reached either of these?

Given that NEWNEWS is mandatory, the only reason I can see for keeping
LISTGROUP optional, is the late stage in the process.

> range as an optional second argument.

IMHO, that would change it from a mere curiosity to a command that some
clients might want to use.  Again timing is a problem.  Solutions to the
syntax snags have been proposed, so I don't have a problem with that,
but IMHO, it definitely can't get away without having a CAPABILITY
argument (e.g. 'RANGE').

Just my 2c.

Regards,

Peter