Subject: [NNTP] Consensus?

From: "Clive D.W. Feather" <[email-masked]>
Date: Tue, 9 Aug 2005 10:31:20 +0100

Russ,

Where are we on reaching consensus on the following items?

(1) Article number limits. In particular, the possibilities of:
  (a) changing the limit from 2^32-1 to 2^31-1;
  (b) adding wording saying the present situation is unsatisfactory but
      is being worked on ready for the next version of this document.

(2) Limits on the length of command name and capability labels.

[For (1), my preference is to reduce the limit and add the wording. For (2),
it is to remove the limit.]

-- 
Clive D.W. Feather  | Work:  <[email-masked]>   | Tel:    +44 20 8495 6138
Internet Expert     | Home:  <[email-masked]>  | Fax:    +44 870 051 9937
Demon Internet      | WWW: http://www.davros.org | Mobile: +44 7973 377646
Thus plc            |                            |

---

From: Russ Allbery <[email-masked]>
Date: Tue, 09 Aug 2005 07:47:16 -0700

> [For (1), my preference is to reduce the limit and add the wording. For
> (2), it is to remove the limit.]

(2) I think we've reached a solid consensus on removing the limit.  If
anyone disagrees, please speak up now.  (Note that even with consensus, my
understanding of the process is that we don't publish a new draft.
Instead, when the RFC Editor contacts Clive for author's 48 hours, he
would tell them to make the change at that time.)

For (1), the discussion has been wide-ranging enough that I'm not sure on
(a).  I know some folks here would like the limit removed entirely, but I
don't have a good feeling on what people think about 2^32-1 versus 2^31-1
assuming that we keep a limit.  More discussion would be valuable there, I
believe.

For (b), had you or someone else already posted a specific wording
suggestion?  (I've been swamped this past week and fairly busy before
that, so may have missed it.)  Would this change include warning people to
use longer data types as well?

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

---

From: Ken Murchison <[email-masked]>
Date: Tue, 09 Aug 2005 11:07:07 -0400

> would tell them to make the change at that time.)

Agreed.

> use longer data types as well?

I proposed wording that made support for 2^32-1 a MUST, and support for 
2^64-1 a SHOULD.

I'm still not sure what the problem is with this suggestion.  We know we 
have at least a couple of years to get deployment before groups start 
hitting the 2^32-1 limit.  When servers finally start spitting out 
article numbers > 2^32, client implementors will have had plenty of time 
to get ready.  Those that haven't will get plenty of complaints from 
users that they can't read their favorite groups and will either be 
forced into fixing their product, or the product will die.

-- 
Kenneth Murchison     Oceana Matrix Ltd.
Software Engineer     2495 Main St. - Suite 401
716-604-0088 x26      Buffalo, NY 14214
--PGP Public Key--    http://www.oceana.com/~ken/ksm.pgp

---

From: Steve Walker <[email-masked]>
Date: Tue, 09 Aug 2005 20:26:11 -0500

> assuming that we keep a limit.  More discussion would be valuable there, I

If you mean remove the limit in the draft and leave the issue 
open for further discussion I'm ok with that.  The draft should 
not be released as-is due to the major problems with the current 
wording.

Personally I would like the wording changed to require (a MUST) 
32 bit support and suggest (a SHOULD) 64 bit support.  It seems 
the least likely to cause major problems.  Text clients could 
standardize on 32 bit, instead of the current mix of 31 and 32 
bit that we currently have, while the binary centered clients 
would upgrade quickly to 64 bits, paving the way for other 
clients as time goes on.  In the next major revision of the 
standard the 64 bit SHOULD could be changed to a MUST.

Steve Walker

---

From: Russ Allbery <[email-masked]>
Date: Tue, 09 Aug 2005 18:15:56 -0700

>> For (1), the discussion has been wide-ranging enough that I'm not sure
>> on (a).  I know some folks here would like the limit removed entirely,
>> but I don't have a good feeling on what people think about 2^32-1
>> versus 2^31-1 assuming that we keep a limit.  More discussion would be
>> valuable there, I believe.

> open for further discussion I'm ok with that.

No, I didn't mean that; I meant discussion of Clive's specific proposal.
There had been a lot of discussion around other things, but not about what
Clive said in particular, so I didn't feel like I could judge consensus on
that proposal.

(Which isn't to say that I can judge consensus on other things as well,
but for other reasons.)

> Personally I would like the wording changed to require (a MUST) 32 bit
> support and suggest (a SHOULD) 64 bit support.

That's two people supporting that option.  What does everyone else think
about that?

---

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 10 Aug 2005 10:56:34 GMT

>I proposed wording that made support for 2^32-1 a MUST, and support for 

No, I think we should leave the 64 bit issue alone for now. If we say
SHOULD support, then we have effectively made the decision that 64 bits
will be the eventual fix.

My personal view is that a wrap around solution would be best long term
(and I think we have discussed enough to see that such a scheme would be
possible, but obviously not to the extent that we can define it now). So I
don't want to make any decision now that would make it difficult later.

Most systems are safe for several years. It is only Steve's unusual scheme
which might fail after 2 years, and I don't think we should be forced into
a premature decision just because of that.

I would have no problem if we want to limit it to 31 bits for now, with a
SHOULD support 32 (i.e. SHOULD use unsigned int).

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: "Scott Hollenbeck" <[email-masked]>
Date: Wed, 10 Aug 2005 07:20:51 -0400

> SHOULD support 32 (i.e. SHOULD use unsigned int).

I would advise against using 2119 directives with any programming language-
or machine architecture-specific data types/limits.  It's OK to mention the
range of numbers that can be expected, but it's best to stay far away from
implementation-specific details unless they are clearly identified as
examples.

-Scott-
(speaking as area advisor)

---

From: [email-masked] (Thomas Gschwind)
Date: Wed, 10 Aug 2005 14:06:36 +0200

Hi all!

> That's two people supporting that option.  What does everyone else think

I am also very much in favor of this solution.

> don't want to make any decision now that would make it difficult later.

Recommending the use of 64 bit article numbers does not preclude us from
specifying a wrap around scheme.  It only postpones it by some more
time.  Enough time till we get out a newer version.

> a premature decision just because of that.

What would be the problem with recommending a 64-bit counter?  So far we
know there are some people having problems if we specified a 32-bit
limit.

> implementation-specific details unless they are clearly identified as

That's actually a good point.  Also considering some "broken" languages
that don't have unsigned data types (eg Java), it might be nice to
specify the ranges maybe as MUST 1..2^31-1 and the SHOULD as 1..2^63-1.
I would be very much in favor of that.

Regards,
Thomas
-- 
Thomas Gschwind                      Email: [email-masked]
IBM Zurich Research Lab
Saeumerstrasse 4                     Tel: +41-1-724-8990
CH-8803 Rueschlikon, Switzerland     Fax: +41-1-724-8953

---

From: "maurizio codogno" <[email-masked]>
Date: Wed, 10 Aug 2005 12:27:22 +0000

>That's two people supporting that option.  What does everyone else think

for me it's fine.
ciao, .mau.

---

From: Richard Clayton <[email-masked]>
Date: Wed, 10 Aug 2005 14:02:10 +0100

In message <[email-masked]>, Russ Allbery
<[email-masked]> writes
>Clive said in particular, so I didn't feel like I could judge consensus on

If you meant Clive's BIGNUM proposal

    http://lists.eyrie.org/pipermail/ietf-nntp/2005-July/001656.html

I thought it was elegant and looked reasonably straightforward to cope
with from an implementation point of view -- and had the advantage that
everyone knew exactly where they were with it.

It does mean that there is no wrap around -- so clients cannot access
groups that have overflowed (but today's clients probably can't cope
with groups with that sort of volume on anything other than a sampling
basis anyway).

The only real problem seems to be this one server design that doesn't
have separate numbers for separate groups (thereby breaking assumptions
about file sizes for clients that thought they could use the space-
saving "10-15, 17-23, 25" style recording for articles they had read,
and also breaking all sorts of heuristics for "catching up" the last 50
articles....)

I'm reluctant to see ANY solution discarded for a single design that
although it doesn't break any rules bends them significantly.

Since Turnpike (a client, albeit one in development hibernation) was
mentioned earlier and I know a few things about it -- I should point out
that it spends most of its time handling message IDs and deals with
article numbers only occasionally (mainly for spotting when the server
has renumbered the newsgroup). However, it does store these values in
its files (which are not flat ASCII files), so it's not just a case of
changing a typedef and recompiling, but changing the file format,
providing a migration tool and such...  I doubt anyone will be
interested in bothering.

As a developer, I think I'm more impressed with a system like Clive's
that allows me to make my own judgment as to what sort of limits I'm
prepared to support (and handle the failure elegantly and with precise
details for the user) than a SHOULD for 64bits, which I'm likely to just
say "I know better, this will ruin my pretty design for something that
will not happen for decades" and get on with something else (ie: it's
not going to make me implement 64 bits unless I'm very motivated or have
personally encountered this one strange server design).

>That's two people supporting that option.  What does everyone else think

The MUST for 32 bits might get people's attention on a slow afternoon
and may prevent a few people using signed values...   but I doubt the
SHOULD will make any impression at all -- after all, they ought to be
starting to worry about what their time_t code will do in 2038...

- -- 
richard                                              Richard Clayton

They that can give up essential liberty to obtain a little temporary
safety deserve neither liberty nor safety.         Benjamin Franklin

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Wed, 10 Aug 2005 16:53:16 +0100

Russ Allbery said:
> anyone disagrees, please speak up now.

Nobody did, so I'll update my master text.

> assuming that we keep a limit.  More discussion would be valuable there, I

The reason for keeping a limit is that it describes existing practice, and
that's what our charter talks about. Recent discussion has shown that some
existing practice uses signed int for this, so 2^31-1 is a genuine limit on
those systems.

> suggestion?

    It is likely that at some stage at least one NNTP server will reach the
    limit of 4,294,967,295 in at least one newsgroup. Ways to deal with
    this while not gratuitously breaking compatibility are still being
    investigated and are likely to result in publication of a revision or
    extension to this specification at some future date.

The intent is to not hold up this document while still giving fair warning.

> Would this change include warning people to

It didn't, as you can see.

I'm loath to add wording that implies that a server can produce a larger
number without the agreement of the client; a client using 32 bit values is
likely to break painfully when it receives that number (either because of
the error from the string-to-number conversion or because the value
silently wraps).

However, something like:

    While servers MUST NOT send article numbers greater than this limit,
    client and server developers are advised to use internal structures
    and datatypes capable of handling larger values in anticipation of
    such a change.

might work.

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Thu, 11 Aug 2005 09:26:54 +0100

> My personal view is that a wrap around solution would be best long term

I'm not at all convinced of this.

> a premature decision just because of that.

I very much DO agree with this. Let's get this document out of the door and
try to find a workable solution before the problem actually becomes major.
[As I said, I have no problem with putting a warning about future
directions in the document.]

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Thu, 11 Aug 2005 09:27:04 +0100

> That's two people supporting that option.  What does everyone else think

I seriously don't like it.

It doesn't represent current practice, which is what we're supposed to be
documenting.

It introduces an interoperability issue. What happens if the server
supports the SHOULD and the client doesn't?

Finally, think how it actually would go into the wording. To remind you,
this is what the text currently says:

   The second type of key is
   composed of a newsgroup name and an article number within that
   newsgroup.
[...]
   Article numbers MUST lie between 1 and 4,294,967,295 inclusive.

[In the following I'll use the "2^N-1" notation for clarity.]

What are the proponents of this proposal actually suggesting for the
wording? Clearly they don't mean:

   Article numbers MUST lie between 1 and 2^32-1 inclusive and SHOULD
   lie between 1 and 2^64-1 inclusive.

because that's a nonsense. Do they mean the inverse?

   Article numbers MUST lie between 1 and 2^64-1 inclusive and SHOULD
   lie between 1 and 2^32-1 inclusive.

What is the effect of breaching the SHOULD?

Let's see some actual wording proposals here, including addressing the
inter-operability issues when any SHOULD or MAY is breached.

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Thu, 11 Aug 2005 09:31:18 +0100

> everyone knew exactly where they were with it.

For the record, I was proposing that as a solution to the overall problem,
but *not* as something to go into the about-to-be-an-RFC. It's the latter
that needs freezing fast.

> groups that have overflowed

It would be possible to design extensions for this as well. The two
solutions need not be incompatible.

> so it's not just a case of
> providing a migration tool and such

This could easily be an issue for other client implementations as well.
It's not necessarily a trivial matter.

---

From: Ken Murchison <[email-masked]>
Date: Thu, 11 Aug 2005 09:30:20 -0400

> It doesn't represent current practice, which is what we're supposed to be

True, but it sounds like current practice is all over the place (2^31, 
2^32, 2^64), so we need to standardize on something.  Standardizing on 
something that we *know* will break in a few years doesn't seem like the 
best move when there is a relatively simple solution which will push the 
problem out far enough where most, if not all, of us on this list won't 
have to worry about it.  ;)

> supports the SHOULD and the client doesn't?

Then the client chokes and can't read the group.  They are no worse off 
then if the server supports your BIGNUM strawman (which I like BTW) and 
the client doesn't.  The bottom line is that if we recommend 2^64 in 
this spec, it gives client authors at *least* two years to upgrade the 
code.  If they don't upgrade (to 2^64 OR BIGNUM) by the time article 
numbers on servers exceed 2^32, I'd consider the clients horribly broken.

> wording?

You must have missed my post on this:

http://lists.eyrie.org/pipermail/ietf-nntp/2005-July/001612.html

We could also add text that states that 2^64-1 will most likely become 
mandatory in a future version of the spec.

Having said all of this, if we can't reach a consensus on 2^64 
relatively quickly, then lets just go with 2^32 and be done with it.  I 
don't really like the idea of catering to 2^31 implementations.

---

From: Steve Walker <[email-masked]>
Date: Thu, 11 Aug 2005 10:22:28 -0500

>directions in the document.]
Considering the decade long periods between revisions it's certainly 
possible that many others will also have overflow problems before 
anything is done. You can say that some people plan on updating the 
RFC's sooner, but past history proves otherwise.  I can see much more 
harm being done by being impatient.  How can spending a couple weeks to 
fix a major flaw not be justified?

Steve Walker

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Fri, 12 Aug 2005 16:31:16 +0100

> RFC's sooner, but past history proves otherwise.

We don't need to rewrite the entire specification to add a solution for
this. So the work on a revision RFC will be rather simpler and faster.

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Fri, 12 Aug 2005 16:44:07 +0100

Ken Murchison said:
> 2^32, 2^64), so we need to standardize on something.

"Least common denominator" or "be conservative in what you send" is the
usual guideline.

> Standardizing on 

I am not ocnvinced that it is "a relatively simple solution". It's
basically saying "an unknown number of existing implementations are hereby
declared non-conforming".

> Then the client chokes

You find that acceptable?

Most protocols with options in are phrased so that the recipient MUST
accept the transaction using the option, even if it then sends back a
rejection. In this case that would require the support of numbers above
2^32 by clients. But we know that isn't current practice.

> and can't read the group.
>
> They are no worse off 
> the client doesn't.

The big difference there is that the client gets a "there's a problem"
response code rather than bad data.

> http://lists.eyrie.org/pipermail/ietf-nntp/2005-July/001612.html

I did.

+   Article numbers MUST be positive integers.  Clients and servers
+   MUST support article numbers up to, and including 4,294,967,295
+   (2^32 - 1).  However, due to the exponential growth of Netnews
+   traffic, it is RECOMMENDED that clients and servers support article
+   numbers up to, and including 18,446,744,073,709,551,615 (2^64 - 1).

Again, this doesn't explicitly address what happens if one end doesn't
implement the RECOMMENDED (which, incidentally, means exactly the same as
SHOULD).

> relatively quickly, then lets just go with 2^32 and be done with it.

As I said, I don't think that changing to 2^64-1 is the sort of thing that
should be "sneaked in" at this late stage.

---

From: [email-masked] (Kai Henningsen)
Date: 12 Aug 2005 22:40:00 +0200

> [In the following I'll use the "2^N-1" notation for clarity.]

How about something like:

  Article numbers MUST be larger or equal to 1.
  Note 1: Some current server and client implementations limit article
          numbers to 2^31-1.
  Note 2: It is expected that some servers will exceed article numbers of
          2^32 a few years after publishing this spec.

That is, do it as advice to implementers, not as a MUST or SHOULD. "This  
is what you will actually encounter."

NB. It so happens that I know of another implementation that seems to use  
global article numbers, though it isn't anywhere near the number limit as  
it doesn't actually take a feed. It is a NNTP access module to a web board  
(and the only name for the software I know is "Webboard"), which  
incidentally is quite buggy as I found out when using my normal NNTP pull  
script on it :-/ I think one of the half a dozen or so problems was  
actually my bug (pipelining too early).

The installation I know about is on bar.baen.com. (For one, it currently  
uses message ids lacking any domain part. Needed to patch my local INN to  
make it accept the things. It's an excellent mispropagation prevention aid  
:-))

MfG Kai

---

From: "Charles Lindsey" <[email-masked]>
Date: Sat, 13 Aug 2005 21:26:04 GMT

>numbers on servers exceed 2^32, I'd consider the clients horribly broken.

But it is totally naive to expect that anything approaching the majority
of clients will be upgraded within two years, even if every known
implementation produces an upgraded version within that timescale, which
is itself most unlikely.

In USEFOR, there are umpteen schemes which we have had to discard because
the client base cannot be assumed to have changed. Even schemes where we
had hoped that the base of servers might be upgraded within a couple of
years are now being removed from the draft (expectations that folded
Newsgroups headers could be used within two years, or so, for example).

And in Email, they go to enormous lengths to ensure that all clients MUST
accept strange things which have not been seen in real emails for many
years, and most of which, I suspect, have never been used in anger at all.

That seems to be the IETF way of doing things. The only way you can get
away with something which will not interoperate with existing software (or
which cannot be worked around with existing software) is to define an
Experimental Protocol, and I don't think that is the way forward for the
present problem.

---

From: "Charles Lindsey" <[email-masked]>
Date: Sat, 13 Aug 2005 21:14:25 GMT

>know there are some people having problems if we specified a 32-bit

I can think of two problems.

1. After a while, the numbers typically used for articles will become
excessively long. Any time one needs to look at one manually (which I do
occasionally, as when I look at some Xref header and them go find the
article in some group), then I have to transcribe a huge number. Huge
numbers are clumsy, and should only be used where there is a need to
distinguish that many objects at one time - but who has a news spool that
actually contains of the order of 2^32 articles, or even has a group where
the high and low water marks differ by that sort of amount? In fact, the
number of articles stored at any one time is many orders of magnitude less
than 2^32.

Yes, this is an appeal to elegance, and I suppose elegance was lost when
article numbers were invented in the first place - but there we are.

2. Existing clients simply will not cope. And existing clients are not
going to go away, and there will be howls of protest and accusations that
the IETF has allowed a non-interooperable change to the basic protocol.

Reason #1 may be simply elegance, but IMO #2 is a show stopper.

So the proper way forward is to leave it as it is, and then embark on a
reasoned discussion of alternatives. If we ask the IETF nicely, and point
out that there is an issue which needs resolution in the next few years,
then I am sure they will permit us to recharter so as to start immediate
work on what should be a fairly small extension.

Ideally, what we want is a solution that allows existing clients to
continue in use indefinitely. That means a solution with two propoerties:

a) When the problem hits, clients that are aware of our extension will
recognise "it" (whatever "it" is) and will do the "right thing".

b) Existing clients will require some manual attention. If we design it
carefully, the action needed will be no more than unsubscribing from the
group and resubscribing to it. That is a thing even the most naive of
users can probably manage to do (more sophisticated users can try manual
editing of their .newsrc files, or other such hacks).

I note that various interesting ideas are now being floated. They will
take time to evaluate, and time is what we do not have at the moment.

---

From: Steve Walker <[email-masked]>
Date: Sun, 14 Aug 2005 05:23:38 -0500

> excessively long. Any time one needs to look at one manually (which I do

This is a very poor reason to allow a complete break down of the 
protocol.  If this was a valid reason then why are Message-ID's, 
which are hand typed much more often, allowed to be 500+ bytes 
long.  By the way, a 4 billing number and a 9 billion number have 
the same length.  Even adding one more digit allows for many more 
years of life to the protocol.

> Reason #1 may be simply elegance, but IMO #2 is a show stopper.

You keep forgetting that RFC977 was completely text based and did 
not specify any binary limits.  The current 32 bit limit is the 
new non-interoperable change to the protocol.  I would be happy 
with just getting the new limit removed as any software that 
really matters is already or will soon be 64 bit compatible. 
While a 32 bit limit may have sounded reasonable 8 years ago, it 
is not a reasonable limit today.

If you can justify breaking existing software and forcing 
everything to use 32 bit binary numbers then you can just as 
easily force everyone to use 64 bit numbers.  There is no 
difference with the exception that one of the choices (64 bit) 
will actually solve a future problem.

Steve.

---

From: Chris Caputo <[email-masked]>
Date: Mon, 15 Aug 2005 21:07:11 +0000 (GMT)

> I am also very much in favor of this solution.

As am I.

The whole wrap-around/roll-over thing is less likely to be implemented 
correctly than just increasing the recommended width of the data type.

On normal servers (as opposed to Steve's) the groups most likely to exceed 
2^32 in our lifetime will tend to be groups that have followers using 
cutting edge clients anyways.  Some of those client authors are already 
using 64-bits and the rest certainly will be some of the first do so when 
the RFC says SHOULD or RECOMMENDED.

On my site two groups have now passed the 200 million mark.  The highest 
numbered group was created in late 2000, so it is averaging in excess of 
40 million posts per year.

But more revealing is this...  if I take the highest volume group for a 
given month the chart looks like this:

Month	Articles/day on average in busiest group
01/1998	10153
01/1999	19146
01/2000	91473
01/2001	124107
01/2002	102270
01/2003	84421
01/2004	218261
01/2005	360728
02/2005	360514
03/2005	380973
04/2005	387267
05/2005	420427
06/2005	371891
07/2005	385895
08/2005	489367 (so far)

If that 489,367 number holds or increases, and it is a safe bet that it 
will, the group that matches it will exceed 365 * 489,367 = 178,618,955 
messages in a year.

As for language, I think this suggestion by Ken Murchison is fine:

  -   Article numbers MUST lie between 1 and 4,294,967,295 inclusive.  The
  -   client and server MAY use leading zeroes in specifying article
  -   numbers, but MUST NOT use more than 16 digits.  In some situations,
  -   the value zero replaces an article number to show some special
  -   situation.
  +   Article numbers MUST be positive integers.  Clients and servers
  +   MUST support article numbers up to, and including 4,294,967,295
  +   (2^32 - 1).  However, due to the exponential growth of Netnews
  +   traffic, it is RECOMMENDED that clients and servers support article
  +   numbers up to, and including 18,446,744,073,709,551,615 (2^64 - 1).
  +   The client and server MAY use leading zeroes in specifying article
  +   numbers.  In some situations, the value zero replaces an article
  +   number to show some special situation.

Chris

---

From: "Charles Lindsey" <[email-masked]>
Date: Wed, 17 Aug 2005 20:29:14 GMT

In <[email-masked]> Steve Walker <[email-masked]> writes:

>not specify any binary limits.

As a consequence of which implementors used their common sense, and used
32 bit integers (or preferably unsigned). There are millions of clients
out there using those 32 bit ints, and a large number of people will still
be using them for many years to come. Therefore, a solution that allows
those to continue is much to be preferred (and yes, such solutions DO
exist).

>  The current 32 bit limit is the 
>new non-interoperable change to the protocol.

No, it was not a change. It just recognized and codified what was already
the de facto practice.

>easily force everyone to use 64 bit numbers.

It is you who has to justify breaking existing software (i.e. the huge
installed base of clients - I could perhaps justify breaking the much
smaller base of installed servers). I don't have to "force" everything to
use 32 bit numbers, as you claim. They are already doing that.

-- 
C. H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: Steve Walker <[email-masked]>
Date: Thu, 18 Aug 2005 07:17:53 -0500

> 32 bit integers (or preferably unsigned). There are millions of clients

No, they didn't.  They used what every they wanted.  Some used 
31, some used 32 and some used 64.  There are existing clients 
and servers using all three data sizes.  This change will break 
the only software that had the foresight to choose a data size 
large enough.

> those to continue is much to be preferred (and yes, such solutions DO

Allowing servers to use 64 bits is backwards compatible with all 
the 31/32 bit clients until the same point that the clients would 
have naturally broken.  A 64 bit server is no different than a 31 
/32bit server until the ~2/4 billion article number.  By 
specifying 64 bits numbers on the servers we will allow the 
clients time to evolve.  Waiting until it's a major problem and 
then trying to get hundreds of clients recompiled last minute is 
foolish.  By making the change today we give the client authors 
and users many years to upgrade to a current 64 bit client version.

It's also only an issue for servers that have binary groups. 
Small text only servers may never need to worry about it in their 
lifetime.  Binary users upgrade their software often.  They will 
be prepared for the 64 bit servers.  Look at how fast yEnc'ing 
become the de-facto standard. Almost 80% of all articles are now 
yEnc'ed.  The binary community is also very resourceful and will 
solve their own problems.  If the IEFT releases an anti-binary 
server RFC, which this is, the people will be forced to come up 
with their own solution.  We will have servers that roll over, 
servers that use 64 bits, server that use 50% shift left 25% 
schemes, etc..  It will be a mess and it's what an RFC should 
solve, not create.

> No, it was not a change. It just recognized and codified what was already

Maybe 8 years ago, but not currently.  Some clients have already 
changed and the ones that haven't yet will be soon due to 64 bit 
processors being the norm.  How many client/servers used 
'unsigned long' as the data type for article numbers?  As soon as 
that client/server is compiled on a EM64 platform it suddenly 
becomes a 64 bit clean.  Servers/clients are going to evolve to 
64 bit on their own over the next couple years.  In case you are 
not aware of it, I don't even think you can buy a new non-64bit 
server today.

> use 32 bit numbers, as you claim. They are already doing that.

While a large number are using 32 bit numbers, another large 
group is using 31 bit numbers, and there is a smaller group that 
uses 64 bits or text based math.  I do not think you can even 
claim 75% of users are using 32 bit clean software/servers.

Steve Walker

---

From: Richard Clayton <[email-masked]>
Date: Thu, 18 Aug 2005 14:14:07 +0100

In message <[email-masked]>, Steve Walker
<[email-masked]> writes

>How many client/servers used 
>'unsigned long' as the data type for article numbers? 

only dumb ones...

> As soon as 
>becomes a 64 bit clean.

... and breaks its on-disk file structure.

This type of code will use some sort of generic name (int32, dword etc)
for the numbers and #define it to unsigned long, possibly by using a
configuration testing system to pick the correct set of #defines for the
machine it is being built on.

So as soon as it compiles it stays as it was! (everyone hopes)

>  Servers/clients are going to evolve to 
>64 bit on their own over the next couple years.

I agree, but it's nothing like as simple as you portray :(

>claim 75% of users are using 32 bit clean software/servers.

10% of posters to Usenet use Google; 71% of distinct sites handling
articles are FIDO nodes ... appealing to numbers is not entirely
illuminating :)

- -- 
richard                                              Richard Clayton

---

From: [email-masked] (Peter Robinson)
Date: Sat, 20 Aug 2005 16:09:26 +0100

> Where are we on reaching consensus on the following items?

I was away when the article number issue came up most recently, but here
is my opinion for the record.

>   (a) changing the limit from 2^32-1 to 2^31-1;

The main benefit of the current work is to present a clear, accurate and
well specified account of the current state of NNTP protocol.  New
software authors should be able to read this spec and have a good idea
of what they will find out there.  This is not the time for significant
new work.

RFC 977 did not specify a limit for article numbers.  That isn't because
it expected implementations to cope with arbitrarily large numbers, it's
because it is a product of its time.  This is just one of the many ill
specified parts of NNTP that we are cleaning up.  Changing the de-facto
limit from 32/1 to 64bits is very far from an insignificant change, and
one that we certainly cannot make at this stage.

Current clients and servers do not cope with 64 bit article numbers.
That will remain the case whatever this document says.  I strongly
believe that the way forward is via a future extension along the lines
of the BIGNUM proposal, where the capacities of both sides are exchanged
as necessary.  We should not be declaring 32 bit implementations
(conditionally) uncompliant with SHOULD or MUST language for 64bit
support.

As for the specific 31/2 bit question, I suggest we should face reality
and change to 2^31-1, but I have no strong feeling either way.

>       is being worked on ready for the next version of this document.

That is probably a good idea.

> (2) Limits on the length of command name and capability labels.

Remove the limits.

Regards,

Peter