Subject: [NNTP] Fwd: Gen-art review of draft-ietf-nntpext-streaming-05

From: Russ Allbery <[email-masked]>
Date: Thu, 02 Jun 2005 09:43:48 -0700

Date: Thu, 02 Jun 2005 10:41:19 +0100
From: Elwyn Davies <[email-masked]>
To: [email-masked]
CC: Mary Barnes <[email-masked]>, [email-masked],
        [email-masked], Russ Allbery <[email-masked]>,
        Scott Hollenbeck <[email-masked]>
Subject: Gen-art review of draft-ietf-nntpext-streaming-05

Document: draft-ietf-nntpext-streaming-05.txt
Trigger: IETF Last call, ending 7 June 2005
Reviewer: Elwyn Davies
AD: Scott Hollenbeck
Document Shepherd: Russ Allbery
Review Date: 2 June 2005

Intended status: Proposed Standard

Review Parameters:  This is a review by a member of the General Area 
Reviewing team, supporting the IETF General Area Director. As the 
submission is from an IETF WG, the review should focus on these 
questions: "Is this document a reasonable contribution to the area of 
Internet engineering which it covers? If not, what changes would make it 
so?"

Summary: This document is almost ready for submission to the IESG.  An 
interesting case of turning a de facto situation (RFC2980) into a de 
jure standard.  I think the document could do with a little more, and 
earlier, explanation of and connection to RFC2980 since it is presumably 
intended that it should interoperate with legacy clients/servers that 
implement the extensions.  The idea of deprecating MODE STREAMING and 
then stating that it was defined in this document boggled me for a 
moment! It is also an interesting question as to whether the reference 
to RFC2980 should be normative: I feel it should be.

On the matter of reponse codes:  There appears to be a wider question of 
whether these should be registered with IANA (as with SIP response 
codes).  The definitions are (or are about to be) spread across several 
documents and ensuring that there is no clash may become more difficult 
if further extensions are made.

I noted that the potentiality for pipelined commands to be used fos DoS 
attacks was not mentioned in the Security Considerations of this or the 
base document (draft-ietf-nntpext-base-26).

Review:
Generally a well written , well thought out document, which is almost 
ready for IESG review.

As I read the document, I got the sneaking suspicion that the document 
was almost ashamed of its ancestry in Sections 1.1 to 1.3 of RFC2980 (a 
*mere* informational document relating to some extensions that don't 
have the dignity of standardisation)!  Providing a short section in the 
introduction (either as part of S1 or a new sub-section) to explain the 
relationship to RFC2980 would reduce the crogglement when MODE STREAM is 
deprecated almost before it has been defined.  It would also help to 
confirm and/or limit the extent to which the current specification is 
expected to interoperate with legacy implementations that 'conform' to 
the corresponding RFC2980 capabilities. I also do't believe that RFC2980 
is an Informative reference: There is a degree of backwards 
compatibility which makes RFC2980 Normative.

Detailed Comments:
S1: Last sentence: I think 'effective' would be better word than 
'consistent'.

S2.1: Advertising Capability: This section might be more easily 
understood if placed after the definition of MODE STREAMING (i.e., 
reorder (2.1, 2.2, 2.3) -> (2.2, 2.3, 2.1)).  I note that the capability 
  STREAMING is actually noted as being in the inital IANA registry of 
NNTP capabilities in the base document, but this document contains the 
definition: I think that means there should be a NORMATIVE reference to 
this document in draft-ietf-nntpext-base-26, rather than the informative 
one there is at present (applies to the authentication document also).

S2.2: Streaming Article Transfer:  A pointer to the section in the base 
spec defining pipelining in this context would be useful:  pipelining 
has a generic meaning and to the naive reader it might not be obvious 
that it has a specific meaning in these documents.  Specifically it has 
implications for how responses should be associated with commands which 
makes some of the later stuff more comprehensible.

S2.3: Defn of MODE STREAM:  It would be worth emphasising that MODE 
STREAM does not actually cause the server to 'change mode' (a good and 
sufficient reason for deprecating it!).  The IANA regsitartion mentions 
that MODE STREAM can be pipelined - it would be worth mentioning that 
here too, since it is perhaps not obvious that something which sounds 
like a mode switch could be pipelined - because of course it doesn't 
actually do anything! Also, presumably (for consistency with the 
CAPABILITY advertising), the repsonse to MODE STREAM, after MODE READER 
has been issued, should match whether the server would advertise 
STREAMING in this state.

S4: Summary of Response Codes: This section in itself is fine.  However 
with the current standards for NNTP spread across several documents, it 
would make it easier to check that there was not a clash between 
response codes in documents (and to enforce the rules in S3.2 of 
draft-ietf-nntpext-base-26) if there was an IANA registry for the 
reponse codes.  Incidentally, I note that 23x and 43x are already rather 
crowded! (It seems this could have been postponed by reusing some of the 
existing response codes that have identical meanings, e.g, using 436 
instead of 431, but backwards comaptibility ties your hands here).

S5: Security Considerations:  I suspect that pipelined commands 
(espcially CHECK) offer an opportunity for DoS attacks on NNTP servers. 
  This is not discussed either in this document or the base 
specification to which it refers.  Maybe some notes about rate limiting 
generation of CHECK requests by clients (over and above the requirement 
to consider buffer sizes), and possibly the use of 431 responses as a 
way to control overloading by servers, might help.

---

From: Russ Allbery <[email-masked]>
Date: Thu, 02 Jun 2005 09:53:31 -0700

Elwyn,

Thank you very much for the comments.  I'm not sure who else on your cc
list would want to receive the ensuing discussion, so I've limited it to
just you; please feel free to forward these comments more widely.

> moment!

It may be worth noting that neither existing servers nor our work really
worked directly from RFC 2980 to a large degree.  RFC 2980 was a
retroactive description of a de facto situation (and is also quite brief
in its descriptions).  However, we could probably say more about the
history of this extension and the use of MODE STREAM in the introduction.

> It is also an interesting question as to whether the reference to
> RFC2980 should be normative: I feel it should be.

RFC 2980 is informational and this document is intended to be standards
track.  Wouldn't that rule out a normative reference?

> if further extensions are made.

This isn't a bad idea, particularly since NNTP has suffered from clashes
in the past (although mostly because NNTP has been developed for quite
some time outside the auspices of the IETF; this is an attempt to bring it
back into the fold).

> base document (draft-ietf-nntpext-base-26).

Is this something that's mentioned in the SMTP documents on the same
topic?  Maybe we could just reuse existing text, since the issues should
be about the same.

> one there is at present (applies to the authentication document also).

The reference to [NNTP] is normative in the copy that I have.

> makes some of the later stuff more comprehensible.

Good point.

> actually do anything!

Also a very good point.

I'm not as sure about the other points you raise, but your message has
been passed along to the working group for further discussion.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Mon, 6 Jun 2005 10:52:54 +0100

Russ Allbery said:
> I also do't believe that RFC2980 

Here and later: my understanding of "normative reference" is that it
indicates documents which need to be understood to implement the document
making the reference. That isn't the case for 2980 - it helps to see *why*
certain decisions were made, but you can implement all the new NNTP
documents without knowing anything inside 2980.

> reorder (2.1, 2.2, 2.3) -> (2.2, 2.3, 2.1)).

I disagree: it might be better swapped with 2.2, but it should come before
the commands defined in the specification.

> I note that the capability 

And the TLS one. However, once again I disagree with your interpretation of
the rules. The Instructions to RFC Authors says:

    Normative references
    specify documents that must be read to understand or implement the
    technology in the new RFC, or whose technology must be present for
    the technology in the new RFC to work.

In the base document, the only purpose of these references is to say "this
entry in the registry is [reserved] for this document". That's informative
as I see it - you can implement all of NNTP without reading those documents.

> reponse codes.

I can sympathise with this viewpoint, but I wouldn't like it to hold us up.
Should we look at doing it as a separate document?

It is worth noting that you can always tie a response to the command that
caused it, so collisions aren't as big an issue as in a protocol that has
unsolicited responses. Ensuring consistency and uniqueness in responses just
allows for other program designs, but is not essential.

> (espcially CHECK) offer an opportunity for DoS attacks on NNTP servers. 

I'm not sure why this is the case over and above any protocol that uses
TCP. The pipelined commands will simply sit in the receiver's buffer until
they are read. There is no requirement for the server to do its own
buffering of these commands while responding to them.

The base document does mention the danger of deadlock, but I'm not sure how
to turn that into an attack.

> way to control overloading by servers, might help.

I still don't follow. A string of 431s won't necessarily cause the client
to stop doing CHECKs, particularly if it's a malicious one.

Can you explain how a string of (pipelined) CHECKs is a particular threat?
Compared with, say, a string of (pipelined) STATs?

-- 
Clive D.W. Feather  | Work:  <[email-masked]>   | Tel:    +44 20 8495 6138
Internet Expert     | Home:  <[email-masked]>  | Fax:    +44 870 051 9937
Demon Internet      | WWW: http://www.davros.org | Mobile: +44 7973 377646
Thus plc            |                            |

---

From: "Andrew - Supernews" <[email-masked]>
Date: Thu, 09 Jun 2005 03:28:53 +0100

I took a guess at the best CC list to use for this response; forward
if appropriate. This response is aimed mainly at an area where I think
the review missed the point, rather than real problems with the
documents.

 >> Subject: Gen-art review of draft-ietf-nntpext-streaming-05

 [...]

 >> I noted that the potentiality for pipelined commands to be used
 >> fos DoS attacks was not mentioned in the Security Considerations
 >> of this or the base document (draft-ietf-nntpext-base-26).

 [...]

 >> (espcially CHECK) offer an opportunity for DoS attacks on NNTP
 >> servers.  This is not discussed either in this document or the
 >> base specification to which it refers.  Maybe some notes about
 >> rate limiting generation of CHECK requests by clients (over and
 >> above the requirement to consider buffer sizes), and possibly the
 >> use of 431 responses as a way to control overloading by servers,
 >> might help.

Both of the above comments rather miss the point that the CHECK and
TAKETHIS commands exist _specifically_ for the purpose of transferring
large volumes of data (the Usenet feed is around two terabytes per day
at the moment) at the fastest possible rate that the receiving server
can handle. It's very important that implementations _not_
artificially impose limits on the rate at the sending (client) end,
and it's up to the receiving end to decide how to handle the load.
Servers have not historically used 431 responses to manage load, and
it's unlikely that it would have been successful (one older
implementation treated 431 as though it were 239, which would have
rather defeated any such attempt).

Real implementations typically allow the number of concurrent
connections (total and/or from a single feed site) to be limited
(which is a precaution not in any way specific to streaming feeds).
Some also allow the accepted bytes/second from a given feed source to
be limited at the server too, but this isn't done to protect the
server, but more typically to provide bandwidth management options to
the administrator (when dealing with a 200 megabit traffic flow such
things are often important).

So I am (and I suspect most of the rest of the NNTP group are)
somewhat puzzled by what you think the specific security issue with
CHECK is.

-- 
Andrew, Supernews
http://www.supernews.com

---

From: Elwyn Davies <[email-masked]>
Date: Thu, 09 Jun 2005 11:39:04 +0100

The sort of thing I was thinking of was sending streams of CHECKs for 
articles and never sending  TAKETHIS, or asking about or sending the 
same  article repeatedly, or sending a stream of CHECKs for the same 
article that the malicious client knows the server has already.  These 
could waste quite a lot of your bandwidth!  I understand that the whole 
point is to provide a wide open pipe, but a wide open pipe can be a wide 
open opportunity for the malicious client - can he exploit it in this 
case?  Your response is fine where the client is well-behaved and 
sending a sensible set of requests... not every 'client' is necessarily 
well-behaved and my feeling was that you needed to at least mention how 
a server could protect itself against a deliberately malicious client, 
and maybe how a well-behaved client could be a better citizen by 
controlling its load.  CHECKs in particular are a way of making a server 
do unproductive work at little cost to the malicious client.

Don't forget I am not an expert in NNTP things - part of the job of 
gen-art is to think the unthinkable from another point of view, to 
challenge (apparently undocumented) assumptions and apply experience 
from other fields.  If you guys are *really* sure that this is not a 
problem and you can show I am talking from the wrong part of my anatomy 
then that is cool with me.  All I (we) ask is that you get past the knee 
jerk, and spend at least a couple of seconds checking that you haven't 
had a collective blind spot!

Regards,
Elwyn

---

From: Russ Allbery <[email-masked]>
Date: Thu, 09 Jun 2005 15:54:55 -0700

Elwyn Davies <[email-masked]> writes:

> articles and never sending TAKETHIS,

This *does* happen and does have a negative impact on the server due to
the standard practice of marking such messages as in progress and
deferring other offers of the same message.  I'm wondering if we should
say something about this, although I'm not quite sure what to say.

> or asking about or sending the same article repeatedly,

This is extremely fast for the server to deal with, so the only concern
would be bandwidth (and starvation in a poorly designed server, but
servers usually deal with that fairly well).

> or sending a stream of CHECKs for the same article that the malicious
> client knows the server has already.

This is also extremely fast for the server to deal with.

> These could waste quite a lot of your bandwidth!

I'm personally not sure it's worth worrying too much about bandwidth
attacks on TCP services, as I can consume lots of bandwidth with pretty
much any arbitrary TCP service (consider sending lots and lots of MAIL,
RCPT, RSET commands to an SMTP server, for instance).  NNTP STREAMING is
actually *less* vulnerable than something like SMTP because it's generally
only used with authenticated peers, so there's someone who's access can be
disabled if they abuse the privilege.

But then, I don't have a broad range of protocol experience.  Would you
raise the same issue with SMTP?  It feels to me like other protocols have
this same issue, so if there's some standard way of dealing with it, maybe
we could just copy that language.

---

From: "Charles Lindsey" <[email-masked]>
Date: Fri, 10 Jun 2005 15:20:54 GMT

In <[email-masked]> Elwyn Davies <[email-masked]> writes:

>article that the malicious client knows the server has already.

I think the whole point of the STREAMING extension is that it is intended
to be used between peers who have agreed to it beforehand. Thus a random
outsider, even if he knows how to authenticate to that server, is unlikely
to find the STREAMING capability open to him. Thus malicious clients can
be largely discounted.

Plain broken clients are another matter, but a server administrator who
finds himself in that situation will simply dis-authorize that client
until it gets its system fixed.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5

---

From: "Andrew - Supernews" <[email-masked]>
Date: Fri, 10 Jun 2005 20:37:18 +0100

>>>>> "Elwyn" == Elwyn Davies <[email-masked]> writes:

 Elwyn> The sort of thing I was thinking of was sending streams of
 Elwyn> CHECKs for articles and never sending TAKETHIS,

2.4.2: "Note however, that the responses to CHECK are advisory; the
server MUST NOT rely on the client to behave as requested by these
responses."

I don't know if this is worth pointing out in the security
considerations too; the fact that you mentioned it might suggest it
_is_ worth it :-)

What practical applications do is usually: if a CHECK or IHAVE is
received from one connection, then send 431 responses to the same
CHECK (or IHAVE) on other connections until either the article is
received or a short timeout expires. This avoids receiving too many
copies of the same article while not reducing feed reliability unless
the 431 is consistently ignored by other feeding clients. In
pathological cases this can still cause a problem, but it's not
common in practice.

 Elwyn> or asking about or sending the same article repeatedly, or
 Elwyn> sending a stream of CHECKs for the same article that the
 Elwyn> malicious client knows the server has already.  These could
 Elwyn> waste quite a lot of your bandwidth!  I understand that the
 Elwyn> whole point is to provide a wide open pipe, but a wide open
 Elwyn> pipe can be a wide open opportunity for the malicious client -
 Elwyn> can he exploit it in this case?  Your response is fine where
 Elwyn> the client is well-behaved and sending a sensible set of
 Elwyn> requests... not every 'client' is necessarily well-behaved and
 Elwyn> my feeling was that you needed to at least mention how a
 Elwyn> server could protect itself against a deliberately malicious
 Elwyn> client, and maybe how a well-behaved client could be a better
 Elwyn> citizen by controlling its load.  CHECKs in particular are a
 Elwyn> way of making a server do unproductive work at little cost to
 Elwyn> the malicious client.

They are not nearly as bad in this respect as some of the commands in
the base spec - STAT, HDR and LISTGROUP for example, which are made
much more widely available than CHECK/TAKETHIS, which are invariably
restricted to specific IP addresses (some servers have tried a
halfhearted approach to password auth for feed connections, but it's
never caught on - however one can hope the AUTHINFO draft may help fix
this by providing a standard alternative to IP-based auth).

A well-behaved streaming client is one that transfers as much data as
possible using as few concurrent connections as possible. Any attempt
to limit things at the sending end is counterproductive.

-- 
Andrew, Supernews
http://www.supernews.com

---

From: Ken Murchison <[email-masked]>
Date: Sun, 12 Jun 2005 20:41:00 -0400

> considerations too; the fact that you mentioned it might suggest it

Suggested text?

I would think that any text we craft regarding CHECK would also apply to 
IHAVE (the first step).

It would seem to me that TAKETHIS is more worrisome if the peer just 
starts flooding the pipe with articles that the server doesn't want. 
Should we say something along the lines like "if the number of rejected 
TAKETHIS commands exceeds an implementation specific threshold, the 
server SHOULD/MAY terminate the session with a 400 response".

-- 
Kenneth Murchison     Oceana Matrix Ltd.
Software Engineer     21 Princeton Place
716-662-8973 x26      Orchard Park, NY 14127
--PGP Public Key--    http://www.oceana.com/~ken/ksm.pgp

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Mon, 13 Jun 2005 09:30:33 +0100

Elwyn Davies said:
> could waste quite a lot of your bandwidth!

However, this applies equally to almost any command in NNTP. Certainly STAT
can be misused in exactly the same way.

But it also applies to almost any command in any protocol. What stops an
SMTP client from repeatedly sending sequences of MAIL, 100 RCPT, and RSET
commands? What stops and HTTP client from sending lots of the check page
last changed date command? All of these make the server do lots of work for
little effort.

I agree that there is a *potential* issue here. I'm just not convinced it's
serious enough to mention here, as opposed to in a general "preventing
malicious use of TCP/IP" document.

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Mon, 13 Jun 2005 09:31:32 +0100

> server SHOULD/MAY terminate the session with a 400 response".

I won't scream if you add it, but as I've just written, I don't see this as
being NNTP-specific.

If the client really is malicious, tarpitting it might be better than just
issuing a 400.

---

From: Russ Allbery <[email-masked]>
Date: Mon, 13 Jun 2005 01:34:39 -0700

> starts flooding the pipe with articles that the server doesn't
> want. Should we say something along the lines like "if the number of
> rejected TAKETHIS commands exceeds an implementation specific threshold,
> the server SHOULD/MAY terminate the session with a 400 response".

The difference with CHECK, and the thing that it allows you to do that
IHAVE doesn't (and TAKETHIS doesn't) is that you can prevent a host from
ever receiving an article by repeatly claiming you're going to send it
with CHECK and then never sending it.  You can't really do that with IHAVE
since you can only offer one article at a time and then either need to
spit out an article or abort the connection.  With CHECK, though, you can
send a whole flood of message IDs without ever stopping.

Whether this is important enough to care about, I'm not sure.  I don't
think it really is, since a peer can also give you a corrupted or fake
article with IHAVE and prevent you from getting the real one.

I suppose the other concern is that CHECK makes the server do more work
per KB of bandwidth than TAKETHIS, since the server has to check each
message ID against the history database.  I think that's more what the
original review comment was about, but I agree with others who have said
that LISTGROUP and the like are even worse.

I'm not seeing anything here that's particularly compelling to comment
about, but if anyone else does, given the above, please speak up.

---

From: Ade Lovett <[email-masked]>
Date: Mon, 13 Jun 2005 09:50:28 -0700

>server SHOULD/MAY terminate the session with a 400 response".

This simply doesn't make sense.  Practically every single command in the
spec can be (ab)used to cause pain for a server, with the amount of pain
being implementation dependent.

In the real world, CHECK/TAKETHIS are also likely to be only available
to a small subset of the set of NNTP connections a "server" has to deal
with, presumably to "well-known" endpoints (other NNTP transit peers).

Finally, if something bad *does* happen, it's very unlikely that a
runaway is likely to be able to do anything useful with a 400 response.
 A more correct approach would be to simply deep six the session with a
close() and move on.

Finally (again.  I lied.) a quick scan of various other major protocols
(in particular (E)SMTP) shows no references there to such runaway
behavior, so I don't see any reason why NNTP has to be different in
explicitly mentioning such behavior.

-aDe

---

From: [email-masked]
Date: Mon, 13 Jun 2005 18:24:56 +0100

Maybe I am missing something here, but the original reason for my raising this
query is that the streaming commands are different from the others.

Unless I have totally misunderstood (and I may have), the non-streaming commands
expect to generate a response from the server before the client sends another
command (that was apparently the whole point of the streaming commands to get
round this synchronicity).  My understanding was therefore that a server would
be able to quickly discard commands (and maybe drop the connection) if another
command arrived before it sent the response for commands other than the
streaming ones because it would be a protocol error.  The streaming ones on the
other hand could be legitimately sent as fast as the client could make them thus
leading to a potential drain on the server which it couldn't easily control.  As
far as I am aware all the counterexamples cited recently are from synchronous
protocols, rather than the asynchronous scheme which the streaming commands
introduce.  

Regards,
Elwyn

--

---

From: Russ Allbery <[email-masked]>
Date: Mon, 13 Jun 2005 10:25:45 -0700

> explicitly mentioning such behavior.

Could you look over the text I posted earlier this morning and see if it
would work for you?  I think there are a few things we can usefully say.

---

From: Ken Murchison <[email-masked]>
Date: Mon, 13 Jun 2005 13:36:44 -0400

> query is that the streaming commands are different from the others.

Different how?  Many of the commands defined in the base document can be 
"pipelined" (e.g. STAT, HEAD, ARTICLE, OVER), in the same fashion as 
CHECK and TAKETHIS.  The only reason for CHECK and TAKETHIS is that 
IHAVE is a two step command where it requires the client to wait for an 
initial response before sending the article, therefore it can not be 
"pipelined".  CHECK and TAKETHIS simply split the two steps into 
separate commands.

So, if you have issues with "pipelined" commands, then you have issues 
with the base NNTP spec as well as IMAP and POP3.

---

From: Russ Allbery <[email-masked]>
Date: Mon, 13 Jun 2005 10:38:08 -0700

elwynd <[email-masked]> writes:

> Maybe I am missing something here, but the original reason for my
> raising this query is that the streaming commands are different from the
> others.

> Unless I have totally misunderstood (and I may have), the non-streaming
> commands expect to generate a response from the server before the client
> sends another command (that was apparently the whole point of the
> streaming commands to get round this synchronicity).  My understanding
> was therefore that a server would be able to quickly discard commands
> (and maybe drop the connection) if another command arrived before it
> sent the response for commands other than the streaming ones because it
> would be a protocol error.

No, pipelining is supported for all NNTP commands except for a small
handful of exceptions.  That was added in the new protocol revision and is
documented in the base draft.

> The streaming ones on the other hand could be legitimately sent as fast
> as the client could make them thus leading to a potential drain on the
> server which it couldn't easily control.  As far as I am aware all the
> counterexamples cited recently are from synchronous protocols, rather
> than the asynchronous scheme which the streaming commands introduce.

I was specifically looking for security considerations in RFC 2920, which
seems equivalent.

However, the commands are a bit different in a few other respects.  I
proposed the following text this morning; would this help?

    A malicious client with knowledge of the message-ids a server will be
    receiving could use a flood of CHECK commands to cause the server to
    defer offers of those articles from its other peers.  If client
    authentication is not sufficient to protect against this attack, the
    server SHOULD watch for clients that send excessive CHECK commands
    without a following TAKETHIS and reject further CHECK commands (with
    438 or 431) from that client.

    TAKETHIS is designed to maximize the speed of article transfer and
    therefore is the only client command followed immediately by a
    multi-line data block.  Servers concerned with possible excessive
    bandwidth utilization by clients using TAKETHIS SHOULD restrict use of
    the STREAMING extension to trusted and authenticated clients.

---

From: [email-masked]
Date: Mon, 13 Jun 2005 19:00:49 +0100

OK...

Thank you for the clarification... I realise that to an extent this is all an
academic discussion since NNTP is out there and has been doing a sterling job
for many years.  As I said at the outset I am not an NNTP expert and so I was
not aware that so many other commands could be pipelined already.  I think Russ'
proposal which he just copied in another email would probably suit.  Given that
much of this is about documenting current practice, we shouldn't agonise too
much over this, just as long as people are aware that there is a risk.

I do note that in the case of IMAP authentication and frequently the use of TLS
have been built in from the beginning. 

Regards,
Elwyn

--

---

From: Ade Lovett <[email-masked]>
Date: Mon, 13 Jun 2005 11:59:31 -0700

> would work for you?  I think there are a few things we can usefully say.

I've read it over and I have to admit that I'm siding with Ken on this
one as being more of an implementation issue than anything.

We're already saying (in nntpext-streaming sec.2.4.2):

	"Note however, that the responses to CHECK are advisory;
	 the server MUST NOT rely on the client to behave as
	 requested by these responses."

To me, the strong MUST NOT covers everything we'd want to say at a
protocol level.  As to exactly how it should be dealt with, that's
definitely treading into implementation specifics.

The most likely approach taken by a server in such a runaway case would
be to simply terminate the TCP session, doing anything more gracefully
is unlikely to work, and would be syntactic sugar anyway.

Client behavior in the case of the TCP session going away is already
covered elsewhere, and it would presumably try to reconnect.  As to what
the server does at this point is again an implementation detail, eg:

	"502 bad runaway client, no cookie, contact [email-masked]"

-aDe

---

From: Russ Allbery <[email-masked]>
Date: Mon, 13 Jun 2005 13:13:09 -0700

> one as being more of an implementation issue than anything.

I really don't understand this at all.  How is the interaction between
multiple streams of CHECKs not a protocol issue?

> 	"502 bad runaway client, no cookie, contact [email-masked]"

Picking a random status code out of a hat is definitely not an
implementation detail.  This problem is not a case where the client needs
to acquire different authentication credentials.  You meant 400.

---

From: "Andrew - Supernews" <[email-masked]>
Date: Mon, 13 Jun 2005 23:32:19 +0100

>>>>> "Russ" == Russ Allbery <[email-masked]> writes:

 Russ> The difference with CHECK, and the thing that it allows you to
 Russ> do that IHAVE doesn't (and TAKETHIS doesn't) is that you can
 Russ> prevent a host from ever receiving an article by repeatly
 Russ> claiming you're going to send it with CHECK and then never
 Russ> sending it.

It's simple to protect against this. What I do is: a precommit entry
for the article can only be created by the _first_ IHAVE, CHECK or
TAKETHIS command to mention the message-id. If that entry is then not
committed within the timeout and it expires, then we _keep it_ in the
precommit cache as an expired entry, and further IHAVEs or CHECKs for
it, from any source, will return a "yes, we want the article"
response.  The entry stays in place until it is overwritten (in our
implementation the precommit and history caches are a single cyclic
buffer, which on machines with inbound feeds from outside is sized to
contain a large fraction of the daily number of articles), or until
we actually receive the article, in which case the entry becomes a
history cache entry instead.

-- 
Andrew, Supernews
http://www.supernews.com

---

From: Russ Allbery <[email-masked]>
Date: Mon, 13 Jun 2005 16:52:10 -0700

> It's simple to protect against this. What I do is: a precommit entry for
> the article can only be created by the _first_ IHAVE, CHECK or TAKETHIS
> command to mention the message-id. If that entry is then not committed
> within the timeout and it expires, then we _keep it_ in the precommit
> cache as an expired entry, and further IHAVEs or CHECKs for it, from any
> source, will return a "yes, we want the article" response.

That's a good technique.  I guess the question that I have is should we
say something about it in the document (probably not to this degree of
specificity but at least to the degree of giving some general advice)?

---

From: "Charles Lindsey" <[email-masked]>
Date: Tue, 14 Jun 2005 11:16:29 GMT

>proposed the following text this morning; would this help?

Not really.

>    the STREAMING extension to trusted and authenticated clients.

I think the last sentence there might be of some help, but steer clear of
floods of CHECKs. I agree with Ade that if you start down that track, then
you have to go the whole way, and I don't think that is necessary.