Subject: Re: [NNTP] Extension docs snapshots

From: Ken Murchison <[email-masked]>
Date: Wed, 01 Dec 2004 11:16:57 -0500

Russ Allbery wrote:

> Ken Murchison <[email-masked]> writes:
> 
>>Clive D.W. Feather wrote:
> 
> 
>>>streaming, section 2.3.1, why is MODE STREAM "MUST NOT be pipelined"? I
>>>understand the reasoning with MODE READER, but I don't understand it
>>>here (of course, the client needs to be careful what it does
>>>immediately afterwards, but if it pipelined a sequence of CHECKs would
>>>it hurt?).
> 
> 
>>Don't know.  Someone more familiar with streaming (Andrew?) needs to
>>chime in here.  I don't recall when this came into the document.  It
>>seems to me that any argument that can be made for MODE READER can be
>>made for MODE STREAM and vice versa.
> 
> 
> You can't make the same argument for MODE STREAM as for MODE READER
> because the thing about MODE READER is that it hands the connection off to
> another binary.  I'm unaware of anyone who implemented streaming that way,
> or of any purpose served in doing so.
> 
> The comment that it can't be pipelined is in there not for any inherent
> protocol reason, but because it doesn't make any sense to do so.  The
> thing you're going to want to do next is issue a CHECK or TAKETHIS
> command, and until MODE STREAM returns, you don't know if you actually can
> or if you need to use IHAVE instead.  It's an extension negotiation
> command, and therefore it doesn't make any sense to queue up use of the
> extension before you're aware that the extension is there.
> 
> It doesn't really matter, but I'd be inclined to leave in the prohibition;
> it's going to quickly become irrelevant since MODE STREAM is only an issue
> when dealing with legacy implementations.

Would it be better if I changed the text to say "SHOULD NOT be pipelined"?

-- 
Kenneth Murchison     Oceana Matrix Ltd.
Software Engineer     21 Princeton Place
716-662-8973 x26      Orchard Park, NY 14127
--PGP Public Key--    http://www.oceana.com/~ken/ksm.pgp

---

From: Russ Allbery <[email-masked]>
Date: Wed, 01 Dec 2004 11:37:52 -0800

>> command, and until MODE STREAM returns, you don't know if you actually
>> can or if you need to use IHAVE instead.  It's an extension negotiation
>> extension before you're aware that the extension is there.  It doesn't
>> really matter, but I'd be inclined to leave in the prohibition; it's
>> going to quickly become irrelevant since MODE STREAM is only an issue

> Would it be better if I changed the text to say "SHOULD NOT be

That works for me.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

---

From: Ken Murchison <[email-masked]>
Date: Wed, 01 Dec 2004 15:14:33 -0500

>>Would it be better if I changed the text to say "SHOULD NOT be

Or would it be better yet to say "SHOULD NOT be pipelined with CHECK or 
TAKETHIS"?

---

From: Russ Allbery <[email-masked]>
Date: Wed, 01 Dec 2004 12:34:11 -0800

> Or would it be better yet to say "SHOULD NOT be pipelined with CHECK or

Actually, if we're going to say that, we could just say MUST NOT.  That
also works for me.

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Thu, 2 Dec 2004 09:25:54 +0000

> protocol reason,

Right there that says to me that a MUST, or even a SHOULD, is the wrong
thing to do.

> but because it doesn't make any sense to do so.  The
> or if you need to use IHAVE instead.

You don't know that the next thing is going to be a CHECK and, even if so,
why does it matter? Suppose the pipeline depth is 3:

    [C] MODE STREAM
    [C] CHECK <[email-masked]>
    [C] CHECK <[email-masked]>
    [S] 501 What is MODE STREAM?
    [S] 500 What is CHECK?
    [S] 500 What is CHECK?

Where's the problem? Whereas:

    [C] MODE STREAM
    [C] STAT <[email-masked]>
    [C] STAT <[email-masked]>
    [S] 501 What is MODE STREAM?
    [S] 223 0 <[email-masked]>
    [S] 430 No such article <[email-masked]>

might be a useful thing to do.

Can we please go back to basic principles on this? The reason for
pipelining restrictions is that some servers hand off certain commands to
other processes and there is then an issue with which process gets which
data from the socket. That justifies the restriction on MODE READER. The
handshaking of things like IHAVE also mean that pipelining can't work.

But unless there is a protocol interoperability (or security) reason for
not pipelining stuff, we should NOT be restricting it. Even if we can't see
why a client might want to pipeline a command, that is not a good reason
for banning it. Pipelining should be the default.

-- 
Clive D.W. Feather  | Work:  <[email-masked]>   | Tel:    +44 20 8495 6138
Internet Expert     | Home:  <[email-masked]>  | Fax:    +44 870 051 9937
Demon Internet      | WWW: http://www.davros.org | Mobile: +44 7973 377646
Thus plc            |                            |

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Thu, 2 Dec 2004 10:59:34 +0000

Russ Allbery said:
> That would also be fine with MODE STREAM, but the difference with MODE
> STREAM is that a client that sends it is likely to send nothing other than
> CHECK, TAKETHIS, and QUIT commands for the whole rest of the session, and
> none of those make sense to pipeline with MODE STREAM.

Nevertheless, as I've already said, that's not a reason to forbid it.

If we do forbid it, ten years down the line some idiot will come up with an
implementation that relies on it being forbidden. I'd rather not leave that
hostage to fortune.

[If you say that no such idiot will appear, that's just proving that
there's no need for the prohibition.]

---

From: Russ Allbery <[email-masked]>
Date: Thu, 02 Dec 2004 10:16:48 -0800

>> but because it doesn't make any sense to do so.  The thing you're going
>> to want to do next is issue a CHECK or TAKETHIS command, and until MODE
>> STREAM returns, you don't know if you actually can or if you need to
>> use IHAVE instead.

> You don't know that the next thing is going to be a CHECK and, even if
> so, why does it matter? Suppose the pipeline depth is 3:

> Where's the problem?

Now try that again where the next command is TAKETHIS.

> pipelining restrictions is that some servers hand off certain commands
> to other processes and there is then an issue with which process gets
> which data from the socket. That justifies the restriction on MODE
> READER. The handshaking of things like IHAVE also mean that pipelining
> can't work.

The reason for the MODE STREAM command is that if you send TAKETHIS
without establishing that the server supports streaming first, really,
really bad things happen.  There is no requirement in the protocol to send
CHECK before TAKETHIS, and in fact servers are sometimes configured not to
do so in particular circumstances.

Really nasty interoperability problems happen if you pipeline MODE STREAM
and TAKETHIS together to a server that doesn't support streaming.  I think
we need to say something about that somewhere.

---

From: Ken Murchison <[email-masked]>
Date: Thu, 02 Dec 2004 13:24:06 -0500

> we need to say something about that somewhere.

Its already done.  The current text reads:

"This command MUST NOT be pipelined with the CHECK or TAKETHIS
commands."

Once we nail down the CAPABILITIES/LIST CAPABILITIES command and MODE 
READER stuff, I will have new pre-2 drafts ready.  I'm working on the 
AUTHINFO SASL changes now.

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Fri, 3 Dec 2004 14:24:36 +0000

> Now try that again where the next command is TAKETHIS.

That's a different problem: you really ought not to do TAKETHIS until you
know it will work. But that does *not* mean that you shouldn't pipeline
MODE STREAM.

> really bad things happen.

Agreed.

> There is no requirement in the protocol to send
> CHECK before TAKETHIS,

Also agreed.

> we need to say something about that somewhere.

Also agreed, but MODE STREAM isn't the problem.

The correct solution is wording something like:

    A client MUST NOT use the TAKETHIS command unless:
    * it has confirmed that the server capability list includes a line
      with the capability name STREAMING, or
    * it has received a 203 response from the MODE STREAM command.

Problem solved. Pipelining unaffected.

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Fri, 3 Dec 2004 14:27:08 +0000

> "This command MUST NOT be pipelined with the CHECK or TAKETHIS

We don't have a definition of "pipelined with" and, in any case, this
doesn't solve the problem:

    [C] CAPABILITIES
    [S] VERSION 2
    [S] TRANSIT
    [S] .
    [C] TAKETHIS <[email-masked]>
    [C] Newsgroup: example.local

is just as broken, even though no pipelining is taking place.

See my response to Russ just posted for better wording.
 
> AUTHINFO SASL changes now.

Please be prepared for cosmetic/admin changes as we negotiate stuff and
nail it down. (I.e. don't say I didn't warn you.)

---

From: Ken Murchison <[email-masked]>
Date: Fri, 03 Dec 2004 09:52:12 -0500

> See my response to Russ just posted for better wording.

Yup, got it.  I'll remove the bit about pipelining (unless Russ objects) 
and use your text.

> nail it down. (I.e. don't say I didn't warn you.)

Right.  I was thinking about putting up some -pre2 drafts today anyways 
so people can look at the more substantive changes.

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Fri, 3 Dec 2004 14:52:03 +0000

> so people can look at the more substantive changes.

Good idea.

---

From: Russ Allbery <[email-masked]>
Date: Fri, 03 Dec 2004 08:37:55 -0800

> Yup, got it.  I'll remove the bit about pipelining (unless Russ objects)

No objections here; Clive's text is great.

---

From: "Charles Lindsey" <[email-masked]>
Date: Sat, 4 Dec 2004 19:31:09 GMT

>implementation that relies on it being forbidden. I'd rather not leave that

I can imagine an implementation that got screwed by relying on it being
allowed, but I would have thought it was provably impossible to rely on it
being forbidden (no, I am not going to construct the proof).

Generally speaking, I would have thought the term "hostage to fortune"
would be more applicable to allowing some capability that was provably
useless.

-- 
Charles H. Lindsey ---------At Home, doing my own thing------------------------
Tel: +44 161 436 6131 Fax: +44 161 436 6133   Web: http://www.cs.man.ac.uk/~chl
Email: [email-masked]      Snail: 5 Clerewood Ave, CHEADLE, SK8 3JU, U.K.
PGP: 2C15F1A9      Fingerprint: 73 6D C2 51 93 A0 01 E7 65 E8 64 7E 14 A4 AB A5