Subject: [NNTP] STREAMING diffs

From: Ken Murchison <[email-masked]>
Date: Thu, 09 Jun 2005 14:24:20 -0400

First crack at STREAMING fixes.  I rewrote the Intro, with more 
references to NNTP-COMMON, I swapped sections 2.1 and 2.2, and I 
reinforced the fact that MODE STREAM does not constitute a mode change 
and can be pipelined.  I didn't do anything w.r.t. the DoS comments 
because nobody on the list seemed to agree that DoS is more of an issue 
with STREAMING.  If anybody would like to do some additional 
word-smithing on the Intro, I'd be happy to consider alternative text.

--- streaming-5.txt	2005-06-09 14:17:54.000000000 -0400
+++ streaming-6.txt	2005-06-09 14:17:48.000000000 -0400
@@ -6,13 +6,13 @@

  NNTP Extensions Working Group                                 J. Vinocur
  Internet Draft                                        Cornell University
-Updates: 2970 (if approved)                                 K. Murchison
+Updates: 2980 (if approved)                                 K. Murchison
  Expires: December 2005                                Oceana Matrix Ltd.
                                                                 June 2005

                     NNTP Extension for Streaming Feeds
-                    draft-ietf-nntpext-streaming-05
+                    draft-ietf-nntpext-streaming-06

  Status of this memo
@@ -46,23 +46,31 @@
  Abstract

       This memo defines an extension to the Network News Transport
-     Protocol [NNTP] to provide asynchronous (otherwise known as
+     Protocol (NNTP) to provide asynchronous (otherwise known as
       "streaming") transfer of articles.  This allows servers to transfer
       articles to other servers with much greater efficiency.

-     Section 1 of [NNTP-COMMON] summarizes some ad-hoc transport
-     extensions currently used in the NNTP protocol.  This document
-     updates and formalizes the CHECK and TAKETHIS commands and
-     deprecates the MODE STREAM command.
+     This document updates and formalizes the CHECK and TAKETHIS
+     commands specified in RFC 2980 and deprecates the MODE STREAM
+     command.
+
+Note to the RFC Editor
+
+     The normative reference to RFC 2234 may be replaced by
+     draft-crocker-abnf-rfc2234bis should it reach RFC status before
+     this one.
+
+     The normative reference to [NNTP] is a document which is expected
+     to be published simultaneously with this one and so can be replaced
+     by reference to the resulting RFC.

  Table of Contents

-     0. Changes from Previous Version ............................  2
       1. Introduction .............................................  3
          1.1. Conventions Used in this Document ...................  3
       2. The STREAMING Extension ..................................  3
-        2.1. Advertising the STREAMING Extension .................  3
-        2.2. Streaming Article Transfer ..........................  4
+        2.1. Streaming Article Transfer ..........................  3
+        2.2. Advertising the STREAMING Extension .................  4
          2.3. MODE STREAM Command .................................  5
             2.3.1. Usage ..........................................  5
             2.3.2. Description ....................................  5
@@ -76,44 +84,38 @@
             2.5.2. Description ....................................  7
             2.5.3. Examples .......................................  8
       3. Augmented BNF Syntax for the STREAMING Extension .........  9
-        3.1. Commands ............................................  9
+        3.1. Commands ............................................ 10
          3.2. Command Datastream .................................. 10
          3.3. Responses ........................................... 10
          3.4. Capability entries .................................. 10
-     4. Summary of Response Codes ................................ 10
+     4. Summary of Response Codes ................................ 11
       5. Security Considerations .................................. 11
       6. IANA Considerations ...................................... 11
       7. References ............................................... 12
          7.1. Normative References ................................ 12
-        7.2. Informative References .............................. 12
+        7.2. Informative References .............................. 13
       8. Authors' Addresses ....................................... 13
       9. Acknowledgments .......................................... 13
       10. Intellectual Property Rights ............................ 13
       11. Copyright ............................................... 14

-0. Changes from Previous Version
-
-     Changed:
-     o  Updated to RFC 3978/3979 boilerplate.
-     o  Uses new ABNF tokens from [NNTP].
-     o  Uses new "multi-line data block" term from [NNTP].
-     o  POST has its own capability.
-
-     Clarified:
-     o  Section 3: This document extends the ABNF in [NNTP], and the
-        [NNTP] ABNF must be imported first before validating the
-        STREAMING ABNF (based on recommendations of AD regarding
-        IMAPEXT I-Ds).
-
  1. Introduction

       According to the NNTP specification [NNTP], a peer uses the IHAVE
       command to query whether a server wants a particular article.
       Because the IHAVE command cannot be pipelined, the need to stop and
       wait for the remote end's response greatly restricts the throughput
-     that can be achieved.  The alternative method of peer-to-peer
-     article transfer described in this document permits a more
-     consistent use of network bandwidth.
+     that can be achieved.
+
+     The ad-hoc CHECK and TAKETHIS commands, documented in
+     [NNTP-COMMON], provide an alternative method of peer-to-peer
+     article transfer which permits a more effective use of network
+     bandwidth.  Due to their ubiquity they are formalized in this
+     specification.
+
+     The ad-hoc MODE STREAM command, also documented in [NNTP-COMMON],
+     is deprecated by this specification, but due to its ubiquity is
+     documented here for backwards compatibility.

  1.1. Conventions Used in this Document

@@ -139,7 +141,36 @@
       This extension provides three new commands: MODE STREAM, CHECK, and
       TAKETHIS.  The capability label for this extension is STREAMING.

-2.1. Advertising the STREAMING Extension
+2.1. Streaming Article Transfer
+
+     The STREAMING extension provides the same functionality as the
+     IHAVE command ([NNTP] section 6.3.2) but splits the query and
+     transfer functionality into the CHECK and TAKETHIS commands
+     respectively.  This allows the CHECK and TAKETHIS commands to be
+     pipelined (see section 3.5 of [NNTP]) and provides for "streaming"
+     article transfer.
+
+     A streaming client will often pipeline many CHECK commands and use
+     the responses to construct a list of articles to be sent by a
+     pipelined sequence of TAKETHIS commands, thus increasing the
+     fraction of time spent transferring articles.  The CHECK and
+     TAKETHIS commands utilize distinct response codes so that these
+     commands can be intermingled in a pipeline and the response to any
+     single command can be definitively identified by the client.
+
+     The client MAY send articles via TAKETHIS without first querying
+     the server with CHECK.  The client SHOULD NOT send every article in
+     this fashion unless explicitly configured to do so by the site
+     administrator based on out-of-band information.  However, the
+     client MAY use an adaptive strategy where it initially sends CHECK
+     commands for all articles, but switches to using TAKETHIS without
+     CHECK if most articles are being accepted (over 95% acceptance
+     might be a reasonable metric in some configurations).  If the
+     client uses such a strategy, it SHOULD also switch back to using
+     CHECK on all articles if the acceptance rate ever falls much below
+     the threshold.
+
+2.2. Advertising the STREAMING Extension

       A server supporting the streaming commands described in this
       document will advertise the "STREAMING" capability label in
@@ -180,35 +211,6 @@
          [S] HDR
          [S] .

-2.2. Streaming Article Transfer
-
-     The STREAMING extension provides the same functionality as the
-     IHAVE command ([NNTP] section 6.3.2) but splits the query and
-     transfer functionality into the CHECK and TAKETHIS commands
-     respectively.  This allows the CHECK and TAKETHIS commands to be
-     pipelined (unlike IHAVE) and provides for "streaming" article
-     transfer.
-
-     A streaming client will often pipeline many CHECK commands and use
-     the responses to construct a list of articles to be sent by a
-     pipelined sequence of TAKETHIS commands, thus increasing the
-     fraction of time spent transferring articles.  The CHECK and
-     TAKETHIS commands utilize distinct response codes so that these
-     commands can be intermingled in a pipeline and the response to any
-     single command can be definitively identified by the client.
-
-     The client MAY send articles via TAKETHIS without first querying
-     the server with CHECK.  The client SHOULD NOT send every article in
-     this fashion unless explicitly configured to do so by the site
-     administrator based on out-of-band information.  However, the
-     client MAY use an adaptive strategy where it initially sends CHECK
-     commands for all articles, but switches to using TAKETHIS without
-     CHECK if most articles are being accepted (over 95% acceptance may
-     be a reasonable metric in some configurations).  If the client uses
-     such a strategy, it SHOULD also switch back to using CHECK on all
-     articles if the acceptance rate ever falls much below the
-     threshold.
-
  2.3. MODE STREAM Command

       Historically this command was used by a client to discover if a
@@ -240,9 +242,10 @@

       If a server supports this extension, it MUST return a 203 response
       to the MODE STREAM command (or 501 if an argument is given).  The
-     MODE STREAM command MUST NOT affect the server state in any way.  A
-     server MUST NOT require that the MODE STREAM command be issued by
-     the client before accepting the CHECK or TAKETHIS commands.
+     MODE STREAM command MUST NOT affect the server state in any way (no
+     mode change), therefore this command MAY be pipelined.  A server
+     MUST NOT require that the MODE STREAM command be issued by the
+     client before accepting the CHECK or TAKETHIS commands.

  2.3.3. Examples

@@ -602,9 +605,9 @@
       [NNTP-COMMON], by Stan Barber.

       Special acknowledgment also goes to Russ Allbery, Clive Feather,
-     and others who commented privately on intermediate revisions of
-     this document, as well as the members of the IETF NNTP Working
-     Group for continual (yet sporadic) insight in discussion.
+     Andrew Gierth, and others who commented privately on intermediate
+     revisions of this document, as well as the members of the IETF NNTP
+     Working Group for continual (yet sporadic) insight in discussion.

  10. Intellectual Property Rights

-- 
Kenneth Murchison     Oceana Matrix Ltd.
Software Engineer     21 Princeton Place
716-662-8973 x26      Orchard Park, NY 14127
--PGP Public Key--    http://www.oceana.com/~ken/ksm.pgp

---

From: "Jeffrey M. Vinocur" <[email-masked]>
Date: Thu, 9 Jun 2005 22:11:24 -0400

On Jun 9, 2005, at 2:24 PM, Ken Murchison wrote:

>       that can be achieved.

> +     documented here for backwards compatibility.

I think this makes all the right points.  A little tweaking...

      The ad-hoc CHECK and TAKETHIS commands, originally documented in
      [NNTP-COMMON], provide an alternative method of peer-to-peer
      article transfer that permits a more effective use of network
      bandwidth.  Due to their proven usefulness and wide deployment,
      they are formalized in this specification.

-- 
Jeffrey M. Vinocur
[email-masked]

---

From: "Clive D.W. Feather" <[email-masked]>
Date: Fri, 10 Jun 2005 05:50:03 +0100

A small wordsmithing for this:

      to the MODE STREAM command (or 501 if an argument is given).  The
+-    MODE STREAM command MUST NOT affect the server state in any way (no
+-    mode change), therefore this command MAY be pipelined.  A server
++    MODE STREAM command MUST NOT affect the server state in any way
++    (that is, it is not a mode change despite the name)
++    and therefore MAY be pipelined.  A server
+     MUST NOT require that the MODE STREAM command be issued by the
+     client before accepting the CHECK or TAKETHIS commands.

-- 
Clive D.W. Feather  | Work:  <[email-masked]>   | Tel:    +44 20 8495 6138
Internet Expert     | Home:  <[email-masked]>  | Fax:    +44 870 051 9937
Demon Internet      | WWW: http://www.davros.org | Mobile: +44 7973 377646
Thus plc            |                            |