Subject: [NNTP] Adding A Quota Command to NNTP (internet-draft)

From: Samuel Kleiner <[email-masked]>
Date: Tue, 6 Feb 2007 05:14:35 +0100

(reposted due to wrong e-mail signup)

---

From: Russ Allbery <[email-masked]>
Date: Mon, 05 Feb 2007 20:27:33 -0800

Samuel Kleiner <[email-masked]> writes:

> Internet-Draft                               S. Kleiner,[email-masked]
> Intended status: Standards Track                      February 5th, 2007
> Expires: August 5th, 2007

>                     Adding A Quota Command to NNTP
>              draft-kleiner-adding-quota-command-to-nntp-00

Interesting.  I can't evaluate the general usefulness of this, since I'm
not at all involved in the binary or commercial provider world where this
is more common, but from a protocol perspective it seems like a reasonable
extension.

I would use full keywords rather than single letters for the types of
quota; it's more extensible.

This does raise a different question of whether there should be some way
for the server to return timeout and maximum article size configuration,
although that's not quite the same problem and probably shouldn't use the
same command.

-- 
Russ Allbery ([email-masked])             <http://www.eyrie.org/~eagle/>

---

From: Ade Lovett <[email-masked]>
Date: Thu, 22 Feb 2007 02:57:30 -0800

> is more common, but from a protocol perspective it seems like a  

My primary concern over this would be whether the proposal would be  
expecting real-time status of quota usage.  In the commercial world,  
it is not uncommon to have a reasonable number of individual front  
end reader boxes behind an L4 load balancer, further, until sessions  
are actually closed and then some time after that, when the  
accounting data has been processed back into a central repository,  
would the usage limits be updated.

With these two issues in mind, it is therefore entirely possible for  
out-of-date (ie: incorrect) information to be passed back to the  
client (think 2 connections to news.example.com, via an L4 balancer,  
going to two completely different machines, the only commonality  
being the same authentication tokens have been passed).  Thus I'm not  
sure that the extra workload of "hey, your quota thing sucks! it told  
me I had <x> GB left, when I really only had <x-dx>!" on a support  
desk is worth the effort.

> quota; it's more extensible.

Absolutely agreed.

> although that's not quite the same problem and probably shouldn't  

Definitely needs to be another extension, if implemented at all.  I  
guess the main question would be what kind of "server information"  
could possibly be made available to the client beyond timeout/ 
maxartsize.

-aDe

---

From: [email-masked] (Peter Robinson)
Date: Thu, 22 Feb 2007 20:15:11 +0000

Ade Lovett <[email-masked]> wrote:

> I guess the main question would be what kind of "server information" could
> possibly be made available to the client beyond timeout/ maxartsize.

Max concurrent connections perhaps?  Not sure how much benefit clients
could make of that information but both text and binary clients often
open multiple simultaneous connections and the only way they currently
have to discover a limit is to wait for a 400.

It certainly causes user complaints, but half the trouble seems to be
caused by clients (OE?) just dropping connection without QUITting
cleanly.

Peter

---

From: Samuel Kleiner <[email-masked]>
Date: Thu, 22 Feb 2007 22:07:51 +0100

> have to discover a limit is to wait for a 400.

That's already in this proposal. I felt that it met the standard of a  
user quota.

---

From: Samuel Kleiner <[email-masked]>
Date: Fri, 23 Feb 2007 03:35:28 +0100

On Feb 22, 2007, at 11:57 AM, Ade Lovett wrote:

> expecting real-time status of quota usage.  In the commercial  
> world, it is not uncommon to have a reasonable number of individual  
> front end reader boxes behind an L4 load balancer, further, until  
> sessions are actually closed and then some time after that, when  
> the accounting data has been processed back into a central  
> repository, would the usage limits be updated.

 From my own experience, some providers do update the central  
repository more
frequently  than that, say, once every gigabyte.

Would it help to have the byte-quota information timestamped, so that  
the client
can make predictions based on it?

In addition, I feel that the slow quota updates of current providers  
have a lot to do
with the current scarcity of methods to provide accurate and timely  
quota information.

---

From: Ade Lovett <[email-masked]>
Date: Thu, 22 Feb 2007 20:05:46 -0800

On Feb 22, 2007, at 18:35 , Samuel Kleiner wrote:
> Would it help to have the byte-quota information timestamped, so  
> that the client
> In addition, I feel that the slow quota updates of current  
> providers have a lot to do

That's not the issue at all.  It's a question (as so much of what  
makes large-scale Usenet provisioning hard) of scale.

Take a simple situation.  Two client threads, connected to  
news.example.com, which happens to be an L4 loadbalancer, and the  
threads actually end up on completely distinct machines.

Getting the state information in an accurate manner to a potential  
request from either thread A or thread B, which has to take into  
account in-progress transactions from the other, requires a  
considerable amount of extra state information to be thrown around  
between the frontend servers and the accounting backend.

Further, since caching of such information is likely to be  
undesirable, every request for quota information received by a  
frontend has to be sent off to the accounting backends, the records  
looked up, and returned, thus also driving transaction rates up.

That's just for a single client, with two threads.  Now scale the  
situation up to that of a decently-sized commercial Usenet  
outsourcer, with tens, possibly hundreds of thousands of active  
connections, with 1-N threads for a specific identifiable customer,  
and that small amount of extra state information being passed around  
rapidly overwhelms the systems.

However, the real deal breaker as I see it is that such an NNTP-based  
quota mechanism is only going to work for folks with direct  
accounts.  Those that get access to Usenet via a corporate outsourced  
agreement with an NSP are not going to have that data available to  
them.  Conveniently ignoring privacy issues for a second, think about  
how it would work where the method of authentication is purely IP- 
based, where the ISP is handing out DHCP addresses.  With only the IP  
address to go on, how are quotas handled by the NSP?  Does the ISP  
(who would be able to map an IP address to a specific customer based  
on DHCP lease) have to put in a suitable proxy in order to provide  
this information?

-aDe

---

From: [email-masked] (Peter Robinson)
Date: Sat, 24 Feb 2007 17:37:47 +0000

Samuel Kleiner <[email-masked]> wrote:

> >> I guess the main question would be what kind of "server information"
> > Max concurrent connections perhaps?

> That's already in this proposal.

So it is.

> I felt that it met the standard of a user quota.

And it does.  But it is something that could also be of interest to
non-binary clients and people not bound by specific usage quotas.
Having said that, I'm not certain how many clients or servers would
bother implementing this 'server information' extension (with or without
max connections) unless it had something more compelling in it.

I can see an application of a timeout parameter, but surely making use
of a max atricle size limit would be very unwise: just because one
server accepts 10MB articles doesn't mean they're going to propagate
elsewhere does it?

I'm not hugely interested in binary/commercial news, but I can see how
your original quota extension could be useful if it's something big
servers can implement reasonably cheaply.

Regards,

Peter

---

From: Samuel Kleiner <[email-masked]>
Date: Sun, 25 Feb 2007 23:41:02 +0100

On Feb 23, 2007, at 5:05 AM, Ade Lovett wrote:

(I've added this objection to draft 01, I hope you don't mind)

> However, the real deal breaker as I see it is that such an NNTP- 
> based quota mechanism is only going to work for folks with direct  
> accounts.

If the ISP really wanted to offer this extension and didn't want to  
require NNTP authentication(which would be the best way), I suppose  
the ISP and NSP could concievably WAN in the outsourced NSP server  
inside the ISP DHCP local address space.

I would think that the real problem with outsourced customers would  
be that the NSP has to be told about each user's quota renewal time.

> Those that get access to Usenet via a corporate outsourced  
> them.

But if outsourced customers have individual news quotas (as opposed  
to total bytes used), it is not the ISP that enforces them, but the  
NSP, right? As long as outsourced customers have to log in, the quota  
data should be available to the NSP.

It is suboptimal for the ISP to be the quota enforcer for an  
outsourcing agreement, because they only know when someone is  
communicating with the NSP when it is done over their network, and  
not when the user is traveling.

> Conveniently ignoring privacy issues for a second, think about how  
> it would work where the method of authentication is purely IP- 
> based, where the ISP is handing out DHCP addresses.
> With only the IP address to go on, how are quotas handled by the NSP?

> Does the ISP (who would be able to map an IP address to a specific  
> customer based on DHCP lease) have to put in a suitable proxy in  
> order to provide this information?

---

From: Ade Lovett <[email-masked]>
Date: Mon, 26 Feb 2007 03:26:54 -0800

> (I've added this objection to draft 01, I hope you don't mind)

Not at all.  It's in the best interests of everyone to develop  
procedures that ultimately have a positive effect on the protocol(s)  
we're discussing.

> inside the ISP DHCP local address space.

Unfortunately, that doesn't solve the problem (see below).

> NSP, right? As long as outsourced customers have to log in, the  
> quota data should be available to the NSP.

Bingo.  Here's the issue.  Corporate outsourced accounts are  
generally sold on one of three pricing models:

(a)  X number of total ports at Y KBps per port.  cost is determined  
by 'X' and 'Y'
(b)  $A per GB.  NSP simply takes total amount of data downloaded at  
the end of the billing period, multiples by $A, and presents bill.
(c)  $N per Mbps (usually on the 95th-percentile, ie: standard IP  
transit billing mechanisms).  An extension of (b) above, as far as  
billing goes it's the same.

In none of these cases does the NSP care (nor should it) about the  
ISP customer base.  It is entirely up to the ISP to set port limits  
(in the case of (a)) along with add-ons such as RADIUS accounting  
packets being sent back at the end of each session.

> not when the user is traveling.

Not really.  Regardless of whether RADIUS *authentication* is in use  
(ie: AUTHINFO USER/PASS), we (SuperNews) can send back RADIUS  
*accounting* packets at the end of each session.  In the case where  
the outsourcing is IP-authentication based, we simply drop in the IP  
address as the Framed-User-Name.  It would then be up to the ISP as  
to map that to a specific user and do whatever they need to do.

In terms of quota enforcement, the ISP *has* to be the enforcer,  
since it has the direct contract with the end-users.  The NSP simply  
has a contract with the ISP.

-aDe