Subject: Re: [p2pi] Follow-Up from Comcast Presentation

From: "Livingood, Jason" <[email-masked]>
Date: Mon, 9 Jun 2008 21:02:23 -0400

Inline below - sorry it took awhile, but it was a long email that
touched on many things.
Jason

________________________________

	From: Robb Topolski [mailto:[email-masked]] 
	
	In particular, I'm concerned with possible scenarios such as
someone who is mid-upload of 1100 European vacation pictures (all at 8
Megapixel) when her husband has a medical emergency.  She dials 9-1-1 on
Vonage or whatever non-Comcast VOIP device. Comcast delays and drops her
packets as the new system places her VOIP call is behind long packet
queues created by other activity in her neighborhood.   
	 
	First, I would say that it seems that this example user should
configure their home gateway router to allocate a specific amount of LAN
bandwidth for their VoIP service, so that their VoIP service would not
be overwhelmed by and negatively affected by the upload of several GB of
digital photos.  Many VoIP services instruct their customers to install
their device directly to the broadband connection, and then connect
their router behind this device.  That device then makes decisions about
traffic flowing from the customer's network and would ensure that the
customer's VoIP service would not be negatively affected by any other
other applications.   

	Secondly, without any form of network management (current or
future), the heavy use of  whatever apps by a few people at times of
congestion would interfere with the same VoIP call.

	Nevertheless,  in terms of when the bits get to our network, I
can tell you that we have asked our engineers and our vendors to address
this concern, which has been foremost in our minds. 

	 <snip>... many consumer gateway-router devices marketed under
different brands use a QoS engine by Ubicom.  These devices determine
the upload stream size either as manually configured by the user or
measured automatically when the router reboots.  These devices will not
function properly if the bandwidth it believes it is reserving is
suddenly made unavailable by the network.  In summary, best-effort
traffic is the baseline traffic expectation and developers of Internet
products do not expect a situation where a network operator has invented
a "worse than best-effort" traffic class -- nor should they. 
	 
	As we stated up at MIT, our trial is NOT applying rate limits to
the subscribers.  So for example, we're NOT taking a 16Mbps/2Mbps
customer down to 1Mbps/512Kbps.  The user's bits we are talking about
are simply assigned a lower QoS for some period of time, assuming (1)
that the network is in a near-congestion state and (2) the user has
crossed a specific threshold of usage for a specific period of time
(both conditions must be met).  
	 
	But perhaps more importantly, devices that "assume" that the
network's bandwidth is a fixed constant are flawed devices.  First,
congestion can occur anywhere along the path through the Internet,
including parts beyond the ISP network and therefore outside of the
ISP's control.  Second, bandwidth can fluctuate for "good" reasons too
(ref: Powerboost).  How would such devices or software adapt to
something like a wireless network, where bandwidth is probably the most
variable due to the user moving in relation to base stations, etc.?  It
would seem that devices and software should, within reason, be able to
adapt to normal ups and downs of bandwidth to a user or device.
	 
	The users are harmed because their network is behaving strangely
and they have no way of knowing why.   
	 
	I am not sure how you know that a user will perceive the network
as 'behaving strangely' at such a time.  It would seem reasonable that
one of the things a trial should ascertain is whether users even notice
this at all.  It is clearly one of the things we'll be studying very
closely and gathering data on.

	 Innovation is harmed because the access and transit networks
continue to behave unexpectedly.  It isn't the network operator who will
get the call about the online game or VOIP application that is behaving
poorly, it's the software publisher.   
	 
	Actually, you may be surprised to discover that the ISP is often
the first call for many of these customers.   On a related item, for
example, we recently observed complaints on our user forums, and on the
forums at BroadbandReports that a certain web video site was performing
poorly (a thread in which you and I both participated).  Despite posts
from users from other ISPs that would indicate the problem was not
limited to Comcast users, we engaged engineering resources to study the
problem, to check our network, and then contact that web site's NOC,
open a ticket with them, and assist them in troubleshooting what appears
to have been a routing problem on their network.  This is in fact a
regular and normal part of business for us, whether it is ensuring that
web video sites work well for our customers, whether they can send email
to certain domains, or their online games and VoIP services work well on
our network.   

	Comcast is sending the signal that the operators of the access
networks are going to tamp down the natural demand-based growth of the
internet is as uninteligent as a chip manufacturer saying that they're
done making faster chips.  So why would someone release a
bandwidth-hungry product into that environment?  Just as the world's
largest chip manufacturers enable innovation by trying to stay as far
ahead of developers ability to suck-up MIPS as possible, I would expect
someone serious about leading in the business of broadband should be
doing the same. 

	Unfortunately, you have made a few statements to the effect that
our investment in our network is largely static, meaning that every few
years we decide whether or not to invest in the network.  That could not
be further from the truth.    As our public financial statements show,
we invest substantial sums of money on an ongoing basis to continuously
improve all parts of our network.  When parts of the network approach
pre-set utilization numbers, a well tested process kicks in
automatically to upgrade these portions of the network.  In addition to
these normal-course-of-business investments, we have large generational
investments in things like DOCSIS 3.0.  Further, we continue to develop
and release new services that benefit our customers as well.  Our
network is a valuable asset and no rational company in our position
would choose not to continue to invest in, care for, and improve such an
asset.   I'd also prefer to stick to the tech talk here, rather than the
econ stuff.  :-) 

	Thanks for asking for feedback.  I'm sorry it is not at all
positive.  While I do recognize efforts at more transparency and user
notification as good things, the method itself fails the standards
compatibility test. 

	I am not sure I understand what standards compatability test you
are referring to.  ??   (Unless of course this is the earlier RFC you
referred to in a different email.) 

	Anyhow Robb, at this stage, I'd like to turn my attention to
working with my colleagues to conduct our trials, find the best way to
manage data flows to provide the best possible experience for all of our
users, and let them be the judge. In the meantime, we will also continue
our efforts as participants in this IETF list (and in the upcoming BoFs
and other WGs at IETF-72) and our continuing effort to have a
constructive technical dialogue with the broader community. 

	Thanks, Jason
	
	Robb Topolski 

_______________________________________________
p2pi mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/p2pi

---

From: "Robb Topolski" <[email-masked]>
Date: Mon, 9 Jun 2008 18:44:25 -0700

2008/6/9 Livingood, Jason <[email-masked]>:

> First, I would say that it seems that this example user should configure
> their home gateway router to allocate a specific amount of LAN bandwidth for
> their VoIP service, so that their VoIP service would not be overwhelmed by
> and negatively affected by the upload of several GB of digital photos.
>

Right, but such devices need to know both numbers in order to reserve it.
What's left is the unreserved bandwidth -- the total minus the reserved.

The protocol-agnostic method has the effect of lowering the total, rendering
any such settings designed to reserve bandwidth meaningless..

Secondly, without any form of network management (current or future), the
> heavy use of  whatever apps by a few people at times of congestion would
> interfere with the same VoIP call.
>
>
As I've said before, network developers expect an Internet with occasional
congestion.  An application that is not tolerant of some congestion won't
last on the Internet very long.

> As we stated up at MIT, our trial is NOT applying rate limits to the
> subscribers.  So for example, we're NOT taking a 16Mbps/2Mbps customer
> down to 1Mbps/512Kbps.  The user's bits we are talking about are simply
> assigned a lower QoS for some period of time, assuming (1) that the network
> is in a near-congestion state and (2) the user has crossed a specific
> threshold of usage for a specific period of time (both conditions must be
> met).
>
>
Which is like saying that instead of pot-holing the road, you're letting air
out of their tires.  Whether they get poor performance due to throttling or
poor performance due to dropping or delaying packets, the result is that
they get poor performance.

> But perhaps more importantly, devices that "assume" that the network's
> bandwidth is a fixed constant are flawed devices.
>
> I'm inclined to agree with you.  However, most of the devices out there
disagree with "us" -- and we should recognize that.

> Actually, you may be surprised to discover that the ISP is often the *first
> *call for many of these customers.
>
> OF COURSE it is.  Let's see, you work for an ISP.  I worked for a
networking software/hardware vendor.  From wherever I sit, the grass is
always greener "over there." :-)

Unfortunately, you have made a few statements to the effect that our
> investment in our network is largely static, meaning that every few years we
> decide whether or not to invest in the network.
>
> Please go back and read again.

When the subject has come up, I've given you credit for spending the most
money out of the list of MSOs on infrastructure growth.  I AM WORRIED that
innovations like Deep Packet Inspection will cause such investment to
flatten -- even in Comcast -- but I have not taken the position that it
stopped investment in it.

I'm a Comcast Fan, and I remain a customer.  I got DSL in a deal for my
roommate, but I choose to use Comcast. I've posted that fact quite
publically.  But a line was crossed here that ISPs shouldn't cross, and I
documented it.  It's important, doubly-so given Comcast's leadership
position in its industry.  I very much want Comcast to remain the
open-ports, low-interference ISP it has always been.

As more and more *ideas *about stuff to put in the middle of the network to
monitor and change and monetize what's happening on OUR Internet, the more
concerned I become about the future.  Because no matter how small
industry-giant Comcast decides it is willing to intrude, the farther it
pushes the bounds that poorer or less disciplined ISPs feel empowered for
their more intrusive interceptions.  Keeping Comcast's large network one
that is conforming to the standards and qualified practices helps keep
everyone's network that way.

I've been a fan.  I am a fan.  And I plan to be a fan.  But I've also been
honest and I've been clearly rooting for you to use your leadership position
for things that are good for the net.

> When parts of the network approach pre-set utilization numbers, a well
> tested process kicks in automatically to upgrade these portions of the
> network.
>
> As I said before, I do know this.  I've quoted Tony Werner in 2-year-old
articles on this very topic.  I know that you upgrade regularly -- so it's
not me that you're arguing with.

Robb

-- 
Robb Topolski ([email-masked])
Hillsboro, Oregon USA
http://www.funchords.com/
_______________________________________________
p2pi mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/p2pi