Subject: [Tsvwg] Re: draft-ietf-tsvwg-quickstart-00: Consideration of QS NONCE.

From: Gorry Fairhurst <[email-masked]>
Date: Thu, 01 Sep 2005 21:24:03 +0100

I read Mark's and Sally's comments on the NONCE question, and have only a
little to add to this:

1) I think the case of Receivers guessing the requested rate *IS* an issue.
- Whether they would use this to their advantage to cheat is a moot point.
- I'm not sure experiments in the general Internet would tell you that,
either, determining the importance is a design decision that needs to be
made.
---

2) I am still unsettled by the idea that the Sender can pick an arbitrary
large request rate, and benefit from this. This simple method does not
provide an incentive for choosing a "realistic" rate - this doesn't seem
stable in arbitrary topologies.
---

3) At the end of Mark's message, he said he thought that to make "Gorry's
method" workable would "need" routers to maintain flow-state. I am not sure
either way.

I agree routers would benefit from keeping state (especially if all their QS
capacity is assigned).

Without per-flow state, some QS capacity may be unused for cases where there
are repeated probes. I can see this is an issue - I don't know how much of a
performance hit.  I suggest this could perhaps be mitigated by the incentive
for senders to pick a realistic request rate, end-hosts could be rewarded
for being more conservative (but that may need more sender machinery).

---

I think we do need to address this. It would be nice to report I had done a
stack of simulations and could report them - alas I have not, some further
work needs to be done to understand this.

Gorry

_______________________________________________
tsvwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tsvwg

---

From: Mark Allman <[email-masked]>
Date: Fri, 02 Sep 2005 13:15:17 -0400

> provide an incentive for choosing a "realistic" rate - this doesn't
> seem stable in arbitrary topologies.

I guess I don't follow the stability thought.  Can you sketch a bit
more? 

> I agree routers would benefit from keeping state (especially if all
> their QS capacity is assigned).
> 
> Without per-flow state, some QS capacity may be unused for cases where
> there are repeated probes. I can see this is an issue - I don't know
> how much of a performance hit.  I suggest this could perhaps be
> mitigated by the incentive for senders to pick a realistic request
> rate, end-hosts could be rewarded for being more conservative (but
> that may need more sender machinery).

The problem with this is that asking senders to pick a "realistic
request rate" is either quite hard or impossible, in general.  We have
these words in the draft:

    "Determining the Rate to Request"

    As discussed in [SAF05], the data sender does not necessarily have
    information about the size of the data transfer at connection
    initiation; for example, in request-response protocols such as HTTP,
    the server doesn't know the size or name of the requested object
    during connection initiation.  [SAF05] explores some of the
    performance implications of overly-large Quick-Start requests, and
    discusses heuristics that end-nodes could use to size their requests
    appropriately.  For example, the sender might have information about
    the bandwidth of the last-mile hop, the size of the local socket
    buffer, or of the TCP receive window, and could use this information
    in determining the rate to request.  Web servers that mostly have
    small objects to transfer might decide not to use Quick-Start at
    all, since Quick-Start would be of little benefit to them.

    In the absence of other information, there could be a configured
    value for the Quick-Start Rate Request.  Quick-Start will be more
    effective if Quick-Start requests are not larger than necessary;
    every Quick-Start request that is approved but not used (or not
    fully used) takes away from the bandwidth pool available for
    granting successive Quick-Start requests.  Therefore, it is
    recommended that the request for the initial sending rate be
    somewhat conservative, in order to improve the chances for more
    Quick-Start requests to be approved.

There are more words in [SAF05].  

I agree that we'd like to see reasonable rate requests - based on a host
or app's ability to use the bandwidth - and not always "as much as
possible".  And, it might be nice to somehow incentivize this behavior,
I suppose.  But, it's really quite a hard problem to put on the end
systems.

> I think we do need to address this. It would be nice to report I had
> done a stack of simulations and could report them - alas I have not,
> some further work needs to be done to understand this.

For my own clarity, do you think this needs done before this document
can be experimental?  Or, as part of the experimental period?  Sally and
I chatted earlier in the week and we were fine with noting that stacks
could use "Gorry's scheme" if they wanted (a MAY) and also the pros and
cons of this approach (which I think we all mostly agree on).  Would
that approach work in your mind?

Thanks!

allman

---

From: Gorry Fairhurst <[email-masked]>
Date: Sat, 03 Sep 2005 17:18:47 +0100

> I guess I don't follow the stability thought.  Can you sketch a bit

It seems safe: It doesn't seem to lead to congestion collapse, and no sender
performs worse than without QS.

I think the capacity a sender gets could vary wildly and unpredictably: This
arises because the rate request is also serving as a soft reservation to the
routers.  The scheme allows any sender to start with a maximum value, and
then wait to see what they actually get. A "greedy" sender strategy is
rewarded by maximum QS capacity available for that particular sender, but
could give poor allocation of QS resources to senders who are unable to
utilise the QS capacity. I've made a very simple example at the end.

My concern did not arise from a specific scenario, just that it seemed the
incentive was wrong.

> I suppose.  But, it's really quite a hard problem to put on the end
That's the section. I agree with all that is written!

> can be experimental?  Or, as part of the experimental period?

Aha - that's a good question. Perhaps we should ask:
"What is the experiment which the RFC method is supposed to enable?"

(1) That the senders can benefit from QS in real environments? That QS does
not predjudice performance of non-QS flows? That QS extensions may be passed
by routers? To find interactions with other IP mechanisms?

(2) Or that QS does not open itself to abuse, sub-optimal usage, and is a
"good" / "best" current method we can provide for accelerating performance
of senders (e.g. that distrust midboxes, receivers, routers) ?

So, it would be good to check the former first. I would support early
publishing of an experimental spec. to get such experience. Especially if it
encouraged experimention to help determine the
vulnerabilities/countermeasures for (2).

This thread seems to me to be only discussing (2). Limited practical
deployment may help understand this, but this alone may not provide great
insight into these aspects - more thinking would help.  IMHO, the case for a
nonce and applicable method needs to be determined before this becomes an
IETF standard, but I agree it is hard to mandate this given CURRENT level of
experience. Does anyone else have other inputs?

> Sally and
Indeed - With more wisdom, tsvwg can decide what finally to put forward as
one "standard" method:-)

> allman

Best wishes,

Gorry

----- A simple example of unstable QS capacity with over-reservation ---

I don't have a particular topology in mind, but we could think about:

A ---> R0 ---> R1 ---> R2 ---> R3 ---> D
                |       |
B ---> R4 ---> R5       C

Assuming all routers have equal QS capacity, except R3 which has limited
capacity. 

    A sends to D. It requests the maximum. R0, R1, R2 allocate their
    own max, and R3 allocates a lower value, which is R3's maximum
    << other routers. R3 controls A's sending rate. R0, R1, R2
    have been over-allocated.

    B sends to C. It requests the maximum. R4 & R5 allocate their
    own max. R1 & R2 have fully allocated their QS capacity
    (by A's QS request) and now have little of no QS capacity left to
    give to B. - Although A's usage will actually be constrained
    by R3's rate. I call this greedy.

Instead, if B were to go first, B would get the maximum QS capacity on the
path to C. A would see reduced QS capacity (that actually corresponds to B's
usage), and would get very little capacity. In this case B->C would actually
be able to utilise the QS capacity that is available. I think this is fine.

If we had more senders, receivers and a more complex topology, what would
happen?  - could there be more complex cases were the awarded QS capacity
varies significantly, because it is consumed by greedy requests with little
actual usage?

---

From: Mark Allman <[email-masked]>
Date: Tue, 06 Sep 2005 16:50:31 -0400

> I think the capacity a sender gets could vary wildly and
> unpredictably: This arises because the rate request is also serving as
> a soft reservation to the routers.  

Yes.  See the "threats" section of [SAF05].  In there we discuss just
the scenario you painted in your email.  I hope we also noted this in
the i-d.

You are quite correct that QS rate-requests can get "lucky" or
"unlucky".

It seems to me that how exactly to deal with that situation in the
routers is an open area for work and could be considered part of the
experiment.

> The scheme allows any sender to start with a maximum value, and then
> wait to see what they actually get. A "greedy" sender strategy is
> rewarded by maximum QS capacity available for that particular sender,
> but could give poor allocation of QS resources to senders who are
> unable to utilise the QS capacity. I've made a very simple example at
> the end.
> 
> My concern did not arise from a specific scenario, just that it seemed
> the incentive was wrong.

Basically you are outlining a fundamental limitation of Quick-Start.
Let's just admit it.  Routers have no idea what happens downstream.
What that basically means is that there is no way for some router to
know whether (a) a particular request is simply being unnecessarily
greedy, (b) whether it can legitimately use the capacity it seeks, (c)
the request is simply a spoofed segment to slurp the soft reservation,
etc.  A router could certainly keep history or some such and try to
figure out what portion of a request a given host generally uses or
something (e.g., see the Extreme QS Router in [SAF05]).  But,
essentially an "egregiously greedy request" has the same impact on the
soft state as a "big legitimate request".  So, what to do?

There is a thin incentive in that if everyone is requesting massive
capacity they will never use then it all becomes a game of hit and
miss.  You might get lucky, but probably you'll end up with little or no
advantage for using Quick Start.  So, maybe it is just like congestion
control in that we all gain advantage by all being reasonable.  I
dunno. 

> > For my own clarity, do you think this needs done before this
> > document can be experimental?  Or, as part of the experimental
> > period?
> 
> Aha - that's a good question. Perhaps we should ask: "What is the
> experiment which the RFC method is supposed to enable?"
> 
> not predjudice performance of non-QS flows? That QS extensions may be
> passed by routers? To find interactions with other IP mechanisms?
> 
> publishing of an experimental spec. to get such experience. Especially
> if it encouraged experimention to help determine the
> deployment may help understand this, but this alone may not provide
> great insight into these aspects - more thinking would help.  IMHO,
> the case for a nonce and applicable method needs to be determined
> before this becomes an IETF standard, but I agree it is hard to
> mandate this given CURRENT level of experience. Does anyone else have
> other inputs?

Likewise I would love to hear more input.

Thanks!

allman