Subject: [Tsvwg] Re: draft quick start comments part 1 of: A nit b vs B

From: Erblichs <[email-masked]>
Date: Fri, 24 Mar 2006 15:51:58 -0800

Include mail alias.

	Mitchell Erblich
	--------------------

Erblichs wrote:
> 
> Mark Allman,
> 
>         Can I suggest a simple way to get where I am going
>         with short but separate emails
> 
> 
>         A) Your 3.1 Request Option table specifies kbps.
> 
>            I do not see in your document whether the b stands
>            for bits or for bytes.
> 
>            I leanered "b" to stand for bits and "B" for bytes.
> 
>            If it is for the bits, may I suggest a third column
>            of bytes equivalent and if it is for bytes to make
>            a comment that it means bytes above the table.
> 
>         Mitchell Erblich
>         -----------------
> 
> 
> 
> 
> Mark Allman wrote:
> >
> > > > What are you asking us to say here?  Isn't the single hop case just a
> > > > subset of the bunch-of-hops case?
> > >
> > >       No. It is a special case that removes a number of issues and
> > >       introduces a number of other issues.
> >
> > I agree that it is different in a number of ways.  I guess I am not sure
> > I'd change anything in this case, however.  You sort of end up back at
> > the "how do I know there is no router between A and B?" question that
> > can't be easily answered before you could do anything differently.
> >
> > But, that's just MHO, of course.
> >
> > > > >       4) I did not see until almost halfway into the doc that
> > > > >          multihop/multipath issues are even mentioned. FYI:
> > > > >          in NON strict/loose explicit route environments like
> > > > >          TCP/IP that any later pkt/seg will traverse the same
> > > > >          path as a previous seg/pkt. Thus, reordering and load-
> > > > >          balancing may be done by a router on the same flow or
> > > > >          two flows with the same src/dst?
> > > >
> > > > I cannot parse this question.
> > >
> > >            * One of the most common issue with a TCP/IP environments is
> > >            that there is a presumption that data packets will follow
> > >            the same path. This shows up in your document at section
> > >            9.2 under the "Multiple path:" section which is halfway
> > >            into the document. I would have expected these statements
> > >            to be addressed in 1. Introduction.
> > >
> > >            * TCP/IP does not support any strict explicit route
> > >             paths for connections.
> > >
> > >            * TCP/IP does not support any loose routing paths.
> > >
> > >            * Because of the above 2 bullets, where alternate paths exist,
> > >             it is POSSIBLE based on the path metric that even when a
> > >             full path supports QS aware entities.
> > >               - that the QS aware path won't be used.
> > >               - that the QS path will be used in connection establishment,
> > >                 and then switched to a different path,
> > >               - will be load balanced between the two paths,
> > >
> > >       ---->  * Thus, IMO, if a router if recieving a QS request and has
> > >              alternate paths where the next hop could be a QS aware
> > >              entity, that path should be preferred for the flow and
> > >              if possible pinned to that path.
> >
> > I agree with that last statement.  But, I am not sure how to get there.
> > We could certainly add some complication so that routers knew about
> > their downstream's capability's.  But, I am not sure that is warrented.
> > And, in practice, studies show that routes are pretty stable over time.
> > So, yeah, bad things can happen.  But, probably not with high frequency.
> >
> > > > >       5) I think their are some implicit references to items
> > > > >          that are identified as "cross-traffic". In my experience
> > > > >          if cross-traffic are UDP based then significant mention
> > > > >          needs to be made of them. FYI: UDP is a non self
> > > > >          regulated flow that is either application limited,
> > > > >          controlled by queue drops, etc.
> > > >
> > > > I don't follow what you want us to do here.  For sure, competing against
> > > > UDP is different from competing against TCP.  But, the document is
> > > > pretty generic on this point because there is no good way to know what a
> > > > flow is competing with.
> > >
> > >               yes, however...
> > >               - You identify "cross-traffic" but make no mention of what
> > >                 it is or could be,
> > >                 a) Is it undocumented/unrequested non QS flows?
> > >                 b) are they older QS flows that are no longer tracked because
> > >                    1 RTT has been sent and recived?
> > >                 c) are they flows entering one slower bandwidth router interface
> > >                    and exiting a faster interface,
> > >                 d) etc..
> >
> > Yes, to all that.
> >
> > >               - UDP is not window / ack flow controlled so no congestion avoidance
> > >                 method other than drop is available,
> >
> > Yes.
> >
> > >               - C. Possible Router Algorithm seems to ignore the unknown/
> > >                    uncontrolled flows. It is hidden in the available
> > > bandwidth
> > >                    calc.
> >
> > Yes.
> >
> > >                - The above algorithm assumes that a previous bandwidth used
> > >                  at a preceeding interval is identifies the expected flow during
> > >                  the next interval. I disagree. WHY?
> > >                       - some non QS TCP flows are in Quick-start and are
> > >                         doubling their inflight segments per RTT.
> > >                       - some UDP flows will be sent at application controlled
> > >                         wirespeed limit
> > >                       - some flows have been idle and will  resume close
> > >                         to their previous flow capacity
> > >                       - some flows will go idle.
> > >                       - Thus the accuracy is partially dependent on the number
> > >                         of flows and the variability of each flow.
> > >                       - some link medias are bursty in nature and may avg
> > >                         at a value that is no way representive of their
> > >                         bandwidths at different time intervals..
> >
> > Yep - all of these things could happen.  If you hit Sally's quick-start
> > web page you'll find a paper that Sally, Pasi and I have worked on.  In
> > there you'll see some exploration of router algorithms.
> >
> > >                - 8.2 Deciding the Permitted Rate Request of a Router
> > >                   Potential bandwidth consists of requsted and non
> > >                   requested flows. I see no mention of non requested
> > >                   flows from TCP and/or UDP
> >
> > I don't understand why I should differentiate.
> >
> > >       ---->     - There also needs "fluff" for older QS flows that are
> > >                   still incrementing the number of inflight segments.
> >
> > We take into account how much approved QS bandwidth has been handed out.
> >
> > I think it is illustrative to pop up a level here.  Quick-Start is not
> > congestion control.  In some sense QS is anti-congestion control.  QS is
> > a quick check to see if a network is underutilized and therefore in an
> > environment where traditional conservative congestion control
> > mechanisms (e.g., slow start probing) can be relaxed to some extent
> > (which is defined by the approved rate).  Can things go wrong?  Yes.
> > Traffic surges can happen, routes can change, etc.  But, can we
> > construct something that is good enough for general use, even if it
> > hoses things up every once in a while?  Without reservations and
> > per-flow state there will always be cases that one can construct whereby
> > "it won't work".  It is quite possible that QS cannot work in the
> > general case.  I don't know.  We're working on an experimental document
> > and we hope that people (you!) go off and play with it in various
> > environments and report where it works and where it breaks down (and,
> > then, whether it can be somehow shored up in that area, or whether QS
> > just can't be used in some regime).
> >
> > >            You mention polciing in the document, however that almost
> > >            seems to be treated as a after thought as to whether it
> > >            is feasible on a wirespeed packet basis.
> >
> > Yep - that'd be part of the grand QS experiment.
> >
> > > > >       8) In a normal slow-start reducing the number of acks by a recvr
> > > > >          because segments are spaced to close together is suspended.
> > > > >          Did i miss and wouldn't QS want to follow the same functionality
> > > > >          until 1 full set of pkts have been ack'ed? Should the recv'r
> > > > >          do it on a byte count, pkt count, or a RTT timer?
> > > >
> > > > I don't follow this.  I think we have different models of how a
> > > > receiver sends ACKs.  The standards call for receivers to send an
> > > > ACK for every second segment that arrives (RFC 2581).  We do not
> > > > change this at all.  So, I don't follow your question.
> > > >
> > >               woooooooow, check with your people
> > >               The common consensus is that in slow-start a single
> > >               ack per segment is done to allow for lost acks and
> > >               to decrease the time until ss threshold is reached.
> > >
> > >               thus the delayed ack is suspended during slow-start.
> > >
> > >               -----> Thus, if ssthresh is reached via QS, then delayed
> > >               acks can resume, else they should be suspended until
> > >               reached.
> >
> > A couple of things...
> >
> >   * I understand why one would want to send more ACKs during slow start.
> >     I wrote a paper about *exactly* that (and a bunch of other
> >     strategies for increasing the speed of the slow start rampup) a long
> >     time ago.  I think it's better if the extra ACKs are not used and
> >     the sender just uses more intelligent cwnd increase, but ...
> >
> >   * I don't need to check with my people.  I study network traffic, on
> >     occasion.  I see some of what you're talking about, but it is
> >     certainly not the behavior of every or even most stacks in my
> >     estimation (from looking at actual live, production traffic).  It is
> >     fairly normal, from my observations, for the first few packets to be
> >     ACKed without delay.
> >
> >   * As a practical matter I have no idea what would / should be put in
> >     the Quick-Start draft.  If folks want to ACK every segment during
> >     slow start then fine (that is not even necessarily against the
> >     standard).  But, they are using some sort of non-standard heuristic
> >     for the receiver to determine when the sender is in slow start.  So,
> >     how are we to address this in a specification?  Put another way,
> >     some idiot changes linux to do some new and different stupid trick
> >     about 3 times a week - what impact should these have on the
> >     standards?  (Of course, that is a huge rat-hole of a question.)
> >
> > > This test shows how TCP with the delayed ACK property cannot get three
> > > duplicate ACKs and has to wait for the retransmission timeout when a
> > > packet is lost (generic Tahoe with delayed ack). Expected output:
> >
> > The situation you played has nothing to do with delayed ACKs as far as I
> > can tell.  I assume that it says "delayed ACKs" at the top for
> > completeness and not to *blame* delayed ACKs for the failures, as you
> > are doing.  That scenario is just the normal very-tiny-cwnd case.  We
> > have tried to address this with Limited Transmit to some degree.  But,
> > basically, it's a crappy network.  If the network can only support 2
> > segments in flight before it starts dropping things then it doesn't make
> > much sense to me to talk about "performance" and it sure doesn't make
> > any sense to talk about something like QS.  And, if you can only get 2
> > segments into the network before loss happens then it doesn't matter how
> > often you ACK.  You can ACK once per packet or once for every second
> > packet and the results are going to be the same.  There is no way to
> > ever get 3 ACKs into the network from 2 data segments unless you
> > introduced some new rule whereby you could emit multiple ACKs per packet
> > arrival!
> >
> > Finally, as far as I can tell it doesn't matter alot whether the QS
> > packets are ACKed immediately or every second segment is ACKed.  If one
> > is using QS then there ought to be enough packets in the network such
> > that ACK loss is not a huge issue (i.e., we'll get our three dups).  If
> > the QS packets do cause loss then ACKing every segment will end up
> > making that problem worse before it is detected than ACKing every second
> > packet.  But, that is at best a second order effect, I think.  That is,
> > I think the first order effect is that we started with a window that was
> > too big for some reason (because QS failed).
> >
> > allman
> >
> >   ------------------------------------------------------------------------
> >    Part 1.2Type: application/pgp-signature

_______________________________________________
tsvwg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/tsvwg

---

From: Erblichs <[email-masked]>
Date: Fri, 24 Mar 2006 16:02:09 -0800

Mark Allman,

        Can I suggest a simple way to get where I am going
        with short but separate emails?

	No small N values:

	B) 3.1 Rate request table.

	I can understand the N = 0 value, but the other
	N=1, N = 2, .... lower values IMO are redundant
	to normal slow start.

	Slow start gives us 4 MTU sized segments which is
	equal to 1,500bytes per normal IP MTU * 4MTU segs * 8 bits/byte
	equals 48k bits.

	If the lower values are not equvalent to slow start
	for wireless or any other type of connection, then why? What 
	do we gain, with small N values vs TCP slow start?

	Mitchell Erblich
	-------------------

---

From: Erblichs <[email-masked]>
Date: Fri, 24 Mar 2006 16:17:34 -0800

Mark Allman, et al,

	  Compressed N values: 

         C) 3.1 Rate request table.

	This request rate table has a major implied over-subscription
	problem as N increases. Why?

	If a rate request of 700,000 kbps is needed a requester
	is forced to specify a N= 15 value. This N value forces
	a waste of approx 610kbps during QS. It is then wasting
	almost 45% of the needed bandwidth.

	In theory, IMO the N values should increase, but significantly
	less dramaticly. If my B suggestion of removing the lower
	N values, then the rate request table could look something
	like this. These numbers were just thown out to "TRY" to
	allow a rate request and minimize the amount of waste.

	7 : 127k
	8 : 163k
	9 : 240k
	10 : 327k
	11 : 455k
	12 : 655
	13 : 960
	14 : 1,1k
	15 : 1,3k

	Mitchell Erblich
	-----------------

---

From: Erblichs <[email-masked]>
Date: Fri, 24 Mar 2006 17:04:33 -0800

Mark Allman, et al,

	Sums of Requested rate

	D) If you accept that requestors are forced to choose
	the best N value (I will cover A.2 Exact request rate
	later), then we must acknowledge that the sums of the
	requested rates must be greater than the amount of
	available bandwidth to fully utilize the link's
	capacity.

	This somewhat clarifies why the 2. Assumptions ...
	should take into account that the "approve a request
	for a higher sending rate if the output link is
	underutilized".

	My suggested added principle:

	A QS aware router should not deny a request based just
	on the current sums of the requested rates.

	Mitchell Erblich
	----------------------

---

From: Erblichs <[email-masked]>
Date: Fri, 24 Mar 2006 17:19:38 -0800

Mark Allman, et al,

	Priority bit

	E) 3.1 Requested rate N field uses 4 of the 8 available
	bits. The issue is that requestors requests are handled 
	on a first-come-first-serve basis. This may prevent later
	important sessions from being able to use QS.

	However, it is also common that the last percentage of
	available resources are normally consumable by
	special requestors.

	To resolve both issues, I suggest that at least a
	single bit be allocated from the rate request field,
	called the priority bit.

	Priority requests SHOULD be accmpanied by small N
	values and is meant as a hint that this session
	establishment has a significant level of importance.

	The amount of last available space reserved for priority
	requestors should be a router implementation issue.

	A router can ignore this priority bit and deal with the
	request as a normal QS rate request.

	Mitchell Erblich
	----------------

---

From: Erblichs <[email-masked]>
Date: Fri, 24 Mar 2006 18:14:34 -0800

Mark Allman, et al,

	Temporary bit.

	F) The assumption with QS is that a QS flow's
	capacity will be be moved from the QS requested
	flow to a the aggregated flow in the next
	time inteval.

	However, a number of flows may have a different
	set of characteristics, 

	Some flows need a short burst of data,
	   then can go quiet for a period of time
	   or will only need to trickle data after
	the burst. The result is that the capacity
	of the link will be under-utilized.

	To maximize the capacity usage of the link,
	3.1 specifies unused bits within the Requested
	rate field. My suggestion is to allocate 1 bit
	to specify that the flow is only requested for
	1 interval period and should not be counted in
	the next time period.

	Thus, F and E allocate two unused bits and
	still allows the other 2 unused bits for
	other means.

	This also means that at a maximum of 6 bits would
	be available and might eliminate A.2's exact rate
	requests from being used as an alternative.

	this concludes the set of 3.1 rate request suggested
	changes.

	Mitchell Erblich
	------------------