Subject: [tcpm] Discussion -- Pause/Resume mechanism on TCP ?

From: Xiangsong Cui <[email-masked]>
Date: Tue, 20 Apr 2010 17:32:10 +0800

Hi folks,

I'm thinking such a question, can we, or should we add a pause/resume mechanism on TCP?

Current TCP senders understand all packet losses as indications of congestion, but this is not always true, especially in the 
scenarios where wireless link is part of the network. Compared with wire link, wireless link is less stable, with higher bit error 
rate and more disconnection. And additionally, packet error and link disconnection can not be indicated by current TCP protocol and 
extensions, do I miss anything?

So I think maybe pause/resume mechanism is helpful to this situation. The outline of pause/resume mechanism is as follows:

* Pause/Resume indication may be transmitted by the receiver or the middle-box. Like ECN on TCP, middle-box may also send 
Pause/Resume indication. The use cases may be (a) wireless host moves to a tunnel or a shadow where the radio coverage is lost, and 
later returns to the radio coverage; (b) the router in the TCP path detects the next hop is invalid (by BFD or other mechanism), and 
later detects the re-connection to the next hop; (c) the wireless host (i.e. the receiver) doesn't want to accept too many traffic. 
A scenario for this use case is a user watches TCP video by wireless device, the user/device doesn't want buffer too many video 
stream because maybe the user only watch a little begin part of the video, but the unwanted video would waste wireless bandwidth and 
user's money. So the device does receiving-buffering-pausing-resuming, following the user's schedule.

* Pause/Resume indication is directional, that means the receiver of Pause should stop transmit traffic packets but the sender of 
Pause may transmit traffic packets, unless the peer host also transmits a Pause indication.

* When the TCP sender receives the Pause indication, it should keep the cwind value, stop transmitting traffic packets, and if the 
retransmission timer is running, the sender should also stop the timer. The TCP sender should accept received incoming TCP packet 
(both traffic packet and control packet) as normal, that means the sender can adjust the receive window, mark the transmitted 
packets as acknowledged, and so on.
Another consideration is the Pause timer, the TCP sender should start the pause timer when it receives the Pause indication.

* After the TCP receiver transmits the Pause indication, it may accept the received traffic packets or discard the packets, but the 
receiver should accept TCP control packets.

* When the TCP sender receives the Resume indication, it should begin to transmit traffic packets (if there are) as normal, if there 
are transmitted packet(s) in the queue, it should also restart the retransmission timer...
If the Pause timer is running, the TCP sender should kill the timer as soon as it receives the Resume indication.

* If the TCP sender doesn't receive the Resume indication before the Pause timer expires, it should close the TCP connection.

The intention of Pause/Resume is to avoid the violent change on the TCP congestion window, does this idea make sense?
If it builds interest, I am going to write a draft, anybody who like to co-work on this topic?

Thanks and best regards

Xiangsong

---

From: Yoshifumi Nishida <[email-masked]>
Date: Tue, 20 Apr 2010 18:58:13 +0900 (JST)

Hi, My first impression is it looks like TCP needs to interact with other 
layers to know when to pasue/resume, which might be a bit tricky..
Also, is thre any reason not to use zero window advertisement?

Thanks,
--
Yoshifumi Nishida
[email-masked]

From: Xiangsong Cui <[email-masked]>
Subject: [tcpm] Discussion -- Pause/Resume mechanism on TCP ?
Date: Tue, 20 Apr 2010 17:32:10 +0800
Message-ID: <[email-masked]>

 > Current TCP senders understand all packet losses as indications of
 > congestion, but this is not always true, especially in the scenarios
 > where wireless link is part of the network. Compared with wire link,
 > wireless link is less stable, with higher bit error rate and more
 > disconnection. And additionally, packet error and link disconnection
 > can not be indicated by current TCP protocol and extensions, do I miss
 > anything?
 > 
 > 
 > So I think maybe pause/resume mechanism is helpful to this
 > situation. The outline of pause/resume mechanism is as follows:
 > 
 > * Pause/Resume indication may be transmitted by the receiver or the
 > * middle-box. Like ECN on TCP, middle-box may also send Pause/Resume
 > * indication. The use cases may be (a) wireless host moves to a tunnel
 > * or a shadow where the radio coverage is lost, and later returns to the
 > * radio coverage; (b) the router in the TCP path detects the next hop is
 > * invalid (by BFD or other mechanism), and later detects the
 > * re-connection to the next hop; (c) the wireless host (i.e. the
 > * receiver) doesn't want to accept too many traffic. A scenario for this
 > * use case is a user watches TCP video by wireless device, the
 > * user/device doesn't want buffer too many video stream because maybe
 > * the user only watch a little begin part of the video, but the unwanted
 > * video would waste wireless bandwidth and user's money. So the device
 > * does receiving-buffering-pausing-resuming, following the user's
 > * schedule.
 > 
 > * Pause/Resume indication is directional, that means the receiver of
 > * Pause should stop transmit traffic packets but the sender of Pause may
 > * transmit traffic packets, unless the peer host also transmits a Pause
 > * indication.
 > 
 > * When the TCP sender receives the Pause indication, it should keep the
 > * cwind value, stop transmitting traffic packets, and if the
 > * retransmission timer is running, the sender should also stop the
 > * timer. The TCP sender should accept received incoming TCP packet (both
 > * traffic packet and control packet) as normal, that means the sender
 > * can adjust the receive window, mark the transmitted packets as
 > * acknowledged, and so on.
 > Another consideration is the Pause timer, the TCP sender should start
 > the pause timer when it receives the Pause indication.
 > 
 > * After the TCP receiver transmits the Pause indication, it may accept
 > * the received traffic packets or discard the packets, but the receiver
 > * should accept TCP control packets.
 > 
 > * When the TCP sender receives the Resume indication, it should begin to
 > * transmit traffic packets (if there are) as normal, if there are
 > * transmitted packet(s) in the queue, it should also restart the
 > * retransmission timer...
 > If the Pause timer is running, the TCP sender should kill the timer as
 > soon as it receives the Resume indication.
 > 
 > * If the TCP sender doesn't receive the Resume indication before the
 > * Pause timer expires, it should close the TCP connection.
 > 
 > 
 > The intention of Pause/Resume is to avoid the violent change on the
 > TCP congestion window, does this idea make sense?
 > _______________________________________________
 > tcpm mailing list
 > [email-masked]
 > https://www.ietf.org/mailman/listinfo/tcpm

---

From: Lars Eggert <[email-masked]>
Date: Tue, 20 Apr 2010 13:09:18 +0300

Hi,

are you aware of the following research paper?

@inproceedings{DBLP
:conf/infocom/GoffMPG00,
  author    = {Tom Goff and
               James Moronski and
               Dhananjay S. Phatak and
               Vipul Gupta},
  title     = {Freeze-TCP: A True End-to-End TCP Enhancement Mechanism
               for Mobile Environments},
  booktitle = {INFOCOM},
  year      = {2000},
  pages     = {1537-1545},
  ee        = {http://www.ieee-infocom.org/2000/papers/501.pdf},
  bibsource = {DBLP, http://dblp.uni-trier.de}

Lars

---

From: rick jones <[email-masked]>
Date: Tue, 20 Apr 2010 07:29:45 -0700

Superficially at least, it sounds like ICMP source quench.

rick jones
there is no rest for the wicked, yet the virtuous have no pillows

---

From: "Scheffenegger, Richard" <[email-masked]>
Date: Tue, 20 Apr 2010 23:51:54 +0100

Hi,

I think it's well known (and well established), that middle-boxes which
intercept and "translate" TCP from wired to wireless give a big
performance boost; I've been involved in a PoC of such an architecture
for a cell provider some 10 years ago (when GPRS was in it's infancy).

Logically, the middlebox would terminate the tcp sessions from each end,
but appear transparent on the L3. Appropriate tunings to the tcp stack
depending on the type of connectivity were then made...

For the wired side, a zero window advertisement was used to throttle
incoming data when the wireless receiver were in a high BER area...

I would have thought, that such boxes are commercially available and
widely deployed these days. (Back then, it was an expirimental feature
of a Linux stack, I believe).

Besides, as others noted, some of the functionality described seems to
require some interaction between L4 and L1/L2; An application wishing to
do that, can already use zero window advertisements (indirectly, by not
fetching the data from the tcp receive buffer).

The only "enhancement" I see here is to now cut back cwnd - but without
an ack clock, you are asking for incast issues there...

Best regards

Richard Scheffenegger

> -----Original Message-----
> From: Xiangsong Cui [mailto:[email-masked]] 
> Sent: Dienstag, 20. April 2010 11:32
> To: [email-masked]
> Current TCP senders understand all packet losses as 
> indications of congestion, but this is not always true, 
> especially in the scenarios where wireless link is part of 
> the network. Compared with wire link, wireless link is less 
> stable, with higher bit error rate and more disconnection. 
> And additionally, packet error and link disconnection can not 
> be indicated by current TCP protocol and extensions, do I 
> miss anything?
> 
> 
> * Pause/Resume indication may be transmitted by the receiver 
> or the middle-box. Like ECN on TCP, middle-box may also send 
> Pause/Resume indication. The use cases may be (a) wireless 
> host moves to a tunnel or a shadow where the radio coverage 
> is lost, and later returns to the radio coverage; (b) the 
> router in the TCP path detects the next hop is invalid (by 
> BFD or other mechanism), and later detects the re-connection 
> to the next hop; (c) the wireless host (i.e. the receiver) 
> doesn't want to accept too many traffic. 
> A scenario for this use case is a user watches TCP video by 
> wireless device, the user/device doesn't want buffer too many 
> video stream because maybe the user only watch a little begin 
> part of the video, but the unwanted video would waste 
> wireless bandwidth and user's money. So the device does 
> receiving-buffering-pausing-resuming, following the user's schedule.
> 
> * Pause/Resume indication is directional, that means the 
> receiver of Pause should stop transmit traffic packets but 
> the sender of Pause may transmit traffic packets, unless the 
> peer host also transmits a Pause indication.
> 
> * When the TCP sender receives the Pause indication, it 
> should keep the cwind value, stop transmitting traffic 
> packets, and if the retransmission timer is running, the 
> sender should also stop the timer. The TCP sender should 
> accept received incoming TCP packet (both traffic packet and 
> control packet) as normal, that means the sender can adjust 
> the receive window, mark the transmitted packets as 
> acknowledged, and so on.
> Another consideration is the Pause timer, the TCP sender 
> should start the pause timer when it receives the Pause indication.
> 
> * After the TCP receiver transmits the Pause indication, it 
> may accept the received traffic packets or discard the 
> packets, but the receiver should accept TCP control packets.
> 
> * When the TCP sender receives the Resume indication, it 
> should begin to transmit traffic packets (if there are) as 
> normal, if there are transmitted packet(s) in the queue, it 
> should also restart the retransmission timer...
> If the Pause timer is running, the TCP sender should kill the 
> timer as soon as it receives the Resume indication.
> 
> * If the TCP sender doesn't receive the Resume indication 
> before the Pause timer expires, it should close the TCP connection.
> 
> 
> The intention of Pause/Resume is to avoid the violent change 
> on the TCP congestion window, does this idea make sense?

---

From: Xiangsong Cui <[email-masked]>
Date: Wed, 21 Apr 2010 09:31:37 +0800

Hi Yoshifumi,

Thank you for your response!
One reason is that zero window advertisement can not stop the sender's RTX timer, in my understand, the zero window advertisement 
can only tell the sender not to send new traffic, but the sender will still keep the timer running, and when the timer expires, the 
cwind is reduced, and some packets are retransmitted unnecessary. When the receiver advertises the new receive window, maybe the 
sender has entered the slow start procedure, is that right?

Regards, Xiangsong

---

From: Xiangsong Cui <[email-masked]>
Date: Wed, 21 Apr 2010 09:35:48 +0800

Hi Lars,

Thank you very much for your reminder!

I didn't know this paper until now, and I have downloaded the paper and will read it carefully.

Regards, Xiangsong

----- Original Message ----- 
From: "Lars Eggert" <[email-masked]>
To: "Xiangsong Cui" <[email-masked]>
Cc: <[email-masked]>
Sent: Tuesday, April 20, 2010 6:09 PM
Subject: Re: [tcpm] Discussion -- Pause/Resume mechanism on TCP ?

Hi,

@inproceedings{DBLP
:conf/infocom/GoffMPG00,
  author    = {Tom Goff and
               James Moronski and
               Dhananjay S. Phatak and
               Vipul Gupta},

Lars

---

From: Xiangsong Cui <[email-masked]>
Date: Wed, 21 Apr 2010 09:38:58 +0800

Hi Rick,

Yes, they are similar, maybe the only difference is that the functionality is moved to TCP layer, affecting the TCP congestion 
control.

Regards
Xiangsong

---

From: Xiangsong Cui <[email-masked]>
Date: Wed, 21 Apr 2010 10:00:05 +0800

Hi Richard,

Thank you very much for your detailed explanation!

I found PILC, a concluded working group, ever studied on this area, and outputted many documents, but it seems that the RFCs are not 
used widely. maybe one reason is that the mechanism destroys the end-to-end principle of internet? I am wondering at this fact. 
Could you please tell me more information?

Another issue I am concerning is the buffer in the middle box. If there is a TCP proxy in the border and it splits the TCP 
connection into two parts, it has to buffer the TCP traffic? I am not sure about the influence.

Regards, Xiangsong

----- Original Message ----- 
From: "Scheffenegger, Richard" <[email-masked]>
To: "Xiangsong Cui" <[email-masked]>; <[email-masked]>
Sent: Wednesday, April 21, 2010 6:51 AM
Subject: RE: [tcpm] Discussion -- Pause/Resume mechanism on TCP ?

Hi,

Best regards

Richard Scheffenegger

---

From: Xiangsong Cui <[email-masked]>
Date: Wed, 21 Apr 2010 16:34:49 +0800

Hi Lars,

Many thanks for your showing me the excellent paper!

As stated in the paper: "the main advantage is that when the sender enters persist mode, it freezes all packet retransmit timers and 
does not drop the congestion window so that the idle time during the slow-start phase can be avoided", I am also in this way.

Some details I am considering now are: who send the indication, middle-box or the receiver? when does host/middle-box send the 
indication? sometimes it is difficult to forecast the disconnection.

I also read some other papers about TCP over wireless link and I am a little confused. It seems that there are many researches and 
energy on this topic for more than ten years, but it is not clear until now which solution is recommended by IETF, or I missed the 
proposal? Since the TCP protocol is product of IETF, should IETF give some official suggestion on the applicability of TCP 
extensions? So the mobile hosts and middle-boxes may follow that...

Thanks and regards
Xiangsong

----- Original Message ----- 
From: "Lars Eggert" <[email-masked]>
To: "Xiangsong Cui" <[email-masked]>
Cc: <[email-masked]>
Sent: Tuesday, April 20, 2010 6:09 PM
Subject: Re: [tcpm] Discussion -- Pause/Resume mechanism on TCP ?

Hi,

@inproceedings{DBLP
:conf/infocom/GoffMPG00,
  author    = {Tom Goff and
               James Moronski and
               Dhananjay S. Phatak and
               Vipul Gupta},

Lars

---

From: Lars Eggert <[email-masked]>
Date: Wed, 21 Apr 2010 13:00:15 +0300

Hi,

> extensions? So the mobile hosts and middle-boxes may follow that...

you're correct in that there is no IETF recommendation in this space, although there is a large body of research work. In my view, the reason for this is mainly that none of the researchers have had the energy to seriously continue their work so that it could be published as an IETF recommendation. This is to some degree understandable, because the effort it takes may need to be spent on something that may not be academically publishable.

For example, whereas it is sufficient for a research paper to show that benefits exists in some scenarios, for an IETF recommendation, we need to be sure that a proposal has no drawbacks, or at the very least that they are well-understood and easy to work around (e.g., by not using a proposed extension in those cases).

Also, TCP is a general-purpose protocol. Optimizing its operation for a specific link technology implicitly assumes that that link technology will be used on the bottleneck link that mostly defines the path characteristics. That's a pretty strong assumption in general. Many folks have shown that TCP can be tuned in this way; not many have tried to determine whether there are drawbacks when a tuned TCP runs over a path where a different link technology is the bottleneck.

Even proposals that are agnostic to link technology have issues, e.g., Freeze TCP, which I cited since it sounded similar to your proposal. For example, there is an implicit assumption that path characteristics (available bandwidth, RTT, etc.) do not change during the freeze period - when they do, the thawed connection can significantly overshoot along the new path, or it can underperform (and will take a long time to probe for extra capacity if it is in congestion avoidance mode). And obviously, there is an assumption that you somehow know when to freeze and thaw. 

In a nutshell, I believe there is no IETF recommendation in this space, because designing one that is often useful and overall not harmful and has sufficient benefits to offset the needed effort is harder than it first appears.

Lars

---

From: Alexander Zimmermann <[email-masked]>
Date: Wed, 21 Apr 2010 14:01:05 +0200

Hi,

Am 21.04.2010 um 12:00 schrieb Lars Eggert:

> ...

Exactly!!! This IMO the reason why TCP freeze would not work (in general).

Copied from RFC 2861:

3. Description
"When a TCP sender has sufficient data available to fill the available network capacity for that flow, cwnd and ssthresh get set to appropriate values for the network conditions. When a TCP sender stops sending, the flow stops sampling the network conditions, and so the value of the congestion window may become inaccurate."

BTW: Read the introduction of this RFC. It gave me an excellent insight why TCP freeze is a bad idea...

Alex

---

From: Xiangsong Cui <[email-masked]>
Date: Thu, 22 Apr 2010 12:10:25 +0800

Hi Lars,

Thanks for your detailed explanation!

I can understand this item better now.

Regards, Xiangsong

----- Original Message ----- 
From: "Lars Eggert" <[email-masked]>
To: "Xiangsong Cui" <[email-masked]>
Cc: <[email-masked]>
Sent: Wednesday, April 21, 2010 6:00 PM
Subject: Re: [tcpm] Discussion -- Pause/Resume mechanism on TCP ?

Hi,

> extensions? So the mobile hosts and middle-boxes may follow that...

you're correct in that there is no IETF recommendation in this space, although there is a large body of research work. In my view, 
the reason for this is mainly that none of the researchers have had the energy to seriously continue their work so that it could be 
published as an IETF recommendation. This is to some degree understandable, because the effort it takes may need to be spent on 
something that may not be academically publishable.

For example, whereas it is sufficient for a research paper to show that benefits exists in some scenarios, for an IETF 
recommendation, we need to be sure that a proposal has no drawbacks, or at the very least that they are well-understood and easy to 
work around (e.g., by not using a proposed extension in those cases).

Also, TCP is a general-purpose protocol. Optimizing its operation for a specific link technology implicitly assumes that that link 
technology will be used on the bottleneck link that mostly defines the path characteristics. That's a pretty strong assumption in 
general. Many folks have shown that TCP can be tuned in this way; not many have tried to determine whether there are drawbacks when 
a tuned TCP runs over a path where a different link technology is the bottleneck.

Even proposals that are agnostic to link technology have issues, e.g., Freeze TCP, which I cited since it sounded similar to your 
proposal. For example, there is an implicit assumption that path characteristics (available bandwidth, RTT, etc.) do not change 
during the freeze period - when they do, the thawed connection can significantly overshoot along the new path, or it can 
underperform (and will take a long time to probe for extra capacity if it is in congestion avoidance mode). And obviously, there is 
an assumption that you somehow know when to freeze and thaw.

In a nutshell, I believe there is no IETF recommendation in this space, because designing one that is often useful and overall not 
harmful and has sufficient benefits to offset the needed effort is harder than it first appears.

Lars