Subject: RE: Transport in 0-RTT connections for high BDP: opt-in

From: <[email-masked]>
Date: Fri, 8 Mar 2019 08:51:00 +0000

My view differs a bit.

In TLS1.3, the early_data extension field specifies the signaling for tuning the egress part of the 0-RTT traffic.

In https://tools.ietf.org/html/draft-kuhn-quic-0rtt-bdp-00, we discuss the usage of the same signaling for tuning the 0-RTT traffic sent by the server. It is a transport concern common to TCP and QUIC.

A client cannot read the content of the opaque ticket it stores on the behalf of the server. By consequence we have to define a new extension field to be sure the client opt in. The client will not use this ticket if it rejects the option.

Emile

-----Message d'origine-----
De : QUIC [mailto:[email-masked]] De la part de Martin Thomson
Envoyé : jeudi 7 mars 2019 22:26
À : [email-masked]
Objet : Re: Transport in 0-RTT connections for high BDP

As you say, this is a congestion control topic, not as much a QUIC-specific one.  And you are equally correct that this does not require standardization of any signaling to use it.

However, we have typically attempted to document congestion control strategies.  So in that spirit, it's a fine thing to do.

All that said, we explicitly decided to not address this particular question early in our process.  Saving information about congestion window from one connection so that it could be used to jump-start the next connection was a feature cut from HTTP/2 as well.

On Fri, Mar 8, 2019, at 08:04, Kazuho Oku wrote:

> Thank you for writing the draft.

> I think the approach is fine, though I wonder if we need to a

> specification for this.

> The server has the freedom to store whatever it wants in a session

> ticket. Therefore, I do not see why we need to define an extension.

> Also, it has been my understanding that token (sent by NEW_TOKEN

> frame) is a better place for storing the RTT observed in last

> connection. This is because token is a per-path information that is

> updated per each path, compared to session tickets that are expected

> to rarely be updated mid-connection.

> 2019年3月7日(木) 19:36 Kuhn Nicolas <[email-masked]>:

> > Hi,

> > We have uploaded a draft that describes a solution to improve QUIC’s performance during 0-RTT establishment in a high BDP context.

> > https://tools.ietf.org/html/draft-kuhn-quic-0rtt-bdp-00

> > Abstract

> >    0-RTT is designed to accelerate the throughput at the establishment

> >    of a connection.  There are cases where 0-RTT alone does not improve

> >    the time-to-service.

> >    This memo discusses a solution where a fundamental characteristic of

> >    the path is learned during the 1-RTT phase and shared with the 0-RTT

> >    phase to accelerate the initial throughput during subsequent 0-RTT

> >    connections.

> > Would it be possible to have a small slot to present it during the next IETF meeting?

> > Kind regards,

> > Nico & Emile

> --

> Kazuho Oku

_________________________________________________________________________________________________________________________

Ce message et ses pieces jointes peuvent contenir des informations confidentielles ou privilegiees et ne doivent donc
pas etre diffuses, exploites ou copies sans autorisation. Si vous avez recu ce message par erreur, veuillez le signaler
a l'expediteur et le detruire ainsi que les pieces jointes. Les messages electroniques etant susceptibles d'alteration,
Orange decline toute responsabilite si ce message a ete altere, deforme ou falsifie. Merci.

This message and its attachments may contain confidential or privileged information that may be protected by law;
they should not be distributed, used or copied without authorisation.
If you have received this email in error, please notify the sender and delete this message and its attachments.
As emails may be altered, Orange is not liable for messages that have been modified, changed or falsified.
Thank you.

---

From: Jana Iyengar <[email-masked]>
Date: Tue, 12 Mar 2019 20:42:00 -0700

Emile,

The client does not need to be told that the server is going to use a large
window in this connection, the server is free to choose any congestion
window without notifying the client. As has been already been noted, the
server can remember and use a large window using an opaque token, and a
token that is opaque to the client does not need standardization.

What else does the client need to know from the server here?

-  jana

On Fri, Mar 8, 2019 at 12:51 AM <[email-masked]> wrote:

> My view differs a bit.
>
>
>
> In TLS1.3, the early_data extension field specifies the signaling for
> tuning the egress part of the 0-RTT traffic.
>
> In https://tools.ietf.org/html/draft-kuhn-quic-0rtt-bdp-00, we discuss
> the usage of the same signaling for tuning the 0-RTT traffic sent by the
> server. It is a transport concern common to TCP and QUIC.
>
>
>
> A client cannot read the content of the opaque ticket it stores on the
> behalf of the server. By consequence we have to define a new extension
> field to be sure the client opt in. The client will not use this ticket if
> it rejects the option.
>
>
>
> Emile
>
>
>
> As you say, this is a congestion control topic, not as much a
> QUIC-specific one.  And you are equally correct that this does not require
> standardization of any signaling to use it.
>
>
>
> However, we have typically attempted to document congestion control
> strategies.  So in that spirit, it's a fine thing to do.
>
>
>
> All that said, we explicitly decided to not address this particular
> question early in our process.  Saving information about congestion window
> from one connection so that it could be used to jump-start the next
> connection was a feature cut from HTTP/2 as well.
>
>
>
> > > We have uploaded a draft that describes a solution to improve QUIC’s
> performance during 0-RTT establishment in a high BDP context.
>
> > >
>

---

From: Mikkel Fahnøe Jørgensen <[email-masked]>
Date: Wed, 13 Mar 2019 06:36:11 +0000

I guess the point is that the client needs to say, with a single bit, if the user is still in the environment where the high bandwidth part of the token is still applicable.

________________________________
Fra: QUIC <[email-masked]> på vegne af Jana Iyengar <[email-masked]>
Sendt: onsdag, marts 13, 2019 4:42 AM
Til: [email-masked]
Cc: [email-masked]; [email-masked]; Martin Thomson; Kazuho Oku
Emne: Re: Transport in 0-RTT connections for high BDP: opt-in

Emile,

The client does not need to be told that the server is going to use a large window in this connection, the server is free to choose any congestion window without notifying the client.. As has been already been noted, the server can remember and use a large window using an opaque token, and a token that is opaque to the client does not need standardization.

-  jana

On Fri, Mar 8, 2019 at 12:51 AM <[email-masked]<mailto:[email-masked]>> wrote:

My view differs a bit.

Emile

-----Message d'origine-----
De : QUIC [mailto:[email-masked]<mailto:[email-masked]>] De la part de Martin Thomson
Envoyé : jeudi 7 mars 2019 22:26
À : [email-masked]<mailto:[email-masked]>
Objet : Re: Transport in 0-RTT connections for high BDP

> 2019年3月7日(木) 19:36 Kuhn Nicolas <[email-masked]<mailto:[email-masked]>>:

---

From: Kazuho Oku <[email-masked]>
Date: Wed, 13 Mar 2019 16:09:58 +0900

> I guess the point is that the client needs to say, with a single bit, if the user is still in the environment where the high bandwidth part of the token is still applicable.

Assume that the token (or the session ticket) contains the CWND (and
RTT) of the previous connection.

To me it seems that the question is if having a client-sent flag that
indicates whether the client is on the same network as the one it
obtained the token helps the server determine the correct send window,
either before path validation or right after path validation.

For data sent before path validation (i.e. 0.5-RTT packets), I think
the answer is no. Because an attacker can use 0.5-RTT packets to mount
DDoS (by spoofing the source address of the Initial packet that
contains the flag and the token). A server should not take such a
signal into consideration when us a token for path validation.

For data sent after path validation, I could see the chance of the
answer becoming yes. Without such a signal, the server needs to solely
rely on the IP address (or the network address) of the client to guess
if it can reuse the CWND of the previous connection.

However the question is, how much do we gain from having that signal,
compared to just relying on the IP address? Do we have data?

-- 
Kazuho Oku

---

From: Lars Eggert <[email-masked]>
Date: Wed, 13 Mar 2019 04:59:49 -0700

Hi,

> either before path validation or right after path validation.

sure, but the server already knows this based on whether the source IP address has changed?

Lars

---

From: Kazuho Oku <[email-masked]>
Date: Wed, 13 Mar 2019 22:23:05 +0900

> sure, but the server already knows this based on whether the source IP address has changed?

Exactly. That's why I stated in my previous mail that the benefit of
the proposed approach depends on the improvement we gain from the
added signal (compared to just relying on the client's address).

-- 
Kazuho Oku

---

From: Kuhn Nicolas <[email-masked]>
Date: Wed, 13 Mar 2019 16:26:03 +0000

Hi, 

This email is just an attempt to clarify the potential gain with the proposed solution. 
In short, the client being also aware of the high BDP connection can help in further improvements of the time-to-service (and does not cost much).

---
Potential classic operation with the proposed BCP 
---
During the 1-RTT connection, the server measures MTU, RTT, BDP, etc.
This information is sent to the client in the NewSessionTicket to the client. 
The reduced ticket_lifetime  guarantee that the client has not moved (the server is also free to discard this information if the client IP address has changed). 
-> The server can adapt the CC to the measured characteristics of the network to improve the time to service

---
Added signal with the proposed BCP 
---
In addition to a classic operation, with our proposal the client is aware of being on a high BDP path.
- > The client can adapt the CC to the measured characteristics of the network to improve the time to service (e.g. ACK management (ratio, delayed, etc), better memory management, reestablishment, etc)

While I agree that more data may be required, the purpose is more to let the door open for further concrete proposals initiated by the client knowing it is on a high BDP path.

Cheers,

Nico

-----Message d'origine-----
De : Kazuho Oku <[email-masked]> 
Envoyé : mercredi 13 mars 2019 14:23
À : Lars Eggert <[email-masked]>
Cc : Mikkel Fahnøe Jørgensen <[email-masked]>; Jana Iyengar <[email-masked]>; Kuhn Nicolas <[email-masked]>; [email-masked]; Martin Thomson <[email-masked]>; [email-masked]
Objet : Re: Transport in 0-RTT connections for high BDP: opt-in

> sure, but the server already knows this based on whether the source IP address has changed?

Exactly. That's why I stated in my previous mail that the benefit of the proposed approach depends on the improvement we gain from the added signal (compared to just relying on the client's address).

--
Kazuho Oku

---

From: Lars Eggert <[email-masked]>
Date: Wed, 13 Mar 2019 09:33:05 -0700

> added signal (compared to just relying on the client's address).

Thank you for clarifying - I had misread your email.

Lars

---

From: Roberto Peon <[email-masked]>
Date: Wed, 13 Mar 2019 17:07:17 +0000

This may be obvious, but I'll lay it out anyway:

The 1st order benefit of predicting the BW to the client/server is bounded by the error between under-predicted BW and actual BW at the start of a connection. We know that this error last for a longer time with larger RTTs.

The 1st order effects on the network is either better utilization or over-utilization. Both are corrected in some amount of time. Depending on the algorithm, it can be corrected in a single RTT, or it can be corrected in something like RTT * lg(error) time.

The 2nd order effects aren't new-- folks may make more connections because the opportunity cost of spinning up a new connection is lower. We've seen from H1->H2 that applications do this more often anyway when congestion control has non-error-proportional feedback loops (e.g. is a direct function of packet-loss, rather than ratio of packet loss).

From what we saw in the SPDY-experiment era, caching of the prior state worked and showed improvements. The experiments that we did then were imperfect-- we recorded BDP (as CWND). IIRC most clients had surprisingly moderate CWNDs (< 40), and saw benefit from an increased INITCWND.  The benefit at the time was not enough to overcome fears in the H2 era, and thus it was dropped then.

Since the SPDY-era experiment failed to record CWND+RTT (or bandwidth directly), there were hypotheses that this could have been much better and that larger cwnds were probably causal of burst-losses that immediately decreased the effectiveness (at those larger cwnd start sizes). 
There are a bunch of heuristics that attempt to deal with this in various transport stuff (how much should you transmit after you're idle for X seconds? Hint: If you just use the last param you had, you'll potentially initiate a large burst and hurt yourself). In other words, this problem is endemic of long-lived bursty traffic (i.e. web and web-like traffic *including* most video right now) within any congestion control loop and not just at connection initiation.
-=R

On 3/13/19, 9:27 AM, "QUIC on behalf of Kuhn Nicolas" <[email-masked] on behalf of [email-masked]> wrote:

    Hi, 

    ---
    
    ---

    Cheers,
    
    Nico
    
    -----Message d'origine-----
    De : Kazuho Oku <[email-masked]> 
    Envoyé : mercredi 13 mars 2019 14:23
    À : Lars Eggert <[email-masked]>
    Cc : Mikkel Fahnøe Jørgensen <[email-masked]>; Jana Iyengar <[email-masked]>; Kuhn Nicolas <[email-masked]>; [email-masked]; Martin Thomson <[email-masked]>; [email-masked]
    Objet : Re: Transport in 0-RTT connections for high BDP: opt-in

    

    --
    Kazuho Oku

---

From: Kuhn Nicolas <[email-masked]>
Date: Wed, 13 Mar 2019 18:42:15 +0000

Thanks for your comment and sharing the issues in sizing the initial CWND for H1/H2.

The core element in the draft is : if the client knows that the BDP of the network is high, client-level adaptations can be envisioned. 
The draft also presents server-level adaptations that could also improve the time to service (with an increased IW and MTU as examples - following some comments we received from the list).  

So : 
1- How do we inform the client know the BDP is high ?
The actual BDP may not be relevant and difficult to measure - as you mentioned. Using the actual value or a binary based on simple heuristics (e.g. RTT value > 500ms) is still under discussion. 

2- Is increasing the IW safe ? 
As mentioned in other emails by others, server implementations decide how to restart connections. The draft indeed mentions the gains we could have in increasing the initCWND, but along with pacing this increase *should* not hurt yourself. When you send a packet once every five milliseconds, a 50ms RTT would give you an initCWND of 10, while a 500ms RTT would result in a initCWND of 100 without burst-related losses. 

Cheers, 

Nico

-----Message d'origine-----
De : Roberto Peon <[email-masked]> 
Envoyé : mercredi 13 mars 2019 18:07
À : Kuhn Nicolas <[email-masked]>; 'Kazuho Oku' <[email-masked]>; Lars Eggert <[email-masked]>
Cc : Jana Iyengar <[email-masked]>; Mikkel Fahnøe Jørgensen <[email-masked]>; [email-masked]; [email-masked]; Martin Thomson <[email-masked]>
Objet : Re: Transport in 0-RTT connections for high BDP: opt-in

    Hi, 

    ---
    
    ---

    Cheers,
    
    Nico
    
    -----Message d'origine-----
    De : Kazuho Oku <[email-masked]> 
    Envoyé : mercredi 13 mars 2019 14:23
    À : Lars Eggert <[email-masked]>
    Cc : Mikkel Fahnøe Jørgensen <[email-masked]>; Jana Iyengar <[email-masked]>; Kuhn Nicolas <[email-masked]>; [email-masked]; Martin Thomson <[email-masked]>; [email-masked]
    Objet : Re: Transport in 0-RTT connections for high BDP: opt-in

    

    --
    Kazuho Oku