Subject: Request forgery in QUIC

From: "Martin Thomson" <[email-masked]>
Date: Wed, 12 Aug 2020 17:14:03 +1000

I've just spent a good portion of my day opening a single issue.  It's a little involved, so I will just point you to the issue:

https://github.com/quicwg/base-drafts/issues/3995

I ran out of time for writing a pull request.  That will happen tomorrow.

In the mean time, the issue describes what I'm recommending is changed.  Unfortunately, my conclusion is that this requires a couple of small design tweaks, mostly just changes to recommended behaviour.  Nothing that breaks compatibility though; it's not that serious an issue.

---

From: "Martin Thomson" <[email-masked]>
Date: Thu, 13 Aug 2020 17:54:13 +1000

Me again.

https://github.com/quicwg/base-drafts/pull/3996 is a first draft of a proposal for this issue.  There are some small normative recommendations, but all of that is negotiable.

At a high level, I would like to understand whether people think the general shape of this response is reasonable.

On Wed, Aug 12, 2020, at 17:14, Martin Thomson wrote:
> I've just spent a good portion of my day opening a single issue.  It's 
> a little involved, so I will just point you to the issue:
> 
> In the mean time, the issue describes what I'm recommending is changed. 
>  Unfortunately, my conclusion is that this requires a couple of small 
> design tweaks, mostly just changes to recommended behaviour.  Nothing 
> that breaks compatibility though; it's not that serious an issue.
> 
>

---

From: Kazuho Oku <[email-masked]>
Date: Thu, 13 Aug 2020 21:37:49 +0900

Hi,

Thank you for raising the issue and the PR.

I think that the direction of the PR is good. As the PR suggests, networks
should be blocking ingress packets that carry invalid source addresses. We
can recommend clients to use NEW_TOKEN tokens only when it reconnects to an
address that is previously known.

Then, the only attack vector that can be used by an attacker to inject UDP
packets with chosen text is SPA, in which case the chosen text would be the
server-chosen CID and the ciphertext. But as the PR recommends, attacks
using ciphertext can be blocked by requiring clients to not send
non-probing packets during path validation. Then the only remaining attack
vector is the server-chosen CID.

It is a bit concerning that the attacker can choose whatever it wants for
the 2nd to 21st byte of a UDP packet. As the first byte is 0b01xxxxxx where
x is effectively random, an attacker could generate a UDP packet that looks
like text at a certain probability. As an example, a UDP packet starting
with "NOTIFY * HTTP/1.1\r\n" would look like a SSDP packet.

But that would probably be okay (at least for SSDP), as it would be
difficult (I think) to build a meaningful SSDP packet when the amount of
bytes that the attacker can choose is merely 20 bytes (max cid length).

Besides, as the draft suggests, when we figure out potential attack
vectors, browser vendors can add blocking rules for suspiciously-looking
bit patterns of server-chosen CIDs.

That's why I tend to think that the proposed solution is acceptable.

FWIW, If we want to be extremely cautious, we could create special encoding
rules. As an example, we could reserve QUIC version number 0x00ba5e64 for
indicating a base-64 encoded QUIC packet, meaning that the actual QUIC
packet is transmitted base64-encoded in the payload part of the long packet
header. By choosing such an approach, we can prevent rouge servers from
injecting arbitrary text. In this example, only the characters used by
base64 would be injectable.

That said, such a defense can be introduced as an protocol extension.
Therefore, we can live with a less-intrusive approach (as proposed by the
PR) until it turns out that we need something more thorough.

2020年8月13日(木) 16:54 Martin Thomson <[email-masked]>:

> Me again.
>
> https://github.com/quicwg/base-drafts/pull/3996 is a first draft of a
> proposal for this issue.  There are some small normative recommendations,
> but all of that is negotiable.
>
> At a high level, I would like to understand whether people think the
> general shape of this response is reasonable.
>

-- 
Kazuho Oku

---

From: "Lubashev, Igor" <[email-masked]>
Date: Thu, 13 Aug 2020 15:06:52 +0000

*   We can recommend clients to use NEW_TOKEN tokens only when it reconnects to an address that is previously known.

This would mostly defeat the purpose of NEW_TOKEN tokens in the context of DNS-mapped CDNs.  Clients would quickly learn to either not to follow this recommendation, or to keep an extensive history of addresses and guess at subnets they come from, or to risk lower handshake performance.

The real attack is not simply getting a client to send some bytes to a server somewhere on the Internet (those should be ready for that or nothing can help them).  The attack is for a server in one network security domain causing the client to send packets to a server in a notionally more secure network domain.

The best recommendation is watching for a notional security escalation between “globally routable IP space” -> “RFC1918/unique-local IP space” -> “link-local” -> “loopback”.  It would be very reasonable to record the “network security domain” a NEW_TOKEN came from and only send it if the new IP address is in the same “network security domain”.  I.e. that will trigger when someone changes A record to 127.0.0.1.

  *   Igor

From: Kazuho Oku <[email-masked]>

Hi,

I think that the direction of the PR is good. As the PR suggests, networks should be blocking ingress packets that carry invalid source addresses. We can recommend clients to use NEW_TOKEN tokens only when it reconnects to an address that is previously known.

Then, the only attack vector that can be used by an attacker to inject UDP packets with chosen text is SPA, in which case the chosen text would be the server-chosen CID and the ciphertext. But as the PR recommends, attacks using ciphertext can be blocked by requiring clients to not send non-probing packets during path validation. Then the only remaining attack vector is the server-chosen CID.

It is a bit concerning that the attacker can choose whatever it wants for the 2nd to 21st byte of a UDP packet. As the first byte is 0b01xxxxxx where x is effectively random, an attacker could generate a UDP packet that looks like text at a certain probability. As an example, a UDP packet starting with "NOTIFY * HTTP/1.1\r\n" would look like a SSDP packet.

But that would probably be okay (at least for SSDP), as it would be difficult (I think) to build a meaningful SSDP packet when the amount of bytes that the attacker can choose is merely 20 bytes (max cid length).

Besides, as the draft suggests, when we figure out potential attack vectors, browser vendors can add blocking rules for suspiciously-looking bit patterns of server-chosen CIDs.

FWIW, If we want to be extremely cautious, we could create special encoding rules. As an example, we could reserve QUIC version number 0x00ba5e64 for indicating a base-64 encoded QUIC packet, meaning that the actual QUIC packet is transmitted base64-encoded in the payload part of the long packet header. By choosing such an approach, we can prevent rouge servers from injecting arbitrary text. In this example, only the characters used by base64 would be injectable.

That said, such a defense can be introduced as an protocol extension. Therefore, we can live with a less-intrusive approach (as proposed by the PR) until it turns out that we need something more thorough.

2020年8月13日(木) 16:54 Martin Thomson <[email-masked]<mailto:[email-masked]>>:
Me again.

https://github.com/quicwg/base-drafts/pull/3996<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_quicwg_base-2Ddrafts_pull_3996&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=Djn3bQ5uNJDPM_2skfL3rW1tzcIxyjUZdn_m55KPmlo&m=PDttBAq_NNpOMfzGqLrlwa4wUkhxUojWnwl7ROVQRLU&s=qj6KgxGtb5pPjPBYvWMg2io_kHh_lhzAmSkjn0f9UFs&e=> is a first draft of a proposal for this issue.  There are some small normative recommendations, but all of that is negotiable.

On Wed, Aug 12, 2020, at 17:14, Martin Thomson wrote:
> https://github.com/quicwg/base-drafts/issues/3995<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_quicwg_base-2Ddrafts_issues_3995&d=DwMFaQ&c=96ZbZZcaMF4w0F4jpN6LZg&r=Djn3bQ5uNJDPM_2skfL3rW1tzcIxyjUZdn_m55KPmlo&m=PDttBAq_NNpOMfzGqLrlwa4wUkhxUojWnwl7ROVQRLU&s=bLohuEW_fNsbmpYh9sZETt_ywmuZ1bMnPak7sQNNqCI&e=>
>

--
Kazuho Oku

---

From: Christian Huitema <[email-masked]>
Date: Thu, 13 Aug 2020 08:06:47 -0700

Martin rightly points out that the attack is also available to the
client. Send an otherwise valid packet to the server from an attacker
specified address. Per spec, the attacker will send a probe to the
specified address, using one of the CID chosen by the client. That seems
at least as bad as SPA.

-- Christian Huitema

On 8/13/2020 5:37 AM, Kazuho Oku wrote:
> I think that the direction of the PR is good. As the PR suggests,
> networks should be blocking ingress packets that carry invalid source
> addresses. We can recommend clients to use NEW_TOKEN tokens only when
> it reconnects to an address that is previously known.
>
> Then, the only attack vector that can be used by an attacker to inject
> UDP packets with chosen text is SPA, in which case the chosen text
> would be the server-chosen CID and the ciphertext. But as the PR
> recommends, attacks using ciphertext can be blocked by requiring
> clients to not send non-probing packets during path validation. Then
> the only remaining attack vector is the server-chosen CID.
>
> It is a bit concerning that the attacker can choose whatever it wants
> for the 2nd to 21st byte of a UDP packet. As the first byte is
> 0b01xxxxxx where x is effectively random, an attacker could generate a
> UDP packet that looks like text at a certain probability. As an
> example, a UDP packet starting with "NOTIFY * HTTP/1.1\r\n" would look
> like a SSDP packet.
>
> difficult (I think) to build a meaningful SSDP packet when the amount
> of bytes that the attacker can choose is merely 20 bytes (max cid length).
>
> vectors, browser vendors can add blocking rules for
> suspiciously-looking bit patterns of server-chosen CIDs.
>
> FWIW, If we want to be extremely cautious, we could create special
> encoding rules. As an example, we could reserve QUIC version number
> 0x00ba5e64 for indicating a base-64 encoded QUIC packet, meaning that
> the actual QUIC packet is transmitted base64-encoded in the payload
> part of the long packet header. By choosing such an approach, we can
> prevent rouge servers from injecting arbitrary text. In this example,
> only the characters used by base64 would be injectable.
>
> Therefore, we can live with a less-intrusive approach (as proposed by
> the PR) until it turns out that we need something more thorough.
>
>     https://github.com/quicwg/base-drafts/pull/3996 is a first draft
>     of a proposal for this issue.  There are some small normative
>     recommendations, but all of that is negotiable.
>
>     At a high level, I would like to understand whether people think
>     the general shape of this response is reasonable.
>

---

From: Kazuho Oku <[email-masked]>
Date: Fri, 14 Aug 2020 00:28:02 +0900

2020年8月14日(金) 0:18 Christian Huitema <[email-masked]>:

> Martin rightly points out that the attack is also available to the client.
> Send an otherwise valid packet to the server from an attacker specified
> address. Per spec, the attacker will send a probe to the specified address,
> using one of the CID chosen by the client. That seems at least as bad as
> SPA.
>
On one hand, I agree that spoofed migration is a concern for both the
client and the server. At the same time, it can be blocked when the network
(in which either endpoint is contained) implements ingress filtering, as
the pull request points out.

I do not think we need to be concerned about networks *not* implementing
ingress filtering, because *active* attacks can be mounted against the
nodes in such networks.

> 0x00ba5e64 for indicating a base-64 encoded QUIC packet, meaning that the
> actual QUIC packet is transmitted base64-encoded in the payload part of the
> long packet header. By choosing such an approach, we can prevent rouge
> servers from injecting arbitrary text. In this example, only the characters
> used by base64 would be injectable.
>

-- 
Kazuho Oku

---

From: "Martin Thomson" <[email-masked]>
Date: Fri, 14 Aug 2020 10:41:13 +1000

Hi Igor.

On Fri, Aug 14, 2020, at 01:06, Lubashev, Igor wrote:
>   * We can recommend clients to use NEW_TOKEN tokens only when it 
> reconnects to an address that is previously known.
>  
> 
> This would mostly defeat the purpose of NEW_TOKEN tokens in the context 
> of DNS-mapped CDNs.  Clients would quickly learn to either not to 
> follow this recommendation, or to keep an extensive history of 
> addresses and guess at subnets they come from, or to risk lower 
> handshake performance.

The PR tries to address this in stages.  It first says that clients MUST NOT send a token if the server address changes, but then it goes on to list some exceptions.  The first is allowing clients to treat "nearby" addresses as equivalent, for some definition of nearby.  The second is allowing the original address and any validated preferred address to be equivalent.

This might not work for you here.  In particular, if you believe that you can share tokens between servers in dramatically different networks, then you those tokens won't be carried between those server instances.  I can think of a couple of cases that are at least plausible:

* if you have an in-ISP CDN node and one in the cloud they will likely have very different IP addresses (assuming that you don't use anycast) so the client will believe them to be different and not transfer tokens from one to the other.

* if the same name is served by multiple CDNs and somehow they are able to consume tokens from each other, then clients wont transfer tokens.

No matter the circumstances, the loss is not serious.  Tokens from NEW_TOKEN are always opportunistic and provide pretty weak address validation anyway.  They might lift the 3x limit, but that mostly only helps you get a large certificate out.

Servers can't rely on getting these tokens anyway, which is why I think this is a reasonable countermeasure.

> The best recommendation is watching for a notional security escalation 
> between “globally routable IP space” -> “RFC1918/unique-local IP space” 
> -> “link-local” -> “loopback”.  

The PR addresses this point specifically under a heading for generic countermeasures.  I can see clients being cautious about these "escalations", but I don't think that anything but the last can be uniformly blocked.  And even that might need exceptions.  If you have an argument to support stronger protection, that would be good.  But I think that dropping NEW_TOKEN tokens is a far safer choice.  

BTW, under the pull request, clients are free to treat all globally routeable space as equivalent for the purposes of token reuse.  However, I don't think that is at all advisable.  On the contrary, I know for certain that it is unsafe.  Maybe this doesn't happen much any more - it certainly shouldn't - but at a previous employer we had unprotected services that used public IP addresses (the company used a class A IPv4 subnet for its services, which might give you a clue...)

---

From: "Lubashev, Igor" <[email-masked]>
Date: Fri, 14 Aug 2020 22:51:28 +0000

Thanks for the reply Martin.  I think this is an important subject to address.  I've seen a number of legacy services that are, indeed, making assumptions about "we only listen to loopback" or "we only listen to RFC1918 addresses" in place of proper authentication mechanisms.

I re-read the PR, and, indeed, it does try to address the right things, including the emphasis on "IP isolation zone" escalations.

As you said, the PR does allow clients "wiggle room" for treating "nearby" addresses as equivalent.  In fact, it does not place any requirements on how large the "wiggle room" is, so MUST NOT in "MUST NOT send a token" becomes a recommendation and not something enforceable by the protocol.  Is this still a good use of MUST NOT?

- Igor

> From: Martin Thomson <[email-masked]>
> 
> > This would mostly defeat the purpose of NEW_TOKEN tokens in the
> > context of DNS-mapped CDNs.  Clients would quickly learn to either not
> > to follow this recommendation, or to keep an extensive history of
> The PR tries to address this in stages.  It first says that clients MUST NOT send
> a token if the server address changes, but then it goes on to list some
> exceptions.  The first is allowing clients to treat "nearby" addresses as
> equivalent, for some definition of nearby.  The second is allowing the original
> address and any validated preferred address to be equivalent.
> 
> This might not work for you here.  In particular, if you believe that you can
> share tokens between servers in dramatically different networks, then you
> those tokens won't be carried between those server instances.  I can think of
> a couple of cases that are at least plausible:
> 
> * if you have an in-ISP CDN node and one in the cloud they will likely have
> very different IP addresses (assuming that you don't use anycast) so the
> client will believe them to be different and not transfer tokens from one to
> the other.
> 
> * if the same name is served by multiple CDNs and somehow they are able to
> consume tokens from each other, then clients wont transfer tokens.
> 
> No matter the circumstances, the loss is not serious.  Tokens from
> NEW_TOKEN are always opportunistic and provide pretty weak address
> validation anyway.  They might lift the 3x limit, but that mostly only helps you
> get a large certificate out.
> 
> The PR addresses this point specifically under a heading for generic
> countermeasures.  I can see clients being cautious about these "escalations",
> but I don't think that anything but the last can be uniformly blocked..  And
> even that might need exceptions.  If you have an argument to support
> stronger protection, that would be good.  But I think that dropping
> NEW_TOKEN tokens is a far safer choice.
> 
> BTW, under the pull request, clients are free to treat all globally routeable
> space as equivalent for the purposes of token reuse.  However, I don't think
> that is at all advisable.  On the contrary, I know for certain that it is unsafe.
> Maybe this doesn't happen much any more - it certainly shouldn't - but at a
> previous employer we had unprotected services that used public IP
> addresses (the company used a class A IPv4 subnet for its services, which
> might give you a clue...)

---

From: "Martin Thomson" <[email-masked]>
Date: Sat, 15 Aug 2020 12:46:40 +1000

Thanks Igor, I will change to SHOULD NOT as you suggest.

On Sat, Aug 15, 2020, at 08:51, Lubashev, Igor wrote:
> Thanks for the reply Martin.  I think this is an important subject to 
> address.  I've seen a number of legacy services that are, indeed, 
> making assumptions about "we only listen to loopback" or "we only 
> listen to RFC1918 addresses" in place of proper authentication 
> mechanisms.
> 
> I re-read the PR, and, indeed, it does try to address the right things, 
> including the emphasis on "IP isolation zone" escalations.
> 
> As you said, the PR does allow clients "wiggle room" for treating 
> "nearby" addresses as equivalent.  In fact, it does not place any 
> requirements on how large the "wiggle room" is, so MUST NOT in "MUST 
> NOT send a token" becomes a recommendation and not something 
> enforceable by the protocol.  Is this still a good use of MUST NOT?
> 
> - Igor
>