Subject: [tsvwg] Review: draft-manner-tsvwg-gut-00

From: Bob Briscoe <[email-masked]>
Date: Wed, 03 Feb 2010 12:23:31 +0000

Jukka, Nuutti,

I've done a detailed review of gut-00 below:

==1. It's worth stating goal(s) more explicitly==
E.g.
a) mitigating problem of existing NATs, while still allowing 
middleboxes that deliberately want to block to do so (see previous 
email in response to Pasi)

==2. No magic number?==

I would prefer GUT without the magic number. Here's the trade-off:
* Magic number wastes header space always
* No magic number wastes receiver processing very occasionally

If the magic number closed a DoS vulnerability, it would be worth it. 
But as you say in the draft, DoS datagrams can just include the magic 
no. So we might as well let the inner protocol throw away junk, and 
just leave GUT completely transparent.

==3. Same source port?==
"  The source port MAY be chosen freely, although
    if the encapsulated protocol had a notion of port numners, the sender
    MAY choose the same source port."

What reason would there ever be for not using the same port?

I think this ought to be a SHOULD. If the two are reliably always the 
same, it gives information when they are not (e.g. a NAT has intervened).

==4. Missing sender & receiver operations?==

Need to say that the GUT daemon both on sending & receiving do 
checksum creation & verification respectively, as normal for UDP. You 
say (S.3.2) that the receiving GUT daemon recomputes the checksum 
when it rebuilds a new IP packet, but you must also say it verifies 
the original checksum before it does this (otherwise GUT will hide 
bit errors by always recomputing a correct checksum!).

Need to say whether the GUT daemon is considered as an IP hop 
(requiring the IP TTL/hop-limit to be decremented).

BTW, all the stuff about ECN in deployment considerations could just 
be said simply as all IP headers are preserved when the UDP & GUT 
shims are inserted or removed (which preserves all ECN functions).

==5. Stateless GUT daemon?==

A stateful GUT daemon is to be avoided if at all possible. Otherwise 
a programmer has to timeout state, and it becomes vulnerable to 
additional DOS attacks, so you start needing GUT cookies, etc.

Also, if you have a GUT daemon on a middlebox (as mentioned in 
deployment considerations), you want to avoid GUT state, otherwise it 
breaks for:
- asymmetric routes
- route changes
I.e GUT state breaks the 'shared fate' design principle.

The receiving GUT daemon doesn't need to store the following items of 
state, that S.3.2 of the draft says it should:
- source IP
- protocol number
- source port (if it's the same as the inner transport source port)

Rationale:
    ===Source IP===
    The source IP still in the IP packet that a receiving GUT daemon 
re-presents to the IP layer of the stack. So the inner protocol (e.g. 
SCTP, DCCP) can store the source IP if it needs to, then it will turn 
it round back to the sending GUT as normal. So a sending GUT can 
always be told the IP address to use by the process sending to it, 
which will effectively send GUT an IP packet, to have the UDP & GUT 
shims inserted.

    ===Protocol number===
    As with the source IP argument above, a GUT daemon can be told 
what protocol number to use whenever it is given a packet to 'shim'.

    ===Source port===
    If a GUT daemon notices the source port is the same as the inner 
transport protocol, it doesn't need to store any state. Then when the 
GUT daemon gets a packet to 'shim' before sending, if it doesn't hold 
any state, it uses the same source port as the protocol it encapsulates.

This just leaves a need for state when the inner & outer source ports 
are different. This still leaves a DoS vulnerability, because an 
attacker can make the inner & outer source ports different. So we 
still need to think harder whether we can avoid the need for this 
state... Effectively, the reason we are having to do this is because 
NAT-PTs create state for their port mapping, so the GUT daemon has to 
hold the same port mapping in reverse to negate it.

Whatever, once you require state, the draft needs to discuss managing it:
- timeouts
- what to do if a packet needs to be sent but the necessary state 
isn't there, yada yada.

[If you change what you've said based on the above, you will also 
need to change the comment about line 10 of Fig 3.]

==6. Firewall above or below?==

I suspect firewalls will want to look at incoming packets before the 
GUT daemon on incoming and after it on outgoing. There's no need to 
say firewalls have to sit higher in the stack than GUT. Putting 
firewalls above GUT would certainly be a way to ensure GUT isn't used 
to bypass firewalls that don't know about it. But once they do, they 
will want to be below it.

==7. Missing main incremental deployment consideration===

A little story:

Sender sends a DCCP packet, but gets no reply. Why?

It may have been black-holed by a NAT because it doesn't recognise 
DCCP. Or perhaps because it used ECN as well? Or perhaps the packet 
got through but the response was black-holed for another reason? Or 
perhaps either the outgoing or the response was just dropped due to 
congestion? Or due to security policy for some different reason?

Assume the code takes the most likely cases at this current time 
(somehow the code understands its place in the history of the 
Internet): it first assumes congestion and retries. Still no reply. 
So it starts to suspect DCCP was the problem. So after backoff timers 
expire it resends DCCP encapsulated in GUT.

Still no reply. What's the most likely cause?: Maybe the receiver 
doesn't yet support GUT. What next? Ug.

My point is, the draft says nothing about:
- incentives to be the first to deploy GUT when no-one else has
- whether it matters that the sender cannot determine if the rcvr supports GUT

A non-GUT receiver might send an ICMP Destination Unreachable (code 
3: Port unreachable) error, but with so much port-scanning, these 
days what's the chance of a port unreachable being sent, let alone 
getting back to the source through a minefield of ICMP filters?

Nits
====
* S.2. Intro
"IP-in-IP encapsulation is also one potential solution. [...] if NATs 
and firewalls probe deeper... they will encounter potentially an 
unknown transport protocol and drop the packet"

GUT is no different (but it doesn't matter if you are only trying to 
meet goal (a) above and not trying to bypass firewalls).
* S.2. Intro
"... and original protocol type"
The meaning of the word 'original' isn't obvious on first pass.
* S.2. Intro
s/meant to be/intended to be/
s/informations/information/

* S.3.1 Sender operation
s/numners/numbers/

* S.3.2. Receiver operation
s/player/layer/
s/flow source/connection originator/

"...will receive upstream packets...": not clear what 'upstream' means here.

* S.4. Deployment Considerations
s/protocol Y's/protocol's/

* S.6. Security Considerations

"...when there are no listening sockets open that require GUT."
Surely any listening socket may require GUT?

Bob

________________________________________________________________
Bob Briscoe,                                BT Innovate & Design

---

From: Jukka Manner <[email-masked]>
Date: Wed, 03 Feb 2010 18:52:25 +0200

Hi Bob,

thank you very much for a detailed review. I mostly agree with your 
analysis, but have some comments below.

regards,
Jukka

On 3.2.2010 14:23, Bob Briscoe wrote:
> a) mitigating problem of existing NATs, while still allowing middleboxes
> that deliberately want to block to do so (see previous email in response
> to Pasi)

Yes, mostly that, but you could play it evil and try to go through 
firewalls against the net admins' wishes by changing the GUT destination 
port #.

> no. So we might as well let the inner protocol throw away junk, and just
> leave GUT completely transparent.

Sounds ok to me to drop the magic number, and save some space.

> same, it gives information when they are not (e.g. a NAT has intervened).

Sure.

> Need to say that the GUT daemon both on sending & receiving do checksum
> creation & verification respectively, as normal for UDP. You say (S.3.2)
> that the receiving GUT daemon recomputes the checksum when it rebuilds a
> new IP packet, but you must also say it verifies the original checksum
> before it does this (otherwise GUT will hide bit errors by always
> recomputing a correct checksum!).
>
> Need to say whether the GUT daemon is considered as an IP hop (requiring
> the IP TTL/hop-limit to be decremented).
>
> BTW, all the stuff about ECN in deployment considerations could just be
> said simply as all IP headers are preserved when the UDP & GUT shims are
> inserted or removed (which preserves all ECN functions).

Agree on all those.

> A stateful GUT daemon is to be avoided if at all possible. Otherwise a
> programmer has to timeout state, and it becomes vulnerable to additional
> DOS attacks, so you start needing GUT cookies, etc.

I agree, yes. But... we need state at the receiver in order to send 
potential return traffic back through the right NAT binding. When the 
first packets were sent from the sender, a potential NAT on the way has 
a certain state, and thus any return traffic must match that state. If 
the receiver sends packets back in the same way as it would when 
initiating a flow, these return packets will not reach the sender.

In order words, when the GUT daemon receives a packet, it can be part of

a) a new flow (send to GUT port #) or

b) an existing flow (send to the port # that the flow was supposedly 
send from).

Without explicit indication, or e.g. DPI done by GUT, we can't know. We 
can deduce this if there is a state stored for the same 5-tuple.

> SCTP, DCCP) can store the source IP if it needs to, then it will turn it
> round back to the sending GUT as normal. So a sending GUT can always be
> told the IP address to use by the process sending to it, which will
> effectively send GUT an IP packet, to have the UDP & GUT shims inserted.
>
> ===Protocol number===
> As with the source IP argument above, a GUT daemon can be told what
> protocol number to use whenever it is given a packet to 'shim'.
>
> ===Source port===
> attacker can make the inner & outer source ports different. So we still
> need to think harder whether we can avoid the need for this state...
> Effectively, the reason we are having to do this is because NAT-PTs
> create state for their port mapping, so the GUT daemon has to hold the
> same port mapping in reverse to negate it.
>
> - what to do if a packet needs to be sent but the necessary state isn't
> there, yada yada.
>
> [If you change what you've said based on the above, you will also need
> to change the comment about line 10 of Fig 3.]
>
> I suspect firewalls will want to look at incoming packets before the GUT
> daemon on incoming and after it on outgoing. There's no need to say
> firewalls have to sit higher in the stack than GUT. Putting firewalls
> above GUT would certainly be a way to ensure GUT isn't used to bypass
> firewalls that don't know about it. But once they do, they will want to
> be below it.

Not sure what you mean, but if I got it right, yes, GUT truly can be 
used to by-pass local firewalls, depending on when the packets are 
inspected. This depends on how the implementation is done:

a) if packets are checked bw GUT and net if, the fw only sees UDP flows.

b) if packets are checked bw GUT and app, the fw will see e.g. DCCP or 
SCTP flows and may kill them.

I will add discussion about this into the draft.

> It may have been black-holed by a NAT because it doesn't recognise DCCP.
> Or perhaps because it used ECN as well? Or perhaps the packet got
> through but the response was black-holed for another reason? Or perhaps
> either the outgoing or the response was just dropped due to congestion?
> Or due to security policy for some different reason?
>
> (somehow the code understands its place in the history of the Internet):
> it first assumes congestion and retries. Still no reply. So it starts to
> suspect DCCP was the problem. So after backoff timers expire it resends
> DCCP encapsulated in GUT.
>
> A non-GUT receiver might send an ICMP Destination Unreachable (code 3:
> Port unreachable) error, but with so much port-scanning, these days
> what's the chance of a port unreachable being sent, let alone getting
> back to the source through a minefield of ICMP filters?

You lost me. Basically, as Pasi I believe has already said, GUT would be 
a fall-back in case the application can not get its communication to 
work. The cases above are things that can harm the original traffic, and 
trigger GUT encapsulation as a last effort.

GUT is only a tunneling scheme, it does not have any notion of 
retransmissions. If any reliability is needed, this is for the 
encapsulated protocol to implement.

The only timers in GUT are about when to discard state at the receiver 
for an incoming flow. A proper implementation could actually live 
without these timers as well, by keeping track of the open sockets in 
the OS and killing any stored incoming flow state when the application's 
socket is closed. Actually, in order to minimize DoS, we could also not 
store any state if there is no matching socket on the host.

Actually, the more I think about taking the local open sockets into 
account in all the state management done by GUT, the more it makes sense.

I'll revise the text also related to those nits below. Thanks again.

> and firewalls probe deeper... they will encounter potentially an unknown
> transport protocol and drop the packet"
>

---

From: Bob Briscoe <[email-masked]>
Date: Thu, 04 Feb 2010 10:20:47 +0000

Jukka,

>analysis, but have some comments below.

[BB]: Thanks. Responses inline (having cut the items we agree on)...

>potential return traffic back through the right NAT binding. When 
>the first packets were sent from the sender, a potential NAT on the 
>way has a certain state, and thus any return traffic must match that 
>state. If the receiver sends packets back in the same way as it 
>would when initiating a flow, these return packets will not reach the sender.
>
>Without explicit indication, or e.g. DPI done by GUT, we can't know. 
>We can deduce this if there is a state stored for the same 5-tuple.

[BB]: Pls read comments below again. I'm not saying you don't ever 
need flow state, just you usually don't need it:
a) you don't need to store anything except src port, not the other 2 items
b) you only need to store src port when it's different from inner src port.
Not storing anything is like storing 1 bit of information without 
storing it: it says the src ports of GUT & inner protocol were the same.

Then for most (?) connections GUT will have no flow state.

>b) if packets are checked bw GUT and app, the fw will see e.g. DCCP 
>or SCTP flows and may kill them.
>

[BB]: OK, no need to say more on this then...

We're now saying the same thing with different emphasis. It depends 
on which of the implementers (GUT or firewall) gets their 
implementation lowest in the stack. I'm saying host-firewalls are 
bound to evolve to sit below GUT and do DPI on GUT datagrams. 
Currently the draft says firewalls will be above GUT in the stack. 
That will only happen for a short while, if at all. So it's best not 
to say it like that. That's all I meant.

>You lost me. Basically, as Pasi I believe has already said, GUT 
>would be a fall-back in case the application can not get its 
>communication to work. The cases above are things that can harm the 
>original traffic, and trigger GUT encapsulation as a last effort.

[BB]: Yes, I should have been clearer. I am talking about the 
question you raised in the deployment section: "when to use GUT?" I'm 
saying two things:
a) It's hard to write code that does a good job of deciding when to 
use GUT, because the only indication that GUT is needed is 'no 
response' and there are loads of other reasons for no response (see 
my story for some).
b) The draft doesn't discuss the main incremental deployment 
challenge: GUT deployed on the sender but not the receiver. If this 
is the case, if the sender decides to use GUT because it has been 
getting no response, it will still get no response (because port 
unreachable ICMP messages are never used these days). So it still 
won't know whether the receiver has GUT deployed - the lack of 
response may be due to another problem.

Importantly, code to handle 'no response' needs to take account of 
black-holing of many different features in different modules of the 
stack (as well as simple congestion loss). Implementing 'no response' 
code within each module is an art - it's probably better collected 
together in one place.

>encapsulated protocol to implement.

[BB]: Agreed. But the code that decides whether to use GUT is a 
special case - a bit like a generic encapsulated protocol. It has to 
decide whether complete lack of response to a specific encapsulated 
protocol requires a different sort of retransmission - a 
retransmission with GUT added. And then, if there's a complete lack 
of response to GUT, it has to decide whether this requires more GUT 
retransmissions, or whether to give up on GUT too.

>The only timers in GUT are about when to discard state at the 
>receiver for an incoming flow. A proper implementation could 
>actually live without these timers as well, by keeping track of the 
>open sockets in the OS and killing any stored incoming flow state 
>when the application's socket is closed. Actually, in order to 
>minimize DoS, we could also not store any state if there is no 
>matching socket on the host.
>

[BB]: Yes, this is good.

>I'll revise the text also related to those nits below. Thanks again.

[BB]: Cheers. Don't forget the nits I added later.

Bob

---

From: Jukka Manner <[email-masked]>
Date: Mon, 08 Feb 2010 22:05:34 +0200

Hi Bob, some answers below.

Thanks for actively pushing me to think about the whole concept over and 
over again. :)

rgs,
Jukka

On 4.2.2010 12:20, Bob Briscoe wrote:
>> first packets were sent from the sender, a potential NAT on the way
>> has a certain state, and thus any return traffic must match that
>> state. If the receiver sends packets back in the same way as it would
>> when initiating a flow, these return packets will not reach the sender.
>>
> [BB]: Pls read comments below again. I'm not saying you don't ever need
> flow state, just you usually don't need it:

[JM]: I draw the situation again on paper, and I believe we still need 
to store on the receiver
1. encapsulated protocol # (because e.g. a DCCP or SCTP flow may use the 
same port #s), we need per-protocol tables
2. The source port in the UDP header, which was (might have been) 
allocated by the NAT when the packet arrived (this may be different from 
the encapsulated source port #), as you mention below. We may live 
without it in the case you describe.
3. Source port in the encapsulated header, if any (GUT can also tunnel 
stuff which does not have a transport header, e.g., RSVP, ICMP, etc.). 
In addition, consider the follow regarding port #s (G=GUT port #):

(source) (GUT)     (NAT without PT)    (GUT)    (destination)
[A->B]  [A->G]         [A->G]          [A->G]     [A->B]
         [A->B]         [A->B]          [A->B]
                     {NAT will not see B!}

return traffic
(destination) (GUT)        (NAT)
[B->A]        [B->G]   [No B->G state, only A->G]
               [B->A]

So, if we don't store the encapsulated source port #, the packet coming 
back from the destination is interpreted as a new one and it will not 
match the state seen by the NAT. I wonder if the above makes sense?

The source IP we don't need to store, sure, because the destination will 
not know the initial source IP before the NAT did it's thing, so any 
locally generated packets already have the correct destination IP.

The thing with GUT is that it doesn't have any interactions with the 
encapsulated protocol. There is no e.g. cross-layer indications between 
DCCP and GUT, so we can only rely on what we see in incoming packets.

If there would be some interactions bw. protocol layers, then some state 
is eventually store by some entity.

I'll draw this again with multiple clients on a very big piece of paper 
and think again whether there is any logic that could be used to figure 
out if a locally generated packet belongs to an existing flow, or not.

> We're now saying the same thing with different emphasis. It depends on
> which of the implementers (GUT or firewall) gets their implementation
> lowest in the stack. I'm saying host-firewalls are bound to evolve to
> sit below GUT and do DPI on GUT datagrams. Currently the draft says
> firewalls will be above GUT in the stack. That will only happen for a
> short while, if at all. So it's best not to say it like that. That's all
> I meant.
>
>
>
>> You lost me. Basically, as Pasi I believe has already said, GUT would
>> be a fall-back in case the application can not get its communication
>> to work. The cases above are things that can harm the original
>> traffic, and trigger GUT encapsulation as a last effort.
>
> [BB]: Yes, I should have been clearer. I am talking about the question
> you raised in the deployment section: "when to use GUT?" I'm saying two
> things:
> a) It's hard to write code that does a good job of deciding when to use
> GUT, because the only indication that GUT is needed is 'no response' and
> there are loads of other reasons for no response (see my story for some).

[JM]: Very true. And I wouldn't like to specify, at least not yet, when 
to activate GUT for a communication, and when not. So far we just want 
to figure out, what is the bare minimum to get the job done, i.e., get 
stuff through an old legacy NAT without requiring changes to the 
protocols to be encapsulated.

> b) The draft doesn't discuss the main incremental deployment challenge:
> GUT deployed on the sender but not the receiver. If this is the case, if
> the sender decides to use GUT because it has been getting no response,
> it will still get no response (because port unreachable ICMP messages
> are never used these days). So it still won't know whether the receiver
> has GUT deployed - the lack of response may be due to another problem.

[JM]: yes, true, we don't know why GUT wouldn't work if employed. For 
some protocols, we could easily implement a state machine that follows 
e.g. DCCP or SCTP connection initiations, and if a few retransmissions 
are seen, try to activate GUT. But this whole artificial intelligence is 
very tricky to even begin with.

> [BB]: Agreed. But the code that decides whether to use GUT is a special
> case - a bit like a generic encapsulated protocol. It has to decide
> whether complete lack of response to a specific encapsulated protocol
> requires a different sort of retransmission - a retransmission with GUT
> added. And then, if there's a complete lack of response to GUT, it has
> to decide whether this requires more GUT retransmissions, or whether to
> give up on GUT too.

[JM]: sounds like my comment above. Yet, we have not designed GUT to 
have retransmissions by itself. If there are retries, those are done by 
the encapsulated protocol. But here we come back to the logic when to 
activate something like GUT for a certain protocol. I fear that at the 
end of the day, activating GUT for a certain flow is triggered by the 
application itself, which forces applications to be aware of GUT. But 
maybe that won't be such a bad thing...

>> the OS and killing any stored incoming flow state when the
>> application's socket is closed. Actually, in order to minimize DoS, we
>> could also not store any state if there is no matching socket on the
>> host.
>>

---

From: Jukka MJ Manner <[email-masked]>
Date: Thu, 11 Feb 2010 23:03:21 +0200 (EET)

Hi Bob,

We had a discussion, Nuutti, I and a colleague, about the state 
management, and I would claim that there is only one subcase, where 
receiver would not need to store state, and that is when the UDP header 
source and destination port numbers are the same and equal to the GUT port 
#.

If all packets are going from GUT port# to GUT port# by default, and we 
don't have a NAT-PT in the middle (a normal NAT wouldn't harm), we can 
work without per-flow state.

For an other case, we actually need most (if not all) of the 5-tuple to 
really match all outgoing packets to a potential existing stream. A simple 
case is a multi-homed server, which talks over different IPs to a wide 
range of clients, who can in addition use a number of NAT unfriendly ;) 
protocols. If we make a mistake in the matching, we don't have enough 
state, the whole stream is blackholed, unfortunately.

regards,
Jukka

On Thu, 4 Feb 2010, Bob Briscoe wrote:

>> potential return traffic back through the right NAT binding. When the first 
>> packets were sent from the sender, a potential NAT on the way has a certain 
>> state, and thus any return traffic must match that state. If the receiver 
>> sends packets back in the same way as it would when initiating a flow, 
>> these return packets will not reach the sender.
>> 
>> b) an existing flow (send to the port # that the flow was supposedly send 
>> from).
>> 
>> Without explicit indication, or e.g. DPI done by GUT, we can't know. We can 
>> deduce this if there is a state stored for the same 5-tuple.
>
> [BB]: Pls read comments below again. I'm not saying you don't ever need flow 
> state, just you usually don't need it:
> Not storing anything is like storing 1 bit of information without storing it: 
> it says the src ports of GUT & inner protocol were the same.
>
>> Not sure what you mean, but if I got it right, yes, GUT truly can be used 
>> to by-pass local firewalls, depending on when the packets are inspected. 
>> This depends on how the implementation is done:
>> 
>> b) if packets are checked bw GUT and app, the fw will see e.g. DCCP or SCTP 
>> flows and may kill them.
>> 
> We're now saying the same thing with different emphasis. It depends on which 
> of the implementers (GUT or firewall) gets their implementation lowest in the 
> stack. I'm saying host-firewalls are bound to evolve to sit below GUT and do 
> DPI on GUT datagrams. Currently the draft says firewalls will be above GUT in 
> the stack. That will only happen for a short while, if at all. So it's best 
> not to say it like that. That's all I meant.
>
>
>
>> You lost me. Basically, as Pasi I believe has already said, GUT would be a 
>> fall-back in case the application can not get its communication to work. 
>> The cases above are things that can harm the original traffic, and trigger 
>> GUT encapsulation as a last effort.
>
> [BB]: Yes, I should have been clearer. I am talking about the question you 
> raised in the deployment section: "when to use GUT?" I'm saying two things:
> a) It's hard to write code that does a good job of deciding when to use GUT, 
> because the only indication that GUT is needed is 'no response' and there are 
> loads of other reasons for no response (see my story for some).
> b) The draft doesn't discuss the main incremental deployment challenge: GUT 
> deployed on the sender but not the receiver. If this is the case, if the 
> sender decides to use GUT because it has been getting no response, it will 
> still get no response (because port unreachable ICMP messages are never used 
> these days). So it still won't know whether the receiver has GUT deployed - 
> the lack of response may be due to another problem.
>
> black-holing of many different features in different modules of the stack (as 
> well as simple congestion loss). Implementing 'no response' code within each 
> module is an art - it's probably better collected together in one place.
>
>
>> retransmissions. If any reliability is needed, this is for the encapsulated 
>> protocol to implement.
>
> [BB]: Agreed. But the code that decides whether to use GUT is a special case 
> - a bit like a generic encapsulated protocol. It has to decide whether 
> complete lack of response to a specific encapsulated protocol requires a 
> different sort of retransmission - a retransmission with GUT added. And then, 
> if there's a complete lack of response to GUT, it has to decide whether this 
> requires more GUT retransmissions, or whether to give up on GUT too.
>
>
>> The only timers in GUT are about when to discard state at the receiver for 
>> an incoming flow. A proper implementation could actually live without these 
>> timers as well, by keeping track of the open sockets in the OS and killing 
>> any stored incoming flow state when the application's socket is closed. 
>> Actually, in order to minimize DoS, we could also not store any state if 
>> there is no matching socket on the host.
>> 
>> Actually, the more I think about taking the local open sockets into account 
>> in all the state management done by GUT, the more it makes sense.
>
> [BB]: Yes, this is good.
>
>