Subject: Re: [Capwap] need clarification on UDP ports

From: "Scott G. Kelly" <[email-masked]>
Date: Fri, 20 Oct 2006 14:54:18 -0700 (GMT-07:00)

Hi Navin,

-----Original Message-----
>From: "Navin (NKA)" <[email-masked]>
>Sent: Oct 20, 2006 1:00 PM
>To: "Scott G. Kelly" <[email-masked]>
>Cc: [email-masked]
>Subject: Re: [Capwap] need clarification on UDP ports
>
>On 10/20/06, Scott G. Kelly <[email-masked]> wrote:
>> >[NKA] Scott, so you agree that the logic which I have explained for handling
>> >discovery/DTLS packets is correct, but you feel that it would result in slower
>> >synchronization between WTP and AC FSM?
>>
>> Actually, no, I don't agree. A discovery packet should be deterministically treated as a discovery packet - not sometimes erroneously treated as a dtls packet. I think introducing this ambiguity is unnecessary, to say the least.
>
>[NKA] Scott, now you are not agreeing to what you said in youre previous email.
>Look at the snippet of the same.

You know, even if I *had* changed my position -- that is allowed in the IETF. But in this case, I didn't. All along I've been disagreeing with the notion that ambiguity and non-deterministic handling of discovery messages is okay. I'm sorry if anything I've said was in any way confusing.

>-----------------------------------------------------------------------------------------------------------------
>>[NKA] Even if a less-robust AC implementation ends up interpreting DTLS
>>message as a Discovery packet, do you think WTP (which unfortunately
>>has passed discovery state) will be able to carry on with its DTLS
>>session establishment and succeed ultimately? :-( I don't think so.  (Imagine
>>this: WTP needs to successfully interpret Discovery response as HelloVerify
>>request, followed by which AC needs to interpret ClientHello (with cookie) as
>>Client Hello without Cookie, and so on).
>
>That's not the point, and this is minor compared to the real issue,
>which is below
>                *
>                *
>                *
>In this case the WTP will often need to go through discovery again
>before reconnecting to the AC. If the AC erroneously feeds these
>packets to its DTLS implementation, they will be rejected, and the WTP
>will get no response.  This means failure recovery will not occur
>until the AC times out its DTLS session, and since these timeouts are
>frequently set as high as 30 seconds in the field,  and since the WTP
>may exhaust its discovery timers and go into sulking state due to AC
>unresponsiveness, this has the potential to signficantly (and
>unfortunately, completely avoidably) exacerbate the outage.
>-----------------------------------------------------------------------------------------------------------------
>
>[NKA] My point is that seperate discovery packet processing via a special port
>is a bad idea. And it doesn't bring any additional value than what can be
>achieved without special port approach. 

Okay, so we know where you stand on this.

>Scott's additional reasoning that two
>port approach would open up scope for DOS atack. I totally disagree with
>the argument. I claim that the same risk exists even with 3-port approach also.

Sorry, don't agree with this. Here's what you suggested:

>You may have a situation in which session information exists for a WTP at AC,
>but WTP is sending non-DTLS packet. Even in such situation, AC can feed such
>packets to DTLS module, which will ultimately reject the packet. AC can use
>this kind of notification to tear down the existing session, which will bring AC's
>state machine in sync with that of WTP.

To paraphrase, if the AC receives a discovery packet that appears to be from a live WTP, the AC should tear down that WTP's session to sync with the WTP.

To me (and maybe it's just me, because I'm so paranoid or something), that sounds an awful lot like an invitation to a DoS party. Go ahead and implement this, and I'll show you how it works :-)

Now, as to whether that vulnerability also exists with 3 ports... well, I guess if you implement the way you described, then it does. But if it were my implementation, I wouldn't *ever* tear down a session based on the receipt of unauthenticated data.

Scott

_________________________________________________________________
To unsubscribe or modify your subscription options, please visit:
http://lists.frascone.com/mailman/listinfo/capwap

Archives: http://lists.frascone.com/pipermail/capwap

---

From: "Hasan Raza" <[email-masked]>
Date: Sat, 21 Oct 2006 10:34:53 -0700

Hi,

I think we are a bit closer than where we were earlier in terms of resolving
this issue.

I agree with the points which Scott has raised. And I feel that there are few
DoS issues which have not been addressed properly in the capwap draft. May be
resolving those issues first might give us a solution for the DTLS/Discovery
packet identification issue (#224).

To start with, we have to answer few questions:
1. How capwap is protected against discovery packet attacks? Even if there
is a port-based or mux-based soltuion for reliably identifying discovery and
DTLS packets, an attacker can send a discovery packet to the AC, which may
result in the tear down of the WTP session. This is because discovery packets
are un-authenticated packets, and are sent in plain. AC needs to blindly
believe on the discovery packets.

2. If an AC employs actions again discovery packet attacks by discarding
discovery packets if WTP session did not timeout, then an attacker may
prevent the WTP joining back to the AC after a WTP crash. Attacker can detect
a sudden absence of echo request messages from WTP, and start replaying the
WTP echo request messages which causes AC not to teardown the WTP session.
And thus genuine discovery packets might be discarded by AC. May be I am wrong
here if DTLS can detect duplicate packets and discard them automatically.

To resolve the above issues, we can impose following rules on AC.

For issue (1) above:
RULE 1:
  (a) AC discards all discovery packets if an active WTP session exists for the
  WTP.
  (b) A discovery packet from WTP can be accepted by AC only when there does
  not exist any active session for that WTP. Whereas, if a discovery packet is
  received if an active WTP session exists at AC, then all such discovery
  packets are discarded without change or update of any of the AC states.

To resolve the second issue (if at all it exists): 
RULE 2:
  AC should discard all duplicate DTLS and CAPWAP packets without change or
  update of any of its states.

RULE (1) above will impose a constant delay in the WTP re-joining. Suppose a
WTP crashes and comes up before WTP session at AC may timeout, all its
discovery packets will be discarded by the WTP. But since AC won't be
receiving any valid/authenticated packet which could refresh the WTP session,
the WTP session would timeout after _fixed_ amount of time. This _fixed_
amount of time is unfortunately not fixed since it can be changed
administratively, by say updating echo interval at WTP or by changing response
timeout values at AC.

To make the WTP re-joining faster, WTP may save the DTLS cookie on a
persistent storage and (after a restart) presents the cookie to the AC to
restore the DTLS connection, and thus avoiding the discovery phase. This
concept is same as the one presented in "Idle to Join (aa)" transition,
except that here presenting a valid cookie resets the state to join state,
discarding any previous WTP state/stats maintained at AC.

I _think_ that the above proposals will avoid discovery packet DoS attacks,
and there is no need to use separate ports or extra mux header after udp
header to differentiate among the discovery and DTLS packets.

Thanks
Hasan

-----Original Message-----
From: Scott G. Kelly [mailto:[email-masked]]
Sent: Fri 10/20/2006 2:54 PM
To: Navin (NKA)
Cc: [email-masked]
Subject: Re: [Capwap] need clarification on UDP ports
 
Hi Navin,

-----Original Message-----

Scott

---

From: Scott G Kelly <[email-masked]>
Date: Sat, 21 Oct 2006 12:24:52 -0700

Hi Hasan,

> believe on the discovery packets.

Actually, I don't think this is necessarily the case. Since discovery 
packets are unauthenticated, the AC should *never* act on them (other 
than to send a discovery response). The draft currently states that the 
AC should not maintain state when it receives a discovery request (for 
this very reason), but we should probably add text noting that it should 
not modify existing state, either.

> here if DTLS can detect duplicate packets and discard them automatically.

Yes, DTLS provides antireplay detection. I inaccurately said in an 
earlier email that this is compulsory for DTLS - what I should have said 
is that it is compulsory for CAPWAP (it's optional in DTLS).

>   packets are discarded without change or update of any of the AC states.

I think it's okay for the AC to respond, but it should not tear down an 
existing association. It should only do this once the WTP re-joins (and 
re-authenticates in the process). I'll think about better language for 
the security considerations and the discovery processing.

>   update of any of its states.

Maybe we need better language on this as well. The AC should be keeping 
statistics relating to such events, at minimum.

> administratively, by say updating echo interval at WTP or by changing response

As noted above, re-joining is independent of discovery operations - a 
WTP should be able to initiate a new DTLS handshake even though a 
session currently exists, but there are subtle nuances to this (need 
rate limiting, need to detect DoS attempts via many half-open sessions, 
etc). I will work on text for this. The only way an existing session 
should be torn down is via timeout, or via a yet-to-be-defined directive 
received through another authenticated channel with the same WTP.

> discarding any previous WTP state/stats maintained at AC.

I agree DTLS session resumption is something we should think about, but 
honestly, I haven't given it much thought yet. We need to think 
carefully about this, though. But here's an important point: discovery 
is optional - it is not required in order to initiate a dtls handshake.

> header to differentiate among the discovery and DTLS packets.

Think this through again in light of my comments above. I still believe 
that erroneously pushing a discovery packet into the dtls engine (and 
re-testing all failed dtls packets to see if they are discovery) is bad 
form, a hack we don't need to allow, and a potential DoS avenue (if dtls 
is handled in hardware but discovery is handled by the control 
processor, then you can stuff the channel between the hw and cp with 
bogus packets, keeping both busy, and maybe causing real packets to be 
discarded in the process).

There are at least two options on the table for avoiding it (a separate 
discovery port or a type value just after the UDP header).

Scott

---

From: "Hasan Raza" <[email-masked]>
Date: Sat, 21 Oct 2006 19:22:38 -0700

Hi Scott,

I am sorry that I missed the point that discovery packets
do not result in any state change at AC. Actually, the DoS
issues I raised were the outcomes of the wrong assumption
that discovery packets change the AC state. Of course,
differentiation of discovery/DTLS packet by looking at the
wtp session would be a bad design, I beleive.

However, I can think of two other ways to make the distinction
among discovery/DTLS packets. These solutions (if they work)
would be less costlier than extra port or mux-based solutions:

1. Checksum at the end of discovery packets:
   All incoming packets are checked for _discovery-like_ packets
   by looking at the head bytes (as Pat proposed in this thread).
   If the packet looks like to be a discovery packet, then it
   is submitted for checksum calculation. The calculated checksum
   is compared with the last 4 bytes of the packet, and a match of
   the two would mark the packet as discovery packet. Otherwise,
   packet is submitted to DTLS module.

   Drawbacks:
     - extra processing at AC for false positives.
     - overhead of checksum calculation for discovery packets at
       WTP/AC.
   Limitations:
     - The mechanism to identify _discovery-like_ packets will affect
       the false positives and thus packet processing performance.

2. Reserved length for discovery packets:
   All discovery request packets sent by WTP are of size L (say 240
   bytes). Padding is added to the discovery packets to make it of
   exact size L. In case a DTLS packet to be sent to AC is of size L,
   then a (new) pad message element is added to increase the packet
   length.

   Limitations:
      - Selection of size L is critical. Note that DTLS layer may also send
        handshake packets of size L. If DTLS handshake packets does not
        following any lower/upper bound on packet length, or packet length
        alignment, then this scheme will not work. Otherwise, a very short
        lengthed marker message (of say 4 bytes) can also be used to mark the
        packet following it as discovery packet.

The latter solution looks like to be more of a hack, but I have put here
to give it also a thought. However, a combination of (1) and (2) may
work.

Thanks
Hasan

-----Original Message-----
From: Scott G Kelly [mailto:[email-masked]]
Sent: Sat 10/21/2006 12:24 PM
To: Hasan Raza
Cc: [email-masked]; [email-masked]
Subject: Re: [Capwap] need clarification on UDP ports
 
Hi Hasan,

Scott

---

From: Scott G Kelly <[email-masked]>
Date: Sun, 22 Oct 2006 10:43:08 -0700

Hi Hasan,

> to give it also a thought. However, a combination of (1) and (2) may

Proposal 2 should be a non-starter - I agree that artificially limiting 
the packet size really is a hack, and in my opinion, it's a 
short-sighted one at that.

Proposal 1 is also a hack, albeit slightly more subtly so than proposal 
2. I can sort of understand why one might think anything which removes 
the need for a 3rd port is "less costly", but in general, I think this 
takes a rather narrow view.

After poking around quite a bit through various uses of TLS and DTLS, I 
can find no example where someone tries to combine unprotected and 
protected data in the same channel, which the exception of DCCP 
(draft-phelan-dccp-dtls-00.txt). In that case, the unprotected data 
precedes the protected data in the packet (i.e. DTLS encapuslates a 
payload).

I guess that here in capwap, we are now and forever doomed to referring 
to the unprotected bytes preceding the payload as a "mux-based 
solution", and with that we'll get all of people's related baggage and 
anxieties. I would suggest that everyone be objective about this, and 
reflect carefully on your reactions ("Search your feelings, Luke - you 
know it to be true!" :-)). But on a more serious note... this does not 
serve us well.

To unambiguously delineate between these two data types is very simple: 
give the capwap discovery protocol its own port, or add a type 
descriptor to the packets and encapsulate the non-discovery payloads in 
DTLS. No protocol gymnastics, no hacking - both are simple solutions 
that allow fast and definitive hardware-based switching of these packets.

Scott