Subject: [sip-clf] SIP-CLF slides for opsarea and possibly ipfix

From: "Vijay K. Gurbani" <[email-masked]>
Date: Fri, 24 Jul 2009 11:22:19 -0500

Hello: Here is a link to the slide show I plan to present
on SIP-CLF at the opsarea meeting on Thu.  An abbreviated
version of this may be presented at the ipfix meeting
on Mon as well.

Please do look at the contents and suggest any changes.

http://ect.bell-labs.com/who/vkg/IETF/75/IETF-75-sip-clf-vkg.ppt

Thank you,

- vijay
-- 
Vijay K. Gurbani, Bell Laboratories, Alcatel-Lucent
1960 Lucent Lane, Rm. 9C-533, Naperville, Illinois 60566 (USA)
Email: vkg@{alcatel-lucent.com,bell-labs.com,acm.org}
Web:   http://ect.bell-labs.com/who/vkg/

---

From: Hadriel Kaplan <[email-masked]>
Date: Fri, 24 Jul 2009 13:41:15 -0400

Hi Vijay,

I would recommend the following changes:

1) Get rid of the two slides on "Need for CLF in current literature".  Neither of those two use-cases are actually addressed by SIP CLF, afaict.  At least not for the purpose of performing anomaly detection by using the CLF. (CLF would be useful to report when anomalous events are found perhaps, but we've said it's not a replacement for Syslog/SNMP event reporting, right?)  Even if you don't agree with my view of that, there's no need to debate it in the meeting, because it's not central to the need for CLF.

2) Maybe summarize the proposed charter in a slide or two?  Like list what's in/out-of-scope, and the deliverables?

3) on the last slide, include opsarea in the parenthesis list, since that's planned for Stockholm as well I believe.

4) I'm not quite sure what the purpose of the "Challenges in defining SIP CLF" slides are, but if you're itemizing them... then add "To get a common view of merged/global CLF's, must choose where in the stack/system to log SIP messages", and "Security and privacy related issues of various forms".

5) editorial nit: slide 3, remove the word "problem" from the top, since this is a slide on benefits, not problems. Unless you're saying the problem is we don't have these benefits. ;)

-hadriel

> -----Original Message-----
> From: [email-masked] [mailto:[email-masked]] On Behalf
> Of Vijay K. Gurbani
>

---

From: "Vijay K. Gurbani" <[email-masked]>
Date: Fri, 24 Jul 2009 15:31:37 -0500

> Hi Vijay,

Hadriel: Thanks for the feedback; more inline.

> 1) Get rid of the two slides on "Need for CLF in current literature".
> Neither of those two use-cases are actually addressed by SIP CLF,
> afaict.  At least not for the purpose of performing anomaly detection
> by using the CLF. (CLF would be useful to report when anomalous
> events are found perhaps, but we've said it's not a replacement for
> Syslog/SNMP event reporting, right?)  Even if you don't agree with my
> view of that, there's no need to debate it in the meeting, because
> it's not central to the need for CLF.

OK; training anomaly detection systems for semantic attacks on
SIP through the CLF may well be classified as a research topic,
so you are right -- I have taken these two slides out.

> 2) Maybe summarize the proposed charter in a slide or two?  Like list
> what's in/out-of-scope, and the deliverables?

OK; will do.

> 3) on the last slide, include opsarea in the parenthesis list, since
> that's planned for Stockholm as well I believe.

OK.

> 4) I'm not quite sure what the purpose of the "Challenges in defining
> SIP CLF" slides are, but if you're itemizing them... then add "To get
> a common view of merged/global CLF's, must choose where in the
> stack/system to log SIP messages", and "Security and privacy related
> issues of various forms".

I have put in security and privacy related issues as a bullet.

The other point -- "to get a common view of merged/global CLFs ..."
may require some discussion.  Are you proposing that there be a
central CLF server where all SIP actors in a network send their
messages for correlation?  At least my view of the SIP CLF was
more along the lines of each SIP actor producing their own
CLF.

> 5) editorial nit: slide 3, remove the word "problem" from the top,
> since this is a slide on benefits, not problems. Unless you're saying
> the problem is we don't have these benefits. ;)

Done.

The updated copy is at the same location
http://ect.bell-labs.com/who/vkg/IETF/75/IETF-75-sip-clf-vkg.ppt

Thank you for your time on this.

- vijay

---

From: Hadriel Kaplan <[email-masked]>
Date: Fri, 24 Jul 2009 17:17:59 -0400

> From: Vijay K. Gurbani [mailto:[email-masked]]
> Sent: Friday, July 24, 2009 4:32 PM
> 

It's not so much that it's a research topic, but rather that some of the needs of anomaly detection literally require getting the entire SIP message (or at least a lot more than what we're proposing as the baseline header info).  Even the currently *deployed* anomaly detectors check more than what the current drafts are proposing to include in a CLF.

> more along the lines of each SIP actor producing their own

Nope, I'm talking about if all SIP actors generate their own CLF's, and one or more central systems read and merge the CLF's generated by the actors.  In that case, then it's potentially important that we specify how/when the data in the CLF was created, so that all actors provide a consistent view.  

For example, if B2BUA-1 generates CLF entries for SIP messages it's sending out, but does so internally before certain manipulations such as number translations - while a second B2BUA-2 receiving the message records its CLF on reception but after its number translations, the view from a common place is not consistent. (number translations just being an example)  And we may decide it doesn't matter in the end - but it's a potential "Challenge".

Even for some sanity/consistency we'll probably need to define whether a CLF entry is generated at both receiving and sending of SIP messages, above vs. in vs. below a transaction layer, before or after Privacy anonymization, etc.

-hadriel

---

From: "Vijay K. Gurbani" <[email-masked]>
Date: Fri, 24 Jul 2009 16:32:12 -0500

Hadriel Kaplan wrote:
> It's not so much that it's a research topic, but rather that some of
> the needs of anomaly detection literally require getting the entire
> SIP message (or at least a lot more than what we're proposing as the
> baseline header info).  Even the currently *deployed* anomaly
> detectors check more than what the current drafts are proposing to
> include in a CLF.

Currently deployed anomaly detection systems that I am aware
of perform mostly at the parsing layer to determine whether or
not a SIP message is well-formed -- hence they need the whole
SIP message.  Are you aware of any anomaly detection systems that
do semantic anomaly detection for SIP above and beyond the
parsing layer?

> Nope, I'm talking about if all SIP actors generate their own CLF's,
> and one or more central systems read and merge the CLF's generated by
> the actors.  In that case, then it's potentially important that we
> specify how/when the data in the CLF was created, so that all actors
> provide a consistent view.

OK, but this merge is an orthogonal step.  Each actor still
produces CLF from the SIP messages that it is processing, yes?

> Even for some sanity/consistency we'll probably need to define
> whether a CLF entry is generated at both receiving and sending of SIP
> messages, 

Would not that depend on the specific role being played by the
SIP actor?  A proxy will, out of necessity, generate incoming
requests, responses sent upstream, requests sent downstream,
and responses received from downstream servers -- and do all
this so that the entire SIP search tree corresponding to the
incoming request can be rapidly recreated for analysis later.
A UAC, OTOH, can simply generate a CLF that has a list of requests
that it sent out and the responses it received for those
requests.

> above vs. in vs. below a transaction layer, 

Below the transaction layer is the parser -- probably need to
generate CLF at or above the transaction layer.

> before or after Privacy anonymization, etc.

Sure.

Thanks,

- vijay

---

From: Hadriel Kaplan <[email-masked]>
Date: Fri, 24 Jul 2009 18:58:08 -0400

> Sent: Friday, July 24, 2009 5:32 PM
> 
> Hadriel Kaplan wrote:

Yes, of course.  You do know I work for an SBC vendor, right? :)
SBC's do anomaly detection both at the parser layer _and_ at higher layers, and have been for years.  Some of us just don't call the higher layer things "anomaly detection" - because we market it more on the actions we take once we detect an anomaly (like reject or log or blacklist), or the benefit to the customer (like "Fraud Prevention", "Hijack prevention", "DDoS protection", "SPIT Protection", etc.).  

But anyway, there are centralized "anomaly detectors" that purport to analyze "anomalies" too - I don't know what all they mean by that, but they've asked us to provide them SIP message feeds (which right now is basically everything).  If I was them and looked at a CLF, I'd want to get quite a bit more than what we define: Via's, Route's, Record-Route's, Refer-To's, P-Asserted/P-Preferred/Remote-Party-ID, Path's, Max-Forwards, Content-Type's, History-Info, and Diversion... just off the top of my head.  

And really, as an "anomaly detector" I would want to filter which headers I *don't* care-about, not which I do. 

-hadriel

---

From: Vijay Gurbani <[email-masked]>
Date: Sun, 26 Jul 2009 04:08:24 -0500

Hadriel Kaplan wrote:
> SBC's do anomaly detection both at the parser layer _and_ at higher
> layers, and have been for years.  Some of us just don't call the
> higher layer things "anomaly detection" - because we market it more
> on the actions we take once we detect an anomaly (like reject or log
> or blacklist), or the benefit to the customer (like "Fraud
> Prevention", "Hijack prevention", "DDoS protection", "SPIT
> Protection", etc.).

Do SBCs use learning algorithms to do such anomaly detection
or do policy based triggers form the foundation of such detection?
I suspect it is the latter.

Can existing SBC-based anomaly detection system be trained on
the infamous SIP PRACK state machine to detect anomalous behavior?
I suspect not -- though I may be wrong.  I am aware of some
preliminary work on training machines to recognize the temporal
association of SIP messages in a dialog (i.e., BYE is preceded by
an INVITE); but so far this work is just starting and the more
complex cases like it is okay to send an UPDATE and CANCEL
before an INVITE finishes, but one must not send a SUBSCRIBE
in similar scenarios are not supported.  For an SBC to do such
analysis, an awful lot of dialog state would be needed; OTOH,
doing such detection at the individual SIP actor level (i.e.,
a proxy receiving such requests from UAs and producing a CLF
record for analysis) may be more manageable.

> But anyway, there are centralized "anomaly detectors" that purport to
> analyze "anomalies" too - I don't know what all they mean by that,
> but they've asked us to provide them SIP message feeds (which right
> now is basically everything).  If I was them and looked at a CLF, I'd
> want to get quite a bit more than what we define: Via's, Route's,
> Record-Route's, Refer-To's, P-Asserted/P-Preferred/Remote-Party-ID,
> Path's, Max-Forwards, Content-Type's, History-Info, and Diversion...
> just off the top of my head.

I suspect that what we have been discussing in the mailing list, i.e.,
having a base set of headers and an extensibility model to account
for other headers is generally a good thing.  Reverting to logging
the whole SIP message seems to be throwing our hands up in the air
and saying that we don't really know what we want so we'll save the
whole kitchen sink just in case.  HTTP CLF has worked without
saving the whole message, and with some judicious thought I think
we can make SIP CLF work as well.

> And really, as an "anomaly detector" I would want to filter which
> headers I *don't* care-about, not which I do.

Possibly; however, for training and subsequent use of an anomaly
system, you would need the headers which you care about (i.e., if
two colluding UAs leak information by subscribing to event package
X but sending notifications for event package Y, I'd want to log
the appropriate headers that I care about.)

Thanks,

- vijay
-- 
Vijay K. Gurbani, Bell Laboratories, Alcatel-Lucent
1960 Lucent Lane, Rm. 9C-533, Naperville, Illinois 60566 (USA)
Email: vkg@{alcatel-lucent.com,bell-labs.com,acm.org}
WWW:   http://ect.bell-labs.com/who/vkg

---

From: Hadriel Kaplan <[email-masked]>
Date: Sun, 26 Jul 2009 08:24:20 -0400

> From: Vijay Gurbani [mailto:[email-masked]]
> Sent: Sunday, July 26, 2009 5:08 AM
> 

As far as I know, many of them still use policy-based triggers in practice - but "learning algorithms" for the parser-layer detection and for some load-based behavior limits have been either claimed or hinted-at by some SBC vendors for a while, including the company I work for. (but what each of us do in that space in practice is not public info for competitive reasons) 

> analysis, an awful lot of dialog state would be needed...

SBC's can and usually do maintain full dialog state for all dialogs (as a b2bua), and enforce protocol compliance for messages and their state machines, with knobs for various control.  That was one of the basic tenets from the beginning.  Of course whether the SBC's do it correctly, or whether they should be doing it to begin with, is a different question. ;)  It's always been a fine line, though, between enforcing protocol rules for security, and letting sessions succeed when they can or even fixing them so they do succeed.  One man's anomalous signaling, is another man's paying customer.

> for other headers is generally a good thing.  

Yup.

> Reverting to logging

It's not that I don't think CLF has value - it's the central claim that's it's for "anomaly detectors" that makes me wince.  Obviously it can be used for some anomaly purposes - just that we shouldn't be saying that's its main purpose/use-case, because it's not good enough for it, imho.

It's sorta like the sip-ipfix draft, which has as one of its use-cases "billing".  While it may be technically possible to do billing for very trivial cases using a CLF formatted as ipfix, it is missing so much meta-data required for billing in the real-world that the claim is distracting because it's so clearly not useable for that.

-hadriel

---

From: Vijay Gurbani <[email-masked]>
Date: Sun, 26 Jul 2009 07:45:51 -0500

Hadriel Kaplan wrote:
> It's not that I don't think CLF has value - it's the central claim
> that's it's for "anomaly detectors" that makes me wince.  Obviously
> it can be used for some anomaly purposes - just that we shouldn't be
> saying that's its main purpose/use-case, because it's not good enough
> for it, imho.

I agree; I don't think it was claimed that anomaly detection is the
main purpose of SIP CLF.  It is but one possible use case if we
were to have a nicely digested information in the form of a CLF.

Thanks,

- vijay