Subject: [sip-clf] SIPCLF - Questions from implementations...

From: Peter Musgrave <[email-masked]>
Date: Sun, 19 Sep 2010 11:14:50 -0400

Hi,

I am doing an implementation experiment of indexed ASCII and IPFIX and
have found a few things which I need to clarify.

1) To/From
IPFIX records the tags and URIs as separate elements (but the problem
statement and indexed ascii record the whole header line).
Issue: Is IPFIX required to record the datamodel element *exactly* or
is it free to represent the information in a more sub-divided form if
it wishes. (Same issue exists for ip:port:xport)

The description in the problem statement says "URI including tag",
however in the indexed ASCII example the display name is included.
Issue: If display name is desired - then the problem statement needs
to state this?  (As I read 3261 the SIP-URI does not include the
display name but does include URI parameters)

2) ip:port:xport
Does it make sense to record transport for both source and
destination? Is the datamodel just trying to record the actual
transport which was used. If so then should there just be one value
for transport and the sent/received field indicates which it applies
to?

3) Indexed ASCII Example
The example shows comma delimiters in the pointer block as well as in
the uuencoded snippet (but the text and definition do not). Are the
commas intended (to facilitate processing)?

4) Retransmission flag in Indexed ASCII
I could not find a requirement for this in the problem statement.
Since it appears to be optional should it be in the TLV block and not
the flags block?

Once I can resolve these questions I can then log in both formats and
report on typical sizes and provide some sample log files. Hopefully
this will help inform the group about the formats.

Thanks,

Peter Musgrave
(as individual)

---

From: Hadriel Kaplan <[email-masked]>
Date: Sun, 19 Sep 2010 20:44:46 -0400

> it wishes. (Same issue exists for ip:port:xport)

It was an open question we had raised on the mailing list as well - whether the uri:tags and ip:port:xport were actually distinct fields or not.  (see http://www.ietf.org/mail-archive/web/sip-clf/current/msg00214.html)
In my view, they are each separate fields, and indexed-ascii just happens to encode/record the separate fields in multi-piece aggregate blocks.  At least for the ip:port:xport ones.  For the URI and tags, see below.

> display name but does include URI parameters)

The problem statement is wrong either way - the To/From tags are not URI params, they're header params.  So either we want the whole To/From header fields, or we want a URI and a tag header param (which would be without the display-name).  I can see an argument for either way, but I thought recording the URI and tags separately made more sense if we want the tags for some specific purpose and we don't expect collectors of the CLF records to understand SIP ABNF parsing rules to get them.

> for transport and the sent/received field indicates which it applies

It makes sense to record both the L3/4 source and destination addr/ports - recording the remote peer's is a no-brainer, and you need the local device's as well since it can easily be a multi-interface device that's recording it and you don't know which interface received/sent it without this. (arguably even an ip:port is insufficient to distinguish interfaces on a system if it supports the same IP on multiple interfaces, but it's close enough I guess)

> Since it appears to be optional should it be in the TLV block and not

That used to be one of the flags from early on, and I don't remember why it wasn't kept as a mandatory flag.  Maybe because stateless proxies won't know if it's a retrans or not?  Or maybe because we didn't want to have to force the CLF recording layer to be implemented in code at a point such a flag could be set? (i.e., if you're recording this during parsing/early on reception you won't know if it's a retrans yet)

-hadriel

---

From: "Vijay K. Gurbani" <[email-masked]>
Date: Mon, 20 Sep 2010 09:01:51 -0500

> have found a few things which I need to clarify.

Peter: Please see inline.

> it wishes. (Same issue exists for ip:port:xport)

 From the data model point of view, these should be separate elements.
A representation draft may choose to concatenate these for better
presentability, as indexed-ASCII does.

> display name but does include URI parameters)

The display name is not included in the examples of problem-statement.
I just scanned them once again and this is indeed the case.  If I
inadvertently missed a display name, please let me know the exact
step of the figure where the display name appears.

Hadriel, in his follow-up email on this subject, indicated:

> The problem statement is wrong either way - the To/From tags are not
> URI params, they're header params.  So either we want the whole
> To/From header fields, or we want a URI and a tag header param (which
> would be without the display-name).  I can see an argument for either
> way, but I thought recording the URI and tags separately made more
> sense if we want the tags for some specific purpose and we don't
> expect collectors of the CLF records to understand SIP ABNF parsing
> rules to get them

Indeed, the tag is a header parameter.  More specifically, to answer
Hadriel's question above, we need the To/From URI and tag header
parameter.  I will make this clear in the problem-statement draft.

> for transport and the sent/received field indicates which it applies

No.  In SIP, a request could come into a proxy over UDP but go out
over TCP.  So, if we want to record the transports, then we
should record them separately for source and destination.  BTW, the
transports were added because an earlier review from you had
indicated an interest in them [1], and I thought it was reasonable
to include them (certainly does not hurt.)  If you'd rather not
have them any more, I am just as happy to take them out of the
problem-statement.

> commas intended (to facilitate processing)?

Offhand, I do not recall why the header block is comma delimited.
I will check with Gonzalo.

> Since it appears to be optional should it be in the TLV block and not

The original indexed-ASCII draft from Adam always had the retransmission
flag.  When I had proposed SIPCLF, I did not see the need to log
retransmissions.  Since the current indexed-ASCII approach is
based on the old version from Adam, the retransmission flag is
carried over.

I am open to whether or not we need to log the retransmission bit.

Thanks,

[1] http://www.ietf.org/mail-archive/web/sip-clf/current/msg00184.html

- vijay
-- 
Vijay K. Gurbani, Bell Laboratories, Alcatel-Lucent
1960 Lucent Lane, Rm. 9C-533, Naperville, Illinois 60566 (USA)
Email: vkg@{alcatel-lucent.com,bell-labs.com,acm.org}
Web:   http://ect.bell-labs.com/who/vkg/

---

From: Peter Musgrave <[email-masked]>
Date: Mon, 20 Sep 2010 10:30:30 -0400

I think I understand both your and Hadriel's responses.

My question about transport was badly phrased. I mean to ask that
since the entry in the log file is for a specific request or response,
should not the transport reflect the actual transport used for sending
or receiving the request or response? It seems to me it is a single
attribute of the log line (and not tied to the remote or local
address).

If you receive it records the transport it came in on.
If you send - it indicates which transport it was sent with.
You are never doing both in the same log line, so I think we only need
one - decoupled from source and dest.

Thanks,

Peter

On Mon, Sep 20, 2010 at 10:01 AM, Vijay K. Gurbani <[email-masked]> wrote:
> _______________________________________________
> sip-clf mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/sip-clf
>

---

From: Hadriel Kaplan <[email-masked]>
Date: Mon, 20 Sep 2010 10:45:31 -0400

Oh, right, now I grok you.  You're saying the transport indication in particular (rather than the ip:port) is always the same, since clearly if the peer's transport was TCP, your transport is TCP, for that message.
Sure.

-hadriel

---

From: "Vijay K. Gurbani" <[email-masked]>
Date: Mon, 20 Sep 2010 11:14:08 -0500

> one - decoupled from source and dest.

OK, I see what you mean.  Yes, in that case the transport is an
invariant on the entire log line and can be specified only once.

Thanks,

- vijay

---

From: Gonzalo Salgueiro <[email-masked]>
Date: Tue, 21 Sep 2010 03:07:19 -0400

Peter - 

Please excuse the delayed response but we just welcomed our third child on Wednesday and that has consumed my days and (sadly) my nights.

Adding to Vijay's comments, please see inline....

> I will check with Gonzalo.

This was a manual decoding error in the last draft and has since been corrected in the -02 draft that I have not published yet.  Here is proper decode of that same message in case you need it for your implementation of indexed-ASCII CLF:

A000174,Ros,
0070000D007800120091001300A5001400BA001800D3002C0100000E01090001011100010113000E0122

1278629563.323  314159 INVITE   sip:[email-masked]      
biloxi.com:5060:udp     atlanta.com:5060:udp    
Bob <sip:[email-masked]>        
Alice <sip:[email-masked]>;tag=1928301774    
a84b4c76e66710  -       -       z9hG4bKnashds8  
0000,001C,<sip:[email-masked]>  
0001,000C,192.168.1.15

begin-base64 644 sip_clf.txt
QTAwMDE2MCxSb3MsMDA3MDAwMEQwMDc4MDAxMjAwOTEwMDEzMDBBNTAwMTQwMEJBMDAxODAwRDMw
MDJDMDEwMDAwMEUwMTA5MDAwMTAxMTEwMDAxMDExMzAwMEUwMTIyCjEyNzg2Mjk1NjMuMzIzCTMx
NDE1OSBJTlZJVEUJc2lwOmJvYkBiaWxveGkuY29tCWJpbG94aS5jb206NTA2MDp1ZHAJYXRsYW50
YS5jb206NTA2MDp1ZHAJQm9iIDxzaXA6Ym9iQGJpbG94aS5jb20+CUFsaWNlIDxzaXA6YWxpY2VA
YXRsYW50YS5jb20+O3RhZz0xOTI4MzAxNzc0CWE4NGI0Yzc2ZTY2NzEwCS0JLQl6OWhHNGJLbmFz
aGRzOAkwMDAwLDAwMUMsPHNpcDphbGljZUBwYzMzLmF0bGFudGEuY29tPgkwMDAxLDAwMEMsMTky
LjE2OC4xLjE1Cg==
====

> I am open to whether or not we need to log the retransmission bit.

I have no real preference for whether the we want the retransmission bit present in the flags field or not. I will note that I see little to no need to log that information, but am open to the group decision.

Regards,

Gonzalo