Subject: Re: [sip-clf] [sip-ops] [dispatch] SIP-CLF: Results on ASCII vs. binary representation

From: Adam Roach <[email-masked]>
Date: Tue, 05 May 2009 00:38:09 -0500

On 05/04/2009 08:44 PM, Dean Willis wrote:
>
> On May 4, 2009, at 2:58 PM, Adam Roach wrote:
>
>> Dean Willis wrote:
>>> Right. In my experience, binary offers only marginal performance 
>>> over structured ASCII.
>>
>> I'll correct this slightly: binary offers only marginal performance 
>> over *properly* structured ASCII. I argue that the Apache log format 
>> doesn't qualify for this description. (I say this after having to 
>> deal with processing relatively large Apache logs, and suffering from 
>> the vast quantities of time such processing takes).
>>
>> I'm okay with Tom's suggestion to add ASCII length (and tag?) fields 
>> to the text format. If people want to take it that direction, I could 
>> re-cast my proposed format so it contains fixed-length ASCII number 
>> representations for pointers, lengths, and tags; and so that it 
>> terminates records with something like an ASCII 0x0D.
>
>
> One could design things such that each record contains an initial 
> "structure" filed with a fixed-format  series of indices (total 
> length, number of fields, lengths for each field, followed by 
> variable--length data fields, tab (or other reserved char) delimited 
> followed by a CRLF terminator (included in total-length index). This 
> gives us records that are fast-parseable, but still human-readable.
>

Yes. That is what I was proposing.

Followups to the sip-clf list (not sip-ops!).

/a

---

From: "Vijay K. Gurbani" <[email-masked]>
Date: Tue, 05 May 2009 11:02:41 -0500

Adam Roach wrote:
dean> One could design things such that each record contains an initial
dean> "structure" filed with a fixed-format  series of indices (total
dean> length, number of fields, lengths for each field, followed by
dean> variable--length data fields, tab (or other reserved char) delimited
dean> followed by a CRLF terminator (included in total-length index). This
dean> gives us records that are fast-parseable, but still human-readable.
> Yes. That is what I was proposing.

Adam: Is it possible to get a draft out on this format for
further discussion?  I am willing to pitch in and help text
out a draft with you, Dean and anyone else interested on the
list.

There are further advantages to such a format over the simple
space-delimited ASCII format, which would need some escaping
to parse the Contact URI in the CLF file:
    sip:<[email-masked]>;param=" value1 value2 "

Thanks,

- vijay
-- 
Vijay K. Gurbani, Bell Laboratories, Alcatel-Lucent
1960 Lucent Lane, Rm. 9C-533, Naperville, Illinois 60566 (USA)
Email: vkg@{alcatel-lucent.com,bell-labs.com,acm.org}
Web:   http://ect.bell-labs.com/who/vkg/

---

From: Adam Roach <[email-masked]>
Date: Tue, 05 May 2009 12:01:18 -0500

> out a draft with you, Dean and anyone else interested on the

Sure. I'm pretty busy until late this week, but I'll try to get 
something out as quickly as other work allows.

/a