Subject: Re: [sip-ops] [dispatch] SIP-CLF: Results on ASCII vs. binary representation

From: "Dean Willis" <[email-masked]>
Date: Sun, 3 May 2009 11:33:35 -0500

Right. In my experience, binary offers only marginal performance over structured ASCII. Structure can be applied either through fixed lengths or through encoded length indicators. The encoded length indicators can also be fixed length, or structured in some other easy way. This can produce the benefits of human-readable messages combined with rapid parsing, with the only overheads relative to binary being slightly larger records and some string-to-number conversions.

--
Dean Willis

-original message-
Subject: Re: [sip-ops] [dispatch] SIP-CLF: Results on ASCII vs. binary representation
From: Tom Taylor <[email-masked]>
Date: 05/02/2009 05:09

How about adding length fields to ASCII?

---

From: Adam Roach <[email-masked]>
Date: Mon, 04 May 2009 14:58:38 -0500

Dean Willis wrote:
> Right. In my experience, binary offers only marginal performance over structured ASCII.

I'll correct this slightly: binary offers only marginal performance over 
*properly* structured ASCII. I argue that the Apache log format doesn't 
qualify for this description. (I say this after having to deal with 
processing relatively large Apache logs, and suffering from the vast 
quantities of time such processing takes).

I'm okay with Tom's suggestion to add ASCII length (and tag?) fields to 
the text format. If people want to take it that direction, I could 
re-cast my proposed format so it contains fixed-length ASCII number 
representations for pointers, lengths, and tags; and so that it 
terminates records with something like an ASCII 0x0D.

/a

---

From: Simon Perreault <[email-masked]>
Date: Mon, 04 May 2009 16:03:26 -0400

> terminates records with something like an ASCII 0x0D.

+1, but with 0x0A (newline) instead of 0x0D (carriage return).

Simon
-- 
STUN/TURN server    --> http://numb.viagenie.ca
Interplanetary news --> http://reeves.viagenie.ca
vCard 4.0           --> http://www.vcarddav.org

---

From: Dean Willis <[email-masked]>
Date: Mon, 4 May 2009 20:44:41 -0500

On May 4, 2009, at 2:58 PM, Adam Roach wrote:

> I'll correct this slightly: binary offers only marginal performance  
> over *properly* structured ASCII. I argue that the Apache log format  
> doesn't qualify for this description. (I say this after having to  
> deal with processing relatively large Apache logs, and suffering  
> from the vast quantities of time such processing takes).
>
> I'm okay with Tom's suggestion to add ASCII length (and tag?) fields  
> to the text format. If people want to take it that direction, I  
> could re-cast my proposed format so it contains fixed-length ASCII  
> number representations for pointers, lengths, and tags; and so that  
> it terminates records with something like an ASCII 0x0D.

One could design things such that each record contains an initial  
"structure" filed with a fixed-format  series of indices (total  
length, number of fields, lengths for each field, followed by  
variable--length data fields, tab (or other reserved char) delimited  
followed by a CRLF terminator (included in total-length index). This  
gives us records that are fast-parseable, but still human-readable.

--
Dean

---

From: Jiri Kuthan <[email-masked]>
Date: Tue, 05 May 2009 08:07:17 +0200

> quantities of time such processing takes).

in my experience, text processing is VERY CPU-consuming. (may be negligible
if the actual application is TREMENDOUSLY CPU-consuming) Both for writing
(can be somewhat improved by a performance-oriented formatting library,
most c-lib implementations are so general-purpose that performance is not
considered) and reading.

> terminates records with something like an ASCII 0x0D.

database guys like to use index file. (I think I see myself as a database
guy in this context :-))

-jiri

> 
> /a
> _______________________________________________
> sip-ops mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/sip-ops
>