Subject: [Trans] Parsing existing logs entries.

From: Kurt Roeckx <[email-masked]>
Date: Thu, 29 May 2014 19:59:12 +0200

Hi,

I've been tryint to parse the existing log file based on the
description in RFC 6962.  I'm using Google's pilot log as example.

The RFC says that GET will give you:
         leaf_input:  The base64-encoded MerkleTreeLeaf structure.

         extra_data:  The base64-encoded unsigned data pertaining to the
            log entry.  In the case of an X509ChainEntry, this is the
            "certificate_chain".  In the case of a PrecertChainEntry,
            this is the whole "PrecertChainEntry".

With MerkleTreeLeaf:
       struct {
           Version version;
           MerkleLeafType leaf_type;
           select (leaf_type) {
               case timestamped_entry: TimestampedEntry;
           }
       } MerkleTreeLeaf;

Which uses the following types:
       enum { v1(0), (255) }
         Version;

       enum { timestamped_entry(0), (255) }
         MerkleLeafType;

       struct {
           uint64 timestamp;
           LogEntryType entry_type;
           select(entry_type) {
               case x509_entry: ASN.1Cert;
               case precert_entry: PreCert;
           } signed_entry;
           CtExtensions extensions;
       } TimestampedEntry;

       enum { x509_entry(0), precert_entry(1), (65535) } LogEntryType;

       opaque ASN.1Cert<1..2^24-1>;

       opaque CtExtensions<0..2^16-1>;

So after base64 decoding things, and turning it to hex, I see:
0000000 00 00 00 00 01 3d a4 07 ab d0 00 00 00 05 09 30
0000020 82 05 05 30 82 03 ed a0 03 02 01 02 02 11 00 91

So my guess of parsing it is:
MerkleTreeLeaf {
	version = 00 (v1)
	leaf_type = 00 (timestamped_entry)
	TimestampedEntry {
		timestamp = 00 00 01 3d a4 07 ab d0 (1364256598992, 2013-03-26 00:09:58.992)
		entry_type = 00 00 (x509_entry)
		ASN.1Cert = 00 05 09 30 82 05 05 30 [...]

And I seem to have been gotton lost somewhere.  That doesn't look
like a valid X509 certificate, nor do I have any idea how long it
is.

If I had to guess "00 05 09" would be a length, and that might
work since it's limited to 2^24-1 bytes, so let's just assume
that's the case for now.

Now let's try to look at the extra_data.  That is:
       struct {
           ASN.1Cert leaf_certificate;
           ASN.1Cert certificate_chain<0..2^24-1>;
       } X509ChainEntry;

And we have this hex data:
0000000 00 09 29 00 04 e9 30 82 04 e5 30 82 03 cd a0 03
0000020 02 01 02 02 10 07 6f 12 46 81 45 9c 28 d5 48 d6

So with what we learned from the previous attempt,  it looks like
we need to parse it like:
certificate_chain {
	length = 00 09 29 (2345)
	ASN.1Cert {
		length = 00 04 e9 (1257)
		data = 30 82 04 e5 30 82 03 cd a0 03 [...] }
	ASN.1Cert { [...]

I think the RFC needs some clarifications on how to parse things,
so that I don't have to guess.

This definition is also confusing:
         extra_data:  The base64-encoded unsigned data pertaining to the
            log entry.  In the case of an X509ChainEntry, this is the
            "certificate_chain".  In the case of a PrecertChainEntry,
            this is the whole "PrecertChainEntry".  

So the X509ChainEntry is not used, you need to directly go to it's
member certificate_chain.  The LogEntry also isn't used anywhere.

Kurt

---

From: Rob Stradling <[email-masked]>
Date: Thu, 29 May 2014 21:18:37 +0100

Kurt, RFC6962 says...

"1.2.  Data Structures

    Data structures are defined according to the conventions laid out in
    Section 4 of [RFC5246]."

So you shouldn't have to guess.  Just read RFC5246 Section 4.

On 29/05/14 18:59, Kurt Roeckx wrote:
> _______________________________________________
> Trans mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/trans
>

-- 
Rob Stradling
Senior Research & Development Scientist
COMODO - Creating Trust Online
Office Tel: +44.(0)1274.730505
Office Fax: +44.(0)1274.730909
www.comodo.com

COMODO CA Limited, Registered in England No. 04058690
Registered Office:
   3rd Floor, 26 Office Village, Exchange Quay,
   Trafford Road, Salford, Manchester M5 3EQ

This e-mail and any files transmitted with it are confidential and 
intended solely for the use of the individual or entity to whom they are 
addressed.  If you have received this email in error please notify the 
sender by replying to the e-mail containing this attachment. Replies to 
this email may be monitored by COMODO for operational or business 
reasons. Whilst every endeavour is taken to ensure that e-mails are free 
from viruses, no liability can be accepted and the recipient is 
requested to use their own virus checking software.

---

From: Kurt Roeckx <[email-masked]>
Date: Thu, 29 May 2014 22:34:04 +0200

> So you shouldn't have to guess.  Just read RFC5246 Section 4.

Thanks, I seems to have overlooked that section.

Anyway, I really don't understand why this is such a mixture of
josn and base64 encoded things.  Why isn't this all just json with
just the certificate in base64?

Kurt

---

From: Rob Stradling <[email-masked]>
Date: Fri, 30 May 2014 11:17:20 +0100

> just the certificate in base64?

Personal preference of the authors of RFC6962?

---

From: Ben Laurie <[email-masked]>
Date: Fri, 30 May 2014 11:18:58 +0100

> just the certificate in base64?

This has come up before. The data appears in multiple contexts. Each of
those contexts has an "obvious" encoding - namely, DER, TLS structure
encoding and JSON. In general, we tried to pick the most natural encodings
depending on the context the data is really "for". Possibly we didn't get
it exactly right. I'm sure whatever we do, someone will be unhappy.

-- 
Certificate Transparency is hiring! Let me know if you're interested.