Subject: [Trans] What's the load on a CT log?

From: Ben Laurie <[email-masked]>
Date: Thu, 13 Mar 2014 16:06:06 +0000

Several people have asked me this recently. Here's a nice way to estimate load.

Let's assume a single log that takes all the load.

Firstly, we see about 5,000 new certificates a day, so that's around
0.06 new certificates per second. Clearly a trivial load.

Next is load from audit (i.e. from browsers that wish to validate SCTs
accompanying certificates they see). Given some assumptions, we can
calculate the load from audit.

* Clients cache audit results.

* There are approximately b = 2.5B browsers in the world
(http://www.internetworldstats.com/stats.htm).

* The average user visits w = 89 websites a month
(http://www.creditloan.com/blog/how-the-world-spends-its-time-online/
quoting a Nielsen report). Assume these are all TLS sites.

* Assume a certificate lifetime of l = 12 months.

So, each user sees w / l new certificates a month. Each new
certificate needs to be audited, which means in practice, three web
operations (fetch STH, fetch STH consistency proof, fetch SCT
inclusion proof) - it might be a good idea to create a new API to do
all three in one go.

So, total average load is 3 * b * w / l ~ 20,000 web fetches per
second. If we optimise the API we can get that down to 7,000 qps. Each
query (in the optimised case) would be around 3 kB, which gives a
bandwidth of around 150 kb/s.

Monitors add extra load, but should only be at around the new
certificate rate - i.e. ~ .06 * number of monitors fetches per second.

IMO, this is achievable on a single machine (modulo reliability), with
some care. Clearly not a vast farm, however its done.

In practice, no one log would have to take this full load, this is a
worst case analysis.

---

From: Daniel Kahn Gillmor <[email-masked]>
Date: Thu, 13 Mar 2014 12:31:37 -0400

On 03/13/2014 12:06 PM, Ben Laurie wrote:
> second. 

This part i follow (you're switching temporal units between months and
years and seconds, but i get roughly the same final figures)

> If we optimise the API we can get that down to 7,000 qps. Each
> query (in the optimised case) would be around 3 kB, 

And i agree this seems like a win.  Why was the API broken into three
parts instead of the complete proof originally?  what (other than
conceptual cleanliness) might we lose by creating the optimized API?

> which gives a bandwidth of around 150 kb/s.

This looks off by a few orders of magnitude to me.  7kqps and 3kB/q
gives me 7000*3000*8 bits per second, which is 168Mbps.  Am i missing
something?

Should we be considering swarm-based distribution of this kind of data,
or hierarchical proxying for load distribution?

	--dkg

---

From: Ben Laurie <[email-masked]>
Date: Thu, 13 Mar 2014 16:51:20 +0000

> gives me 7000*3000*8 bits per second, which is 168Mbps.  Am i missing

Sorry, you are correct - I meant 150,000 kb/s!

> or hierarchical proxying for load distribution?

Maybe :-)

One thing we're working on is distributing the proofs via DNS, which
is obviously of exactly that nature - which would definitely reduce
bandwidth at the servers.

---

From: Ben Laurie <[email-masked]>
Date: Thu, 13 Mar 2014 17:54:15 +0000

> conceptual cleanliness) might we lose by creating the optimized API?

Forgot to answer this part - it was broken up purely for conceptual
cleanness. I'd probably leave the independent APIs but introduce a new
do-everything-at-once API.

There are actually good reasons to combine them anyway when you're
looking at a load-balanced setup (skew between servers may cause some
calls to fail - e.g. I get an STH from one server, then try to get
audit against that STH from another that is slightly behind, which
will fail).

---

From: Rob Stradling <[email-masked]>
Date: Thu, 13 Mar 2014 20:27:05 +0000

I'm not sure average load tells the whole story.

Won't there be a surge in audit traffic in the aftermath of a busy site 
installing a new cert?

On 13/03/14 16:06, Ben Laurie wrote:
> _______________________________________________
> Trans mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/trans
>

-- 
Rob Stradling
Senior Research & Development Scientist
COMODO - Creating Trust Online
Office Tel: +44.(0)1274.730505
Office Fax: +44.(0)1274.730909
www.comodo.com

COMODO CA Limited, Registered in England No. 04058690
Registered Office:
   3rd Floor, 26 Office Village, Exchange Quay,
   Trafford Road, Salford, Manchester M5 3EQ

This e-mail and any files transmitted with it are confidential and 
intended solely for the use of the individual or entity to whom they are 
addressed.  If you have received this email in error please notify the 
sender by replying to the e-mail containing this attachment. Replies to 
this email may be monitored by COMODO for operational or business 
reasons. Whilst every endeavour is taken to ensure that e-mails are free 
from viruses, no liability can be accepted and the recipient is 
requested to use their own virus checking software.

---

From: Ben Laurie <[email-masked]>
Date: Thu, 13 Mar 2014 21:48:05 +0000

> I'm not sure average load tells the whole story.

Agreed.

> Won't there be a surge in audit traffic in the aftermath of a busy site

Yes.

> This e-mail and any files transmitted with it are confidential and intended
> solely for the use of the individual or entity to whom they are addressed.
> If you have received this email in error please notify the sender by
> replying to the e-mail containing this attachment. Replies to this email may
> be monitored by COMODO for operational or business reasons. Whilst every
> endeavour is taken to ensure that e-mails are free from viruses, no
> liability can be accepted and the recipient is requested to use their own
> virus checking software.

---

From: Eran Messeri <[email-masked]>
Date: Fri, 14 Mar 2014 10:02:42 +0000

> > Won't there be a surge in audit traffic in the aftermath of a busy site
This is where separate calls may be beneficial - there's a higher chance
that a response to each of the individual calls will be cached rather than
a single call that is a response to a client request which contains an STH.

> >> Several people have asked me this recently. Here's a nice way to
> estimate
> >> load.
> >>
> intended
> You received this message because you are subscribed to the Google Groups
> "certificate-transparency" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to [email-masked].
> For more options, visit https://groups.google.com/d/optout.
>