Subject: Re: Thoughts on firewall performance draft

From: [email-masked] (Kevin Dubray)
Date: Fri, 26 Jun 1998 15:25:00 -0400

>From [email-masked] Fri Jun 26 15:12:50 1998

> > Bit forwarding rate
> > The number of bits per second of allowed traffic a firewall can be observed
> > to transmit successfully to the correct destination interface in response
> > to a specified offered load.
> >
>
> Alternatively, you might try:
>
>  The number of bits per second of traffic a firewall can be observed
>  to appropriately transmit to the correct destination interface(s) in response
>  to a specified offered load.
>
> This wording may help the move the burden from the user identifying classes of
> traffic to a firewall discriminating what is to be forwarded - illegal,
> blocked, malformed, wonderfully hunky-dory, or whatever.
>
> What constitutes appropriateness for the collection of this metric can be
> defined in the companion methodology document.
>
Either definition works for me. Note that the terminology draft does
define three classes of traffic: allowed, rejected, and (in the
forthcoming version) illegal.

I've received several inquiries in private e-mail about whether the bits
in question constitute payload only or entire datagrams. My strong
preference is that the latter be measured and I'd suggest adding
language to that effect in the discussion of this definition.

> >
> > Connection
> > A state in which nodes agree to exchange data using a known protocol.
>
>
> This is wonderful as clarifying concept or term on which to build.  As a
> stand-alone metric, I'm not so sure .  You referred to measuring connection
> concurrency.  This is something that I can and probably should measure wrt
> firewalls.  A connection overhead metric may yield better insight how an IUT
> "lifts" or handles connection maintainance.
>
Kevin, I'd like a clarification of what's meant by "connection
overhead." The memory cost of opening a socket is
implementation-specific, and a DUT/SUT that uses 100 kbytes of memory
for each connection may or may not move bits on the wire more slowly
than one requiring only 10 kbytes. That's a firewalls internal issue,
and probably not all that relevant to an end-to-end benchmark. Or did
you mean something else here?

Another option is to make the connection definition(s) very specific to
the workings of TCP, UDP, and ICMP, and count only those conversations
which have successfully navigated the setup procedures of each protocol.
I'm reluctant to do that; better to have a more general definition.

dn

---

From: "Michael C. Richardson" <[email-masked]>
Date: Fri, 26 Jun 1998 16:22:26 -0400

>>>>> "Kevin" == Kevin Dubray <[email-masked]> writes:
    >> This is wonderful as clarifying concept or term on which to build.  As
    >> a stand-alone metric, I'm not so sure .  You referred to measuring
    >> connection concurrency.  This is something that I can and probably
    >> should measure wrt firewalls.  A connection overhead metric may yield
    >> better insight how an IUT "lifts" or handles connection maintainance.

    Kevin> Kevin, I'd like a clarification of what's meant by "connection
    Kevin> overhead." The memory cost of opening a socket is

  Given n current connections, how much time does it take for the n+1'th
connection to be opened.

    Kevin> implementation-specific, and a DUT/SUT that uses 100 kbytes of
    Kevin> memory for each connection may or may not move bits on the wire
    Kevin> more slowly than one requiring only 10 kbytes. That's a firewalls

  The internals issue is more to do with the kind of data structure that
is used to store the connection state. Also, whether or not a new process
needs to be created to handle that connection.

    Kevin> Another option is to make the connection definition(s) very
    Kevin> specific to the workings of TCP, UDP, and ICMP, and count only
    Kevin> those conversations which have successfully navigated the setup
    Kevin> procedures of each protocol.  I'm reluctant to do that; better to
    Kevin> have a more general definition.

  ICMP isn't a protocol in the same way TCP and UDP are. TCP and UDP make use
of IP services. Some of these services are carried by ICMP datagrams.
  A firewall that doesn't consider an ICMP MUST FRAGMENT to be associated
with a TCP connection (i.e. Path MTU discovery) is violating the IP protocol.
[The firewall needn't let it in: it could process it locally, adjusting
its own view of fragment size and perhaps emitting an ICMP out the other
end.]

  The only ICMP that may consistute a protocol *may* be ECHO/ECHO RESPONSE.

   :!mcr!:            |  Sandelman Software Works Corporation, Ottawa, ON  
   Michael Richardson |	SSH IPsec: http://www.ssh.fi/. Secure, strong, international
 Personal: <A HREF="http://www.sandelman.ottawa.on.ca/People/Michael_Richardson/Bio.html">[email-masked]</A>. PGP key available.
 Corporate: <A HREF="http://www.sandelman.ottawa.on.ca/SSW/">[email-masked]</A>.