Subject: [tsvwg] Brian Haberman's Discuss on draft-ietf-tsvwg-circuit-breaker-13: (with DISCUSS)

From: "Brian Haberman" <[email-masked]>
Date: Mon, 14 Mar 2016 07:06:02 -0700

Brian Haberman has entered the following ballot position for
draft-ietf-tsvwg-circuit-breaker-13: Discuss

When responding, please keep the subject line intact and reply to all
email addresses included in the To and CC lines. (Feel free to cut this
introductory paragraph, however.)

Please refer to https://www.ietf.org/iesg/statement/discuss-criteria.html
for more information about IESG DISCUSS and COMMENT positions.

The document, along with other ballot positions, can be found here:
https://datatracker.ietf.org/doc/draft-ietf-tsvwg-circuit-breaker/

----------------------------------------------------------------------
DISCUSS:
----------------------------------------------------------------------

This is a "let's talk about it" DISCUSS (i.e., may not require any
document changes) to make sure I understand what is being recommended...

1. Rules 12 and 13 in section 4 seem more pertinent to sender-based flows
than to tunnels. Is it best current practice to terminate all tunnel
traffic in the face of long-term congestion rather than attempt to
terminate/regulate the worst offenders within the tunnel? If the circuit
breaker capability is implemented within the tunnel ingress point (as is
recommended), it would seem straightforward to penalize the flow(s)
causing the congestion rather than all the flows. Or am I misinterpreting
the rules 12 and 13?

2. Does the need for these types of circuit breakers imply that raw UDP
(and UDP-Lite) is no longer sufficiently functional and that DCCP (or an
equivalent) needs to be used?

3. If this BCP is expecting to influence the implementation of traffic
sources and sinks, should there be some guidance on what should be used
(e.g., ECN) prior to invoking circuit breaker logic? The text says
numerous times that CBs are a last resort, but I don't see much in the
way of what should be used prior to this last resort.

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Mon, 14 Mar 2016 10:49:32 -0500

Hi, Brian,

> https://datatracker.ietf.org/doc/draft-ietf-tsvwg-circuit-breaker/

I'll let the shepherd/author field the other two questions, but on this
one, speaking as an interested observer - yes, if DCCP had seen significant
deployment, that would be brilliant. At least at first blush, it provides
additional information that would be helpful in detecting loss, which we
would interpret as congestion.

But we seem to be talking about tunnel endpoints on network equipment that
doesn't know how to do DCCP, and isn't likely to change on that.

In my opinion only, of course.

Spencer

---

From: Brian Haberman <[email-masked]>
Date: Mon, 14 Mar 2016 12:00:54 -0400

Hi Spencer,

> doesn't know how to do DCCP, and isn't likely to change on that.

So, two responses...

1. Would such a recommendation hold for hosts/endpoints using raw UDP?

2. Has anyone discussed such a recommendation with vendors who make
these intermediate tunnel ingree/egress devices?

Regards,
Brian

---

From: "Black, David" <[email-masked]>
Date: Mon, 14 Mar 2016 23:22:23 +0000

Picking up all three questions in one thread - inline ...

Thanks, --David

> -----Original Message-----
> From: Brian Haberman [mailto:[email-masked]]
> Sent: Monday, March 14, 2016 12:01 PM
> To: Spencer Dawkins at IETF
> Cc: The IESG; Black, David; [email-masked]; draft-ietf-tsvwg-circuit-
> [email-masked]; [email-masked]
> Subject: Re: Brian Haberman's Discuss on draft-ietf-tsvwg-circuit-breaker-13:
> (with DISCUSS)
> 
> Hi Spencer,
> 

That "it would seem straightforward" statement assumes that the individual
flows that are the worst offenders can be identified and individually terminated/
regulated by the tunnel endpoint implementations.  That assumption can easily
be incorrect for a tunneled traffic aggregate.  

I would say that if the tunnel implementation can identify and regulate
individual flows within a tunnel, then doing so is an alternative to regulating
the aggregate as a whole.  Even then, if the circuit breaker operates on (e.g.,
is triggered by the behavior of) the tunneled aggregate (and not each
individual tunneled-flow), then the overall requirements of Rules 12 and 13
apply to regulation of the aggregate, even if that is realized by regulating
individual "worst offender" flows within the aggregate.

As an alternative, there could be a circuit breaker per tunneled flow in the
aggregate, but that involves much more state for an aggregate consisting of
a large number of flows - see Section 3.1 for what is needed for each flow.

> 1. Would such a recommendation hold for hosts/endpoints using raw UDP?

Yes, middleboxes that won't pass DCCP traffic are common.  One does not need
a tunnel to encounter this problem.  DCCP-in-UDP (RFC 6773) is a possible approach
to this class of problem, but that adds another tunnel encapsulation header.

> these intermediate tunnel ingree/egress devices?

It's end systems in addition to intermediate devices.  Based on what I see, current
protocol design preference is to assume a Controlled Environment (see the rfc5405bis
draft in tsvwg), and not add congestion control or another header (e.g., see RFC 7510
on MPLS/UDP).  When that is done, a circuit breaker can help when unexpected
problems occur in that controlled environment - see Section 5.3 of this draft for a
couple of examples, and in particular this text from 5.3.1:

   Part of the rationale, was that high loss
   ratios typically indicated that something was "broken" and ought to
   have already resulted in operator intervention, and therefore need to
   trigger this intervention.
 
> >> way of what should be used prior to this last resort.

There are at least two senses of "last resort," design and operational. 

For protocol design, the preferred technique is congestion control, e.g., a congestion
controlled transport.  This is discussed in (among other places) the last two paragraphs
of Section 1, but could be made more explicit.  Protocol designers ought to be dealing
with congestion, but we have a lot of non-congestion-controlled protocols and traffic.

For network operations, an exhaustive list of specific mechanisms is infeasible, but
the general concept of operator action before excessive congestion becomes persistent
is discussed in the first new paragraph on p.4 ("Other mechanisms may also be available
...") as well as in the text quoted above from the example in section 5.3.1.

---

From: Brian Haberman <[email-masked]>
Date: Tue, 15 Mar 2016 08:07:07 -0400

Hi David,

> be incorrect for a tunneled traffic aggregate.  

Are you thinking of cases where the ingress' decision to tunnel is
software driven, but the forwarding is actually done in hardware?

> individual "worst offender" flows within the aggregate.

I agree that if the circuit breaker can only work on the aggregate, full
termination or drastic reduction may be the only solution. And upon
re-reading the rules, I do see that rule 10 says all contributors need
to be punished for persistent congestion, which contradicts my
hypothesis that only the worst contributors should be punished.

> to this class of problem, but that adds another tunnel encapsulation header.

So, are we resigning ourselves to not developing any new transport
protocols due to the current state of firewall platforms/vendors? If so,
perhaps we should say that somewhere.

>    trigger this intervention.

Sigh... I have been involved in more than few conversations on drafts
that start with "<foo> is designed for use in a highly controlled
environment" and end with "so don't make us do the hard work". I feel
the pain you are describing.

> ...") as well as in the text quoted above from the example in section 5.3.1.

I would support any decision that would make the protocol design points
more explicit, but I won't hold up the draft for that.

Thanks for the discussion!

Regards,
Brian

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Tue, 15 Mar 2016 11:51:31 -0500

Hi, Brian,

On Tue, Mar 15, 2016 at 7:07 AM, Brian Haberman <[email-masked]>
wrote:

> >>>> Please refer to
> https://www.ietf.org/iesg/statement/discuss-criteria.html

Speaking only for myself, and not as AD ...

TSV managed to get squeezed between "need new transport protocols beyond
TCP and UDP" and "transport protocols get implemented in OS kernels that
already know how to do TCP and UDP, and stuff at least sort of works" for
decades. That was really damaging for SCTP deployment, and worse for DCCP
deployment - and almost certainly worse for other transport protocols that
predated me.

We now have SCTP-in-UDP in RFC 6951, published in 2013, and we now have
DCCP-in-UDP in RFC 6773, published in 2012, and (not predicting the future,
but) I'd expect the trend of "transport protocol-in-UDP" to continue for a
while. It's certainly what Google did with QUIC (same problem, so
encapsulate it in UDP).

So, I think we've been resigned to not doing much with new transport
protocols for a while, but we're been un-resigning over the past couple of
years, especially in places like the IAB Stack Evolution program and
projects like SPUD.

Does that make sense?

Now, back to T2TRG,

Spencer

---

From: Brian Haberman <[email-masked]>
Date: Tue, 15 Mar 2016 13:27:36 -0400

Hi Spencer,

> deployment - and almost certainly worse for other transport protocols that

Was the latter squeeze an implementation issue?  Where does the impact
of firewalls not playing nice with new transport protocols fall in this?

> years, especially in places like the IAB Stack Evolution program and

Encapsulating new things over old things certainly sounds like resigning
ourselves. :)

Regards,
Brian

---

From: Spencer Dawkins at IETF <[email-masked]>
Date: Tue, 15 Mar 2016 12:37:29 -0500

Hi, Brian,

> of firewalls not playing nice with new transport protocols fall in this?

I'd say that's orthogonal - it's also true, and together with "must run
directly over IP/be implemented in the kernel" made a nice chicken and egg
combination (no reason to implement in the kernel because firewalls block
unknown transport protocols, no reason to support on firewalls because
nobody can send that protocol now).

> Encapsulating new things over old things certainly sounds like resigning

I'd say we may be resigned to encapsulating in UDP, but not resigned
applications using UDP and TCP almost exclusively ;-)

But this is probably a better topic for another thread.

Spencer