Subject: [DNSOP] Consensus check on underscore names and draft-ietf-dnsop-rfc7816bis

From: Warren Kumari <[email-masked]>
Date: Wed, 7 Jul 2021 13:54:37 -0400

Hi there all,

I wanted to check the consensus on a point brought up during IETF LC /
OpsDir review of draft-ietf-dnsop-rfc7816bis.

Please see:
https://mailarchive.ietf.org/arch/msg/last-call/fuDyx2as6QsK8CT_7Nvci5d7XQQ/
and
https://mailarchive.ietf.org/arch/msg/dnsop/_H4aM5AquCSRlz0Pz3ncwl7Plpk/

If resolving " _ldap._tcp.ad.example.com", once you hit the _tcp label
you are quite likely in ENT territory, and some implementations
(especially those behind firewalls / middleboxes) are still broken.
There is also a performance hit.

Version 10 of the document added:
"Another potential, optional mechanism for limiting the number of
queries is to assume that labels that begin with an underscore (_)
character do not represent privacy-relevant administrative
boundaries. For example, if the QNAME is "_25._tcp.mail.example.org"
and the algorithm has already searched for "mail.example.org", the
next query can be for all the underscore-prefixed names together,
namely "_25._tcp.mail.example.org"."

(unsurprisingly the document does a much better job of explaining the
issue than I did :-))

Viktor is suggesting that QNAME Minimization should be stopped when
you run into an underscore ("_") label, instead of this being worded
as a potential, optional mechanism.

Obviously there is a tradeoff here -- privacy vs deployment.
1: while it's **possible** that there is a delegation point at the
underscore label, (IMO) it is unlikely. If there is no delegation, you
will simply be coming back to the same server again and again, and so
you are not leaking privacy sensitive information.

2: some recursives are less likely to enable QNAME minimization
because of the non-zero ENT and slight performance issues.

What does the WG think? Does the privacy win of getting this deployed
and enabled sooner outweigh the potential small leak if there *is* a
delegation inside the _ territory of the name?

Should the advice above be strengthened to SHOULD / RECOMMENDED?

This is after IETF LC, but as the question was raised during LC, I
think it needs to be discussed and addressed.

I'd like a **clear** input, on this question only, by Tuesday August 13th.

Much thanks!
W

-- 
Perhaps they really do strive for incomprehensibility in their specs.
After all, when the liturgy was in Latin, the laity knew their place.
-- Michael Padlipsky

---

From: Paul Vixie <[email-masked]>
Date: Wed, 7 Jul 2021 17:57:46 +0000

On Wed, Jul 07, 2021 at 01:54:37PM -0400, Warren Kumari wrote:
> ...

+1.

> Should the advice above be strengthened to SHOULD / RECOMMENDED?

+1.

-- 
Paul Vixie

---

From: Viktor Dukhovni <[email-masked]>
Date: Wed, 7 Jul 2021 14:01:13 -0400

> Should the advice above be strengthened to SHOULD / RECOMMENDED?

Thanks, Indeed I'm arguing for RECOMMENDED (synonymous with SHOULD IIRC,
but sounds less intrasigent).

-- 
    Viktor.

---

From: Brian Dickson <[email-masked]>
Date: Wed, 7 Jul 2021 11:32:37 -0700

> delegation inside the _ territory of the name?

Yes.

> Should the advice above be strengthened to SHOULD / RECOMMENDED?

Yes.

Brian

> _______________________________________________
> DNSOP mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dnsop
>

---

From: Tim Wicinski <[email-masked]>
Date: Wed, 7 Jul 2021 14:35:10 -0400

I am with Victor on the use of RECOMMENDED.

---

From: Peter Thomassen <[email-masked]>
Date: Wed, 7 Jul 2021 20:46:17 +0200

> you are not leaking privacy sensitive information.

There are deployments with delegations at _openpgpkey, such as _openpgpkey.posteo.de. This is how I would do such a deployment myself. As there are only hashes exposed, I don't see a privacy issue if no QNAME Minimization is done. (The point is that I would argue that such a delegation points is not unlikely.)

Another example is _acme-challenge. About 0.1% of zones hosted at desec.io have a delegation point there. (Ok, that's not very frequent.)

The draft for authenticated bootstrapping of DNSSEC delegations (https://datatracker.ietf.org/doc/draft-thomassen-dnsop-dnssec-bootstrapping/) does suggest a zone cut at an underscore name. However, that's not to say it would be a privacy-relevant one, so it seems fine to me when QNAME Minimization is not done in this case.

It is unclear whether the future will bring underscore labels under which privacy-sensitive owner names may reside.

> Should the advice above be strengthened to SHOULD / RECOMMENDED?

Especially because of the last reason above, I tend towards MAY.

However, I would endorse SHOULD / RECOMMENDED if the wording is changed such that "skipping a split" is done "up to the lowest-level" underscore label. In other words, jumping from example.com to _25._tcp.example.com would be RECOMMENDED, but jumping from example.com to foobar._openpgpkey.example.com would not, because "foobar" is no an underscore label. Generally, if there are N consecutive underscore labels, minimization SHOULD be skipped for the N-1 of them which are closest to the root.

Cheers,
Peter

-- 
Like our community service? 💛
Please consider donating at

https://desec.io/

deSEC e.V.
Kyffhäuserstr. 5
10781 Berlin
Germany

Vorstandsvorsitz: Nils Wisiol
Registergericht: AG Berlin (Charlottenburg) VR 37525

---

From: Viktor Dukhovni <[email-masked]>
Date: Wed, 7 Jul 2021 15:00:48 -0400

On Wed, Jul 07, 2021 at 08:46:17PM +0200, Peter Thomassen wrote:

> However, I would endorse SHOULD / RECOMMENDED if the wording is
> changed such that "skipping a split" is done "up to the lowest-level"
> underscore label. In other words, jumping from example.com to
> _25._tcp.example.com would be RECOMMENDED, but jumping from
> example.com to foobar._openpgpkey.example.com would not, because
> "foobar" is no an underscore label. Generally, if there are N
> consecutive underscore labels, minimization SHOULD be skipped for the
> N-1 of them which are closest to the root.

I appreciate the caution, but resuming qname minimisation after skipping
a few labels does look rather complex, it also defeats the main goal of
avoiding known issues with likely ENTs at a name that is rarely a zone
cut, and even if a zone cut, likely not privacy relevant.

There are upcoming drafts on DANE client auth where the leaf label will
not be an "_" special-use label, but its parent is.

For the same reason that asking for "_tcp.smtp.example.com IN A" is
likely to run into trouble or at least impose excessive latency when the
leaf label is "_25", I'd like to *recommend* that qname minimisation
will do more harm than good even if the leaf label is "some-client-id".

I'd like to the see the tradeoff lean heavily towards "practical"
privacy enhancement.

-- 
    Viktor.

---

From: Paul Wouters <[email-masked]>
Date: Wed, 7 Jul 2021 16:54:19 -0400 (EDT)

> namely "_25._tcp.mail.example.org"."

The description is correct....

> as a potential, optional mechanism.

I agree. The _underscore directly conveys information about the qname
parts up to that label. There is no distinct privacy realm here, and
query minimalization SHOULD stop, and the entire QNAME should be
requested.

> you are not leaking privacy sensitive information.

It is possible there is a delegation. For example, at Fedora we ran
into an issue that _openpgp reocrds got quite big, and a delegation
to a separate name server would have been nice. And while this may
be under a different IT branch, from a privacy point of view, it is
still within the same privacy realm.

> because of the non-zero ENT and slight performance issues.

I doubt that, but sure. auth servers could stuff it in the Additional
section too :)

> delegation inside the _ territory of the name?

I don't think there realistically is a "potential small leak".

Yes.

Paul

---

From: Tony Finch <[email-masked]>
Date: Wed, 7 Jul 2021 23:04:53 +0100

> as a potential, optional mechanism.

This sounds sensible to me.

We have some _underscore delegations, because our VOIP phone setup
requires distinct internal and external views, and our main zones don't
support views. But, as in most of the other cases mentioned in this
thread, it isn't a privacy-relevant delegation point: there are only one
or two predictable SRV records in each delegated _underscore zone.

Tony.
-- 
f.anthony.n.finch  <[email-masked]>  https://dotat.at/
Southeast Fitzroy: Northwesterly 3 to 5, occasionally 6 later in
south. Moderate, occasionally rough at first in far north. Showers.
Good, occasionally moderate.

---

From: Petr Špaček <[email-masked]>
Date: Thu, 8 Jul 2021 16:28:29 +0200

> Should the advice above be strengthened to SHOULD / RECOMMENDED?

With my implementer hat on, I say "no", I don't see a compelling reason 
to "mandate" it. Keep it at MAY/optional level and leave it to 
implementers to decide what's best for their implementation and use-cases.

Thanks.
Petr Spacek  @  ISC

---

From: Viktor Dukhovni <[email-masked]>
Date: Thu, 8 Jul 2021 12:00:07 -0400

> On 8 Jul 2021, at 10:28 am, Petr Špaček <[email-masked]> wrote:
> 
> With my implementer hat on, I say "no", I don't see a compelling reason to "mandate" it. Keep it at MAY/optional level and leave it to implementers to decide what's best for their implementation and use-cases.

Just wanted to check what you mean by *mandate*, I don't quite see
RECOMMENDED as a mandate, my understanding is that SHOULD/RECOMMENDED
means "do this unless you have good reason to do otherwise".  So there
is certainly enough rope to ignore the advice.

How would you strongly suggest that stopping qname minimisation at the
first special-use label is probably a good idea, more strongly than just
mentioning as a possible optional optimisation?

-- 
	Viktor.

---

From: Brian Dickson <[email-masked]>
Date: Thu, 8 Jul 2021 09:15:39 -0700

On Thu, Jul 8, 2021 at 7:29 AM Petr Špaček <[email-masked]> wrote:

> to "mandate" it.

Did you actually read what Viktor wrote?

It is a known fact that there ARE implementations where ENT handling (on
the authoritative side) are broken.
Given that the vast majority of the first underscore queries would be
hitting ENTs, this would seem to me, at least, to be compelling.

Why would you not strongly suggest to other implementers to avoid this
problem (by stopping the QNAME minimization)?
Even if you, as an implementer, choose to ignore the problem?
(Also, does your resolver code actually handle the situation during QNAME
minimization of broken ENT handling by authoritative servers?)

Respectfully,
Brian Dickson

> Keep it at MAY/optional level and leave it to

---

From: Wes Hardaker <[email-masked]>
Date: Thu, 08 Jul 2021 14:44:44 -0700

Paul Wouters <[email-masked]> writes:

> There is no distinct privacy realm here, and query minimalization
> SHOULD stop, and the entire QNAME should be requested.

I think this is the primary point and the proposal should go forward
with the stronger wording.

But...  you know some future-dnsop will decide to create some new
special _label that does a privacy separation.  Because we hack DNS
never-endingly.
-- 
Wes Hardaker
USC/ISI

---

From: Peter van Dijk <[email-masked]>
Date: Mon, 12 Jul 2021 10:19:25 +0200

tl;dr: No.

> (especially those behind firewalls / middleboxes) are still broken.

Then they shall suffer. It is not the job of the resolver vendors and
operators to keep working around broken auths. We'd like to remove more
workarounds from the resolver, not add more.

> There is also a performance hit.

This is fair.

> namely "_25._tcp.mail.example.org"."

This is good text, with the note that I like Peter Thomassen's
modification of only jumping to the next non-underscore label, instead
of immediately to the end the moment an underscore is found.

> delegation inside the _ territory of the name?

This looks like a false choice to me. I am unconvinced deployment is
hindered by the difference between MAY and SHOULD for this
recommendation. I also don't think the leak potential is very
interesting.

> Should the advice above be strengthened to SHOULD / RECOMMENDED?

No. MAY is a perfect level for this advice. It is good to let
implementers know that somebody thought of this trick, and it might
make sense for many implementations, but we should not be overly
prescriptive.

Kind regards,
-- 
Peter van Dijk
PowerDNS.COM BV - https://www.powerdns.com/

---

From: Petr Špaček <[email-masked]>
Date: Mon, 12 Jul 2021 11:19:57 +0200

On 08. 07. 21 18:15, Brian Dickson wrote:
> 
> 
> On Thu, Jul 8, 2021 at 7:29 AM Petr Špaček <[email-masked] 
> <mailto:[email-masked]>> wrote:
> 
>     <https://mailarchive.ietf.org/arch/msg/last-call/fuDyx2as6QsK8CT_7Nvci5d7XQQ/>
>      > and
>      >
>     <https://mailarchive.ietf.org/arch/msg/dnsop/_H4aM5AquCSRlz0Pz3ncwl7Plpk/>
>      >
>      > If resolving " _ldap._tcp.ad.example.com
>     <http://tcp.ad.example.com>", once you hit the _tcp label
>      > boundaries. For example, if the QNAME is
>     "_25._tcp.mail.example.org <http://tcp.mail.example.org>"
>      > and the algorithm has already searched for "mail.example.org
>     <http://mail.example.org>", the
>      > namely "_25._tcp.mail.example.org <http://tcp.mail.example.org>"."
>      >

Of course, I did, and I'm not too fond of the tone you used above. Let's 
skip to the constructive part, please.

 > the authoritative side) are broken.

I agree that there are some, but anecdotal evidence of existence tells 
us nothing about
a) prevalence
b) significance of these broken auths and domains hosted on them.
AFAIK both of these are parameters taken into account by resolver 
vendors when deciding what types of non-compliance need to be worked around.

I'm arguing against mandating workarounds. The recommendation might be 
sound in 2021, but that's not a good enough reason for me to mandate it 
as it might change next month when one major player fixes its deployment.

Historical context:
Based on experience with Knot Resolver, various workarounds present in 
older resolver implementations were not "needed" by the time Knot 
Resolver came about. Multiple types of formerly-prevalent protocol 
non-compliance became so marginal that it was not worth the complexity 
to implement and maintain workarounds for them anymore.

 > problem (by stopping the QNAME minimization)?

When you reach this line, I think it should be clear that I consider 
this a wrong question. We should be asking why RFC should mandate a 
workaround that might get obsolete in an inestimable amount of time.

Need for workarounds changes, and for this reason I don't consider RFCs 
to be a good place to _mandate_ workarounds which are by definition 
dependent on current (and constantly changing) situation. (To be clear, 
describing workarounds without mandate is fine with me!)

I hope it clears up why I'm against mandating this.

Petr Špaček @ ISC

> (Also, does your resolver code actually handle the situation during 
> QNAME minimization of broken ENT handling by authoritative servers? >
> Respectfully,
> Brian Dickson
>

---

From: Petr Špaček <[email-masked]>
Date: Mon, 12 Jul 2021 13:52:12 +0200

> mentioning as a possible optional optimisation?

When I look at RFC 2119 ...

--------
3. SHOULD   This word, or the adjective "RECOMMENDED", mean that there 
may exist valid reasons in particular circumstances to ignore a 
particular item, but the full implications must be understood and 
carefully weighed before choosing a different course.

...

5. MAY   This word, or the adjective "OPTIONAL", mean that an item is 
truly optional.  One vendor may choose to include the item because a 
particular marketplace requires it or because the vendor feels that it 
enhances the product while another vendor may omit the same item. An 
implementation which does not include a particular option MUST be 
prepared to interoperate with another implementation which does include 
the option, though perhaps with reduced functionality. In the same vein 
an implementation which does include a particular option MUST be 
prepared to interoperate with another implementation which does not 
include the option (except, of course, for the feature the option provides.)
--------

I think MAY describes exactly what workarounds are:
A completely optional hack which _nobody_ can rely on.

When I read definitions SHOULD and MAY together, I believe the text 
implies problems caused by ignoring SHOULD lie on the party which choose 
to ignore that particular instance of SHOULD:
IMHO this is very wrong when describing workarounds.
Workarounds are temporary "crutches for broken implementations", and 
anything stronger than MAY is just wrong.

As a resolver implementer, I do not want to get into a position when a 
broken auth vendor uses the SHOULD requirement in RFC as an argument to 
keep their auth-brokenness because "resolvers SHOULD implement the 
workaround!"

-- 
Petr Špaček

---

From: Paul Hoffman <[email-masked]>
Date: Mon, 12 Jul 2021 15:18:05 +0000

The current text is sufficient to tell resolver developers, and resolver operators, why they should even think about underscore labels when they create a QNAME minimisation strategy. Elevating such a strategy to a SHOULD as a work-around for broken middleboxes that might (hopefully!) be fixed in the future seems like a very wrong direction for the WG. 

--Paul Hoffman

---

From: Petr Špaček <[email-masked]>
Date: Mon, 12 Jul 2021 17:37:24 +0200

> The current text is sufficient to tell resolver developers, and resolver operators, why they should even think about underscore labels when they create a QNAME minimisation strategy. Elevating such…

Oh, thank you for summarizing my thoughts into concise yet easier to 
understand message!

-- 
Petr Špaček

---

From: Viktor Dukhovni <[email-masked]>
Date: Mon, 12 Jul 2021 17:57:31 -0400

> On 12 Jul 2021, at 11:18 am, Paul Hoffman <[email-masked]> wrote:
> 

If this were just a work-around for breakage, I'd be more inclined
to agree, but it is also a solid opportunity to improve performance,
because privacy-relevant changes of administrative control across
special-use labels should be very rare to non-existent.

So short-circuiting qname minimisation when a special-use label is
encountered seems like a win-win.

Measuring qname 

-- 
	Viktor.

---

From: Brian Dickson <[email-masked]>
Date: Mon, 12 Jul 2021 15:13:19 -0700

On Mon, Jul 12, 2021 at 2:20 AM Petr Špaček <[email-masked]> wrote:

> >     <
> >     <
> https://mailarchive.ietf.org/arch/msg/dnsop/_H4aM5AquCSRlz0Pz3ncwl7Plpk/>
> >      >
> >      > namely "_25._tcp.mail.example.org <http://tcp.mail.example.org
> >"."
> >      >

Let me start by apologizing to you (and the group) for the tone (whether
intended or not).
(I was literally inquiring as opposed to intending to having tone.)

> vendors when deciding what types of non-compliance need to be worked

So, let me counter by illustrating the scope of the problem, if it occurs,
and the challenges created depending on whether or not any work-around is
RECOMMENDED, or not.

This is specifically concerned only with underscore names which are
registered with IANA, and the implications of ENT-based resolution errors.
For reference, the list is published here:
https://www.iana.org/assignments/dns-parameters/dns-parameters.xhtml#underscored-globally-scoped-dns-node-names

Not all entries in the underscore table have the same degree of
utilization, or relevance, etc.

The ones that jump out from that list, as well as not-yet-published
entries, at least in my opinion, are:

   - TLSA
   - SVCB
   - DMARC
   - DKIM
   - ACME
   - validation-contactemail
   - validation-contactphone
   - SPF use of _spf
   - X.509 URIs
   - DNS SD

The reason I'm interested in ensuring that the REASON (rationale) matches
the CHOICE we make in the WG, is precisely that the collective knowledge
about the deployment prevalence and issues is little better than anecdotal.
And, that the impact of picking one (MAY vs SHOULD) might affect lots of
other applications and protocols.

DNS is a necessary and fundamental service, used by not only the various
parties (end-users and servers, zone owners) but also the services which
rely on the DNS itself and proper resolution of entries in the DNS.

The problem MAY be present in middle-boxes (of all sorts of functions and
flavors), but MAY NOT be anything under operator control, and might not be
easy to identify.
The problem MAY ALSO exist in other software, firmware, or hardware, which
is acting as an actual DNS device (rather than multi-function middlebox).

So, basically, my suggestion is, that if the WG wants to use "MAY", it
should ensure the guidance to implementers is clear, on what might happen,
and if it does, how to detect it, report it, and what documentation to
provide to operators/users.

The old adage of, "Never test for an error condition you don't know how to
handle" has relevant corollaries here:

   - If you do know how to handle an error condition, test for it and
   handle it
   - If you are unable to test for an error condition, maybe don't create
   it in the first place
   - Don't blame the victim

Examples of problems that aren't easy or even possible to fix, and may not
be possible to work around, include (but are not limited to):

   - Mandated services or equipment
   - Service-provider (ISP) provided equipment
   - Consumer-grade equipment without support
   - Third party operators of services (e.g. DNS)
   - MITM functionality (e.g. government-operated, or public WiFi operated,
   or hotel service, etc)

The implications to borked stuff, based on the underscore registry entries
highlighted could include:

   - Loss of privacy
   - Inability to validate certificates or to discover revoked certificates
   - Loss of spam-prevention ability to receivers
   - Loss of spam-prevention protection to domain owners
   - Inability to discover services
   - Degradation of web performance
   - Inability to auto-renew certificates

The current text does not provide any guidance on the broken-ENT-NXDOMAIN
problem, and basically ignores the problem entirely.

If keeping MAY is the approach chosen, I would like to suggest that some
text be added to the effect that (a) the specific ENT problem may exist,
and (b) what to do to detect it and handle it, or whether to add a "knob"
to add support for operators to enable such support.

I'm not sure if Viktor has already supplied language that would work.

If not, something like this (but maybe easier to read, parse, implement,
etc) would be helpful:

"When doing QNAME minimization, if an iterative query whose first label
begins with an underscore "_", results in an NXDOMAIN, the owner name may
be an ENT (empty non terminal). A signed response would include a proof of
non-existence, and no additional action is needed in that situation.
However, an unsigned response MAY be treated as a possible indicator of an
ENT, and the algorithm MAY choose to ignore the NXDOMAIN if the current
QNAME is not the same as the original QNAME, and proceed to the next label
in the algorithm."

There is a difference between mandating workarounds, and supporting use
cases where the impacted party is not in a position to change the outcome
of failed DNS resolution.

Brian

P.S. I am genuinely interested in what implementers plan on doing,
regardless of whether that is mandated or not. If anyone is willing to
share, please do so.

---

From: Viktor Dukhovni <[email-masked]>
Date: Mon, 12 Jul 2021 18:16:53 -0400

[ Resending complete message, previous draft was incomplete... ]

Measuring qname minimisation for TLSA RRs I see that today breakage
of qname minimisation is rare.  An example is:

	https://dnsviz.net/d/_tcp.u24.altospam.com/YOx4nQ/dnssec/
	https://dnsviz.net/d/_25._tcp.u24.altospam.com/YOx4IA/dnssec/

In which many (but not all) of the nameservers return NXDOMAIN for the
ENT.  Out of 150k RRsets, O(10) have ENT-related issues.

So one might reasonably neglect the breakage, but it is not clear that
we need to go looking for it, just to "punish" the operators in question.
There's an opportunity here to make qname minimisation more performant for
SRV, TLSA, ... lookups, speeding up Domain Control and LDAP server lookups,
email delivery, ...

Of course if the WG cannot come to consensus on "SHOULD"/"RECOMMENDED", I'll
gratefully settle for the current "MAY" (thanks for the document update)...

-- 
	Viktor.

---

From: Warren Kumari <[email-masked]>
Date: Mon, 12 Jul 2021 18:28:57 -0400

"

> gratefully settle for the current "MAY" (thanks for the document update)...

Another option would be to keep the current text, and simply add
another sentence describing why implementations may want to do this;
the current paragraph starts off with:
" Another potential, optional mechanism for limiting the number of
queries is to assume that labels that begin with an underscore (_)
character do not represent privacy-relevant administrative
boundaries." - the context around this is mainly around limiting the
many labels attack, but it could also mention the ENT / other
performance gain.

Whatever the case, this conversation is still ongoing, so I'd like to
keep it open for a few more days...

W

-- 
The computing scientist’s main challenge is not to get confused by the
complexities of his own making.
  -- E. W. Dijkstra

---

From: Petr Špaček <[email-masked]>
Date: Tue, 13 Jul 2021 12:15:46 +0200

>> special-use labels should be very rare to non-existent.

I hear you Viktor, but I fail to see why performance optimization cannot 
be optional. Why this specific optimization is so important that it 
cannot be left to vendors to decide?

>> SRV, TLSA, ... lookups, speeding up Domain Control and LDAP server lookups,

Please note I do not object to the method by itself - I agree it might 
be useful. My disagreement is limited to using stronger requirement than 
MAY. Let vendors have some leeway, please.

Petr Špaček

---

From: Petr Špaček <[email-masked]>
Date: Tue, 13 Jul 2021 12:22:01 +0200

On 13. 07. 21 0:13, Brian Dickson wrote:
> 
> 
> On Mon, Jul 12, 2021 at 2:20 AM Petr Špaček <[email-masked] 
>     <mailto:[email-masked]>
>      > <mailto:[email-masked] <mailto:[email-masked]>>> wrote:
>      >
>      >      > Hi there all,
>      >      >
>      >      > I wanted to check the consensus on a point brought up
>     during IETF
>      >     LC /
>      >      > OpsDir review of draft-ietf-dnsop-rfc7816bis.
>      >      >
>      >      > Please see:
>      >      >
>      >
>       <https://mailarchive.ietf.org/arch/msg/last-call/fuDyx2as6QsK8CT_7Nvci5d7XQQ/ <https://mailarchive.ietf.org/arch/msg/last-call/fuDyx2as6QsK8CT_7Nvci5d7XQQ/>>
>      >      > and
>      >      >
>      >
>       <https://mailarchive.ietf.org/arch/msg/dnsop/_H4aM5AquCSRlz0Pz3ncwl7Plpk/ <https://mailarchive.ietf.org/arch/msg/dnsop/_H4aM5AquCSRlz0Pz3ncwl7Plpk/>>
>      >      >
>      >      > If resolving " _ldap._tcp.ad.example.com
>     <http://tcp.ad.example.com>
>      >     <http://tcp.ad.example.com <http://tcp.ad.example.com>>",
>     once you hit the _tcp label
>      >      > you are quite likely in ENT territory, and some
>     implementations
>      >      > (especially those behind firewalls / middleboxes) are
>     still broken.
>      >      > There is also a performance hit.
>      >      >
>      >      > Version 10 of the document added:
>      >      > "Another potential, optional mechanism for limiting the
>     number of
>      >      > queries is to assume that labels that begin with an
>     underscore (_)
>      >      > character do not represent privacy-relevant administrative
>      >      > boundaries. For example, if the QNAME is
>      >     "_25._tcp.mail.example.org <http://tcp.mail.example.org>
>     <http://tcp.mail.example.org <http://tcp.mail.example.org>>"
>      >      > and the algorithm has already searched for
>     "mail.example.org <http://mail.example.org>
>      >     <http://mail.example.org <http://mail.example.org>>", the
>      >      > next query can be for all the underscore-prefixed names
>     together,
>      >      > namely "_25._tcp.mail.example.org
>     <http://tcp.mail.example.org> <http://tcp.mail.example.org
>     <http://tcp.mail.example.org>>"."
>      >      >
>      >      > (unsurprisingly the document does a much better job of
>     explaining the
>      >      > issue than I did :-))
>      >      >
>      >      > Viktor is suggesting that QNAME Minimization should be
>     stopped when
>      >      > you run into an underscore ("_") label, instead of this
>     being worded
>      >      > as a potential, optional mechanism.
>      >      >
>      >      > Obviously there is a tradeoff here -- privacy vs deployment.
>      >      > 1: while it's **possible** that there is a delegation
>     point at the
>      >      > underscore label, (IMO) it is unlikely. If there is no
>      >     delegation, you
>      >      > will simply be coming back to the same server again and
>     again, and so
>      >      > you are not leaking privacy sensitive information.
>      >      >
>      >      > 2: some recursives are less likely to enable QNAME
>     minimization
>      >      > because of the non-zero ENT and slight performance issues.
>      >      >
>      >      > What does the WG think? Does the privacy win of getting
>     this deployed
>      >      > and enabled sooner outweigh the potential small leak if
>     there *is* a
>      >      > delegation inside the _ territory of the name?
>      >      >
>      >      > Should the advice above be strengthened to SHOULD /
>     RECOMMENDED?
>      >
>       > It is a known fact that there ARE implementations where ENT
>     handling (on
>       > the authoritative side) are broken.
> 
> So, let me counter by illustrating the scope of the problem, if it 
> occurs, and the challenges created depending on whether or not any 
> work-around is RECOMMENDED, or not.

I'm sorry for not replying in full, but I think it's more efficient this 
way:

I agree that DNS is complex and has tons of failure modes, but this 
specific case does not seem special enough to me to warrant special 
treatment.

As Viktor pointed out in 
https://mailarchive.ietf.org/arch/msg/dnsop/w7JBD4czpGKr46v-DlycGbOv9zs/ 
, it seems that this problem plagues roughly tens out of 150k domains he 
surveyed. I think this makes further discussion about _necessity_ of the 
workaround kind of moot.

This leaves us with performance optimization, to which I replied already 
in 
https://mailarchive.ietf.org/arch/msg/dnsop/h9BlBI0hQ0MtDAQAK_s8Tmj2bUc/ .

Thank you for your time.
Petr Špaček @ ISC

> <https://www.iana.org/assignments/dns-parameters/dns-parameters.xhtml#underscored-globally-scoped-dns-node-names>
> 
>   * TLSA
>   * SVCB
>   * DMARC
>   * DKIM
>   * ACME
>   * validation-contactemail
>   * validation-contactphone
>   * SPF use of _spf
>   * X.509 URIs
>   * DNS SD
> 
> The reason I'm interested in ensuring that the REASON (rationale) 
> matches the CHOICE we make in the WG, is precisely that the collective 
> knowledge about the deployment prevalence and issues is little better 
> than anecdotal.
> The problem MAY be present in middle-boxes (of all sorts of functions 
> and flavors), but MAY NOT be anything under operator control, and might 
> not be easy to identify.
> The problem MAY ALSO exist in other software, firmware, or hardware, 
> which is acting as an actual DNS device (rather than multi-function 
> middlebox).
> 
> should ensure the guidance to implementers is clear, on what might 
> happen, and if it does, how to detect it, report it, and what 
> documentation to provide to operators/users.
> 
> The old adage of, "Never test for an error condition you don't know how 
> to handle" has relevant corollaries here:
> 
>   * If you do know how to handle an error condition, test for it and
>     handle it
>   * If you are unable to test for an error condition, maybe don't create
>     it in the first place
>   * Don't blame the victim
> 
> Examples of problems that aren't easy or even possible to fix, and may 
> not be possible to work around, include (but are not limited to):
> 
>   * Mandated services or equipment
>   * Service-provider (ISP) provided equipment
>   * Consumer-grade equipment without support
>   * Third party operators of services (e.g. DNS)
>   * MITM functionality (e.g. government-operated, or public WiFi
>     operated, or hotel service, etc)
> 
> The implications to borked stuff, based on the underscore registry 
> entries highlighted could include:
> 
>   * Loss of privacy
>   * Inability to validate certificates or to discover revoked certificates
>   * Loss of spam-prevention ability to receivers
>   * Loss of spam-prevention protection to domain owners
>   * Inability to discover services
>   * Degradation of web performance
>   * Inability to auto-renew certificates
> 
> The current text does not provide any guidance on the 
> broken-ENT-NXDOMAIN problem, and basically ignores the problem entirely.
> 
> and (b) what to do to detect it and handle it, or whether to add a 
> "knob" to add support for operators to enable such support.
> 
> begins with an underscore "_", results in an NXDOMAIN, the owner name 
> may be an ENT (empty non terminal). A signed response would include a 
> proof of non-existence, and no additional action is needed in that 
> situation. However, an unsigned response MAY be treated as a possible 
> indicator of an ENT, and the algorithm MAY choose to ignore the NXDOMAIN 
> if the current QNAME is not the same as the original QNAME, and proceed 
> to the next label in the algorithm."
> 
> cases where the impacted party is not in a position to change the 
> outcome of failed DNS resolution.
> 
> Brian
> 
>       > Given that the vast majority of the first underscore queries
>     would be
>       > hitting ENTs, this would seem to me, at least, to be compelling.
>       >
>       > Why would you not strongly suggest to other implementers to
>     avoid this
>       > problem (by stopping the QNAME minimization)?
>

---

From: Viktor Dukhovni <[email-masked]>
Date: Tue, 13 Jul 2021 13:01:08 -0400

> On 13 Jul 2021, at 6:22 am, Petr Špaček <[email-masked]> wrote:
> 
> As Viktor pointed out in https://mailarchive.ietf.org/arch/msg/dnsop/w7JBD4czpGKr46v-DlycGbOv9zs/ , it seems that this problem plagues roughly tens out of 150k domains he surveyed. I think this makes further discussion about _necessity_ of the workaround kind of moot.

Full disclosure, I only tested TLSA records.  I can't speak to what
one might expect with SRV or other record types.  Yes, failures are
not that common, for what is worth another example:

	https://dnsviz.net/d/_tcp.mail.ncsc.de/YO3DpQ/dnssec/
	https://dnsviz.net/d/_25._tcp.mail.ncsc.de/YO3Bsw/dnssec/

Here the "A" query for the ENT was unexpectedly "REFUSED". :-(

If implementations at least seriously consider the advice to treat
special-use labels *specially*, I'll declare victory...

-- 
	Viktor.

---

From: Brian Dickson <[email-masked]>
Date: Tue, 13 Jul 2021 20:13:23 -0700

> surveyed. I think this makes further discussion about _necessity_ of the
The plural of anecdotes is not data... unfortunately.

So, I don't recall who presented it, where, or when, but it was some time
in the last couple of years, maybe at an IETF or OARC meeting.
But, the gist of the presentation was that the presenters studied query
sources from a number of vantage points, over a period of years, and
concluded there are approximately 3 million resolvers.

Those resolvers are not all directly connected to the Internet, and
some/many are configured as forwarders (with potentially multiple
forwarders in chains/trees).

The point here is, that Viktor represents one out of three million vantage
points, which undoubtedly have different characteristics in terms of
resolver software and version, and intermediate servers (e.g. forwarding to
resolvers, or forwarding to forwarders).

Additionally, Viktor's data set was TLSA, which is already a niche set that
is self-selected to be on DNSSEC-signed zones, meaning relatively new and
mainstream code bases (possibly over-generalizing admittedly.)
Examining larger data sets on domains that are not DNSSEC-signed, may show
a different prevalence of the ENT problem.

The other distinction is that the problems leading to bad ENT results, may
not be on the authoritative servers, but may be in front of them (close to
the authoritatives), or may be other middle-boxes closer to the resolvers,
or between forwarders and resolvers, etc. Thus the scope of the ENT problem
may vary based on the vantage point being used to collect results.

Thus, in the absence of any statistically meaningful data, I disagree with
the mootness of the issue.

That doesn't mean we can't reach consensus, of course, and given that this
is a -bis document, and we are discussing how to handle the corner case of
underscore ENTs, some focus should be given to the impact rather than the
prevalence. This includes both the impact on code paths, and on the effects
to client apps affected by ENT broken-ness.

It may be helpful to note that the draft itself already differentiates
behavior by the number of labels, in section 2.3 (in a MAY context) using
the MAX_MINIMIZE_COUNT logic. Thus, if an implementation is already doing
that work, adding underscore handling might not be a large burden (and
might mesh nicely in terms of coding). For example, in evaluating the
break-points when partitioning the labels to limit the total number of
queries, the sequence COULD treat any contiguous sequence of underscore
labels as if it were a single label, and then do its partitioning of labels
using the same relative logic.

The main point being, if the implementer is already doing anything other
than literally iterating over all the labels one at a time, under all
circumstances, then adding underscores into its handling isn't likely
significantly burdensome.
The difference between the many-labels problem and the underscore
situation, is the difference between weakness against attacks and
inefficiency (many labels), versus actual brokenness (for some
indeterminate fraction of the namespace from some indeterminate portion of
the resolvers on the Internet).

I'm hoping to advance the discussion even if no one is persuaded.

Brian

---

From: Viktor Dukhovni <[email-masked]>
Date: Tue, 13 Jul 2021 23:53:54 -0400

> On 13 Jul 2021, at 11:13 pm, Brian Dickson <[email-masked]> wrote:
> 
> For example, in evaluating the break-points when partitioning the labels to limit the total number of queries, the sequence COULD treat any contiguous sequence of underscore labels as if it were a single label, and then do its partitioning of labels using the same relative logic.

FWIW, my take is that treating consecutive special-use labels as a single label
only to resume minimisation at some subsequent non-special-use label sounds too
complex to me.  Intermediate special-use labels are just as likely to be ENTs
and to not be privacy-relevant zone cuts even when some labels below are not
special-use.

So whether it is MAY or SHOULD, my sense is that if an implementation chooses
to implement the proposed strategy it should just make the final query at
that point.

-- 
	Viktor.

---

From: Petr Špaček <[email-masked]>
Date: Wed, 14 Jul 2021 10:17:15 +0200

On 14. 07. 21 5:13, Brian Dickson wrote:
> 
> 
> On Tue, Jul 13, 2021 at 10:01 AM Viktor Dukhovni <[email-masked] 
> <mailto:[email-masked]>> wrote:
> 
>      > On 13 Jul 2021, at 6:22 am, Petr Špaček <[email-masked]
>     <https://mailarchive.ietf.org/arch/msg/dnsop/w7JBD4czpGKr46v-DlycGbOv9zs/>
>     , it seems that this problem plagues roughly tens out of 150k
>     domains he surveyed. I think this makes further discussion about
>     _necessity_ of the workaround kind of moot.
> 
> 
> So, I don't recall who presented it, where, or when, but it was some 
> time in the last couple of years, maybe at an IETF or OARC meeting.
> The point here is, that Viktor represents one out of three million 
> vantage points, which undoubtedly have different characteristics in 
> terms of resolver software and version, and intermediate servers (e.g. 
> forwarding to resolvers, or forwarding to forwarders).
> 
> Additionally, Viktor's data set was TLSA, which is already a niche set 
> that is self-selected to be on DNSSEC-signed zones, meaning relatively 
> new and mainstream code bases (possibly over-generalizing admittedly.)
> Examining larger data sets on domains that are not DNSSEC-signed, may 
> show a different prevalence of the ENT problem.
> 
> The other distinction is that the problems leading to bad ENT results, 
> may not be on the authoritative servers, but may be in front of them 
> (close to the authoritatives), or may be other middle-boxes closer to 
> the resolvers, or between forwarders and resolvers, etc. Thus the scope 
> of the ENT problem may vary based on the vantage point being used to 
> collect results.
> 
> Thus, in the absence of any statistically meaningful data, I disagree 
> with the mootness of the issue.
> 
> That doesn't mean we can't reach consensus, of course, and given that 
> this is a -bis document, and we are discussing how to handle the corner 
> case of underscore ENTs, some focus should be given to the impact rather 
> than the prevalence. This includes both the impact on code paths, and on 
> the effects to client apps affected by ENT broken-ness.
> 
> behavior by the number of labels, in section 2.3 (in a MAY context) 
> using the MAX_MINIMIZE_COUNT logic. Thus, if an implementation is 
> already doing that work, adding underscore handling might not be a large 
> burden (and might mesh nicely in terms of coding). For example, in 
> evaluating the break-points when partitioning the labels to limit the 
> total number of queries, the sequence COULD treat any contiguous 
> sequence of underscore labels as if it were a single label, and then do 
> its partitioning of labels using the same relative logic.
> 

Maybe, or maybe not. We cannot know for sure until it's implemented in a 
real resolver. In my experience, major resolver implementations contain 
little but important details that make even seemingly simple tweaks way 
harder to implement than it looks on paper.

Not having running code to support this proposal this late in 
publication process is IMHO another reason why it should stay at MAY level.

Petr Špaček

> indeterminate fraction of the namespace from some indeterminate portion 
> of the resolvers on the Internet).
> 
>     <https://dnsviz.net/d/_tcp.mail.ncsc.de/YO3DpQ/dnssec/>
>     <https://dnsviz.net/d/_25._tcp.mail.ncsc.de/YO3Bsw/dnssec/>
>

---

From: Warren Kumari <[email-masked]>
Date: Fri, 23 Jul 2021 16:18:17 -0400

> publication process is IMHO another reason why it should stay at MAY level.

Thank you everyone for the discussion.

As this is after WGLC and IETF LC, I'd want to see a very strong and
clear consensus before requesting a change from MAY to
SHOULD/RECOMMENDED... and I simply don't see that.

However, one of the big things that we wanted from this discussion was
to make sure that implementers consider if this is an optimization
that they want to perform, and seems that that has occurred :-)

Unfortunatly I made a stupid error when sending the initial note - I
said: "I'd like a **clear** input, on this question only, by Tuesday
August 13th.".
I intended to ask for feedback by **July 13th**, but August 13th is my
birthday, and so it seems that mental autocorrect took over...

Whatever the case, I'll progress this to IESG Eval. What with IETF
scheduling and such, it won't show up on the telechat before the 13th
(if there is somehow clear consensus to change before that, we can
adjust the dates, etc...)

Thanks again for the useful (and constructive!) discussion,
W