Subject: [ietf-dkim] DNS wildcarding behavior scenarios

From: Michael Thomas <[email-masked]>
Date: Fri, 08 Jun 2007 09:26:20 -0700

Hi all,

I think that there is a huge amount of confusion about how DNS wildcards
work, and in particular how they might come to bear on the discovery
problem for ssp-requirements, 5.1.4.

Executive summary: Wildcards, either TXT of a new one DO NOT meet this
requirement.

Here is the proof using my own zone (using bind 9.3):

0) Pertinent Parts of my Zone:

mtcc.com.         IN      TXT     "v=spf1 a mx include:cisco.com 
include:volcano.net ~
all"
mtcc.com.*.       IN      TXT     "v=spf1 a mx include:cisco.com 
include:volcano.net ~
all"
gate              IN      A       64.142.29.1

alltheway.kirkwood._domainkey.mtcc.com.   3600   IN    TXT   "v=DKIM1; 
k=rsa; g=*; s
=email; t=y; " 
"p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCo0kNKsffbCSZ8mqsaApvPYyDzRf
GYXZBmg4VhvugbQ4wYrZXMGn9m/V7XJxKpJj9cHE2VHUemZUPlOto8FgerCrQMaQC0tHgBw0DRjFCurMtS+4
EOfXcjn5LQ308BmVr2lUwl7QHuIFBE9Ls/66xld4AJxLrb9+" "yeTIAUZdX7WwIDAQAB;"

1) Top level query

    fugu$ host -t txt mtcc.com
    mtcc.com descriptive text "v=spf1 a mx include:cisco.com 
include:volcano.net ~all"

    As you would expect: the exact match worked.

2) Query for nonexistent node adjacent to top level

    fugu$ host -t txt asdfasd.mtcc.com
     asdfasd.mtcc.com descriptive text "v=spf1 a mx include:cisco.com 
include:volcano.net ~all"

    This works as expected because of the top level wildcard.

3) Query for nonexistent node arbitrarily far from the top node

    fugu$ host -t txt asdfsdf.asdf.mtcc.com
    asdfsdf.asdf.mtcc.com descriptive text "v=spf1 a mx 
include:cisco.com include:volcano.net ~all"

    This works as you'd expect since there is a clear line to the top
    of the wildcard.

4) Node which has a valid A record

    fugu$ host -t txt gate.mtcc.com
    gate.mtcc.com has no TXT record

    Here, the wildcard ceases to work and the resolver returns an 
ancount of zero.
    This case still *must* be handled somehow by SSP.

5) Subnode from a valid non-TXT bearing node

    fugu$ host -t txt asdfsdf.gate.mtcc.com
    Host asdfsdf.gate.mtcc.com not found: 3(NXDOMAIN)

    Here it returns NXDOMAIN. This the behavior you expect if there wasn't
    the *.mtcc.com wildcard.

6) As it relates to the _domainkey subnode

    fugu$ host -t txt _domainkey.mtcc.com
    _domainkey.mtcc.com has no TXT record

    Note again that the wildcard at mtcc.com does not cover this since 
there are
    subnodes that bear RR's. This is really another case of 4 but it 
works even
    when it's an interior node that bears no RR's at its node.

7) A TXT record that overrides the top level wildcard

    fugu$ host -t txt alltheway.kirkwood._domainkey.mtcc.com
    alltheway.kirkwood._domainkey.mtcc.com descriptive text "v=DKIM1\; 
k=rsa\; g=*\; s=email\; t=y\; " 
"p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCo0kNKsffbCSZ8mqsaApvPYyDzRfGYXZBmg4VhvugbQ4wYrZXMGn9m/V7XJxKpJj9cHE2VHUemZUPlOto8FgerCrQMaQC0tHgBw0DRjFCurMtS+4EOfXcjn5LQ308BmVr2lUwl7QHuIFBE9Ls/66xld4AJxLrb9+" 
"yeTIAUZdX7WwIDAQAB\;"

    Note that here, we can take the place of the top level wildcard by
    simply putting a TXT record in. This is true of any place we might
    want to put a TXT record.

8) Subnodes of valid TXT nodes

    fugu$ host -t txt asdf.alltheway.kirkwood._domainkey.mtcc.com
    Host asdf.alltheway.kirkwood._domainkey.mtcc.com not found: 3(NXDOMAIN)

    This is nothing more than case 5 again.

_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: Hector Santos <[email-masked]>
Date: Fri, 08 Jun 2007 13:28:28 -0400

> problem for ssp-requirements, 5.1.4.

Michael, rather than continue this path of trying to tell people how 
confused they are and that you know better,  just show us your results. 
   Honestly, there are alot of SMART people here - we are NOT stupid.

I'm glad you did this similar exercise that I did.  I only did so get 
DNS people to explore how it can be done, and if there certain bottle 
necks, hiccups, how do we get around it.

> Executive summary: Wildcards, either TXT of a new one DO NOT meet this

I'm not sure 100% that I agree. Maybe the requiremnent is wrong? If I 
read your testing right, it is also based on not using a prefix. So 
maybe that method is not right. Maybe you need to have a cut off.

Case in point:

>    Here, the wildcard ceases to work and the resolver returns 
 >    an ancount of zero. This case still *must* be handled somehow
 >    by SSP.

Right.  This can be handled some way.

>    Note again that the wildcard at mtcc.com does not cover 
 >    this since there are subnodes that bear RR's. This is really
 >    another case of 4 but it works even when it's an interior
 >    node that bears no RR's at its node.

Right, especially if you have a prefix for the SSP record that is 
different than the KEY record.

I think the difference with my exercise is that here, you use the entire 
domain where in my exercise, I borrowed the logic used in the LMAP 
protocol "DMP" to use a prefix with a split of the domain:

     *._SSP.<domain.tld>     		global answer
     _SSP.<domain.tld>       		main domain answer
     [subdomains.]_SSP.<domain.tld>	subdomain answer

This works!

The only problem that I see with this style is that the client has to be 
aware of the zone cuts.  It needs knowledge of the gTLD and ccTLDS.

Lets not throw out the baby with the bath water yet.

-- 
Sincerely

Hector Santos, CTO
http://www.santronics.com
http://santronics.blogspot.com

---

From: John Levine <[email-masked]>
Date: 8 Jun 2007 19:05:11 -0000

>Executive summary: Wildcards, either TXT of a new one DO NOT meet this

Wildcards only cover nodes with no records at all.  You need to put an
explicit record at any node that has any other kind of node.  This is
a well-documented aspect of DNS wildcards that we usually assume will
be addressed by some sort of semi-automated process for generating all
the required records.

>0) Pertinent Parts of my Zone: ...

I sure hope that was supposed to be *.mtcc.com. since a trailing star
only matches a literal star.

>    This case still *must* be handled somehow by SSP.

This is a case where you need an explicit record.

>    Here it returns NXDOMAIN. This the behavior you expect if there wasn't

This is the other case where you need an explicit record as well as a
wildcard to cover subdomains.

>6) As it relates to the _domainkey subnode

Since host names can't contain underscores, that's not a valid e-mail
domain name, so it doesn't matter.

>7) A TXT record that overrides the top level wildcard

Somewhat similar to #4, although if the SSP record is a TXT record,
you're depending on clients that use anything encoded as a TXT record,
including SPF/S-ID and dkim-base to look at the version strings and
throw away the ones that aren't the right kind.

Nearly everyone agrees that in retrospect, it would have been better
to design DNS wildcards differently.  But if you control the whole
zone, particularly if it's a new RR so it doesn't contaminate existing
uses of TXT records, you can mechanically add extra records to get the
effect of the wildcards you actually want.

Personally, I agree that this is really klunky, and this problem is
one of the reasons that SSP is unlikely to be of practical use to mail
receivers.

R's,
John

---

From: "Hallam-Baker, Phillip" <[email-masked]>
Date: Fri, 8 Jun 2007 12:05:25 -0700

I don't quite understand the point here.

Why do you think it is necessary to define a signing policy for a node that does not exist and does not therefore send mail?

You certainly need to define a signature policy but its not a signing policy.

> -----Original Message-----
> From: [email-masked] 
> [mailto:[email-masked]] On Behalf Of Michael Thomas
> Sent: Friday, June 08, 2007 12:26 PM
> To: IETF-DKIM
> Subject: [ietf-dkim] DNS wildcarding behavior scenarios
> 
> 
> Hi all,
> 
> I think that there is a huge amount of confusion about how 
> DNS wildcards work, and in particular how they might come to 
> bear on the discovery problem for ssp-requirements, 5.1.4.
> 
> Executive summary: Wildcards, either TXT of a new one DO NOT 
> meet this requirement.
> 
>      asdfasd.mtcc.com descriptive text "v=spf1 a mx 
>     Here, the wildcard ceases to work and the resolver 
> returns an ancount of zero.
> this since there are
>     subnodes that bear RR's. This is really another case of 4 
> but it works even
>     alltheway.kirkwood._domainkey.mtcc.com descriptive text 
> "v=DKIM1\; k=rsa\; g=*\; s=email\; t=y\; " 
> "p=MIGfMA0GCSqGSIb3DQEBAQUAA4GNADCBiQKBgQCo0kNKsffbCSZ8mqsaApv
> PYyDzRfGYXZBmg4VhvugbQ4wYrZXMGn9m/V7XJxKpJj9cHE2VHUemZUPlOto8F
> gerCrQMaQC0tHgBw0DRjFCurMtS+4EOfXcjn5LQ308BmVr2lUwl7QHuIFBE9Ls
/66xld4AJxLrb9+"

---

From: Michael Thomas <[email-masked]>
Date: Fri, 08 Jun 2007 12:44:52 -0700

> Why do you think it is necessary to define a signing policy for a node that does not exist and does not therefore send mail?

case 4 demonstrates that a wildcard DOES NOT cover a node that DOES
EXIST. This is absolutely and explicitly in scope of requirement 5.1.4

		Mike

---

From: Michael Thomas <[email-masked]>
Date: Fri, 08 Jun 2007 17:41:14 -0700

> only matches a literal star.

Yes, my screwup when cutting and pasting.
> effect of the wildcards you actually want.

Right, I just wanted remind people once again what wildcards do and
don't provide. Synthesizing the wildcard by mechanically populating
the record at existing nodes is possible, though extremely kludgey --
at least with the resolvers I'm familiar with.

One question though: is this an artifact of the actual protocol, or an
artifact of the resolvers themselves? I've never been clear on that.
As far as the bits on the wire, aren't wildcards actually invisible
to the querier?

       Mike

---

From: "Hallam-Baker, Phillip" <[email-masked]>
Date: Fri, 8 Jun 2007 18:15:58 -0700

OK add a TXT record:

gate.mtcc.com TXT "Here I am"

Problem solved.

> From: Michael Thomas [mailto:[email-masked]] 
> Sent: Friday, June 08, 2007 3:45 PM
> To: Hallam-Baker, Phillip
> Cc: IETF-DKIM
> Subject: Re: [ietf-dkim] DNS wildcarding behavior scenarios
> 
> > Why do you think it is necessary to define a signing policy 
> for a node that does not exist and does not therefore send mail?
> >
> 
> case 4 demonstrates that a wildcard DOES NOT cover a node 
> that DOES EXIST. This is absolutely and explicitly in scope 
> of requirement 5.1.4
> 
> 		Mike
> 
> >> I think that there is a huge amount of confusion about how DNS 
> >> wildcards work, and in particular how they might come to 
> bear on the 
> >> discovery problem for ssp-requirements, 5.1.4.
> >>
> >> Executive summary: Wildcards, either TXT of a new one DO NOT meet 
> >> this requirement.
> >>
> >>     Here it returns NXDOMAIN. This the behavior you expect 
> if there 
> >> wasn't
> >>     the *.mtcc.com wildcard.
> >>
> >>     Note again that the wildcard at mtcc.com does not cover this 
> >> since there are

---

From: Steve Atkins <[email-masked]>
Date: Fri, 8 Jun 2007 18:22:38 -0700

> As far as the bits on the wire, aren't wildcards actually invisible

Yes. Wildcards don't actually exist in DNS. The behavior
being discussed here is an implementation detail of one
(commonly used) authoritative server[1], though other
servers often have similar functionality, often mostly
compatible.

Wildcards are something of a hack, and I understand
there are significant incompatibilities in wildcard handling of
some records between different authoritative servers. I
think we'd be fine with simple usage, but anything subtle
should not[2] be suggested by anyone who isn't intimately
familiar with the operational issues with the various authoritative
servers out there.

Cheers,
   Steve

[1] For the RFC lawyers, not of any actual operational value:
There are some RFCs that specify some of this behaviour,
I believe, in the context of zone transfers rather than DNS queries
but those RFCs really just document the behaviour of that
one authoritative server, and most people consider them
a mistake worth forgetting as far as being a Standard with
a capital S is concerned, rather than just documentation
of current practice with that one server.

[2] This email is RFC 2119 compliant.
_______________________________________________
NOTE WELL: This list operates according to 
http://mipassoc.org/dkim/ietf-list-rules.html

---

From: Hector Santos <[email-masked]>
Date: Fri, 08 Jun 2007 22:01:43 -0400

> familiar with the operational issues with the various authoritative

hmmmmmm, interesting comment.

I think you got the wrong perception of me.  Nonetheless, from the 
start, I said I wasn't a "DNS administrator," but I am an seasoned 
systems engineer and manager and when I don't see the "DNS 
administrators" offering a solution themselves, then I will attempt to 
provide input myself.

I think the "suggestions" I provided is the best so far.

First you said it doesn't work "at all."  You were proven wrong.

Client resolving the gTLD/ccTLD cut off for the query question would be 
the ideal solution for these requirements.

You say it is impossible.

Well, you haven't convince me that it is impossible.

I also said I will be studying the wildcard behavior of servers, 
starting with Bind, by digging straight into the source.

Now you are basically implying we are beating on a dead horse because 
the authoritative servers may not behave the same in this regard.

Well, if all this is true, we should stop wasting time and go straight 
to a Wildcard/Masking definition concept at the TXT data itself, similar 
to a SPF commands.

Do you have a suggestion for a TXT record format that offers subdomain 
policy masking?

BTW, I wouldn't call it a hack if popular server like BIND has wildcard 
masking logic built-in all over its lookup code.  It is simple though:

file:  bin\named\ns_init.c:

int
ns_wildcard(const char *name) {
     if (*name != '*')
	return (0);
     return (*++name == '\0');
}

ns_wildcard() is used quite extensively,  I would not call that a hack.

-- 
Sincerely

Hector Santos, CTO

---

From: Michael Thomas <[email-masked]>
Date: Fri, 08 Jun 2007 19:08:51 -0700

> gate.mtcc.com TXT "Here I am"

Having spent time with our enterprise management folks who manage
our DNS, I can safely say "were it so simple". This is not a straightforward
proposition for sites who have 100 and 1000's of labels under their top
domain. This would require significant investment in a wildcard
synthesis infrastructure that does not currently exist.        

       Mike
>>> You certainly need to define a signature policy but its not 
>>>       
>> a signing policy.
>>     
>> include:cisco.com 
>>>>     This works as you'd expect since there is a clear line 
>>>>         
>> to the top
>>     
>>>>     of the wildcard.
>>>>
>>>>     subnodes that bear RR's. This is really another case 
>>>>         
>> of 4 but it 
>>     
>>>> works even
>>>>     Note that here, we can take the place of the top level 
>>>>         
>> wildcard by
>>     
>>>>     simply putting a TXT record in. This is true of any 
>>>>         
>> place we might
>>

---

From: John Levine <[email-masked]>
Date: 9 Jun 2007 11:52:46 -0000

>[1] For the RFC lawyers, not of any actual operational value: There
>are some RFCs that specify some of this behaviour, I believe, in the
>context of zone transfers rather than DNS queries but those RFCs
>really just document the behaviour of that one authoritative server,
>and most people consider them a mistake worth forgetting as far as
>being a Standard with a capital S is concerned, rather than just
>documentation of current practice with that one server.

Unfortunately, there seems to be a critical mass of people in the DNS
part of the IETF that believe otherwise, that the only problem with
wildcards is that there are servers that don't do exactly what BIND
does, and the solution is to fix them so they do.

R's,
John

---

From: Hector Santos <[email-masked]>
Date: Sat, 09 Jun 2007 10:18:54 -0400

> servers often have similar functionality, often mostly

Steve,

Please excuse any perceived ignorance.  Maybe I am being to technical 
here, and I am not a DNS administrator but STD 13, has specific and 
clear information about wildcards:

   4.3.3. Wildcards

   In the previous algorithm, special treatment was given to RRs
   with owner names starting with the label "*".  Such RRs are
   called wildcards.  Wildcard RRs can be thought of as instructions
   for synthesizing RRs. When the appropriate conditions are met,
   the name server creates RRs with an owner name equal to the query
   name and contents taken from the wildcard RRs.

So I am not sure what it meant by "Wildcards don't actually exist in 
DNS."

It seems you are saying since one server had offered it, and other 
followed, that it been "kludged" in and now a common behavior but its 
not actually part of the specification, therefore unreliable?

Seems pretty clear in STD 13.

What am I missing here?

-- 
Sincerely

Hector Santos, CTO

---

From: "Hallam-Baker, Phillip" <[email-masked]>
Date: Mon, 11 Jun 2007 13:33:17 -0700

That is somewhat unfair.

Until DNSSEC there was no need to standardize behavior at all unless you were using different software flavors to publish your DNS information. Even the zone transfer issue was irrelevant since the semantics of the woldcard were defined in the publishing server.

DNSSEC reqired there be a single interpretation of a signed wildcard. The advantage of chosing the narrow matching rules was that it was always possible for someone to add in extra records if broader semantics were desired but not possible to take records out if the match was too loose.

As you point out in your other messages, DNS does provide support for this particular use case, even though it might not be what we want.

> [mailto:[email-masked]] On Behalf Of John Levine
> Sent: Saturday, June 09, 2007 7:53 AM
> To: [email-masked]
> >[1] For the RFC lawyers, not of any actual operational 
> value: There are 
> >some RFCs that specify some of this behaviour, I believe, in the 
> >context of zone transfers rather than DNS queries but those 
> RFCs really 
> >just document the behaviour of that one authoritative 
> server, and most 
> >people consider them a mistake worth forgetting as far as being a 
> >Standard with a capital S is concerned, rather than just 
> documentation 
> Unfortunately, there seems to be a critical mass of people in 
> the DNS part of the IETF that believe otherwise, that the 
> only problem with wildcards is that there are servers that 
> don't do exactly what BIND does, and the solution is to fix 
> them so they do.
> 
> R's,
> John
>

---

From: "Hallam-Baker, Phillip" <[email-masked]>
Date: Mon, 11 Jun 2007 13:36:57 -0700

On the contrary, most DNS servers can handle it today.

The only difficulty is if you also have DNSSEC, and that is not much of a problem since your signer process can do the expansion.

> Sent: Friday, June 08, 2007 10:09 PM
> Having spent time with our enterprise management folks who 
> manage our DNS, I can safely say "were it so simple". This is 
> not a straightforward proposition for sites who have 100 and 
> 1000's of labels under their top domain. This would require 
> significant investment in a wildcard
> that DOES

---

From: Frank Ellermann <[email-masked]>
Date: Mon, 18 Jun 2007 20:16:56 +0200

> Seems pretty clear in STD 13.

Nothing, it's only not obvious, and Michael just stumbled over it:

Queries for this.is.a.wild.example won't "see" the wild.example
if it's "blocked" by an "existing" a.wild.example, where "existing"
means "any RR".  

He'd have to "reinsert" the blocked wildcard at is.a.wild.example.

I hope I got that right, because I've no way to verify it.  The
intuitive ideas would be "only the same RR can block a wildcard",
or "only a SoA can block a wildcard", but that's not how it works.

Please correct me, 

 Frank

---

From: Douglas Otis <[email-masked]>
Date: Mon, 18 Jun 2007 13:37:00 -0700

On Jun 18, 2007, at 11:16 AM, Frank Ellermann wrote:

> Queries for this.is.a.wild.example won't "see" the wild.example if  
> it's "blocked" by an "existing" a.wild.example, where "existing"  
> means "any RR".
>

Both wildcard and non-wildcard records need to be placed at _every_  
valid node existing within the zone.

Deprecating the use of A records for discovery extends RFC 1123's  
exclusions of CNAMEs that are problematic with wildcards.

Deprecating the use of A records even eliminates a need for wildcards.

As DNSSEC is deployed, wildcard abuse must be carefully considered.

-Doug

---

From: Frank Ellermann <[email-masked]>
Date: Tue, 19 Jun 2007 00:03:22 +0200

> valid node existing within the zone.

Yes, thanks for the correction, the wildcard counts as "at", not as 
"below".  I had that wrong.  But the opposite was hopefully okay, a
wildcard above an existing node isn't visible at or below this node.

IOW to cover everything below x.example you'd need wildcards at all
existing nodes below x plus x itself.  For SPF it was simpler to
ignore the issue, nodes without MX and without IP anyway can't send
mail, or rather they can try, but it's possible to reject this crap.
For SPF you only need wildcards where they already are (MX, A, or
AAAA).  

No recipe for SSP unfortunately, nobody checks 2822-From addresses
for plausibility at the MX, rejecting anything that can't be okay.

Frank

---

From: Douglas Otis <[email-masked]>
Date: Mon, 18 Jun 2007 17:06:55 -0700

On Jun 18, 2007, at 3:03 PM, Frank Ellermann wrote:

> Douglas Otis wrote:
>
>> Both wildcard and non-wildcard records need to be placed at  
>> _every_ valid node existing within the zone.
>
> existing nodes below x plus x itself.

See RFC 4592.  A wildcard will blocks itself.  Both a wildcard RR and  
a non-wildcard RR record is therefore required.  Synthesis is  
prevented at any existing node, terminal or not.

> For SPF it was simpler to ignore the issue, nodes without MX and  
> without IP anyway can't send mail, or rather they can try, but it's  
> possible to reject this crap. For SPF you only need wildcards where  
> they already are (MX, A, or AAAA).
>

Sender-ID depends upon these SPF records and refers to email- 
addresses within 2822-From addresses.  It seems many limit email- 
addresses to the Internet namespace.  While some systems may utilize  
a different namespace, whether this namespace is accepted and what  
determines the policy associated with this namespace is outside the  
scope of SSP.  An email-address within the Internet namespace must  
resolve a discovery record, or it is not a valid Internet email-address.

DKIM uses only an Internet namespace, however SSP should be able to  
make exceptions.  The general goal behind DKIM is to curtail invalid  
email messages.  A message containing what the recipient considers an  
invalid email-address is also likely to be considered an invalid  
message and refused.  Establishing what is a valid email-address  
establishes a basis for acceptance, and hence limits where policy  
records must also be published.

There are significant advantages in allowing SSP to selectively apply  
to just parameters seen within the RFC 2821 envelope, the SMTP client  
hostname and the MAIL FROM.  This could then allow namespaces to  
safely mix and still simplify where policy must be checked.

To further minimize locations where policy records are checked,  
deprecate A record discovery.

This deprecation offers two significant benefits:

  1- Policy can be referenced from only MX record locations.
  2- CNAME discovery (allowed with A records) is also deprecated.

-Doug