Subject: [dmarc-ietf] Been Quiet Around Here - Org Domain? Tree Walk?

From: Todd Herr <[email-masked]>
Date: Mon, 14 Mar 2022 10:50:16 -0400

Greetings.

Haven't seen any list traffic in over a week, which either means things
have gone dormant or I've been unsubscribed.

In hopes that it was the former, not the latter...

My most recent understanding of where things stood with regard to the org
domain/tree walk discussions was that Mr. Kitterman had planned to propose
some text to share with the group.

Is this still where we are on things?

-- 

*Todd Herr * | Technical Director, Standards and Ecosystem
*e:* [email-masked]
*m:* 703.220.4153

This email and all data transmitted with it contains confidential and/or
proprietary information intended solely for the use of individual(s)
authorized to receive it. If you are not an intended and authorized
recipient you are hereby notified of any use, disclosure, copying or
distribution of the information included in this transmission is prohibited
and may be unlawful. Please immediately notify the sender by replying to
this email and then delete it from your system.

---

From: Scott Kitterman <[email-masked]>
Date: Tue, 15 Mar 2022 01:51:04 +0000

>Is this still where we are on things?

Yes.  My schedule has not gone at all like I expected the last couple weeks, so I have not done this yet.  Actually, tomorrow is looking good, so maybe news soon.

Scott K

---

From: Scott Kitterman <[email-masked]>
Date: Wed, 16 Mar 2022 00:40:45 -0400

On Monday, March 14, 2022 9:51:04 PM EDT Scott Kitterman wrote:
> On March 14, 2022 2:50:16 PM UTC, Todd Herr 
<todd.herr=[email-masked]> wrote:
> Yes.  My schedule has not gone at all like I expected the last couple weeks,
> so I have not done this yet.  Actually, tomorrow is looking good, so maybe
> news soon.

Sorry for the delay.  Here's my proposed update.

I think I've made the changes to capture the discussion on updating the DNS 
Tree Walk for Organizational Domain determination and policy discovery.

It took a fair amount of editing and I expect you all will have further 
suggestions, so instead of getting up to my elbows in XML, I took the 
published DMARCbis-05 text and updated it directly.  The modified version and 
an rfcdiff are attached.

There is some duplication in the text I put in 4.5 and 4.6 that could be 
improved.  I wasn't sure which location was best for some information, so I 
was probably overly verbose.

You are going to do what you are going to do and I have no kind of authority 
to tell anyone what they should or should not send to the list, but my 
personal preference is that we focus on addressing questions of clarity or if 
the proposed language captures the end point of the last discussion.  I have a 
hope (sure to be crushed) that people won't take this as an opportunity to re-
litigate the entire discussion.

Scott Kitterman

---

From: "John Levine" <[email-masked]>
Date: 16 Mar 2022 23:02:00 -0400

>published DMARCbis-05 text and updated it directly.  The modified version and 

It's closer, but I think it still needs some reorganization.  I think this is where we
want to end up:

Policy domain: if a domain has a dmarc record, that's the policy, otherwise use the org
domain's policy or if no org domain policy, PSD policy.

You need to find org domains if
a) the domain has no DMARC record so you use the org domain's instead, OR
b) the DKIM domain doesn't match the from header domain and policy adkim=r. OR
c) the SPF domain doesn't match the from header domain and policy aspf=r.

To find the org domain for a domain:
  chop the domain to the last five labels and walk up the tree.
  stop when you find a DMARC record with psd or you hit the root.
  if a record has psd=n, that's the org domain
  if a record has psd=y and it isn't the original domain, the org domain is the one below it
  otherwise the org domain is the last (highest) DMARC record you found

Relaxed alignment doesn't change, if two domains have the same org domain, they're aligned.

Minor nit: if a name has two or more DMARC records, that's invalid so pretend it had none.

R's,
John

---

From: Scott Kitterman <[email-masked]>
Date: Wed, 16 Mar 2022 23:27:09 -0400

On Wednesday, March 16, 2022 11:02:00 PM EDT John Levine wrote:
> >published DMARCbis-05 text and updated it directly.  The modified version
> >and an rfcdiff are attached.
> 
> It's closer, but I think it still needs some reorganization.  I think this
> is where we want to end up:
> 
> Policy domain: if a domain has a dmarc record, that's the policy, otherwise
> use the org domain's policy or if no org domain policy, PSD policy.
> 
> b) the DKIM domain doesn't match the from header domain and policy adkim=r.
> OR c) the SPF domain doesn't match the from header domain and policy
> aspf=r.
> 
>   if a record has psd=y and it isn't the original domain, the org domain is
> the one below it otherwise the org domain is the last (highest) DMARC
> record you found
> 

Thanks.  I think that makes sense.  Based on an offlist comment I got, I think 
it needs to say has an explicit psd=n in the record or something like that.  
Since psd=n is the default, it's implicitly true for any record that doesn't 
have psd=y in it.

I'll take another shot at it, but I'm unlikely to finish it tonight.

Scott K

---

From: Alessandro Vesely <[email-masked]>
Date: Thu, 17 Mar 2022 11:02:26 +0100

> c) the SPF domain doesn't match the from header domain and policy aspf=r.

Good point.  It may deserve its own subsection.

>    otherwise the org domain is the last (highest) DMARC record you found

Fine.  DMARC /specifies/ that the org domain MUST publish a record.  Whereas it 
started off allowing to publish a record at the org level only, as a 
simplification w.r.t. SPF, that record is now requisite.

> Relaxed alignment doesn't change, if two domains have the same org domain, they're aligned.

No, we don't repeat the walk for each identifier.  On a mail From:[email-masked], 
assume we have already determined that the org domain is c.d.  Then there is a 
signature with d=e.f.c.d.  It is aligned based on string comparison.

Repeating the tree walk, we'd get a different result if we find psd=y at 
_dmarc.f.c.d.  Is that realistic?

> Minor nit: if a name has two or more DMARC records, that's invalid so pretend it had none.

Does it make sense to continue the walk in that case?

Best
Ale
--

---

From: Todd Herr <[email-masked]>
Date: Thu, 17 Mar 2022 11:35:32 -0400

> I'll take another shot at it, but I'm unlikely to finish it tonight.
I've been working to incorporate the proposed text that Scott submitted the
other day, and following along with the discussion since, and I think I've
got a new draft ready that incorporates the latest comments.

Happy to post that if it'll save you some work, Scott.
--

---

From: "John Levine" <[email-masked]>
Date: 17 Mar 2022 15:50:39 -0400

It appears that Alessandro Vesely  <[email-masked]> said:
>Fine.  DMARC /specifies/ that the org domain MUST publish a record.

No, that's not what it says.  If you encounter psd=y and go back one,
that can be an org domain without a record.  It is currently possible
to have org domains without records and I see no reason to change that.

They're not terribly useful, but they're not useless.

>On a mail From:[email-masked], 

>_dmarc.f.c.d.  Is that realistic?

Yes, of course it is.  Look at some of the PSL entries.

R's,
John

---

From: Douglas Foster <[email-masked]>
Date: Thu, 17 Mar 2022 18:40:37 -0400

In the general case, we allow for this possible configuration:
public suffix domain segments
    <organization boundary>
organization domain for public registration organization
subdomains of the organization
registration point for private registration clients
    <organization boundary>
organization domain for private registration clients
subdomains of the client organization

If the candidate domain is a parent directly above the FROM organization
domain, the names are not aligned and no tree walk is necessary.

If the candidate MAILFROM or DKIM domain matches anything between the FROM
domain and its organization domain, they are aligned without a second tree
walk.

If the candidate domain is a child directly below the FROM domain, then a
tree walk is necessary, to check for an organization boundary.  The search
stops at the FROM domain.  If a boundary is found, they are not aligned, if
none is found, they are aligned.

If the candidate domain is not in a parent-child relationship with the FROM
domain, then a string compare can identify the junction domain where the
siblings come together.   If the junction point is above the organization
domain, the names are not aligned and no tree walk is needed.   Otherwise,
a tree walk is needed to check for an organization boundary.  The walk
proceeds until it reaches the higher of the junction domain or the FROM
domain.

Doug

On Thu, Mar 17, 2022 at 6:03 AM Alessandro Vesely <[email-masked]> wrote:

> _______________________________________________
> dmarc mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/dmarc
>

---

From: Douglas Foster <[email-masked]>
Date: Fri, 18 Mar 2022 02:52:07 -0400

With this comment, we see the pernicious effect of using a binary psd=(y|n)
token to represent a multi-valued role.    When the role indicator is
missing, there is no default value, instead it must be inferred from the
context.    Since the author of the proposed text is confused on this
point, it is certain that the general community will be confused as well.

On Wed, Mar 16, 2022 at 11:27 PM Scott Kitterman <[email-masked]>
wrote:

> > Relaxed alignment doesn't change, if two domains have the same org
> domain,
> > they're aligned.
> >

---

From: Alessandro Vesely <[email-masked]>
Date: Fri, 18 Mar 2022 10:39:28 +0100

> to have org domains without records and I see no reason to change that.

The reason to require an org record is that filters cannot determine that a 
domain is an org domain without it.  The only exception is domains registered 
right below a PSD publishing psd=y.  Otherwise the org domain is the last 
(highest) DMARC record you found.

> Yes, of course it is.  Look at some of the PSL entries.

I may be dumb, but I stared at the PSL for a few minutes and couldn't think of 
non-aligned subdomains of an org domain.  Yes, there are PSDs which are a 
substring of another PSD.  For example .it and .milano.it, or *.kobe.jp and 
!city.kobe.jp.  However, prepending the org domain label you obviously get a 
unique suffix.  Everything at or below the org domain belongs to that org.  Any 
two identifiers belonging to the same org are mutually aligned.

Best
Ale
--

---

From: Alessandro Vesely <[email-masked]>
Date: Fri, 18 Mar 2022 10:40:07 +0100

> subdomains of the client organization

I think you mean hybrid cases like us.com, where X.us.com and Y.us.com are 
distinct organizations, and mail from [email-masked] is also possible.

> domain and its organization domain, they are aligned without a second tree walk.
> 
> If the candidate domain is a child directly below the FROM domain, then a tree 
> walk is necessary, to check for an organization boundary.

For example, From: [email-masked], org=us.com, d=X.us.com should not be aligned 
because there is a boundary in between.  Yet, we're unable to find that 
boundary.  Even using the PSL, us.com, like com alone, shouldn't be a From: domain.

Presumably, we should state that if the From: domain has psd=y, then alignment 
must be strict.

Best
Ale
--

---

From: "John R Levine" <[email-masked]>
Date: 18 Mar 2022 12:11:56 -0400

> (highest) DMARC record you found.

Ah, we agree.  That's what I said.

> I may be dumb, but I stared at the PSL for a few minutes and couldn't think 
> of non-aligned subdomains of an org domain.

Look at these two:

repl.co
id.repl.co

so the example would be a.b.repl.co and c.id.repl.co

There aren't a vast number like that but I see no reason to break them.
In the common case where one name is an ancestor of the other, the normal
DNS cache behavior will make the multiple tree walks effectively free.

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
Please consider the environment before reading this e-mail. https://jl.ly

---

From: Barry Leiba <[email-masked]>
Date: Fri, 18 Mar 2022 18:05:50 +0100

> With this comment, we see the pernicious effect of using a binary psd=(y|n) token to represent a multi-valued role.    When the role indicator is missing, there is no default value, instead it must be inferred from the context.    Since the author of the proposed text is confused on this point, it is certain that the general community will be confused as well.

I don't see that that follows at all.  Regardless of how the tag is
formed, the text needs to specify what happens when it's not present.
Saying, explicitly, that the default is "psd=n" if the psd tag is
absent makes the situation quite clear, with no inference needed.

Barry

> On Wed, Mar 16, 2022 at 11:27 PM Scott Kitterman <[email-masked]> wrote:
>>

---

From: "John Levine" <[email-masked]>
Date: 18 Mar 2022 13:43:52 -0400

It appears that Barry Leiba  <[email-masked]> said:
>> With this comment, we see the pernicious effect of using a binary psd=(y|n) token to represent a multi-valued role.    When the role indicator is missing, there is no default
>value, instead it must be inferred from the context.    Since the author of the proposed text is confused on this point, it is certain that the general community will be confused
>as well.
>

Making explicit psd=n the same as no psd would screw up the tree walk since
it would mean you always stop at the first DMARC record.  On the other hand,
there are plenty of letters, and we can make the default psd=u for unspecified.
You can even put that in your DMARC record if you want.

R's,
John

---

From: Alessandro Vesely <[email-masked]>
Date: Fri, 18 Mar 2022 20:12:07 +0100

> so the example would be a.b.repl.co and c.id.repl.co

Let me use the PSL, as it works with fictitious domains.  I could find the org domain like so:

ale@pcale:~/tmp/zdkimfilter/svn/src$ ./TESTpublicsuffix ../no-svn/public_suffix_list.dat a.b.repl.co
a.b.repl.co -> b.repl.co

So b.repl.co is the org domain.  c.id.repl.co doesn't end with that string, therefore it is not aligned.

I could also have run this:

ale@pcale:~/tmp/zdkimfilter/svn/src$ ./TESTpublicsuffix ../no-svn/public_suffix_list.dat c.id.repl.co
c.id.repl.co -> c.id.repl.co

and conclude that it is not aligned because its org domain differs.  However, string comparison is easier, no?

Best
Ale
--

---

From: "John R Levine" <[email-masked]>
Date: 18 Mar 2022 15:30:09 -0400

You're right, those weren't a good example.

Try these from the PSL:

com.br
users.scale.virtualcloud.com.br

virtualcloud.com.br is not algned with a.users.scale.virtualcloud.com.br

(For the latter name it would start at users.scale.virtualcloud.com.br and 
find psd=y)

cz
usr.cloud.muni.cz

cloud.muni.cz is not aligned with a.usr.cloud.muni.cz

co.uk
cust.retrosnub.co.uk

retrosnub.co.uk is not aligned with a.cust.retrosnub.co.uk

Like I said, there aren't a lot of these but there is no need to break 
them.  The performance issue is insignificant.

Regards,

---

From: Alessandro Vesely <[email-masked]>
Date: Sun, 20 Mar 2022 20:21:32 +0100

On Fri 18/Mar/2022 20:30:09 +0100 John R Levine wrote:
> (For the latter name it would start at users.scale.virtualcloud.com.br and find 
> psd=y)
> 
> cz
> usr.cloud.muni.cz
> 

Good spot, very clever.  In fact, I get: (blank lines inserted)

ale@pcale:~/tmp/zdkimfilter/svn/src$ ./TESTpublicsuffix ../no-svn/public_suffix_list.dat virtualcloud.com.br  a.users.scale.virtualcloud.com.br cloud.muni.cz  a.usr.cloud.muni.cz retrosnub.co.uk a.cust.retrosnub.co.uk

virtualcloud.com.br -> virtualcloud.com.br
a.users.scale.virtualcloud.com.br -> a.users.scale.virtualcloud.com.br

cloud.muni.cz -> muni.cz
a.usr.cloud.muni.cz -> a.usr.cloud.muni.cz

retrosnub.co.uk -> retrosnub.co.uk
a.cust.retrosnub.co.uk -> a.cust.retrosnub.co.uk

That outcome clearly shows that string comparison would fail when applied to org domains /found via PSL/.

> Like I said, there aren't a lot of these but there is no need to break them.

To better understand what those domains are, let me note that if I restrict the PSL to just the ICANN part, as has been sometimes suggested, I get: (still inserting blank lines for readability)

ale@pcale:~/tmp/zdkimfilter/svn/src$ ./TESTpublicsuffix ../no-svn/public_suffix_list_ICANN_only.dat virtualcloud.com.br  a.users.scale.virtualcloud.com.br cloud.muni.cz  a.usr.cloud.muni.cz retrosnub.co.uk a.cust.retrosnub.co.uk

virtualcloud.com.br -> virtualcloud.com.br
a.users.scale.virtualcloud.com.br -> virtualcloud.com.br

cloud.muni.cz -> muni.cz
a.usr.cloud.muni.cz -> muni.cz

retrosnub.co.uk -> retrosnub.co.uk
a.cust.retrosnub.co.uk -> retrosnub.co.uk

By the whole PSL, the same domain can be characterized as an org domain and as a PSD.

> The performance issue is insignificant.

Maybe performance doesn't matter.  However, what do we expect to find out by a tree walk?  We'd come to the same conclusion as using the ICANN only list unless their record contains psd=y.  Correct?

Then, suppose we have the following:
_dmarc.a.users.scale.virtualcloud.com.br NO DATA
_dmarc.users.scale.virtualcloud.com.br   v=DMARC1
_dmarc.scale.virtualcloud.com.br         NO DATA
_dmarc.virtualcloud.com.br               v=DMARC1; psd=y

If we have From: [email-masked], the org domain is users.scale.virtualcloud.com.br, according to Scott's writing.  A signature with d=virtualcloud.com.br is not aligned.

If, instead, we have From: [email-masked], the hybrid org domain is virtualcloud.com.br.  Scott's writing doesn't cover this case.  What about a signature with d=scale.virtualcloud.com.br?  It looks like it should be aligned, since there are no other org domains in between.  Otherwise, we could solve that case by simply stating that hybrid domains can only use strict alignment.  However, we said that dkim.bank cannot sign for whatever.bank.  How do they differ?  We must tune the tree walk in order to allow hybrid domains and their internal subdomains.

Best
Ale
--

---

From: Douglas Foster <[email-masked]>
Date: Sun, 20 Mar 2022 15:45:47 -0400

Terms
We can use "hybrid" organizations if you prefer, but I thought the
public/private distinction matched the PSL list   The important point is
that any DNS tree can have more than one organizational boundary, so we
need to worry about a possible boundary between the FROM domain and a child
domain, as well as a possible boundary between a sibling domain and the
substring where the sibling names converge.   All of that makes the second
tree walk necessary, and means that a walk in the upward direction is
always preferable.

--

Relaxed Alignment and Second tree walk
For relaxed alignment, the rule needs to be that two domains are in relaxed
alignment, if the tree walk up, applied independently to each one, finds
the same organization domain.   This is all that the main body of the
document needs to say.

Because the tree walk can be relatively expensive, and because a
message may have multiple domains to be tested for alignment, it is
desirable to optimize the process.   The words that you copied from my
previous post were a first attempt at describing those optimizations, but
it may not need to be completed.   We could either develop an appendix
which suggests a DNS-optimizing algorithm, or simply observe that DNS
optimization should be performed, letting developers decide how it should
be done.

One of the optimizations is that the alignment test can exit as soon as one
aligned domain is found.
--

Does a "psd" indicator mean that mail is possible or not?

We need to be careful about overloading assumptions into a single token.
 A public suffix domain will never be used for mail.   A private
registration domain may or may not be used for mail.  Ideally, we want to
entity publishing a DMARC "psd" record to clearly indicate whether the
related domain sends mail or not.  Announcing that a domain never sends
mail will provide a stronger repudiation than the next-best option of DMARC
fail with p=reject.
--

Boundaries

We may have three types of organization boundaries:
A public registration, the boundary between a PSD and a registered
organization.
A private registratiion, the boundary between a publicly-registered
organization and subdomains that are independent organizations.
A partitioned subdomain, where a specific subdomain tree wishes to be
treated as independent of the parent organization for alignment purposes,
but sibling domains continue to be evaluated as part of the parent..

The first two configurations can be detected by a DMARC record published by
the registrar, indicating an organization boundary below, or by the
registered organization publishing a DMARC record indicating an
organization boundary above.  The last configuration can only be identified
using a top-of-organization record indicating a boundary above.

Defaults
To repeat what has been stated previously, a single DMARC record can appear
in several roles:
- a subdomain-specific record which is not adjacent to an organization
boundary
- an organization domain record which has a boundary above and organization
subdomains below
- a registrar record which has an organization boundary below and does not
accept mail itself
- a private registrar record which has an organization boundary below, is
not an organization domain so there is no boundary above, and accepts mail
- a private registrar record which has an organization boundary above and
below, and also accepts mail.

When the record is not flagged, any of these roles are possible, so there
is no default.  The actual role must be assigned by heuristic.

How does anyone propose to explain this complexity with clarity using only
"psd=(y,n)"?   The statement "psd=n is the default" assumes a simplicity
that does not exist.

This complexity occurs because not all organization boundaries are PSO
registrations.   Therefore the fixation on "psd" is inappropriate.   We
need to explain the different types of organization boundaries that can
affect DMARC, and adopt neutral terminology that only refers to PSDs when
we are talking about PSO registration points.

Doug

---

From: "John R Levine" <[email-masked]>
Date: 20 Mar 2022 15:47:57 -0400

On Sun, 20 Mar 2022, Alessandro Vesely wrote:
> To better understand what those domains are, let me note that if I restrict 
> the PSL to just the ICANN part, as has been sometimes suggested, ...

Not any more.  The people suggesting it misunderstood what the PSL 
contained.  There are applications where only the ICANN part is useful but 
DMARC is not one of them.

> Maybe performance doesn't matter.  However, what do we expect to find out by 
> a tree walk?  We'd come to the same conclusion as using the ICANN only list 
> unless their record contains psd=y.  Correct?

No, that is completely wrong.  Please review the previous thousand 
messsages about why we are switching to a tree walk.

R's,
John

---

From: Scott Kitterman <[email-masked]>
Date: Sun, 20 Mar 2022 21:54:28 +0000

>messsages about why we are switching to a tree walk.

For the record, I was the one that suggested using only the ICANN part.  It was a bad suggestion that would cause more problems than it would solve.  Dumping PSL and moving to something else is the right plan 

Scott K

---

From: Barry Leiba <[email-masked]>
Date: Sun, 20 Mar 2022 23:47:07 +0100

> messages about why we are switching to a tree walk.

Rather than being snarky about reviewing the previous thousand
messages, perhaps you could write up a one-paragraph summary of the
result?  Hhere's why:  If it took us a thousand messages, literally or
metaphorically, and if some are still showing a lack of clarity about
why we got here, then (1) a brief reminder posted here wouldn't hurt
and (2) readers in the future are going to want to know as well, and
having something we can put in the document to explain it will be
really useful.

Then the text we settle on can go into the draft and it will be
clearly documented so that others -- future implementors, the IESG
during the approval process, whoever -- will know the story and will
have fewer questions.

Barry

---

From: Scott Kitterman <[email-masked]>
Date: Sun, 20 Mar 2022 18:53:48 -0400

> Happy to post that if it'll save you some work, Scott.

I missed this when you sent it, sorry.

I've started work on fixing things up based on John's feedback.  I'll provide a 
diff to what I sent last time to make it easier for you to continue from where 
you are.  At this point I'd prefer to finish what I've got and then we do a new 
draft with that in it.

Scott K

---

From: Scott Kitterman <[email-masked]>
Date: Sun, 20 Mar 2022 19:52:44 -0400

> Minor nit: if a name has two or more DMARC records, that's invalid so

I think this addresses the above as well as any other comments.  Unless anyone 
has any strong objections, I'd suggest Todd incorporate this into his planned 
update and then we see where we are.

Scott K

---

From: Scott Kitterman <[email-masked]>
Date: Sun, 20 Mar 2022 20:23:58 -0400

> during the approval process, whoever -- will know the story and will

Generically, I think that's quite reasonable and I'll give it a shot.  
Specifically, I am also feeling some frustration about this.  My impression is 
that whenever this topic gets mentioned, there are certain participants 
attempting to rehash the argument over again and unwilling or unable to accept 
the arguments that have been made.  I didn't look back and see who wrote this 
or who it was written to, so I have no idea about the conversation previous to 
this on the thread.

Working from memory (so chime in if I miss something):

The PSL was created to deal with the lack of a clear way to distinguish DNS 
boundaries related to web technologies.  It's purpose is unrelated to email.  
While it served reasonably well for the initial DMARC deployment, it has a 
number of limitations that lead it to not being an optimal choice for 
determining DMARC Organizational Domains:

1.  It is by design, incomplete.  Due to lack of resources, the PSL 
maintainers have stated that not all domains can be included.

2.  Due to the differing purpose of the PSL, there are cases where the correct 
data for the PSL is not the correct data for DMARC (see the recent message 
from John Levine with examples).

3.  PSL is maintained by a single private organization with limited external 
oversight and no documented configuration management process.  It could 
disappear tomorrow or be completely changed if the people running it decided 
to do so.  A protocol based on an unstable reference such as this is not 
suitable for standardization by the IETF.

The alternative Tree Walk mechanism that the working group is currently 
finishing work on has none of these disadvantages.  We believe it will produce 
more correct results and be more maintainable than continued used of the PSL.  
We also believe that, in practice, it is backward compatible with existing 
usage.

To the extent we have found differences between the results with PSL versus 
Tree Walk, the Tree Walk results have all been more correct.

My predication is we will now get to redo the argument about this, just like 
we do every time it comes up.

Scott K

---

From: "John R Levine" <[email-masked]>
Date: 20 Mar 2022 21:47:18 -0400

> perhaps you could write up a one-paragraph summary of the result

Good thought.

The PSL is intended for web browsers, mostly to prevent "supercookies" and 
secondarily to help browsers display domain names more intuitively. It is 
basically a coincidence that it more or less works to find DMARC 
organizational domains.  It is manually maintained, chronically out of 
date, and there are org domains it cannot describe at all, the ones that 
are vanity TLDs like .BANANAREPUBLIC.  I have great respect for the PSL 
maintainers, several of whom I know personally, but they are quite clear 
that they're just a handful of volunteers, and browsers are its main 
target.

The goal of switching to a tree walk is to allow domains make their own 
statements about what their org domains are and find those org domains 
more reliably.  Given that software changes slowly, we need to keep in 
mind that there will be DMARC records published under the current rules, 
and software interpreting them using the PSL for a long time, so a new 
design should produce results no worse than what we get now.  They need 
not be identical, since the PSL can get things wrong, which is why I say 
no worse.  The tree walk MUST (in the English sense, not the RFC sense) 
work wthout reference to the PSL in view of the limits described above, 
and if we were to tell people to keep using the PSL, what's the incentive 
to change anything?

I believe that the tree walk design that Scott and I are closing in on 
will produce good results efficiently.  The only change to the DMARC 
record format is psd=u/y/n which will be needed in only an infinitesimal 
fraction of DMARC records, the rest being unchaanged from what they are now.

Regards,

---

From: Douglas Foster <[email-masked]>
Date: Mon, 21 Mar 2022 07:05:38 -0400

The tree walk needs introductory material that explains the complexities
created by public registries, private registries, and partitioned
organization sub-units.   We cannot expect readers to accept our solution,
or correctly apply it, if they do not understand the problem we are trying
to solve.   RFC 7489 implies that there are only PSO registrations, and
that language has not been fixed.

The tree walk section needs a redesign.  The basic tree walk is a
function:   Given one domain, it returns the organization domain and
default policy associated with that domain.   So it should be explained
along those lines.   There are multiple points in the evaluation where the
tree walk function must be invoked;  the description of those circumstances
and those inputs needs to be explained separately.  By mixing the first
tree walk with the subsequent ones makes the entire section obscure.

Doug Foster

---

From: Alessandro Vesely <[email-masked]>
Date: Mon, 21 Mar 2022 12:34:08 +0100

On Sun 20/Mar/2022 20:47:57 +0100 John R Levine wrote:
> No, that is completely wrong.

I guess you didn't read what I wrote.  Let me restate.

Currently, we have this situation:

ale@pcale:~/tmp/zdkimfilter/svn/src$ dig _dmarc.virtualcloud.com.br txt

; <<>> DiG 9.16.22-Debian <<>> _dmarc.virtualcloud.com.br txt
;; global options: +cmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 52124
;; flags: qr rd ra ad; QUERY: 1, ANSWER: 1, AUTHORITY: 0, ADDITIONAL: 1

;; OPT PSEUDOSECTION:
; EDNS: version: 0, flags:; udp: 1232
; COOKIE: 16cef50e4a9c56f301000000623854b73b6e24c1798b712e (good)
;; QUESTION SECTION:
;_dmarc.virtualcloud.com.br.	IN	TXT

;; ANSWER SECTION:
_dmarc.virtualcloud.com.br. 3600 IN	TXT	"v=DMARC1; p=none; rua=mailto:[email-masked]; ruf=mailto:[email-masked]; rf=afrf; sp=none; fo=0:1:d:s; pct=100; ri=86400; aspf=s"

Hence, a tree walk starting at a.users.scale.virtualcloud.com.br will end up there.  Since com.br and br have no DMARC record, virtualcloud.com.br is going to be the actual org domain for any subdomain thereof.  That's what I meant by "come to the same conclusion as using the ICANN only list".

Since you strove to find an example where string comparison doesn't determine alignment, I assume that the tree walk should be specified so as to mirror the results of the full PSL.

> Please review the previous thousand messages about why we are
> switching to a tree walk.

I'm not so much interested in the why as in the how, therefore I think that reading just Scott's message of last Wednesday[*] suffices.  His rewriting of Section 4.6 considers two cases:

    1.  If a DMARC record contains the psd= tag set to 'n' (psd=n), this

        is the Organizational Domain and the selection process is

        complete.

    2.  From the DMARC records that do not contain the psd= tag set to

        'y' (psd=y), select the record for the domain with the smallest

        number of labels.  This is the Organizational Domain and the

        selection process is complete.

Given that, it seems that, if virtualcloud.com.br sets psd=y, it cannot become the org domain for d=scale.virtualcloud.com.br.  OTOH, if it does not set psd=y, users.scale.virtualcloud.com.br will not result the org domain of a.users.scale.virtualcloud.com.br, as happens using the full PSL.

Best
Ale
-- 

[*] https://mailarchive.ietf.org/arch/msg/dmarc/H_PfAuJP95OQ7WSA-rIxtl3XOx4/

---

From: Alessandro Vesely <[email-masked]>
Date: Mon, 21 Mar 2022 12:58:18 +0100

> from John Levine with examples).

No, the PSL results are correct for those examples.  The point of those 
examples is to show that the relaxed alignment rule:

   Two identifiers are aligned iff their org domains coincide

cannot be shortened to string comparison.  It is necessary to find the org 
domain of each identifier and then compare them.  However, it is not clear 
under what conditions the org domains found by the tree walk are the same as 
those found by the (full) PSL.

Best
Ale
--

---

From: Alessandro Vesely <[email-masked]>
Date: Mon, 21 Mar 2022 17:04:32 +0100

Ouch, forget my previous message(s).

> _dmarc.virtualcloud.com.br               v=DMARC1; psd=y

As I said, the above doesn't work.  After thinking once more, it 
should be:

_dmarc.a.users.scale.virtualcloud.com.br v=DMARC1 (possibly)
_dmarc.users.scale.virtualcloud.com.br   v=DMARC1; psd=y
_dmarc.scale.virtualcloud.com.br         NO DATA
_dmarc.virtualcloud.com.br               v=DMARC1

Maybe it's me, but it doesn't look straightforward at all.

Best
Ale
--

---

From: "John R Levine" <[email-masked]>
Date: 21 Mar 2022 12:46:43 -0400

> Ouch, forget my previous message(s).

Done.

> As I said, the above doesn't work.  After thinking once more, it should be:
>
And maybe
  _dmarc.com.br                             v=DMARC1; psd=y

> Maybe it's me, but it doesn't look straightforward at all.

It's straightforward enough.

It's not obvious to me whether the org domain for 
users.scale.virtualcloud.com.br should be users.scale.virtualcloud.com.br 
or virtualcloud.com.br but I don't think it matters.  Situations like 
that, a PSD which is also a mail domain, are very rare and since the PSD 
has a DMARC record, one probably doesn't need to use its org domain to do 
DMARC checks.

Perhaps we can say that the result of doing relaxed alignment on a domain 
with psd=y is undefined, i.e., "don't do that."

Regards,

---

From: Alessandro Vesely <[email-masked]>
Date: Mon, 21 Mar 2022 18:42:42 +0100

On Mon 21/Mar/2022 17:46:43 +0100 John R Levine wrote:
> 
>> After thinking once more, it should be:
>>

I think I was fooled by the idea that psd=y belongs to domains nearer to the root.

> users.scale.virtualcloud.com.br should be users.scale.virtualcloud.com.br or 
> virtualcloud.com.br but I don't think it matters.  Situations like that, a PSD 
> which is also a mail domain, are very rare and since the PSD has a DMARC 
> record, one probably doesn't need to use its org domain to do DMARC checks.
> 
> Perhaps we can say that the result of doing relaxed alignment on a domain with 
> psd=y is undefined, i.e., "don't do that."

Hm, apparently yes.  According to the definition, two identical domains having 
psd=y are in strict alignment but not in relaxed alignment, which is somewhat 
counterintuitive.

Best
Ale
--

---

From: Scott Kitterman <[email-masked]>
Date: Mon, 21 Mar 2022 22:02:03 +0000

>psd=y are in strict alignment but not in relaxed alignment, which is somewhat 

Actually, no:

"If this process does not determine the Organizational Domain, then
   the initial target domain is the Organizational Domain."

This text in DMARCbis06 addresses that case.

ScottK

---

From: "John Levine" <[email-masked]>
Date: 21 Mar 2022 22:47:37 -0400

I took a look and I think we're pretty close.  I have two related
suggestions.

It is confusing that the default for psd is psd=n but an explicit
psd=n isn't the same as no psd.  So I suggest allowing threee values
for psd=y/n/u with "u" for unspecified being the default.

In the three numbered steps in sec 4.8 it says first to look for any psd=n,
then any for psd=y.  Consider this somewhat contrived case:

x.a.b.c.d.e  v=DMARC1;
b.c.d.e      v=DMARC1; psd=y
c.d.e        (nothing)
d.e          v=DMARC1; psd=n
e            (nothing)

The current text says the org domain would be d.e but I think we want
it to be a.b.c.d.e.

The fix is simple, combine the first two steps so you stop at the
longest label with psd=y OR psd=n, and it's that label if psd=n or
the one below if it's psd=y.  Arcane sort of special case, if the
original domain has a psd=y/n, that's the org domain and no tree walk
is needed.

R's,
John

---

From: Scott Kitterman <[email-masked]>
Date: Tue, 22 Mar 2022 04:19:48 +0000

>original domain has a psd=y/n, that's the org domain and no tree walk

Thanks.  I agree that's the right answer.  I'll take another shot at it.

I had considered just saying there's no default, but I guess using "u" works too if you think that's better.

Scott K

---

From: Barry Leiba <[email-masked]>
Date: Tue, 22 Mar 2022 12:27:46 +0100

Yes, I thought about that too: just omit psd rather than explicit
psd=u.  Is there a reason not to do that?  What's the value of
explicit psd=u, as long as the spec says what absent psd means?

b

---

From: Scott Kitterman <[email-masked]>
Date: Tue, 22 Mar 2022 11:42:13 +0000

I haven't come up with anything.  Anyone?

Scott K

---

From: "John Levine" <[email-masked]>
Date: 22 Mar 2022 12:15:09 -0400

>explicit psd=u, as long as the spec says what absent psd means?

I don't think there is any other place where the default is not one
of the explicit options.  The benefit of psd=u, such as it might be,
is to make it more consistent, and to be clear that we really mean
it when we say that psd=y, psd=n. and psd=u mean three different
things.

This is not a big deal but I do think I've seen confusion, e.g.,
people wrongly concluding that all existing DMARC records will have to
have psd=n added. (I suppose those people will now demand psd=u, so
you can't win.)

R's,
John

---

From: Ken O'Driscoll <[email-masked]>
Date: Tue, 22 Mar 2022 17:35:03 +0000

> 
> I don't think there is any other place where the default is not one of
> the explicit options.  The benefit of psd=u, such as it might be, is to
> make it more consistent, and to be clear that we really mean it when we
> say that psd=y, psd=n. and psd=u mean three different things.
> 
> This is not a big deal but I do think I've seen confusion, e.g., people
> wrongly concluding that all existing DMARC records will have to have
> psd=n added. (I suppose those people will now demand psd=u, so you can't
> win.)

+1 

Having different behaviour for the absence of the tag and the default value will be unnecessarily confusing and not intuitive.

Ken.

---

From: Douglas Foster <[email-masked]>
Date: Tue, 22 Mar 2022 21:47:24 -0400

This is not a new question, nor should it be a difficult one.

Absence of an indicator means only one of these possibilities:  that the
DMARC policy was implemented before DMARCbis was published, or implemented
without reading DMARCbis, or implemented by a lazy domain owner.

If the specification says that absence of an indicator means that the
domain owner chose to communicate a non-boundary status for the policy,
then the specification is lying.   The only way a domain owner communicates
intent is by publishing information.

Missing data is always NULL.   You can guess intent from other clues, but
not from the missing data.   There is no default value, only missing data.

Doug

---

From: Alessandro Vesely <[email-masked]>
Date: Wed, 23 Mar 2022 11:37:32 +0100

> Having different behaviour for the absence of the tag and the default value will be unnecessarily confusing and not intuitive.

+1, I don't agree so much on obscuring the actual meaning, but since there are 
(at least) three states, enumerating three values is clear-headed.

Best
Ale
--

---

From: Douglas Foster <[email-masked]>
Date: Wed, 23 Mar 2022 07:08:16 -0400

At one point, Ale had suggested a fourth state, for "Both", when an
organization boundary exists above (to a PSO) and below (for private
registrations.)   We can finesse that by ignoring the "boundary below"
indicator when the evaluation does not move up from below.

But we do have a difference between PSOs, which never send mail, and
private registrars, which may or may not send mail from the domain or
subdomain used as a private registration point.  It seems desirable to
resolve this ambiguity so that we can reliably know that a true PSO cannot
be impersonated, while allowing private registrars to document their
configuraion.

A "sendsmail=(y,n)" indicator would accomplish this purpose.   For "psd=y"
records, the default will be "sendsmail=n", while for other records the
default will be "sendsmail=y".   This ensures that most participants do not
need to publish the indicator.   It has two expected uses:
- "sendsmail=y" For private registrars to indicate that they also send mail
from the "psd=y" domain name, and
- "sendsmail=n" For an organization that only sends mail from subdomains,
and wants to strongly protect the organizational domain from impersonation.

Doug

On Wed, Mar 23, 2022 at 6:37 AM Alessandro Vesely <[email-masked]> wrote:

> > Having different behaviour for the absence of the tag and the default
> value will be unnecessarily confusing and not intuitive.
>
>

---

From: Alessandro Vesely <[email-masked]>
Date: Thu, 24 Mar 2022 13:01:39 +0100

On Wed 23/Mar/2022 12:08:16 +0100 Douglas Foster wrote:
> But we do have a difference between PSOs, which never send mail, and private 
> registrars, which may or may not send mail from the domain or subdomain used as 
> a private registration point.  It seems desirable to resolve this ambiguity so 
> that we can reliably know that a true PSO cannot be impersonated, while 
> allowing private registrars to document their configuration.
> 
> A "sendsmail=(y,n)" indicator would accomplish this purpose.

For documentation purposes, although I'd have preferred meaningful, explicit 
tokens, if people much more experienced than me insist that obscurity is 
advisable in this case, I don't agree but I accept it.

For security, a private registrar should set psd=y.  If it sets psd=n, it 
forces all registrants below that point to do the same.  If the From: domain 
has psd=y, you know that they send mail because you received it.  In that case, 
it can only authenticate by strict alignment.

Perhaps, we could advise private registrars that they had better use an 
intermediate label with psd=y as a registration point if they want more DMARC 
flexibility at their base domain.

Best
Ale
--

---

From: Douglas Foster <[email-masked]>
Date: Thu, 24 Mar 2022 09:01:24 -0400

We know that DMARC Fail is a weak result because in-transit changes may
cause a message that was verifiable at first hop to be unverifiable at last
hop.

  Consequently, it is desirable to distinguish between Fail and
Never-sends-mail.   We should not assume that a domain sends mail simply
because a message arrives with their name on it.

Real PSOs never send mail, and I suspect that many private registries do
not send mail from the specific subdomain used for client registration.  So
my default expectation for registrars is that they do not send mail, but we
need a way for private registrars to announce that they are an exception

On Thu, Mar 24, 2022, 8:02 AM Alessandro Vesely <[email-masked]> wrote:

> private

---

From: "Murray S. Kucherawy" <[email-masked]>
Date: Thu, 24 Mar 2022 10:45:14 -0700

> value will be unnecessarily confusing and not intuitive.

I'm confused.  In the absence of the tag, don't you apply the default?
That is, aren't these necessarily the same thing?

-MSK

---

From: "John Levine" <[email-masked]>
Date: 24 Mar 2022 14:53:13 -0400

It appears that Murray S. Kucherawy  <[email-masked]> said:
>-=-=-=-=-=-

Currently, no.  psd=n means one thing, psd=y means another thing, and no psd at all means a third.

My suggestion is to allow explicit or default psd=u for the third thing.

R's,
John

---

From: Douglas Foster <[email-masked]>
Date: Thu, 24 Mar 2022 18:04:44 -0400

Since some of you have a default value, and I do not, we must not have a
shared understanding of the underlying algorithm.   I will document mine,
perhaps one of you can explain your alternative.

Since we lack consensus about psd=(y|n|u), I will use symbolic tokens, with
these equivalences:

<registrar policy> (psd=y)

<organization default policy> (psd=n)

<single-domain policy> (psd=u)

My Algorithm:

Primary Tree Walk

The primary tree walk searches for the Organization Domain of the From
address.

When an untagged policy is found, the name and policy are cached as
candidates for the organizational domain and default policy.   The walk
proceeds up the tree.

If the next domain upward is untagged, it is ignored and the walk proceeds
up the tree with the cached information from previous steps.

If the next policy found is also untagged, the previous policy is
interpreted as a <single domain policy> and therefore not of interest.
The current domain and policy are cached as the new candidates for
organizational domain and default policy.  The walk proceeds.

If the next policy found is tagged as <organizational default policy>, then
the cached domain and policy are interpreted as a <single domain policy>
and therefore not of interest.   The current domain and policy become the
confirmed organizational domain and default policy.  The walk terminates.

After an untagged policy, if the next domain up the tree is tagged as a
<registrar policy>, then the cached domain is the organization domain and
the current domain is the default policy.   However, if there is
an intervening domain with no policy, the results are ambiguous.   The
organization and its registrar are not compliant with DMARCbis.  Either
way, the walk terminates.

After an untagged policy, if the walk proceeds all the way to the TLD
without finding another policy, then the cached domain and policy are
selected as the organizational domain and policy.

Secondary Tree Walk

The secondary tree walk checks for presence or absence of a private
registrar between the candidate domain and the From address organizational
domain.   Private registrations can only be detected if their policies are
explicitly tagged.

Consequently, a domain without a policy, a domain tagged with a
<single-domain> policy, or a domain with an untagged policy are treated
equally.   Alignment is not disproven, so the walk continues up the tree to
the termination point.

If any domains are tagged as <organization default policy> or <registrar
policy>, then the domains are not aligned and the walk terminates so that
the next candidate domain can be evaluated.

Doug Foster

---

From: Douglas Foster <[email-masked]>
Date: Thu, 24 Mar 2022 20:02:55 -0400

Couple of corrections and extensions to the previous note:

The <registrar policy> applies only if it is the first policy found, and
the next lower name, which has no policy, is the organization domain.

If an untagged policy immediately precedes the <registrar policy>, then the
cached domain is the organization domain and the cached policy is the
default policy.

If a gap exists between an untagged policy and the <registrar policy>, then
the results are ambiguous.

- - -
If an exact-match policy is found, and the organization domain is still
needed for relaxed alignment, then the current domain is cached as a
candidate organization domain.   The default policy is not needed.

If an exact-match policy is not found, then the cache begins empty.   It
will be initialized when the first untagged policy is found.

If no policy is found at any level, then the domain does not participate in
DMARC and the DMARC result is NULL.

DF

On Thu, Mar 24, 2022 at 6:04 PM Douglas Foster <
[email-masked]> wrote:

> Since we lack consensus about psd=(y|n|u), I will use symbolic tokens,
> with these equivalences:
>
> If the next policy found is tagged as <organizational default policy>,
> then the cached domain and policy are interpreted as a <single domain
> policy> and therefore not of interest.   The current domain and policy
> become the confirmed organizational domain and default policy.  The walk
> terminates.
>

---

From: Scott Kitterman <[email-masked]>
Date: Fri, 25 Mar 2022 17:10:13 +0000

>My suggestion is to allow explicit or default psd=u for the third thing.

I've revised my opinion based on the discussion.  I agree this is the way to go.  I'll put together some words in the next day or three.

Scott K

---

From: Scott Kitterman <[email-masked]>
Date: Fri, 25 Mar 2022 17:13:23 +0000

>flexibility at their base domain.

Based on the current draft, this is not correct.  An exact match is the org domain, even if PSD=y, so even if the policy uses the relaxed alignment approach, it will still be aligned.

Scott K