Subject: [Uta] Scanresults mta-sts policies

From: Hanno Böck <[email-masked]>
Date: Sun, 30 Sep 2018 20:11:34 +0200

Hi,

I did now some more scans for MTA-STS and I thought it might be
interesting for the list to learn the results.

A very effective way of finding hosts that support mta-sts is to scrape
the Certificate Transparency logs. (With the exception of hosts that
use wildcard certificates.)

This gave me 697 hosts with an mta-sts subdomain. Of those 416 served
something that looked like an mta-sts policy file, indicating that a
large number (281) are either in the process of deploying MTA-STS and
haven't finished yet or have wrongly implemented it, e.g. by using the
wrong filename/path.

I found a few syntax issues:
* The most worrying one is that 24 hosts use policies like
  "mx: .example.org" which was valid in older drafts. I say this is the
  most worrying, because it may actually lead to delivery failures.
  It'd be good to get them converted quickly before this creates
  hassle. However only 4 of them have "mode: enforce" (with "mode:
  testing" I'm not overly worried).
* 11 hosts use "mode: report", which is also from earlier drafts and
  should be "mode: testing". However a number of hosts have already been
  fixed before I made this scan.
* Some have trailing or leading spaces. It's not entirely clear to me
  if this is something to be considered an invalid policy.
* I also wonder about line endings. 263 use Unix line endings (LF
  only), 154 use Windows line endings (CRLF). The RFC reads like CRLF
  is correct (3.2), which would indicate a large number of bad policies.
  However the formal definition in the RFC also lists <LF>. I am not
  familiar with how to read the formal definition. (It also says
  something about spaces.)

I'd appreciate if someone could clarify whether leading/trailing spaces
and unix line breaks should be considered policies that should be
fixed or if this is okay.

Some stats about modes:
211 mode: enforce
194 mode: testing
11 mode: report (invalid)
no mode: none found.

-- 
Hanno Böck
https://hboeck.de/

mail/jabber: [email-masked]
GPG: FE73757FA60E4E21B937579FA5880072BBB51E42

---

From: Viktor Dukhovni <[email-masked]>
Date: Sun, 30 Sep 2018 14:50:28 -0400

> On Sep 30, 2018, at 2:11 PM, Hanno Böck <[email-masked]> wrote:
> 

Thanks for the data collection!

> This gave me 697 hosts with an mta-sts subdomain.

A small comment on terminology, I find your use of "host" here rather
confusing.  I'm used to speak of "domains" that receive email, and
a given domain's MX hosts ("MX host" is not a formal IETF term, but
is widely used and understood).  The domain part of an email address
need not correspond to any host, and many don't.  The domain names
that appear as the "exchange" field of an MX record RDATA (i.e.
MX hosts) are of course names of hosts, since they are expected to
terminate TCP connections to the SMTP (port 25) service.

Thus, with MTA-STS, the policy is associated with a domain, and is
provided via HTTPS by the host whose name is constructed by prefixing
the label "mta-sts" the policy domain.

-- 
	Viktor.

---

From: James Cloos <[email-masked]>
Date: Sun, 30 Sep 2018 15:48:25 -0400

>>>>> "HB" == Hanno Böck <[email-masked]> writes:

HB> * I also wonder about line endings. 263 use Unix line endings (LF
HB>   only), 154 use Windows line endings (CRLF). The RFC reads like CRLF
HB>   is correct (3.2), which would indicate a large number of bad policies.
HB>   However the formal definition in the RFC also lists <LF>. I am not
HB>   familiar with how to read the formal definition. (It also says
HB>   something about spaces.)

It would be much easier if <LF> works.  I also got the impression
that <CRLF> is required, and manually had to add <CR> to the end
of each line in the file to ensure that it would be served with
them.  A proper, simple text file would be easier.

-JimC
-- 
James Cloos <[email-masked]>         OpenPGP: 0x997A9F17ED7DAEA6

---

From: Viktor Dukhovni <[email-masked]>
Date: Sun, 30 Sep 2018 15:53:58 -0400

> On Sep 30, 2018, at 3:48 PM, James Cloos <[email-masked]> wrote:
> 

https://tools.ietf.org/html/rfc8461#section-3.2

sts-policy-record        = sts-policy-field *WSP
                           *(sts-policy-term sts-policy-field *WSP)
                           [sts-policy-term]

sts-policy-term          = LF / CRLF

-- 
	Viktor.