Subject: [Uta] MTA STS and HTTPS fetch timeouts

From: Daniel Margolis <[email-masked]>
Date: Sun, 11 Sep 2016 11:56:55 +0000

There was some discussion of this at IETF96: Should we be specifying
timeouts on the HTTPS GET during policy fetch?

I was looking at RFC 2821's timeouts for comparison (section 4.5.3.2):

"Based on extensive experience with busy mail-relay hosts, the minimum
per-command timeout values SHOULD be as follows..."

The initial 220 timeout is recommended to be 5 minutes. But then I took a
look at qmail and postfix, and if I am reading the docs correctly their
default timeouts on connect are 60 and 30 seconds respectively.

So this sort of makes me think that we should not bother to suggest
specific timeouts in the RFC, as I am not sure how useful they will be.
(Now, granted, the world has changed since RFC 2821 was written; it's
hardly a sign of failure if 15 years later your timeouts aren't quite the
norm!)

Any contrary opinions on this? And if so, any specific thoughts on what
those timeouts should be?

Dan

---

From: Viktor Dukhovni <[email-masked]>
Date: Sun, 11 Sep 2016 16:17:37 +0000

> those timeouts should be?

I, for one, would like some guidance.  

I know what to expect in the SMTP space, where with Sendmail, Qmail,
Postfix or Exim each connection consumes considerable resources,
and MTA listeners can fall behind the connection rate under high
load.  Even if the TCP 3-way handshake completes, a busy MTA might
not return a 220 banner immediately (reverse DNS lookup latency or
just not enough resources to immediately accept the connection).
(The Postfix "connect" timeout is not only the timeout for completing
the TCP handshake, but is also overloaded as the timeout for the
220 banner).

I am not sure what to expect with HTTPS.  Guidance on the timeouts
would not only help senders to avoid unreasonably aggressive short
timeouts, but may guide receiving systems in selecting an HTTPS
server with appropriate response characteristics.

My instinct is that with (typically) non-forking HTTPS servers,
much higher connection concurrency can be sustained than with
SMTP, and that responses will likely exhibit much lower latency
(in large part because no reverse DNS lookups take place, but
also because HTTP servers are generally scalable to very high
connection counts).

So my sense is that ~5s is about right for a (1 RTT) TCP connection
(assuming both peers are on Earth).  Any suggestions about the TLS
handshake?  What's a reasonable minimum time-out for that?  Unlike
TCP TLS can be 2 RTTs for a full handshake, and TLS requires some
CPU on the server.  Returning the policy in response to the HTTP
request should be quite fast, so perhaps 5s is generous enough for
that?

-- 
	Viktor.

---

From: Alexey Melnikov <[email-masked]>
Date: Mon, 12 Sep 2016 10:30:46 +0300

Hi Daniel,

> On 11 Sep 2016, at 14:56, Daniel Margolis <[email-masked]> wrote:
> 
> There was some discussion of this at IETF96: Should we be specifying timeouts on the HTTPS GET during policy fetch?
> 
> "Based on extensive experience with busy mail-relay hosts, the minimum per-command timeout values SHOULD be as follows..."
> 
> The initial 220 timeout is recommended to be 5 minutes. But then I took a look at qmail and postfix, and if I am reading the docs correctly their default timeouts on connect are 60 and 30 seconds respectively. 
> 
> So this sort of makes me think that we should not bother to suggest specific timeouts in the RFC, as I am not sure how useful they will be. (Now, granted, the world has changed since RFC 2821 was written; it's hardly a sign of failure if 15 years later your timeouts aren't quite the norm!) 
> 
> Any contrary opinions on this? And if so, any specific thoughts on what those timeouts should be? 

I am on the fence about specific timeouts, but I think saying something like "HTTP timeouts MUST be configurable" would be a good thing, in order to draw implementors attention to this.

---

From: Daniel Margolis <[email-masked]>
Date: Mon, 12 Sep 2016 09:02:15 +0000

Thanks to both of you for the feedback.

Viktor, I certainly agree that that's a *reasonable* ballpark number, based
on my experience. But I still have a slightly odd feeling about specifying
*specific* numbers in the spec for the reasons I gave.

Alexey, I like your idea. This seems more in line with what I had been
imagining, i.e. we say to implementors, "here be dragons; think twice!"
rather than telling them what specifically they should do. Though (to be
slightly pedantic) this seems slightly odd for a MUST, since
configurability isn't per se a protocol feature. Could we just say,
"Implementors should consider what values may be reasonable for HTTPS
request timeouts; in our initial implementation we expect timeouts of 5
seconds to be sufficient?"

What do you think? There's a tiny touch of bike-shedding to my comment
above, I admit, but if we didn't bikeshed we'd never decide what color to
paint the damn thing, amiright?

Dan

On Mon, Sep 12, 2016 at 9:18 AM Alexey Melnikov <[email-masked]>
wrote:

> > The initial 220 timeout is recommended to be 5 minutes. But then I took
> a look at qmail and postfix, and if I am reading the docs correctly their
> I am on the fence about specific timeouts, but I think saying something
> like "HTTP timeouts MUST be configurable" would be a good thing, in order
> to draw implementors attention to this.
>
>

---

From: Alexey Melnikov <[email-masked]>
Date: Mon, 12 Sep 2016 17:55:14 +0300

Hi Daniel,

> On 12 Sep 2016, at 12:02, Daniel Margolis <[email-masked]> wrote:
> 
> Viktor, I certainly agree that that's a reasonable ballpark number, based on my experience. But I still have a slightly odd feeling about specifying specific numbers in the spec for the reasons I gave. 
> 
> Alexey, I like your idea. This seems more in line with what I had been imagining, i.e. we say to implementors, "here be dragons; think twice!" rather than telling them what specifically they should do. Though (to be slightly pedantic) this seems slightly odd for a MUST, since configurability isn't per se a protocol feature.

It is a manageability feature and such features can be described with RFC 2119 language. The main point I was trying to convey is that implementations shouldn't have hardcoded values, as these might need to be tweaked over time.

> Could we just say, "Implementors should consider what values may be reasonable for HTTPS request timeouts; in our initial implementation we expect timeouts of 5 seconds to be sufficient?" 

That would be a fine thing to say, but, IMHO, it should be said in addition to the above.
> 
> What do you think? There's a tiny touch of bike-shedding to my comment above, I admit, but if we didn't bikeshed we'd never decide what color to paint the damn thing, amiright?
> 
> Dan
> 
>> On Mon, Sep 12, 2016 at 9:18 AM Alexey Melnikov <[email-masked]> wrote:
>> Hi Daniel,
>>

---

From: Viktor Dukhovni <[email-masked]>
Date: Mon, 12 Sep 2016 16:05:07 +0000

> *specific* numbers in the spec for the reasons I gave.

The problem with not giving any specific guidance, is that server
operators no longer know what latency metric they are to be measured
by.  If a particular server takes 3 seconds to respond, but a client
times out after 2 seconds, which one is out of spec?  Is the server
under-provisioned, or is the client timing out prematurely?

You said that Postfix timeouts are shorter than RFC 5321, but this
is not the case, you've confused the TCP connection timeout (30s)
with the command timeouts from RFC5321, which in Postfix are in
fact as specified:

    $ postconf -d | grep '^smtp_.*_timeout = '  # output reordered and FPs pruned
    smtp_connect_timeout = 30s
    smtp_helo_timeout = 300s			# https://tools.ietf.org/html/rfc5321#section-4.5.3.2.1
    smtp_starttls_timeout = 300s
    smtp_xforward_timeout = 300s
    smtp_mail_timeout = 300s			# https://tools.ietf.org/html/rfc5321#section-4.5.3.2.2
    smtp_rcpt_timeout = 300s			# https://tools.ietf.org/html/rfc5321#section-4.5.3.2.3
    smtp_data_init_timeout = 120s		# https://tools.ietf.org/html/rfc5321#section-4.5.3.2.4
    smtp_data_xfer_timeout = 180s		# https://tools.ietf.org/html/rfc5321#section-4.5.3.2.5
    smtp_data_done_timeout = 600s		# https://tools.ietf.org/html/rfc5321#section-4.5.3.2.6
    smtp_quit_timeout = 300s
    smtp_rset_timeout = 20s

    $ postconf -d smtpd_timeout
    smtpd_timeout = ${stress?{10}:{300}}s	# https://tools.ietf.org/html/rfc5321#section-4.5.3.2.7

    (The server timeout for idle client connectiosn is temporarily
     reduced to 10s while and for a short time after the SMTP server
     connection count is maxed out, i.e. the server is under "stress").

The TCP 3-way handshake timeout is not specified in RFC5321, and
the O/S default TCP connect timeout is much too long on multiple
platforms, so Postfix uses 30s as still generous, but more reasonable
bound.

Perhaps with HTTP timeouts are so much less of an issue than with
SMTP, that setting clear expectations is not a requirement?  My
guess is that some sort of "contract" would be helpful.

-- 
	Viktor.

---

From: Daniel Margolis <[email-masked]>
Date: Tue, 13 Sep 2016 11:23:17 +0000

Ah, thanks for explaining the Postfix bit.

It's a fair point about determining who is out of spec. Like I said,
though, it wasn't clear to me what is generally done in such specs; some
(e.g. 2821) seem to specify specific values, whereas others (821, AFAICT)
do not. I agree with the point (which I think you and Alexey are
suggesting) that in the MTA STS case, a timeout on the HTTPS connection has
a sort of cascading impact (i.e. the recipient domain now cannot assume the
sender actually has the policy if the recipient was not reliably serving
it, etc) whereas in some other cases (e.g. HTTP by itself) the connection
timeout manifests in a user-facing way. So that may be a reason to err on
the side of specifying it.

http://kb.mozillazine.org/Network.http.connect.timeout makes me think that
someone who has previously worked on a browser thought 30s was a reasonable
connection timeout (which I suppose doesn't include any additional time
post-connect). Libcurl seems to default to no request timeout, though there
are some other timeouts for specific things (connects, I guess?), if I read
correctly: https://curl.haxx.se/libcurl/c/CURLOPT_TIMEOUT.html. The connect
timeout seems to default to 300s (
https://curl.haxx.se/libcurl/c/CURLOPT_CONNECTTIMEOUT.html).

So given that, 5s seems a bit on the low side to me. But I'm really doing
the handwavy thing here. Half a minute to a minute sort of seems roughly
reasonable.

Thoughts?

On Mon, Sep 12, 2016 at 6:29 PM Viktor Dukhovni <[email-masked]>
wrote:

>     smtp_helo_timeout = 300s                    #
> https://tools.ietf.org/html/rfc5321#section-4.5.3.2.1
>     smtp_mail_timeout = 300s                    #
> https://tools.ietf.org/html/rfc5321#section-4.5.3.2.2
>     smtp_rcpt_timeout = 300s                    #
> https://tools.ietf.org/html/rfc5321#section-4.5.3.2.3
>     smtp_data_init_timeout = 120s               #
> https://tools.ietf.org/html/rfc5321#section-4.5.3.2.4
>     smtp_data_xfer_timeout = 180s               #
> https://tools.ietf.org/html/rfc5321#section-4.5.3.2.5
>     smtp_data_done_timeout = 600s               #
> https://tools.ietf.org/html/rfc5321#section-4.5.3.2.6
>     smtp_quit_timeout = 300s
>     smtp_rset_timeout = 20s
>
>     smtpd_timeout = ${stress?{10}:{300}}s       #
> https://tools.ietf.org/html/rfc5321#section-4.5.3.2.7
>
> _______________________________________________
> Uta mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/uta
>

---

From: [email-masked]
Date: Tue, 13 Sep 2016 09:43:53 -0700 (PDT)

I just took a look at this draft with an eye to implementing it and  I think it
suffers from several more fundamental problems than a failure to recommend
https timeout values.

Apologies is some/most of this has already been pointed out.

I'm going to start in an unusual place: The validation pseudocode in Appendix
1. From there I'll go on to other stuff. But I think the pseudocode, done
right, will be enormously helpful to implementors, so that's going to be
my starting point.

That pseudocode isn't even close to suffciently detailed, making it much more
difficult than it should be to go from specification to implementation. It also
overlaps, and in one place, conflicts, with the steps listed in section 3.5,
which also suffer from being insufficiently detailed.

The pseudocode doesn't properly account for the case where a cached policy says
reject, the mx or tls checks fail, and the policy is reacquired but now says
report.

If I understand the proposal correctly, the pseudocode should look something
like this:

policy_domain = recipient_address_domain()                              [a]
policy = policy_from_cache(policy_domain)
if not policy or is_expired(policy):
  txt_record_name = "_mta_sts." . policy_domain                         [3.3]
  txt_record = dns_lookup_txt(txt_record_name)
  if txt_record and txt_record_valid(txt_record):                       [3.1.1]
    https_url = "https://policy.mta-sts." . 
                policy_domain . "/.well-known/mta-sts/current"          [3.3]
    policy = policy_from_https_endpoint(https_url)
    update_cache = true
policy_rechecked = false;
:policy_recheck:
if policy and policy_valid(policy) and                                  [3.1.2]
   ids_match(policy, txt_record):                                       [3.3]
  if invalid_mx_or_tls(policy):  // check MX and TLS cert               [3.4, 4]
    if mode = report:                                                   [b]
      generate_report()
    if mode = enforce:
      if not policy_rechecked:
        policy = policy_from_https_endpoint(https_url)
        policy_rechecked = true
        update_cache = true
        goto :policy_recheck:
      reject_message()                                                  [4]
      update_cache = false                                              [c]
  if update_cache:
    update_cache(policy)

In regards to the missing references marked with letters:

[a] This document needs to be expicit about how to turn a recipient address
    into a policy domain, even if that process is simply "everything after
    the last @". (And on a side note - are we 100% sure this is the process
    we want?)

[b] rua isn't defined in this document any more so I removed it. Various
    references to draft-ietf-uta-smtp-tlsrpt need to be inserted as noted
    in section 4.

[c] There's nothing in section 4 (or anywhere else AFAICT) to justify turning
    off caching at this point. And even if there was some justification, I
    nevertheless object because disabling caching at this point in the process
    makes it impossible to cleanly tease the policy acquisition mechanism away
    from the SMTP client, since you don't know if you're going to update the
    cache until you've fetched the MX records and made the SMTP connection.

In fact I see no point in reacquiring the policy unconditionally in the reject
case - you should only do so if the policy came from the cache. Why look
something up you've looked up only a fraction of a second before?

And if you are reacquiring policy, I think you should also redo the TXT
record lookup.

One thing this pseudocode now makes clear that you have to choose between
reporting and enforcement. What if someone wants both?

Another thing the psuedocode makes clear is how the sections in the document
don't really follow the actual steps used to implement this protocol. I think
some consideration should be given to reordering the document to more
closely match how an implementation actually works, and there probably should
be more discussion of caching in various places.

Next up: The ABNF specification of the JSON policy format is incorrect in
several places. For exmaple:

sts-record      = WSP %x7B WSP  ; { left curly bracket
                  sts-element   ; comma-separated
                  [             ; list
                  WSP %x2c WSP  ; of
                  sts-element   ; sts-elements
                  ]
                  WSP %x7d WSP  ; } right curly bracket

Keeping in mind that RFC 5234 defines WSP as:

 WSP            =  SP / HTAB

What this says is that a policy statement must consist of a single space/tab
followed by a { followed by a single space/tab followed by a list of elements
separated by space/tab delimited commas followed by ...

I think you get the idea - the white space as specified lacks sufficient
flexibility and fails to allow for line breaks. It certainly doesn't match
the example in section 9.1, which uses normal JSON formatting.

One way to fix this would be to use the convention that's used in 
in the JSON specification itself (RFC 7159). Define ws as follows:

      ws = *(
              %x20 /              ; Space
              %x09 /              ; Horizontal tab
              %x0A /              ; Line feed or New line
              %x0D )              ; Carriage return

And then use if in all the places where you're currently using either WSP
or *WSP. This will fix most, if not all of the ABNF problems.

But I really cannot recommend creating a syntax for a particular JSON instance
that overlaps the more general JSON ABNF. We have a lot of experience with
this approach, including RFC 822 and RFC 1341/RFC 1342, and have found it
almost never works out well. (Which is why this approach is no longer used
in the later versions of those specifications.)

The alternative is to specify your format using higher level specification
tools than ABNF. With XML this is a no-brainer: Use XML Schema or Relax NG.

JSON has a schema language and IMO it's easy to read and use. The problem
is it's still at draft status and hasn't come out as an RFC. (I don't follow
the JSON work so I don't know why this is the case, but since the latest draft
is three years old I'm going to assume it's ratholed for whatever reason.)

It would be one thing if the JSON being used here was complicated - it is
was I'd say use JSON schema and ask for an exception to reference the draft
definitions.

But there's nothing remotely complicated about policy objects, so IMO the way
to go is a prose description along the lines of what was used in RFC 7033, RFC
7266, RFC 7483, and RFC 7484.

You probably want to register a media type for the policy format. Something
like application/mta-sts-policy+json.

This text:

   A size limitation in a sts-uri, if provided, is interpreted as a
   count of units followed by an OPTIONAL unit size ("k" for kilobytes,
   "m" for megabytes, "g" for gigabytes, "t" for terabytes).  Without a
   unit, the number is presumed to be a basic byte count.  Note that the
   units are considered to be powers of two; a kilobyte is 2^10, a
   megabyte is 2^20, etc.

appears to be a discussion of something that's no longer in this draft or
in the tlsrcpt draft. As such, it should be removed.

Finally, the material comparing MTA-STS with DANE, while useful, has no value
to someone implementing this specification. As such, it has  no business being
the first thing in the document. I therefore strongly recommend relegating it
to the final appendix.

That's it for now. There are undoubtedly more issues but I think a new draft
that cleans up some of this stuff will make it much easier to proceed.

				Ned

---

From: Viktor Dukhovni <[email-masked]>
Date: Wed, 14 Sep 2016 02:07:09 +0000

> right, will be enormously helpful to implementors, so that's going to be

My repeated admonitions to write up a more detailed state-machine
were an attempt to make the same point you're making.  The description
of the requisite client-side logic is far too sketchy.  

The description needs to be quite explicit, modulo flexibility on
whether failure policy refresh and retries are synchronous (fetch
new policy and retry immediately) or asynchronous (schedule policy
refresh and defer the mail).  Either approach is legitimate.

-- 
	Viktor.

---

From: Janet Jones <[email-masked]>
Date: Wed, 14 Sep 2016 17:25:59 +0000

Thanks Ned for the general feedback. We will review and get back with you.

Did you have specific feedback for the HTTPS fetch timeouts discussion that this thread is intended to address?

Thanks
Janet

-----Original Message-----
From: Uta [mailto:[email-masked]] On Behalf Of [email-masked]
Sent: Tuesday, September 13, 2016 9:44 AM
To: Daniel Margolis <[email-masked]>
Cc: [email-masked]
Subject: Re: [Uta] MTA STS and HTTPS fetch timeouts

I just took a look at this draft with an eye to implementing it and  I think it suffers from several more fundamental problems than a failure to recommend https timeout values.

I'm going to start in an unusual place: The validation pseudocode in Appendix 1. From there I'll go on to other stuff. But I think the pseudocode, done right, will be enormously helpful to implementors, so that's going to be my starting point.

That pseudocode isn't even close to suffciently detailed, making it much more difficult than it should be to go from specification to implementation. It also overlaps, and in one place, conflicts, with the steps listed in section 3.5, which also suffer from being insufficiently detailed.

The pseudocode doesn't properly account for the case where a cached policy says reject, the mx or tls checks fail, and the policy is reacquired but now says report.

If I understand the proposal correctly, the pseudocode should look something like this:

policy = policy_from_cache(policy_domain) if not policy or is_expired(policy):
    https_url = "https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fpolicy.mta-sts&data=02%7c01%7cJanet.Jones%40microsoft.com%7c09f2173d05b547f49d0e08d3dc1f09cd%7c72f988bf86f141af91ab2d7cd011db47%7c1%7c0%7c636093998283563674&sdata=NEKeLr9rbS4VCzNYIhqsE%2bsQdT%2bgFmblIkqB9LeNY8Y%3d." . 

In fact I see no point in reacquiring the policy unconditionally in the reject case - you should only do so if the policy came from the cache. Why look something up you've looked up only a fraction of a second before?

And if you are reacquiring policy, I think you should also redo the TXT record lookup.

One thing this pseudocode now makes clear that you have to choose between reporting and enforcement. What if someone wants both?

Another thing the psuedocode makes clear is how the sections in the document don't really follow the actual steps used to implement this protocol. I think some consideration should be given to reordering the document to more closely match how an implementation actually works, and there probably should be more discussion of caching in various places.

Next up: The ABNF specification of the JSON policy format is incorrect in several places. For exmaple:

 WSP            =  SP / HTAB

What this says is that a policy statement must consist of a single space/tab followed by a { followed by a single space/tab followed by a list of elements separated by space/tab delimited commas followed by ...

I think you get the idea - the white space as specified lacks sufficient flexibility and fails to allow for line breaks. It certainly doesn't match the example in section 9.1, which uses normal JSON formatting.

One way to fix this would be to use the convention that's used in in the JSON specification itself (RFC 7159). Define ws as follows:

      ws = *(
              %x20 /              ; Space
              %x09 /              ; Horizontal tab

And then use if in all the places where you're currently using either WSP or *WSP. This will fix most, if not all of the ABNF problems.

But I really cannot recommend creating a syntax for a particular JSON instance that overlaps the more general JSON ABNF. We have a lot of experience with this approach, including RFC 822 and RFC 1341/RFC 1342, and have found it almost never works out well. (Which is why this approach is no longer used in the later versions of those specifications.)

The alternative is to specify your format using higher level specification tools than ABNF. With XML this is a no-brainer: Use XML Schema or Relax NG.

JSON has a schema language and IMO it's easy to read and use. The problem is it's still at draft status and hasn't come out as an RFC. (I don't follow the JSON work so I don't know why this is the case, but since the latest draft is three years old I'm going to assume it's ratholed for whatever reason.)

It would be one thing if the JSON being used here was complicated - it is was I'd say use JSON schema and ask for an exception to reference the draft definitions.

But there's nothing remotely complicated about policy objects, so IMO the way to go is a prose description along the lines of what was used in RFC 7033, RFC 7266, RFC 7483, and RFC 7484.

You probably want to register a media type for the policy format. Something like application/mta-sts-policy+json.

This text:

appears to be a discussion of something that's no longer in this draft or in the tlsrcpt draft. As such, it should be removed.

Finally, the material comparing MTA-STS with DANE, while useful, has no value to someone implementing this specification. As such, it has  no business being the first thing in the document. I therefore strongly recommend relegating it to the final appendix.

That's it for now. There are undoubtedly more issues but I think a new draft that cleans up some of this stuff will make it much easier to proceed.

				Ned

_______________________________________________
Uta mailing list
[email-masked]
https://na01.safelinks.protection.outlook.com/?url=https%3a%2f%2fwww.ietf.org%2fmailman%2flistinfo%2futa&data=02%7c01%7cJanet.Jones%40microsoft.com%7c09f2173d05b547f49d0e08d3dc1f09cd%7c72f988bf86f141af91ab2d7cd011db47%7c1%7c0%7c636093998283573688&sdata=Qwsi9%2fHwKV%2fk3elCI4DGYSce845cqzUQp4spIcW9UHM%3d

---

From: Janet Jones <[email-masked]>
Date: Wed, 14 Sep 2016 17:37:14 +0000

Thanks Viktor for the additional feedback.  Trying to close out on the HTTPS fetch timeout issue.  Do you have any additional feedback to Dan's question? 

Reposting Dan's comments and question.

It's a fair point about determining who is out of spec. Like I said, though, it wasn't clear to me what is generally done in such specs; some (e.g. 2821) seem to specify specific values, whereas others (821, AFAICT) do not. I agree with the point (which I think you and Alexey are suggesting) that in the MTA STS case, a timeout on the HTTPS connection has a sort of cascading impact (i.e. the recipient domain now cannot assume the sender actually has the policy if the recipient was not reliably serving it, etc) whereas in some other cases (e.g. HTTP by itself) the connection timeout manifests in a user-facing way. So that may be a reason to err on the side of specifying it. 

http://kb.mozillazine.org/Network.http.connect.timeout makes me think that someone who has previously worked on a browser thought 30s was a reasonable connection timeout (which I suppose doesn't include any additional time post-connect). Libcurl seems to default to no request timeout, though there are some other timeouts for specific things (connects, I guess?), if I read correctly: https://curl.haxx.se/libcurl/c/CURLOPT_TIMEOUT.html. The connect timeout seems to default to 300s (https://curl.haxx.se/libcurl/c/CURLOPT_CONNECTTIMEOUT.html). 

So given that, 5s seems a bit on the low side to me. But I'm really doing the handwavy thing here. Half a minute to a minute sort of seems roughly reasonable. 

Thoughts?

-----Original Message-----
From: Uta [mailto:[email-masked]] On Behalf Of Viktor Dukhovni
Sent: Tuesday, September 13, 2016 7:07 PM
To: [email-masked]
Subject: Re: [Uta] MTA STS and HTTPS fetch timeouts

On Tue, Sep 13, 2016 at 09:43:53AM -0700, [email-masked] wrote:

> I'm going to start in an unusual place: The validation pseudocode in 
> Appendix 1. From there I'll go on to other stuff. But I think the 
> pseudocode, done right, will be enormously helpful to implementors, so 
> that's going to be my starting point.

My repeated admonitions to write up a more detailed state-machine were an attempt to make the same point you're making.  The description of the requisite client-side logic is far too sketchy.  

The description needs to be quite explicit, modulo flexibility on whether failure policy refresh and retries are synchronous (fetch new policy and retry immediately) or asynchronous (schedule policy refresh and defer the mail).  Either approach is legitimate.

-- 
	Viktor.

---

From: [email-masked]
Date: Wed, 14 Sep 2016 10:57:42 -0700 (PDT)

> Did you have specific feedback for the HTTPS fetch timeouts discussion that this thread is intended to address?

Not at this time.

TBH, I'm not really interested in getting in to such issues until the draft is
in a bit better shape and it's easier to visualize how the sequence of
operations fits into the SMTP client, how much of the MTA-STS procedure can be
moved to a coprocess, and what that relationship would look like. (There's no
way we're going to put a full-on HTTP client in our SMTP client, and I
intend to ignore anything that makes that a requirement.)

In regards to timeouts, once we get down to it I'm likely to be very concerned
about the need to access DNS TTL information as well. IME a lot of DNS APIs
don't provide access to this information.

				Ned

---

From: Janet Jones <[email-masked]>
Date: Thu, 15 Sep 2016 20:20:30 +0000

Thanks again Ned for the feedback. Once we finish incorporating the latest round of feedback we will publish a new draft for review. 

-----Original Message-----
From: Ned Freed [mailto:[email-masked]] 
Sent: Wednesday, September 14, 2016 10:58 AM
To: Janet Jones <[email-masked]>
Cc: [email-masked]; Daniel Margolis <[email-masked]>; [email-masked]
Subject: RE: [Uta] MTA STS and HTTPS fetch timeouts

Not at this time.

TBH, I'm not really interested in getting in to such issues until the draft is in a bit better shape and it's easier to visualize how the sequence of operations fits into the SMTP client, how much of the MTA-STS procedure can be moved to a coprocess, and what that relationship would look like. (There's no way we're going to put a full-on HTTP client in our SMTP client, and I intend to ignore anything that makes that a requirement.)

In regards to timeouts, once we get down to it I'm likely to be very concerned about the need to access DNS TTL information as well. IME a lot of DNS APIs don't provide access to this information.

				Ned

---

From: Daniel Margolis <[email-masked]>
Date: Wed, 28 Sep 2016 18:40:33 +0200

(Sorry for the late reply. I was on vacation.)

As Janet said, thanks for the feedback. A few specific points:

1. Agreed on the psuedocode. I had actually removed it entirely from the
latest draft in the Git repo (github.com/mrisher/smtp-sts) because I also
didn't find it to be of much value. I do agree with you and Viktor that
fleshing out a hypothetical implementation can make things a lot easier for
implementors, but doing so in a way that is sufficiently specific to be a
useful guide yet sufficiently generic as to be applicable in most cases
isn't trivial. I had punted on this in the current version in head because
I didn't have a great idea of how to express this--as Viktor notes,
diagramming the state machine may in some form be helpful; I'm not 100%
sure pseudocode is in fact the right way to do it. Thoughts?

2. In what case is DNS TTL information needed?

Thanks.

---

From: [email-masked]
Date: Wed, 28 Sep 2016 10:30:43 -0700 (PDT)

> didn't find it to be of much value.

Actually, there's no agreement here whatsoever since I regard simply removing
the pseudocode as the worst thing you could possibly do.

> I do agree with you and Viktor that
> isn't trivial.

Did you take a look at the code I came up with? I don't believe there
was anything in it that made it specific to any particular implementation.

> I had punted on this in the current version in head because

I could not care less about the format you use. But the fact is right now you
have a document that defines a process consisting of a precise series of steps
which presents those steps out of order and which, without some psuedocode, a
flowchart, an outline, whatever to tie it all together is pretty much
guaranteed not to be implemented correctly.

> 2. In what case is DNS TTL information needed?

Section 3.2, first paragraph. There's also been discussion of making more
use of DNS TTL info on the list.

				Ned