Subject: [ietf-smtp] Valid RFC5322 address

From: Hector Santos <[email-masked]>
Date: Sun, 03 May 2020 10:51:48 -0400

Is this a valid address?

08:04:33.934 C: MAIL 
FROM:<github.agent:.github-hookshot/[email-masked]>
08:04:34.712 S: 250 2.1.0 Ok

with the ':' and also '/' ??

It appears to be accepted by various servers, including my own.

This is related to a new project of handling a GitHub Webhook HTTP 
notifications sent to my web server.  The handler will the payload as 
a local message using the address:

From: github agent: github-hookshot/7431eee

That comes from an operator defined macro:

From: github agent: {GITHUB-AGENT}

Which is read from the JSON payload.

If the setup happens to send the notification to an email address, 
then the mail is exported for SMTP to sent out.

So I need to know if the address is valid. Its hard to decipher 
whether it is or not from the RFC5322/RFC5321 BNF. It seems to read 
the ':' and '/' characters are valid.

Thanks

-- 
HLS

---

From: Claus Assmann <[email-masked]>
Date: Sun, 3 May 2020 17:11:29 +0200

> FROM:<github.agent:.github-hookshot/[email-masked]>

Not valid in 5321.

> with the ':' and also '/' ??

':' is not valid in a local part (unless quoted).

> It appears to be accepted by various servers, including my own.

Well, most mail servers accept all kind of garbage ("be liberal ... ").

Mine doesn't:
Mail from:<github.agent:.github-hookshot/[email-masked]>
501 5.1.7 Bad sender's mailbox address syntax.

Just quote it and you're ok:
<"github.agent:.github-hookshot/7431eee"@winserver.com>

> So I need to know if the address is valid. Its hard to decipher whether it
> is or not from the RFC5322/RFC5321 BNF. It seems to read the ':' and '/'
> characters are valid.

Where do you see an unquoted ':' as valid in "Local-part"?  ':' is
used to separate "At-Domain" parts.

---

From: Ned Freed <[email-masked]>
Date: Sun, 03 May 2020 08:03:10 -0700 (PDT)

> with the ':' and also '/' ??

No. The / is fine, but : is a special, and has to be quoted to be used in a
local part. RFC 5322 section 3.2.3.

> It appears to be accepted by various servers, including my own.

Our server doesn't allow it.

> the ':' and '/' characters are valid.

   addr-spec       =   local-part "@" domain

   local-part      =   dot-atom / quoted-string / obs-local-part

   dot-atom        =   [CFWS] dot-atom-text [CFWS]

   dot-atom-text   =   1*atext *("." 1*atext)

   atext           =   ALPHA / DIGIT /    ; Printable US-ASCII
                       "!" / "#" /        ;  characters not including
                       "$" / "%" /        ;  specials.  Used for atoms.
                       "&" / "'" /
                       "*" / "+" /
                       "-" / "/" /
                       "=" / "?" /
                       "^" / "_" /
                       "`" / "{" /
                       "|" / "}" /
                       "~"

Note tha absence of ":" in atext. The only way a ":" in a local-part
is within a quoted-string:

   quoted-string   =   [CFWS]
                       DQUOTE *([FWS] qcontent) [FWS] DQUOTE
                       [CFWS]

   qcontent        =   qtext / quoted-pair

   qtext           =   %d33 /             ; Printable US-ASCII
                       %d35-91 /          ;  characters not including
                       %d93-126 /         ;  "\" or the quote character
                       obs-qtext

				Ned

---

From: John C Klensin <[email-masked]>
Date: Sun, 03 May 2020 11:57:41 -0400

> with the ':' and also '/' ??

Technically, no.  The "/" is not a problem.  Colons are (they
are not in the list of valid characters for <atext> in RFC
5322), I imagine at least in part because they were part of the
now-discouraged Source Route syntax (see Sections 3.6 and
Appendix C of RFC 5321 (and 5321bis, at least so far)) and so
would need to be quoted.

You also need to be very careful with the "." following the
colon.  I haven't looked carefully enough at the syntax today to
have any confidence how that would be handled with the
Dot-string production -- you might have to quote the whole
business to make it valid.

I also note that, while the example you have given is well under
the 64 octets specified in RFC 5321 Section 4.5.3.1.1, I would
not be surprised if some addresses constructed that way would
push that limit and that there has been a recent discussion
about those limits.

> It appears to be accepted by various servers, including my own.

Yeah, no surprise.  I would be equally unsurprised if some
servers followed the spec and generated a syntax error because
of the colon.  There is also a very long history of various
systems messing up local-parts that use quoting as the quoted
strings pass through assorted processing steps.  And I'd also be
unsurprised if some servers are rejecting or messing up
local-parts much shorter than the specified maximum.

> This is related to a new project of handling a GitHub Webhook
> HTTP notifications sent to my web server.  The handler will
> the payload as a local message using the address:
> 
> If the setup happens to send the notification to an email
> address, then the mail is exported for SMTP to sent out.

Noting that you seem to be turning blanks into ".", I'd also
want to be careful to be sure that any value "." characters to
the right of the "/" in your example don't get broken by any
de-conversion process. 

Also note that, if this is going to be used in conjunction with
web facilities, someone is likely to want to turn whatever you
produce into a MAILTO: URL and restrictions on those (both
restrictions imposed by the spec and those imposed by sloppy
implementations) are even worse.

> So I need to know if the address is valid. Its hard to
> decipher whether it is or not from the RFC5322/RFC5321 BNF. It
> seems to read the ':' and '/' characters are valid.

See above.  However, even if you quote the stuff using either
<qtextSMTP> or  <quoted-pairSMTP>, I predict that some servers
either won't accept it or will thoroughly botch it.  I'd also
predict that, when you complain, at least some of them will tell
you that no one has needed this in 37 years (or whenever their
mail system project started), that it would be a lot of work to
fix, and that you shouldn't expect it to work... or, in a few
cases, that they would be happy to accept a purchase order from
you or your customers to investigate a fix.  You could also
attempt to summon the Protocol Police, but I can assure you that
they are not accepting enforcement requests from this list.

Recommendation: Don't push it.  Remembering that only the
delivery server is permitted to interpret addresses and in spite
of the "%" routing syntax being out of favor these days, mapping 
   From: github agent: github-hookshot/7431eee

into something akin to

   MAIL FROM:<github-hookshot/[email-masked]>

is a great deal less likely to cause you, and your users, pain
and suffering in the long term.   And, of course, if you could
do it,

  MAIL FROM:<github-hookshot/[email-masked]>

would be even less likely to cause problems.

best,
   john

---

From: Hector Santos <[email-masked]>
Date: Sun, 03 May 2020 12:07:07 -0400

Thanks Ned and Claus:

> 501 5.1.7 Bad sender's mailbox address syntax.

The icloud.com SMTP server accepted it. How I saw the issue was with 
Apple's web-based icloud.com mail reader/viewer (MUA).   It showed a 
blank From: field when first reading it. But after viewing a previous 
email, i.e. Apple News, it now used Apple's From address as a filler 
for the notification email from field display. The bugs were reported 
to Apple.

> <"github.agent:.github-hookshot/7431eee"@winserver.com>

Ok. I am probably to change it so isn't a problem with MUAs as well -- 
be "conservative" on sending out. :-)

> used to separate "At-Domain" parts.

I guess I was reading multiple things between RFC5322 and RFC5321. 
Jumped around a lot.

As Ned showed, I see now RFC5222 better defines the local-part. But 
with RFC5321, it wasn't quite clear.  It seems to stop with only one 
occurrence of atext, no BNF for it and no reference to atext being 
defined in RFC5322.

I suppose this might be another clarification issue for RFC5321bis.

Thanks

-- 
HLS

---

From: Hector Santos <[email-masked]>
Date: Sun, 03 May 2020 12:27:55 -0400

> Appendix C of RFC 5321 (and 5321bis, at least so far)) and so

Just from a reader pov, it wasn't quote clear between the two docs. I 
see now RFC5322 is defining atext clearly.  RFC5321 probably needs a 
reference to RFC5322 for atext.

> Noting that you seem to be turning blanks into ".",

Yes, from the old MHS methods that took local user names, i.e. "Hector 
Santos" and dotted/underscored them when networking came to BBSes.

char *MakeDotName(char *psz)
{
   char *p = psz;
   while (*p) {
     switch (*p) {
       case ' ': *p = '.'; break;
       case '.': *p = '_'; break;
     }
     p++;
   }
   return psz;
}

>I'd also

I see.

> ...
> Recommendation: Don't push it.

+1, I plan to change it.  It was initially meant to be more for local 
MUAs (Not networking). Then the notify by email was added where there 
is a MakeDotName() translation and no consideration for special 
characters.  I think I will first try this for the local name to email 
name translation.

   "Github Agent: github-hookshot/xxxxxxxxx" [email-masked]

And check how Apple's web-based MUA handles it.  Btw, the Apple iPhone 
Mail App display the field fine.

I am also going to an wcSMTP server option to validate the local part 
of the return address.

Thanks

-- 
HLS

---

From: John C Klensin <[email-masked]>
Date: Sun, 03 May 2020 13:45:12 -0400

--On Sunday, May 3, 2020 12:07 -0400 Hector Santos
<hsantos=[email-masked]> wrote:

> As Ned showed, I see now RFC5222 better defines the
> local-part. But with RFC5321, it wasn't quite clear.  It seems
> to stop with only one occurrence of atext, no BNF for it and
> no reference to atext being defined in RFC5322.
> 

The first paragraph of Section 4.1.2 of RFC 5321 (and
rfc5321bis-03) concludes with

	"Terminals not defined in this document, such as ALPHA,
	DIGIT, SP, CR, LF, CRLF, are as defined in the "core"
	syntax in Appendix B of RFC 5234 [5] or in the message
	format syntax in RFC 5322 [11]."

So you can't say "no reference..."   However, it has been clear
to me for some time that turning that statement into a specific
list of productions defined in 5334 (not "such as" handwaving)
and a separate list of productions defined in 5322 and then
indexing both would be a considerable improvement.   If someone
wants to do the work of preparing those lists, I'd be happy to
fold them into 5321bis and provide an acknowledgment.

(An absence of volunteers might be interpreted by the editor,
who is known to be error-prone where ABNF is concerned, as
consensus that what is there is good enough and/or that no one
really cares.  :-) )

    john

---

From: Hector Santos <[email-masked]>
Date: Sun, 03 May 2020 13:51:57 -0400

> <"github.agent:.github-hookshot/7431eee"@winserver.com>

I just tried this, easy to explore without code changes, also without 
the dotted spaces. The Apple web-based MUA still can't handle it. It 
shows a blank field. For this MUA, removing the colon ':' seems fix it 
for their display.

TBird was ok.

I will be conservative with this.

Thanks

-- 
HLS

---

From: cketti <[email-masked]>
Date: Sun, 3 May 2020 22:22:38 +0200

>From RFC 5234:
- ALPHA
- DIGIT
- SP
- CRLF
- HEXDIG

>From RFC 5322:
- atext
- FWS
- CFWS

CR and LF are not used in ABNF rules.

---

From: John C Klensin <[email-masked]>
Date: Sun, 03 May 2020 16:37:49 -0400

Many thanks.   Is "cketti" the name you want in the
acknowledgments?  Otherwise, please supply a name (send to me
directly if you prefer).  

This will be in rfc5321bis-04 when it is posted (although that
still probably won't be until after we have a WG).

Stay safe and well,
   john

--On Sunday, May 3, 2020 22:22 +0200 cketti <[email-masked]> wrote:

>> So you can't say "no reference..."   However, it has been
>> clear to me for some time that turning that statement into a
>> specific list of productions defined in 5334 (not "such as"
>> handwaving) and a separate list of productions defined in
>> 5322 and then indexing both would be a considerable
>> improvement.   If someone wants to do the work of preparing
>> those lists, I'd be happy to fold them into 5321bis and
>> provide an acknowledgment.
> 
> 
>> From RFC 5234:
> - ALPHA
> - DIGIT
> - SP
> - CRLF
> - HEXDIG
> 
>> From RFC 5322:
> - atext
> - FWS
> - CFWS
> 
> _______________________________________________
> ietf-smtp mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ietf-smtp

---

From: Viktor Dukhovni <[email-masked]>
Date: Mon, 4 May 2020 15:09:17 -0400

> On May 3, 2020, at 11:57 AM, John C Klensin <[email-masked]> wrote:
> 

Unquoted "." is only valid in a dot-atom, and so:

	[email-masked]
	/////.////@example.org

are both OK, but

	"/../"@example.org
	".///"@example.org
	"///."@example.org

all need quoting.  So long as both characters around the "." are "atext",
the "." does not force quoting of a local-part that would not otherwise
require quoting.

In this case, given that the ":" forces quoting, the "." requirements are moot.

-- 
	Viktor.