Subject: [ietf-smtp] Draft: SMTP Extension for Longer Email Address

From: Viruthagiri Thirumavalavan <[email-masked]>
Date: Thu, 5 Dec 2019 05:10:24 +0530

Hey all,

This is my first internet draft I'm sharing here. So I hope you all will
bear with me on this.

I didn't share my initial draft
<https://tools.ietf.org/html/draft-viruthagiri-email-address-length-00>
here. But, Dr. Klensin had given me feedback for that draft off-list. Turns
out I was listing outdated references in the normative references section
among other things.

I have version 1
<https://tools.ietf.org/html/draft-viruthagiri-email-address-length-01>
now. It focuses only on the SMTP extension.

Here is the abstract.

============

https://tools.ietf.org/html/draft-viruthagiri-email-address-length-01

Abstract

   This memo defines an SMTP extension with keyword "EAML" whereby an
   SMTP server can declare that it is capable of handling longer email
   addresses without any local-part or domain-part restriction set by
   RFC 5321.

   EAML stands for "Email Address Maximum Length".

============

This is how it works.

                  S: 220 smtp.example.com ESMTP Ready
                  C: EHLO bob.example.org
                  S: 250-smtp.example.com
                  S: 250-EXPN
                  *S: 250-EAML 500*
                  S: 250-PIPELINING
                  S: 250 HELP

The parameter omitted EAML keyword says:

                  [*] No local-part limitation.
                  [*] No domain-part limitation.
                  [*] 254 maximum characters

============

Any kind of feedback will be really helpful.

Thanks

-- 
Best Regards,

Viruthagiri Thirumavalavan
Dombox, Inc.

---

From: Michael Peddemors <[email-masked]>
Date: Wed, 4 Dec 2019 15:45:52 -0800

This might again be a question for the extra group to consider, but if 
looking at something like email length, that would apply to both IMAP 
and SMTP in a typical situation.

How should things that apply across multiple protocols be addressed in 
general, I know that is one of the things mentioned when trying to get 
CLIENTID into the working group..

On 2019-12-04 3:40 p.m., Viruthagiri Thirumavalavan wrote:
> here. But, Dr. Klensin had given me feedback for that draft off-list. 
> Turns out I was listing outdated references in the normative references 
> section among other things.
> 
> I have version 1 
> https://tools.ietf..org/html/draft-viruthagiri-email-address-length-01 
>                    S: 220 smtp.example.com <http://smtp.example.com> 
> ESMTP Ready
>                    C: EHLO bob.example.org <http://bob.example.org>
>                    S: 250-smtp.example.com <http://250-smtp..example.com>
>                    S: 250-EXPN
> *S: 250-EAML 500*
>                    S: 250-PIPELINING
>                    S: 250 HELP
> 
> _______________________________________________
> ietf-smtp mailing list
> [email-masked]
> https://www.ietf.org/mailman/listinfo/ietf-smtp
> 

-- 
"Catch the Magic of Linux..."
------------------------------------------------------------------------
Michael Peddemors, President/CEO LinuxMagic Inc.
Visit us at http://www.linuxmagic.com @linuxmagic
A Wizard IT Company - For More Info http://www.wizard.ca
"LinuxMagic" a Registered TradeMark of Wizard Tower TechnoServices Ltd.
------------------------------------------------------------------------
604-682-0300 Beautiful British Columbia, Canada

This email and any electronic data contained are confidential and intended
solely for the use of the individual or entity to which they are addressed.
Please note that any views or opinions presented in this email are solely
those of the author and are not intended to represent those of the company.

---

From: "Kurt Andersen (IETF)" <[email-masked]>
Date: Wed, 4 Dec 2019 15:48:09 -0800

And JMAP too...

On Wed, Dec 4, 2019, 15:45 Michael Peddemors <[email-masked]> wrote:

> >                    S: 250-smtp.example.com <http://250-smtp..example.com
> >
> >                    S: 250-EXPN
> > *S: 250-EAML 500*
> >                    S: 250-PIPELINING
> >                    S: 250 HELP
> >

---

From: Viruthagiri Thirumavalavan <[email-masked]>
Date: Thu, 5 Dec 2019 05:24:21 +0530

The draft focuses on the MAIL FROM email address length for better bounce
handling. Not the RCPT TO length.

-- 
Best Regards,

---

From: Michael Peddemors <[email-masked]>
Date: Wed, 4 Dec 2019 17:45:58 -0800

For the record, want to applaud you for tackling the DRAFT, just the 
effort alone should be applauded, but no matter what discussions occur, 
it is a handy consolidated document of all the related topics..

...

However..

If it is specific to MAIL FROM: then maybe consider that it isn't an 
'email address' length problem.  If it is, then it needs to be 
considered in a wider scope.

If it isn't then it should be considered a change to MAIL FROM handling, 
where it can accept more than an email address.. eg a 'routing' 
instruction, (which historically has became thought of as an 'email 
address', and in some implementations the 'only' acceptable routing 
instruction) .. simply because most implementations would not accept a 
message from a "sender" or route  that you could not route delivery back to.

It 'appears' that the problem you are trying to solve is a 'routing' 
issue, rather than an 'email address' issue.  VERP and others are not 
really email addresses, even though they are used in an email notation.

What you REALLY have to be saying is that the sender accepts long 
routable information in it's sender information at it's MX..

But again, I might have questions.. I see a few bumping roads given the 
amount of MAIL FROM validation out there.. IF the MAIL FROM is not 
valid, IMHO it means rejections.. in connection.. so not sure where the 
value is, but assuming you have have one.. once it passes the MTA 
acceptance layer, you expect it to be treated as an 'email address'.. 
(or in rare cases a literal bounce/double bounce <>)

Imagine a cloud filtering device, should it accept non-email addresses 
<sic> in the MAIL FROM, if it has to then deliver to other mail servers 
which it does not know if they will accept such?

And what to do in the case of the remote server refusing the message? 
The cloud filtering device (CFD) now has an obligation to notify the 
sender that it could not complete the delivery, and while it might have 
'routing' instructions, it can't be seen as an email address. And of 
course, now the CFD needs to know if the MX record associated with that 
domain can handle 'non-email' strings at it's MTA..

What would it do if it found that the original sending MTA could not 
accept long routing instructions.. what does it do?

Personally, I think you might find that another approach to the problem 
might be in order.. eg notifying a regular length email address as 
presently permitted, that a message with this long string could not be 
delivered.. we have that already in several forms..

I think more discussion in the DRAFT on the problem it is attempting to 
solve is in order..

And BTW..

    |          TO THE MAXIMUM EXTENT POSSIBLE, IMPLEMENTATION           |
    |          TECHNIQUES WHICH IMPOSE NO LIMITS ON THE LENGTH          |
    |          OF THESE OBJECTS SHOULD BE USED.

I personally appose such an 'ideal' in these contexts.. while many might 
be open to a 'longer' user part or domain being legal, no limits would 
be disastrous IMHO ..

MAIL FROM: <me@longencylopiadesignedtoeffectivelyddosaserver..xxxx>

Using MAIL FROM as a routing instruction.. fine.. it fundamentally 
always was.. even though IMHO it SHOULD be the actual sender's address, 
not a routing instruction..

But trying to use the MAIL FROM to embed other information is probably 
the wrong direction, and if the purpose of this is strictly to allow the 
embedding of more data in the string, it probably will get less support, 
both in draft form, and in implementations..

If you need to communicate more information that just routing (or email 
address) then maybe another method is more suitable.

Hope the comments are welcome..

On 2019-12-04 3:54 p.m., Viruthagiri Thirumavalavan wrote:
> The draft focuses on the MAIL FROM email address length for better 
> bounce handling. Not the RCPT TO length.
> 
> On Thu, Dec 5, 2019 at 5:18 AM Kurt Andersen (IETF) 
> <[email-masked] <mailto:[email-masked]>> wrote:
> 
>     And JMAP too...
> 
>     On Wed, Dec 4, 2019, 15:45 Michael Peddemors <[email-masked]
>     <mailto:[email-masked]>> wrote:
> 
>          >                    S: 220 smtp.example.com
>         <http://smtp.example.com> <http://smtp.example.com>
>          > ESMTP Ready
>          >                    C: EHLO bob.example.org
>         <http://bob.example.org> <http://bob.example.org>
>          >                    S: 250-smtp.example.com
>         <http://250-smtp.example.com> <http://250-smtp..example.com
>         <http://example.com>>
>          >                    S: 250-EXPN
>          > *S: 250-EAML 500*
>          >                    S: 250-PIPELINING
>          >                    S: 250 HELP
>          >
>          > [email-masked] <mailto:[email-masked]>
>         [email-masked] <mailto:[email-masked]>
>     [email-masked] <mailto:[email-masked]>

---

From: "Valdis Klētnieks" <[email-masked]>
Date: Wed, 04 Dec 2019 21:13:25 -0500

> handling. Not the RCPT TO length.

You really want to apply it to both.

Otherwise, it's impossible to do a loopback test from your own long address to
that same long address to verify that the MTA in fact doesn't choke.

There's also the part where there's an assumption that the recipient's MTA can
send a bounce message to the MAIL FROM - which means your inbound RCPT TO: has
to be able to accept the long address as well.

---

From: Viruthagiri Thirumavalavan <[email-masked]>
Date: Thu, 5 Dec 2019 15:39:59 +0530

Thanks for the feedback.

> issue, rather than an 'email address' issue.

 Yes, you are right.

> accept long routing instructions.. what does it do?

Isn't such issues also applicable to other SMTP extensions? e.g. Message
size declaration

And BTW..
> be open to a 'longer' user part or domain being legal, no limits would

I think you were looking at my old version. My version 1
<https://tools.ietf.org/html/draft-viruthagiri-email-address-length-01>
focuses only on the SMTP extension.
The part you mentioned originally coming from RFC 821.
But yes, I agree with you on this. This is one of the reasons why my
proposal have an upper cap. (900 octets)

> Hope the comments are welcome..

Yes, your comments are always welcome.

Thanks
-- 
Best Regards,

---

From: Viruthagiri Thirumavalavan <[email-masked]>
Date: Thu, 5 Dec 2019 15:44:30 +0530

> that same long address to verify that the MTA in fact doesn't choke.

I didn't know that. Thanks for pointing it out.

There's also the part where there's an assumption that the recipient's MTA
> can

Makes sense. I'll include this part in my next version.

Thanks for the feedback. Really appreciate it.

---

From: Viruthagiri Thirumavalavan <[email-masked]>
Date: Fri, 6 Dec 2019 14:56:09 +0530

> Isn't such issues also applicable to other SMTP extensions? e.g. Message

While reading the rfc5321bis draft, I noticed this section which is the
same as 5321.

2.2.3.  Special Issues with Extensions

   Extensions that change fairly basic properties of SMTP operation are
>    permitted.  The text in other sections of this document must be
>    understood in that context.  In particular, extensions can change the
>    minimum limits specified in Section 4.5.3, can change the ASCII
>    character set requirement as mentioned above, or can introduce some
>    optional modes of message handling.
>

   In particular, if an extension implies that the delivery path
>    normally supports special features of that extension, and an
>    intermediate SMTP system finds a next hop that does not support the
>    required extension, it MAY choose, based on the specific extension
>    and circumstances, to requeue the message and try later and/or try an
>    alternate MX host.  If this strategy is employed, the timeout to fall
>    back to an unextended format (if one is available) SHOULD be less
>    than the normal timeout for bouncing as undeliverable (e.g., if
>    normal timeout is three days, the requeue timeout before attempting
>    to transmit the mail without the extension might be one day).

-- 
Best Regards,

---

From: Hector Santos <[email-masked]>
Date: Fri, 06 Dec 2019 09:30:39 -0500

On 12/4/2019 6:40 PM, Viruthagiri Thirumavalavan wrote:> Hey all,

> This is my first internet draft I'm sharing here. So I hope you all
> will bear with me on this.
>

Hi, Congrats! :)

A quick review.

- Abstract

The Abstract is good. Please consider adding a summary clause, 
sentence indicating first a) the "problem" and b) how this proposed 
extension can resolve it.   For example, the document indicate VERP 
and SRS support can be a problematic.   This problem statement should 
be in the abstract. It will help catch people who may have a known 
issue with the extended protocols and this proposal may offer a 
possible solution.  I normally don't read submissions unless the 
abstract catches my interest.

- Background

Good, but this is about VERP and SRS. You explained how VERP/SRS 
violated SMTP limits.  So is it really not an SMTP problem?

Have you explored other proposed solutions to the VERP or SRS problem? 
   Is there a better solution for what VERP/SRS is trying to address?

Personal implementation note: I never believed in solutions that 
modified or altered the SMTP REQUIRED persistency of the Return Path. 
  I never supported  SRS which addresses the SPF Multi-hop MTA IP 
transition problem.  VERP has a good reason to exist because it deals 
with bulk mail and the common bulk distribution need to recognized 
specific bounce transactions, i.e. a subscriber does not exist.  I 
don't feel the same with SRS and never used it, and never needed it 
since 2003.

Finally, as a proposed solution, I wonder how compliant EAML clients 
will adjust to varying mail servers with different EAML values.  For 
example,  given different servers limits that a client can potentially 
see:

server-A  has EAML 512
server-B  has EAML

If the client is sending an address with 325 characters, server-A will 
accept the input, but server-B will reject it.

I am not sure if this level of strictness at the TRANSPORT level is 
desirable.  We don't even know how the interface points will handle 
it.  The key goal is targeting the message for delivery, and the 
secondary goal, and equally important, is to make sure a bounce is 
possible.

So where is the exact limit?

Putting VERP and SRS aside and just considering SMTP, can it be clarified?

Going back to the documented limits, the RFC821 limit was 129:

user 64
domain 64
path 256

The path limit supported the actual limit of 129 - length(user+"@"+ 
domain)

RFC2821/5321 updates it to 320:

user 64
domain 255

There is a technical conflict just with "user@domain" address size 
possibly exceeding the path 256 limit.   So we probably want to make 
sure we cover the Extension Independent issue to clarify the actual 
INPUT limits.

When you backtrack, mail storage is created with 998 text line limits. 
So a mail sender has to start with the extraction the Network 
Transport address fields.

If it sees sizes that potentially exceeds SMTP limits, then it needs 
to probably work out a mapping solution.  VERP is not it if still 
exceeds the SMTP limit.

Finally, I agree SMTP documentation should be updated for 
clarification regarding limits, its not just addresses, but also 
Message Sizes in an era of multi-media.

Imo, this is more of a VERP problem or SPF problem when SRS is 
considered with SPF.  I don't see this as an SMTP issue.

Having a EAML keyword extension is good, just like we have a SIZE 
extension, but the client needs to support it.  I wonder how a 
compliant EAML client can be designed so that messages can be a) 
single source (one message), and b) be deliverable to different 
servers with different EAML limits.  What does the client do if one 
system, server-B, rejects it?  Bounce it to the author showing which 
servers were successful and which one failed?

To me, the VERP specification should be updated. In fact, from what I 
can see, there is no RFC for it.  It is an expired individual DRAFT 
1999 submission.

I think we would be better off updating VERP making an an official 
RFC, probably as Standards Track item.

-- 
HLS

---

From: Hector Santos <[email-masked]>
Date: Fri, 06 Dec 2019 13:05:25 -0500

On 12/6/2019 9:30 AM, Hector Santos wrote:
> I think we would be better off updating VERP making an official

I only see an 1999 I-D submission:

https://tools.ietf.org/html/draft-varshavchik-verp-smtpext-01

This spec still refers to RFC821/822 and makes no mentioning of 
dealing with VERP addresses exceeding 821 limits.

I would like to suggest that this be updated to current operational 
considerations and RFC5321 limits, including exploring the addition of 
using an encryption method, i.e. TripleDES or AES, to possibly reduce 
and help keep VERP-encoded addresses sizes within the SMTP address limits.

-- 
HLS

---

From: Viruthagiri Thirumavalavan <[email-masked]>
Date: Sat, 7 Dec 2019 01:31:07 +0530

> Hi, Congrats! :)

Thanks :)

===============

> abstract catches my interest.

Will correct it in the next version.

===============

>    Is there a better solution for what VERP/SRS is trying to address?

I found two drafts. The first one you already aware of.

VERP  [July 1999]:

This proposal seems like trying to solve the bandwidth wastage problem by
making the receiver generate VERP addresses.  [Personal note: I like this
idea]

Quoting a part from that proposal

Unfortunately, VERPs require much more bandwidth and network
>    resources than DSNs because VERPs cannot be used to send one copy of
>    a mailing list message addressed to all the recipients in the same E-
>    mail domain.

SRS [June 2003]:
https://web.archive.org/web/20170708142134/http://www.openspf.org/svn/project/specs/drafts/draft-mengwong-sender-rewrite-01.txt

I couldn't find this proposal in IETF. I had to use waybackmachine to view
it.

This proposal recommends the deprecation of the 64 character maximum
>    for localparts defined in RFC2821 section 5.

So this SRS proposal had the same 64 character limitation problem 15 years
back.

===============

> don't feel the same with SRS and never used it, and never needed it

On the contrary, I support SRS.  SRS solves two problems for the
intermediaries.

1) Surviving SPF failures
2) Bounce handling just like VERP

We need solutions for intermediaries too.
We have ARC <https://tools.ietf.org/html/rfc8617> an experimental RFC that
helps intermediaries. But it's for RFC 5322.
As far as I know, the closest solution similar to ARC we have for RFC 5321
is SRS.

e.g. SRS1=HHH=example.com==HHH=TT=example.org=[email-masked]

===============

> accept the input, but server-B will reject it.

I hope you are talking about the multiple hops here, where hop A accepts
512 characters but hop B accepts only 254 characters.
This is a problem with SMTP extension model. They even clarified this part
in section 2.2.3 of RFC 5321.

===============

> Putting VERP and SRS aside and just considering SMTP, can it be clarified?

If you need an exact limit, I would always go with 900 since that's the
value of Gmail. They probably came to that number from their experience.

===============

> sure we cover the Extension Independent issue to clarify the actual

Ok

===============

> servers were successful and which one failed?

If you are referring to multiple hops, then yes bounce to the author.
Alternatively, we can add words like "intermediaries MUST always use value
254"

===============

> RFC, probably as Standards Track item.

Even if we update VERP specification, it still needs to address the
local-part 64 character limitation issue in order to work properly.

So either we have to update RFC 5321 at least to remove the local-part 64
character limitation OR we need to introduce a new extension like EAML. If
we go with the extension, then we always gonna have problems when multiple
hops involved since that's a problem with SMTP extension model.

Thanks

-- 
Best Regards,

---

From: Hector Santos <[email-masked]>
Date: Mon, 09 Dec 2019 10:26:21 -0500

I tried to contact the original editor/author of the VERP draft:

550 5.1.1 <[email-masked]>... User unknown

Anyone know if the author is still around the IETF?  A different address?

Thanks

--
HLS

-- 
HLS

---

From: Viruthagiri Thirumavalavan <[email-masked]>
Date: Mon, 9 Dec 2019 21:02:49 +0530

https://github.com/svarshavchik

-- 
Best Regards,

---

From: Keith Moore <[email-masked]>
Date: Mon, 9 Dec 2019 14:04:02 -0500

> and help keep VERP-encoded addresses sizes within the SMTP address 

I'm at a loss to understand how encryption can reduce address sizes.    
Hashing, yes.  Compression, yes.   But not encryption.

Keith

---

From: Sam Varshavchik <[email-masked]>
Date: Mon, 09 Dec 2019 20:48:24 -0500

Hector Santos writes:

> Hi Sam,
>
> There is some discussion in the IETF-SMTP list regarding VERP. I would like  
> to see it updated, codified, modernized, lessons leaned, but also dealing  
> with technical length limits.  See below for my proposal to be updated.  But  
> please check out the IETF-SMTP list.
>
> What do you think?

Now that's a nice blast from the past. I don't even remember actually  
submitting the draft, but I guess I did. My main intent was mostly public  
documentation of the work that I did at that time (and still do).

I sifted through my archives and found my original working nroff source,  
which I'm attaching below, if it helps. I have no idea if 'roff is still the  
suggested working format. The following version includes a few changes I  
made to the working code after the original draft.

#! /bin/sh
nroff -ms <<EOF | perl -e '

while (<>) {                            # Read the entire input file.
    s/FORMFEED(\[Page\s+\d+\])\s+/        \1\n\f\n/g;
    s/\f\n$//;
    print;
}'
.pl 10.0i
.po 0
.ll 7.2i
.lt 7.2i
.nr LL 7.2i
.nr LT 7.2i
.ds RF FORMFEED[Page %]
.ds LF S. Varshavchik
.ds CF Expires XXX XX, XXXX
.ds LH VERP SMTP Extension
.ds RH XXX XX, XXXX
.ds CH S. Varshavchik
.hy 0
.ad l
.in 0
.\".nf
.\"INTERNET-DRAFT                                            S. Varshavchik
.\"Expires XXX XX, XXXX                              Double Precision, Inc.
.\"                                                            XXX XX, XXXX
.\"
.\".ce
.\"Variable Envelope Return Path SMTP Extension
.\".ce
.\"draft-varshavchik-verp-smtpext-04.txt
.\"
.\".ti 0
.\"Status Of This Memo
.\"
.\".fi
.in 3
.\"This document is an Internet-Draft and is in full conformance
.\"with all provisions of Section 10 of RFC2026.
.\"
.\"Internet-Drafts are working documents of the Internet Engineering
.\"Task Force (IETF), its areas, and its working groups.  Note that
.\"other groups may also distribute working documents as
.\"Internet-Drafts.
.\"
.\"Internet-Drafts are draft documents valid for a maximum of six
.\"months and may be updated, replaced, or obsoleted by other
.\"documents at any time.  It is inappropriate to use Internet-
.\"Drafts as reference material or to cite them other than as
.\""work in progress."
.\"
.\"The list of current Internet-Drafts can be accessed at
.\"
.\"http://www.ietf.org/ietf/1id-abstracts.txt
.\"
.\"The list of Internet-Draft Shadow Directories can be accessed at
.\"http://www.ietf.org/shadow.html.
.\"
.\".ti 0
.\"0. Revision history
.\"
.\"01 - added additional comments on DSNs not being very successful in
.\"automatically handling mailing list functions.  Additional section
.\"added with comments regarding vacation autoresponders.
.\"
.\"02 - expanded section 6 to describe the proper way to handle local
.\"aliasing, or address explosion.
.\"
.\"03 - added the characters -, [, and ], to the list of characters that
.\"must be hex-encoded in a VERP, as well as requiring the domain portion
.\"of the return address to be encoded.  This resolves certain ambiguities in
.\"potential return addresses, as well as allow network address literals
.\"to be used.  Adjusted example SMTP conversation to reflect proper reply
.\"codes.
.\"
.\"04 - clarify that upper and lowercase hexadecimal characters are permitted.
.\"Update contact information.
.\"
.ti 0
1. Abstract

This document describes an extension to the SMTP service [1],
called Variable Envelope Return Path (VERP).
The VERP extension implements a way of automatically identifying
undeliverable mail recipients, even
when non-delivery reports originate from mail systems that do
not implement delivery status notifications, as specified in [2] and [3].

.ti 0
2. Introduction

All E-mail software can expect to deal with undeliverable mail.
[2] and [3] specify a machine-readable format for delivery status
notifications (DSNs, or non-delivery reports).
DSNs allow
undeliverable mail to be handled in a totally automatic fashion, without
requiring manual intervention.
For example, mailing list managers
can automatically identify addresses that are no longer deliverable,
and remove them from the mailing list.

Although [2] and [3] are now widely implemented,
there are still many systems
that do not use them.
This makes it impractical to completely rely on DSNs for
automatic mailing list management.
Undeliverable addresses accumulate quickly even from
a very small percentage of non-DSN systems.
This results in a non-trivial amount of manual work to identify undeliverable
addresses and purge them from the mailing list.

Mailing list software began to use VERPs
(the acronym stands for Variable Envelope Return Path)
after DSNs were found to be impractical for totally
automatic mailing list management.
VERPs are an alternative way to handle non-delivery notices.
The advantage of VERPs is that they can be made to work
automatically, even when non-delivery
notices are not in the format specified by [2].

Unfortunately, VERPs require much more bandwidth and network resources
than DSNs because VERPs cannot be used to send one copy of a
mailing list message addressed to all the recipients in the same E-mail domain.

This SMTP service extension allows E-mail software to send a single
VERP message to all addresses in the same mail domain,
for as long as mail servers, which relay the message, support the VERP
SMTP extension.

The VERP message may be eventually relayed to a mail server that does not
support this extension.
Separate messages -
with variable envelope return paths -
will be sent when this happens.

So the worst case scenario results in the same situation where
traditional VERPs are used right from the start.
The best case scenario results in significant savings of
network resources and bandwidth, from eliminating hundreds (or
more) copies of the same message.

Essentially, the VERP extension postpones the generation of multiple
messages with different return paths as much as possible, until
it is absolutely required.

.ti 0
2.1 VERP overview

The traditional VERP message encodes
the recipient address as a portion of the return address.
When undeliverable mail comes back, the mail software
decodes the return address (now the recipient address) and
obtains the address responsible for the non-delivery notice.

For example: mail sent by a mailing list
manager to the address <[email-masked]>
carries a return address of <mlist-return-john=[email-masked]>.
The mailing list software at domain.com handles all mail with the local
portion of the address starting with "mlist-return-".
If a non-delivery notice is generated because the address is not deliverable,
the mailing list software
takes the address where the non-delivery report was sent, retrieves the
remaining portion of the local address, "john=example.org", and determines
that the undeliverable address was <[email-masked]>.

This does not rely on RFC 1894, and will work for all non-delivery notices.

.ti 0
3. Framework for the VERP SMTP transport extension

This SMTP transport extension [1] is laid out as follows.

.in +7
.ti -4
(1) The name of the SMTP transport extension defined here is Variable
Envelope Return Path.

.ti -4
(2) The EHLO keyword associated with this extension is VERP.

.ti -4
(3) The VERP EHLO keyword takes no parameters.

.ti -4
(4) One optional ESMTP keyword VERP is associated with the
MAIL FROM command. This parameter takes no values.

.ti -4
(5) No additional ESMTP verbs are defined by this extension.

.ti -4
(6) The next section specifies how support for this extension affects
the behavior of a server and client SMTP.
.in -7
.\"The IANA registry is ammended as follows.  "ESMTP MAIL KEYWORDS"
.\"are ammended as follows:
.\"
.\".nf
.\"Keywords Description                    Reference
.\"-------- -----------------------------  --------------------------------
.\"VERP     Variable Envelope Return Path  [draft-varshavchik-verp-smtpext]
.\".fi
.\"
.\""MAIL EXTENSION TYPES" are ammended as follows:
.\"
.\".nf
.\"Service Ext   EHLO Keyword Parameters Verb   Reference
.\"-----------   ------------ ---------- ------ --------------------------------
.\"Var Env RP       EHLO         none     none
.\".fi

.ti 0
4. The VERP SMTP extension

When a VERP keyword is present in the MAIL FROM command, [4],
some additional restrictions are imposed
on the RFC 822 address [5], specified by that
MAIL FROM
command, and on
all RFC 822 addresses in the subsequent RCPT TO
commands that refer to the same message (that is, until the next DATA,
RSET, or QUIT command).
The term "VERP message" refers
to any E-mail message whose MAIL FROM command includes the VERP keyword.
The term "VERP-compliant server" refers to any
E-mail server that supports the Variable Envelope Return Path SMTP
extension.
When a VERP keyword is present in the MAIL FROM command:

.in +8
.ti -4
(1) The address specified by the MAIL FROM verb MUST contain
at least one @ character.

.ti -4
(2) The address in every RCPT TO verb referring to the same
message MUST contain at least one @ character.

.ti -4
(3) The domain portion of the address
in the MAIL FROM and RCPT TO verbs
MUST be compliant with the definition of <domain>
in [6].  That is, it MUST contain only letters, digits, hyphens, and
periods.
The domain portion of the address is the one
that follows
the last @ character.
.in -8

.ti 0
4.1 Delivery failures

When a VERP-compliant
server is unable to deliver a VERP message to one or more
recipients, the VERP server MUST do one of the following:

.in +8
.ti -3
1) Return an RFC 1891 delivery status notification to the
return address, or:

.ti -3
2) Transmit a separate non-delivery notice for each failed recipient.
The return address for each non-delivery notice MUST be
the address that's formed by applying the procedure
described in section 7 of this
document to the return address of the message and the
failed recipient's address.  If more than one recipient was undeliverable
a separate notice MUST be sent for each undeliverable address.
.in -8

.ti 0
5. Final delivery

Section 4.3.1 of [5] specifies that the mail server performing final
delivery of a message will generate a Return-Path: header containing
the return address of the message.

This return address MUST be formed by applying the procedure
described in section 7 of this document to the return address and
the recipient's address.

This also applies if the mail server invokes some other external
process to handle final delivery, instead of placing the message into
the recipient's mailbox. In all cases, the return address specified
by the mail server to any external environment or process MUST be
derived by applying the procedure in section 7 to the return address
and the recipient's address.

.ti 0
6. Relaying

When a VERP-compliant server determines that a recipient
of a VERP message is not a local mailbox, and the message
must be relayed to another server, the VERP-compliant server MUST:

.in +8
.ti -4
(1) If the VERP-compliant server's local policies require
the return and/or recipient addresses are to be rewritten,
the VERP-compliant server MUST make sure that delivery notices
MUST NOT be sent to a rewritten return address, or reflect
a rewritten recipient address, unless separate arrangements are
made with the sender of the VERP message.
This is because the sender expects to be able to
resolve non-delivery notices to some recipient address it knows
about (irrespective whether the addresses are resolved from a VERP,
or from a delivery status notification in the format specified by [2]).
When local policies of the relaying server require the
recipient or the sender address
to be rewritten,
the sender will obviously be not aware of this rewriting or expansion.
If a
subsequent delivery failure results in a non-delivery notice being sent
to a VERP containing the rewritten address, the sender will not be able
to associate the address encoded in the VERP with any recipient address
it has on file.

A suggested way to handle this situation is to set specify a delivery
status notification of "NEVER" - as specified by [3] -
for any recipient address generated by local aliasing or
rewriting, and treat the original address as being
either delivered or relayed, resulting in a
"delivered" or "relayed" delivery status notification if the original
recipient address specified a "SUCCESS" notification.

Note that this clause does not prohibit the relaying server to
have an established agreement with the sender to act as a mailing list
exploder, as long as this agreement is handled in a transparent way.
For example, the relaying server can set its own return address on all
VERP messages it exploded, and handle non-delivery notices all by itself.

.ti -4
(2) If the VERP-compliant server determines that the remote server is
also a VERP compliant server, the VERP keyword MUST be included in the
MAIL FROM command used to relay the VERP message to the remote server.

.ti -4
(3) If the remote server is not a VERP compliant server,
The VERP compliant server SHOULD
send a separate copy of the message for every recipient.
The return address of each
copy of the message MUST be formed by applying the procedure
described in section 7 of this document to the original return address,
and the address of each individual recipient.
Although the message SHOULD NOT be returned as undeliverable,
if it is then the rules defined in section 4.1 MUST be applied.

These rules also apply
if the SMTP-compliant server determines that the VERP message must be
forwarded via some other protocol to a non-SMTP gateway, unless the
non-SMTP
protocol has equivalent features that are completely identical in
function to Variable Envelope Return Path SMTP service extension
(including
any translations of E-mail addresses to and from the non-RFC822 format).
.in -8

.ti 0
7. Variable envelope return path encoding

This encoding method starts with a return address and one recipient
address.  As
mentioned previously, both addresses MUST be valid RFC822 addresses, [5],
and MUST
contain at least one @ character.
The portion of each address following the last @ character MUST
be compliant with [6], or must be an explict network address literal,
for example: john43@[192.68.0.4].

Let "sdomain" represent the portion of the return address that
follows the last @ character.

Let "slocal" represent the portion of the return address that
precedes the last @ character.

Let "rdomain" represent the portion of the recipient address that follows
the last @ character.

Let "rlocal" represent the portion of the recipient address that precedes
the last @ character.

To encode the recipient address within the envelope sender address,
create an address of the following form:

.ti +4
   slocal-encodedrlocal=encodedrdomain@sdomain

Where "encodedrlocal" and "encodedrdomain" are formed by taking
rlocal and rdomain, respectively, and encoding each one as follows:

.in +8
.ti -3
1) Each @, :, %, !, -, [, ], and + character in rlocal
is replaced by a single '+' character
followed by two uppercase hexadecimal characters whose value is the ASCII
code of the replaced character.

Taking into account poorly-written mail relays that ignore case-sensitivity,
both uppercase and lowercase hexadecimal characters SHOULD be recognized
when decoding envelope return paths.

.ti -3
2) All other characters are unchanged.  Other characters MAY, but SHOULD NOT
be also encoded in the same fashion.
.in -8

This can be represented using BNF as follows:

.in +2
.nf
encodedverp: slocal "-" encodedrlocal "=" encodedrdomain "@" sdomain

encodedrlocal: * (char-literal / char-encoded )

encodedrdomain: * (char-literal / char-encoded )

char-literal: any character valid in an RFC821 address [4],
              except @, :, %, !, [, ], -, and +

char-encoded: "+" hexdigit hexdigit

hexdigit: ("0" / "1" / "2" / "3" / "4" / "5" / "6" / "7" / "8" /
           "9" / "A" / "B" / "C" / "D" / "E" / "F" /
                 "a" / "b" / "c" / "d" / "e" / "f")
.fi
.in -2

.ti 0
8. Variable envelope return path decoding

Non-delivery notices for VERP messages
will be sent to either the original address, <slocal@sdomain>, or to the
VERP-encoded address, <slocal-encodedrlocal=encodedrdomain@sdomain>.

Messages sent to <slocal@sdomain> will be RFC 1891-compliant delivery
status notifications.  These messages will be machine-readable, and the
mail software will be able to
identify failed addresses from the RFC 1891 delivery report.

Non-delivery notices will also be sent to
the VERP-encoded address, and the mail software will be able to reconstruct
the failed address from the VERP-encoded address
by simply reversing the steps used in encoding:

.in +8
.ti -3
1) Extract encodedrlocal and rdomain from the recipient address.
There will be at least one = character in the encoded portion of the
return address.  encodedrlocal is everything up to the last =
character.
Everything following the last = character is encodedrdomain.

.ti -3
2) Replacing all occurrences of "+" followed by two hexadecimal digits
in encodedrlocal and encodedrdomain with the equivalent ASCII character.

.ti -3
3) Using the decoded rlocal, @, then rdomain.
.in -8

.ti 0
9. Examples

Suppose that a VERP-compliant server named "example.com" receives a message
via the following SMTP conversation (for brevity, non-relevant headers have
been omitted):

.in +4
.nf
220 example.com ESMTP
EHLO domain.com
250-example.com ESMTP
250-SIZE
250-DSN
250-VERP
250 HELP
MAIL FROM:<[email-masked]> VERP SIZE=100
250 Ok
RCPT TO:<[email-masked]>
250 Ok
RCPT TO:<[email-masked]>
250 Ok
RCPT TO:<[email-masked]>
250 Ok
RCPT TO:<[email-masked]>
250 Ok
RCPT TO:<[email-masked]>
250 Ok
DATA
354 Ok
From: "John" <[email-masked]>
Date: Thu, 16 Jan 1997 14:49:31 -0500 (EST)
Subject: Meeting canceled.

Today's 2pm meeting has been rescheduled for tomorrow, 9am, due
to a scheduling conflict.
\&.
.fi
.in -4

The message is delivered to the local mailbox for <[email-masked]>.
The message looks like this:

.in +4
.nf
Return-Path: <itny-out-alex=[email-masked]>
From: "John" <[email-masked]>
Date: Thu, 16 Jan 1997 14:49:31 -0500 (EST)
Subject: Meeting canceled.

Today's 2pm meeting has been rescheduled for tomorrow, 9am, due
to a scheduling conflict.
.fi
.in -4

The VERP-compliant server at example.com connects to the mail server
for old.example.com.
old.example.com does not support the
Variable Envelope Return Path extension.
Therefore, old.example.com receives two messages.  The SMTP
conversation for the first message is as follows:

.in +4
.nf
220 old.example.com ESMTP
EHLO example.com
250-old.example.com ESMTP
250-SIZE
250-DSN
250 HELP
MAIL FROM:<itny-out-node42+21ann=[email-masked]>
250 Ok
RCPT TO:<[email-masked]>
250 Ok
DATA
354 Ok
From: "John" <[email-masked]>
Date: Thu, 16 Jan 1997 14:49:31 -0500 (EST)
Subject: Meeting canceled.

Today's 2pm meeting has been rescheduled for tomorrow, 9am, due
to a scheduling conflict.
\&.
.fi
.in -4

The SMTP conversation for the second message is as follows:

.in +4
.nf
MAIL FROM:<itny-out-tom=[email-masked]>
250 Ok
RCPT TO:<[email-masked]>
250 Ok
DATA
354 Ok
From: "John" <[email-masked]>
Date: Thu, 16 Jan 1997 14:49:31 -0500 (EST)
Subject: Meeting canceled.

Today's 2pm meeting has been rescheduled for tomorrow, 9am, due
to a scheduling conflict.
\&.
.fi
.in -4

example.com connects to new.example.com and determines that
new.example.com runs a modern ESMTP server that supports the VERP
keyword. The SMTP conversation then goes like this:

.in +4
.nf
220 new.example.com ESMTP
EHLO example.com
250-new.example.com ESMTP
250-SIZE
250-DSN
250-VERP
250 HELP
MAIL FROM:<[email-masked]> VERP SIZE=100
250 Ok
RCPT TO:<[email-masked]>
250 Ok
RCPT TO:<[email-masked]>
250 Ok
DATA
354 Ok
From: "John" <[email-masked]>
Date: Thu, 16 Jan 1997 14:49:31 -0500 (EST)
Subject: Meeting canceled.

Today's 2pm meeting has been rescheduled for tomorrow, 9am, due
to a scheduling conflict.
\&.
.fi
.in -4

.ti 0
10. Security concerns

All the usual security considerations applicable to SMTP are also
applicable to this extension.
Relay of VERP messages to non-VERP servers requires a single
message with many recipients
to be exploded into many messages with
one recipient.
In all cases, however, there will never be any additional overhead
beyond the resources that are required when VERPs
are manually implemented by the mail sender, instead of
the VERP SMTP extension.

Mail systems which support the VERP extension SHOULD have adequate
security measures, including blocks against unauthorized access and
relaying.

.ti 0
10.1 Vacation programs, and other autoresponders

"Vacation" type autoresponders are often used in practice.
A vacation autoresponder is a program that automatically replies to
every message, informing the sender that the
recipient is on vacation, or is generally unavailable at this time.

Vacation autoresponders MUST NOT generate autoresponses to mailing
list messages, but people often forget to do set them up to do so.
Because autoresponses are sent to the same address that's used to
receive non-delivery reports, malfunctioning autoresponders result
in the recipient being removed from mailing lists.

Advanced autoresponders send automatic replies in the format specified
by [2], as a "delayed" notification.  DSN-aware software will not
remove addresses from mailing lists due to delayed notifications.

Section 5 of this document specifies that the mail server MUST
replace the original return address with a VERP-modified address
when delivering the message to a mailbox or an external process.

Therefore it is possible
that RFC 1891 reports may also be sent to
a VERP-encoded address, as specified by sections 5 and 7 of this document.
Mail software SHOULD ignore any RFC 1891 "delayed" or "success" reports that
sent to a VERP-encoded address.
If it is a "failed" report, note that the VERP address will be
more reliable than the address specified in the report itself.

.bp
.ti 0
11. References

.in +8
.ti -4
[1] Klensin, J., Freed, N., Rose, M., Stefferud, E., Crocker, D.
"SMTP Service Extensions", RFC 1425, United Nations University,
Innosoft International, Inc., Dover Beach Consulting, Inc.,
Network Management Associates, Inc., The Branch Office,
February 1993

.ti -4
[2] Moore, K., and G. Vaudreuil, "An Extensible Message Format for
Delivery Status Notifications", RFC 1894, University of Tennessee,
Octel Network Services, January 1996.

.ti -4
[3] Moore, K. "SMTP Service Extension for Delivery Status
Notifications", RFC 1891, University of Tennessee, January 1996.

.ti -4
[4] Postel, J., "Simple Mail Transfer Protocol", STD 10, RFC 821,
USC/Information Sciences Institute, August 1982.

.ti -4
[5] Crocker, D., "Standard for the Format of ARPA Internet Text
Messages", STD 11, RFC 822, UDEL, August 1982.

.ti -4
[6] Mockapetris, P., "Domain Names - Implementation and
Specification", RFC 1035, ISI, November 1987

.in -8
.ti 0
12. Author's address

.nf
Sam Varshavchik
Double Precision, Inc.
402 Main Street Suite 100-241
Metuchen, NJ 08840
<[email-masked]>
.fi
EOF

---

From: Hector Santos <[email-masked]>
Date: Mon, 09 Dec 2019 21:39:54 -0500

On 12/9/2019 2:04 PM, Keith Moore wrote:
>> considerations and RFC5321 limits, including exploring the addition
>> of using an encryption method, i.e. TripleDES or AES, to possibly
>> reduce and help keep VERP-encoded addresses sizes within the SMTP
>> address limits.
>

+1.

Yes, hashing would normally be considered for reduced key size 
indexing. But its not reversible without some sort of mapping. So I 
asked google:

"two way compression without a mapping dictionary"
"reversible hash"

It lead me to an interesting StackOverFlow thread. See Solution #3 here

https://stackoverflow.com/questions/4273466/reversible-hash-function

which also leads you to this with some C# code to explore.

https://stackoverflow.com/questions/165808/simple-insecure-two-way-obfuscation-for-c-sharp

Of course, the goal is to see if we can "correct" the limit 
incompatibilities using a simple two-way encoder/decoder.

If the 5321.SMTP max path length is 320:

user 64
@    1
domain 255

Can a longer VERP-encoded address be reduced to <= 320?

-- 
HLS

---

From: Keith Moore <[email-masked]>
Date: Mon, 9 Dec 2019 21:54:51 -0500

> Can a longer VERP-encoded address be reduced to <= 320? 

Not reliably.

Keith

---

From: Hector Santos <[email-masked]>
Date: Tue, 10 Dec 2019 13:45:34 -0500

Sam,

Well, It looks like your 1999 VERP ESMTP add-on has been well-received 
and widely implemented.  Great work! Would you be interested in 
updating it?  Does posting the nroff source suggest you wish to pass 
it on?

Others here can tell you better, XML2RFC appears to the current 
official IETF tool for I-D submissions. It gets all the IETF process 
related stuff prepared for you, including new legalese stuff.

https://xml2rfc.tools.ietf.org/experimental.html

Maybe the nroff source can be used as input to XML2RFC where you can 
use the outliner/editor to output IETF I-D submission ready-file 
formats. There is also new GitHub work going to help with 
collaboration and version control. Your email address was found at 
your github page, so I thought GitHub work may interest you.

https://www.ietf.org/mailman/listinfo/ietf-and-github

Thanks

-- 
HLS

On 12/9/2019 8:48 PM, Sam Varshavchik wrote:
> Hector Santos writes:
>
>> Hi Sam,
>>
>> There is some discussion in the IETF-SMTP list regarding VERP. I
>> would like to see it updated, codified, modernized, lessons leaned,
>> but also dealing with technical length limits.  See below for my
>> proposal to be updated.  But please check out the IETF-SMTP list.
>>
>> What do you think?
>
> submitting the draft, but I guess I did. My main intent was mostly
> public documentation of the work that I did at that time (and still do).
>
> I sifted through my archives and found my original working nroff
> source, which I'm attaching below, if it helps. I have no idea if
> 'roff is still the suggested working format. The following version
> includes a few changes I made to the working code after the original
> draft.
>
>
> #! /bin/sh
> ..pl 10.0i
> ..po 0
> ..ll 7.2i
> ..lt 7.2i
> ..nr LL 7.2i
> ..nr LT 7.2i
> ..ds RF FORMFEED[Page %]
> ..ds LF S. Varshavchik
> ..ds CF Expires XXX XX, XXXX
> ..ds LH VERP SMTP Extension
> ..ds RH XXX XX, XXXX
> ..ds CH S. Varshavchik
> ..hy 0
> ..ad l
> ..in 0
> ..\".nf
> ..\"INTERNET-DRAFT                                            S.
> Varshavchik
> ..\"Expires XXX XX, XXXX                              Double
> Precision, Inc.
> ..\"                                                            XXX
> XX, XXXX
> ..\"
> ..\".ce
> ..\"Variable Envelope Return Path SMTP Extension
> ..\".ce
> ..\"draft-varshavchik-verp-smtpext-04.txt
> ..\"
> ..\".ti 0
> ..\"Status Of This Memo
> ..\"
> ..\".fi
> ..in 3
> ..\"This document is an Internet-Draft and is in full conformance
> ..\"with all provisions of Section 10 of RFC2026.
> ..\"
> ..\"Internet-Drafts are working documents of the Internet Engineering
> ..\"Task Force (IETF), its areas, and its working groups.  Note that
> ..\"other groups may also distribute working documents as
> ..\"Internet-Drafts.
> ..\"
> ..\"Internet-Drafts are draft documents valid for a maximum of six
> ..\"months and may be updated, replaced, or obsoleted by other
> ..\"documents at any time.  It is inappropriate to use Internet-
> ..\"Drafts as reference material or to cite them other than as
> ..\""work in progress."
> ..\"
> ..\"The list of current Internet-Drafts can be accessed at
> ..\"
> ..\"http://www.ietf.org/ietf/1id-abstracts.txt
> ..\"
> ..\"The list of Internet-Draft Shadow Directories can be accessed at
> ..\"http://www.ietf.org/shadow.html.
> ..\"
> ..\".ti 0
> ..\"0. Revision history
> ..\"
> ..\"01 - added additional comments on DSNs not being very successful in
> ..\"automatically handling mailing list functions.  Additional section
> ..\"added with comments regarding vacation autoresponders.
> ..\"
> ..\"02 - expanded section 6 to describe the proper way to handle local
> ..\"aliasing, or address explosion.
> ..\"
> ..\"03 - added the characters -, [, and ], to the list of characters that
> ..\"must be hex-encoded in a VERP, as well as requiring the domain
> portion
> ..\"of the return address to be encoded.  This resolves certain
> ambiguities in
> ..\"potential return addresses, as well as allow network address literals
> ..\"to be used.  Adjusted example SMTP conversation to reflect proper
> reply
> ..\"codes.
> ..\"
> ..\"04 - clarify that upper and lowercase hexadecimal characters are
> permitted.
> ..\"Update contact information.
> ..\"
> ..ti 0
> 1. Abstract
>
> ..ti 0
> ..ti 0
> ..ti 0
> ..in +7
> ..ti -4
> ..ti -4
> ..ti -4
> ..ti -4
> ..ti -4
> ..ti -4
> ..in -7
> ..\"The IANA registry is ammended as follows.  "ESMTP MAIL KEYWORDS"
> ..\"are ammended as follows:
> ..\"
> ..\".nf
> ..\"Keywords Description                    Reference
> ..\"-------- -----------------------------
> --------------------------------
> ..\"VERP     Variable Envelope Return Path
> [draft-varshavchik-verp-smtpext]
> ..\".fi
> ..\"
> ..\""MAIL EXTENSION TYPES" are ammended as follows:
> ..\"
> ..\".nf
> ..\"Service Ext   EHLO Keyword Parameters Verb   Reference
> ..\"-----------   ------------ ---------- ------
> --------------------------------
> ..\"Var Env RP       EHLO         none     none
> ..\".fi
>
> ..ti 0
> ..in +8
> ..ti -4
> ..ti -4
> ..ti -4
> ..in -8
> ..ti 0
> ..in +8
> ..ti -3
> ..ti -3
> ..in -8
> ..ti 0
> ..ti 0
> ..in +8
> ..ti -4
> ..ti -4
> ..ti -4
> ..in -8
> ..ti 0
> ..ti +4
> ..in +8
> ..ti -3
> ..ti -3
> ..in -8
> ..in +2
> ..nf
> ..fi
> ..in -2
> ..ti 0
> ..in +8
> ..ti -3
> ..ti -3
> ..ti -3
> ..in -8
> ..ti 0
> ..in +4
> ..nf
> ..fi
> ..in -4
> ..in +4
> ..nf
> ..fi
> ..in -4
> ..in +4
> ..nf
> ..fi
> ..in -4
> ..in +4
> ..nf
> ..fi
> ..in -4
> ..in +4
> ..nf
> ..fi
> ..in -4
> ..ti 0
> ..ti 0
> ..bp
> ..ti 0
> ..in +8
> ..ti -4
> ..ti -4
> ..ti -4
> ..ti -4
> ..ti -4
> ..ti -4
> ..in -8
> ..ti 0
> ..nf
> ..fi

---

From: Sam Varshavchik <[email-masked]>
Date: Wed, 11 Dec 2019 07:14:29 -0500

Hector Santos writes:

> Sam,
>
> Well, It looks like your 1999 VERP ESMTP add-on has been well-received and  
> widely implemented.  Great work! Would you be interested in updating it?   
> Does posting the nroff source suggest you wish to pass it on?

I didn't know if someone was already working on the updated draft, or not,  
and might find my old work useful. If so, that's fine; otherwise I can  
certainly spend some time working on this.

> Others here can tell you better, XML2RFC appears to the current official  
> IETF tool for I-D submissions. It gets all the IETF process related stuff  
> prepared for you, including new legalese stuff.
>
> Maybe the nroff source can be used as input to XML2RFC where you can use the  
> outliner/editor to output IETF I-D submission ready-file formats. There is  
> also new GitHub work going to help with collaboration and version control.  
> Your email address was found at your github page, so I thought GitHub work  
> may interest you.
>

Took a brief look at the xml2rfc page, I can work with that. It shouldn't be  
difficult to convert my old nroff source to XML. Is there a standalone tool  
to generate HTML or text from the XML source, somewhere?

I couldn't quite make out what's going on with ietf-and-github, but, in  
general, I can collaborate via github too.

For now, I'll take my nroff source and convert it to XML, then figure out  
what to do next.

---

From: Alexey Melnikov <[email-masked]>
Date: Wed, 11 Dec 2019 12:28:15 +0000

Hi Sam,

On 11/12/2019 12:14, Sam Varshavchik wrote:
> Took a brief look at the xml2rfc page, I can work with that. It 
> shouldn't be difficult to convert my old nroff source to XML. Is there 
> a standalone tool to generate HTML or text from the XML source, 
> somewhere?

I haven't used command line tool, but I use the web serice: 
<https://xml2rfc.tools.ietf.org/>

> I couldn't quite make out what's going on with ietf-and-github, but, 
> in general, I can collaborate via github too.
>
> For now, I'll take my nroff source and convert it to XML, then figure 
> out what to do next. 

Best Regards,

Alexey

---

From: Viruthagiri Thirumavalavan <[email-masked]>
Date: Wed, 11 Dec 2019 21:00:12 +0530

Hi Sam,

I was working on a draft which is why this whole discussion started.  You
can read the full discussion here
<https://mailarchive.ietf.org/arch/msg/ietf-smtp/5i8QNwqL90KSiVDXVNNdM29PYiw>
.

My draft
<https://tools.ietf.org/html/draft-viruthagiri-email-address-length-01>
mainly focuses on relaxing the local-part email address 64 character
limitation set by RFC 821 / 2821 / 5321, but covers both VERP and SRS for
informational purposes.

Your draft <https://tools.ietf.org/html/draft-varshavchik-verp-smtpext-01>
seems like trying to solve the bandwidth wastage problem introduced by
VERP. It also covers VERP address structure. I like your proposal by the
way.

In my opinion, the problem with VERP addresses is not the overall email
address length limitation, but it's the 64 character local-part limitation.
VERP addresses becomes problematic when the end user pick longer
local-part.

If implementations start to adopt your VERP extension, that's a good thing
especially for large mail services since it's saves plenty of bandwidth. So
feel free to use the content from my draft in your proposal.

Thanks.

On Wed, Dec 11, 2019 at 5:44 PM Sam Varshavchik <[email-masked]>
wrote:

> and
> official

-- 
Best Regards,

---

From: Sam Varshavchik <[email-masked]>
Date: Thu, 12 Dec 2019 07:09:37 -0500

> feel free to use the content from my draft in your proposal. 

Thanks, I'll take a look at it, shortly.

I have no recollection that the 64 char limit on local E-mail addresses was  
ever mentioned as an issue in front of me. Mine is not the only mailing list  
software that uses the same approach of encoding mailing list bounces  
addresses. Looking through my mailbox, one of my mailing lists' return  
addresses:

<nut-upsuser-bounces+mrsam=[email-masked]>

According to that one's headers, this is GNU mailman. Which is much more  
widespread than anything that I wrote. Furthermore, my VERP extension did  
not pioneer this encoding. It's already been in use, back in 1999; I merely  
parroted it. It's been in use for at least 20 years. I do not remember  
reading any reports that it caused any issues, anywhere.

I count 41 characters already, in that local address, right there.  
"firstname.lastname@domain" is a fairly popular format of corporate E-mail  
addresses. And if some of us had to use this E-mail address convention, it's  
a fair bet that GNU mailman's bounce addresses will exceed the 64 char limit  
on the local part. But I'll be surprised if it ever causes any problems,  
anywhere; and I never heard of this ever happening.

---

From: Viruthagiri Thirumavalavan <[email-masked]>
Date: Fri, 13 Dec 2019 04:15:30 +0530

> anywhere; and I never heard of this ever happening.

Most servers don't enforce that 64 character limit in the local-part. But
some minority servers certainly do. What happens when those minority
servers enforces that limit and the end user has
the "firstname.lastname@domain" type address?

For the record, plenty of folks in this mailing list use corporate e-mail
addresses. I have a lengthy name. If I go for "firstname.lastname@domain"
format, then the bounce address may exceed the 64 character limit. Since
your proposal deal with SMTP extension, you can explicitly mention not to
enforce the 64 character local-part limitation when the server support your
extension.

However, the real problem starts when multiple hops involved.  For example
I have a forwarding address in Godaddy which catches all the mails and
forwards them to a @gmail.com address. This is how the Return-Path looks
like in my Gmail account when I send an email from [email-masked] to the
forwarding address.

<SRS0=JUYF=2A=dombox.org=[email-masked]>

Since Godaddy acts as an intermediary here, they need to rewrite the MAIL
FROM address.

If Godaddy use the same [email-masked] in the MAIL FROM

        (1) SPF is gonna fail. So the mail may end up in spam.
        (2) There's gonna be bounce issues due to SPF failure. Gmail have
to silently drop the mail in order to prevent backscatter spam.

To use your address as an example.

Original: <[email-masked]>

Debian: <nut-upsuser-bounces+mrsam=[email-masked]>

Godaddy: <SRS0=2wQu=2C=alioth-lists.debian.net=nut-upsuser-bounces+mrsam=
[email-masked]>

So when intermediaries get involved, local-part limitation gonna cause
issues at least for few cases.  I think you need to address that part in
your draft.

Thanks

-- 
Best Regards,

---

From: Sam Varshavchik <[email-masked]>
Date: Thu, 12 Dec 2019 23:33:10 -0500

Viruthagiri Thirumavalavan writes:

> some minority servers certainly do. What happens when those minority servers  
> enforces that limit and the end user has the "firstname.lastname@domain" type  
> address? 

I expect users of those servers to have occasional problems subscribing to  
mailing lists that use GNU mailman, and other software that already uses  
VERP return addresses.

> However, the real problem starts when multiple hops involved.  For example I  
> have a forwarding address in Godaddy which catches all the mails and forwards  
> them to a @<URL:http://gmail.com>gmail.com address. This is how the Return- 
> Path looks like in my Gmail account when I send an email from  
> <URL:mailto:[email-masked]>[email-masked] to the forwarding address.

Well, I break down this situation in several smaller pieces.

An SMTP sender that implements VERP implicitly acknowledges that it accepts  
E-mail addresses with longer local parts.

An SMTP server that advertises a VERP SMTP extension acknowledges the same.

So now you have a VERP SMTP sender connecting to a non-VERP SMTP server.

Mail that does not have a VERP - nothing changes to that.

Mail with a VERP return address - that's the only combination where there  
may be an issue, but that's only a possibility, and by no means a certainty.  
The sender can proactively refuse to send mail when it constructs a return  
address with a local part that exceeds 64 characters. It would be a shame if  
it did that, only to discover later that the non-VERP receiver can receive  
mail with longer local-parts.

The sender can roll the dice and attempt to send the mail anyway. Chances  
are pretty good that it will be fine. The possibility of a rejection are  
quite small.

I don't see any way around this. I see no other ways to handle this.

> Godaddy: <SRS0=2wQu=2C=<URL:http://alioth-lists.debian.net>alioth- 
> lists.debian.net=nut-upsuser-bounces+mrsam=<URL:mailto:courier- 
> [email-masked]>[email-masked]>
>
>
> So when intermediaries get involved, local-part limitation gonna cause issues  
> at least for few cases.  I think you need to address that part in your  
> draft. 

Sure. My old draft already has a "Security" section that waxed eloquently on  
lesser topics. I just completed a first pass on converting the last version  
of my draft to XML, without any further changes. xsltproc balked on  
rfc2629.xslt's version=2.0, but seemed to work if I manully change it to  
1.0, the end results look reasonable, but I want to look through the whole  
thing, and then try the online converter, and if all looks good I'll dump it  
on github and start a discussion about concrete changes.

But, back to the above: I looked at what's in my mailbox. I see mass mail  
from Youtube and Linked in, with local parts that are 55 and 59 characters  
long. So, according to this, all mass mail that Godaddy forwards from  
Youtube and Linked-In already exceeds 64 characters in local-part. Looks to  
me like that ship has already sailed.

---

From: Viruthagiri Thirumavalavan <[email-masked]>
Date: Fri, 13 Dec 2019 20:24:39 +0530

> I don't see any way around this. I see no other ways to handle this.

I'm not sure how viable this method is. I think the client should look for
MAIL FROM command response and then retry with a non VERP address when the
local-part exceed 64 characters.

501 Syntax error in parameters or arguments
455  Server unable to accommodate parameters
555  MAIL FROM/RCPT TO parameters not recognized or not implemented

On Fri, Dec 13, 2019 at 10:03 AM Sam Varshavchik <[email-masked]>
wrote:

> servers
> issues

-- 
Best Regards,

---

From: Hector Santos <[email-masked]>
Date: Fri, 13 Dec 2019 12:57:17 -0500

On 12/12/2019 11:33 PM, Sam Varshavchik wrote:

>> So when intermediaries get involved, local-part limitation gonna
>> cause issues at least for few cases.  I think you need to
>> address that part in your draft.
>
> Sure. My old draft already has a "Security" section that waxed
> eloquently on lesser topics. I just completed a first pass on
> converting the last version of my draft to XML, without any further
> changes. xsltproc balked on rfc2629.xslt's version=2.0, but seemed to
> work if I manully change it to 1.0, the end results look reasonable,
> but I want to look through the whole thing, and then try the online
> converter, and if all looks good I'll dump it on github and start a
> discussion about concrete changes.

Sam,

I am interested in reviewing the XML file. I give it a quick look over 
the weekend. I need to catch up with XMK2RFC myself.

Regarding section 7,  currently the Verp-01 draft section 7 describes 
the very simple algorithm to generate the verp-encoded address:

slocal-encodedrlocal=rdomain@sdomain

where

slocal is the left hand side of the return path,
sdomain is the right hand side of the return path,
rlocal is the left hand side of the recipient path,
rdomain is the right hand side of the recipient path,
encodedrlocal is the VERP-01 specific character encoding of rlocal.

The encodedrlocal encodes the characters: @, :, %, !, and +

I noted in the updated nroff source version, you updated it to include 
dashes and squared brackets. This is good. Many slocal and rlocal 
values will include the dash so it was needed to help with parsing the 
parts by the MDA Verp Bounce Processor.

Do you have any examples of rlocal where squared brackets needed to be 
encoded?

Also, are you aware of any other VERP variations? I am particularly 
interested in adding an optional hashed or another encoded version, 
but also in detecting them.  I did a grep of all the "C: MAIL FROM" 
for the past three day, and there appears to be a wide range of 
VERP-like encoding formats.  I can see the VERP-01 formats as well, 
but I see many with hashes, guids, etc.

Thanks

-- 
HLS

---

From: Keith Moore <[email-masked]>
Date: Fri, 13 Dec 2019 13:58:52 -0500

To nobody in particular:

IMO, it is a Bad Idea to try to embed one address in another, and an 
especially Bad Idea to extend SMTP to try to facilitate that. If you 
want to know whose mail bounced, sure, add a small token to the return 
address when sending the mail.   Associate that small token with the 
original recipient address in a database, so when you get the NDN you 
can look up that token and see whose mail bounced.   But don't try to 
make it invertible.   You're only making the service less reliable by 
trying to do so, and also tempting people to try to decode information 
which should be opaque to them.

Keith

---

From: Sam Varshavchik <[email-masked]>
Date: Fri, 13 Dec 2019 19:09:49 -0500

> 555  MAIL FROM/RCPT TO parameters not recognized or not implemented

I see this as a trade-off. This is gambling that if this gets forwarded on  
with the original list of recipients, as is, any eventual bounce will be a  
parsable DSN. But if it's not, the original purpose of using VERPs – which  
is always being able to identify a bouncing address – has been defeated.  
Now, the mail robots is going to get stuck with undeliverable addresses that  
will bounce forever, and cannot be identified.

---

From: Sam Varshavchik <[email-masked]>
Date: Fri, 13 Dec 2019 19:16:40 -0500

Hector Santos writes:

> I am interested in reviewing the XML file. I give it a quick look over the  
> weekend. I need to catch up with XMK2RFC myself.

I put it up now, at https://github.com/svarshavchik/verp-smtpext

The online tool currently chokes on the bibxml library's RFC 1425 reference,  
it doesn't like one of its stub author entries. I removed it and manually  
pulled the reference in, rather than reference it from the bibxml library.

>
> Regarding section 7,  currently the Verp-01 draft section 7 describes the  
> very simple algorithm to generate the verp-encoded address:
>
> dashes and squared brackets. This is good. Many slocal and rlocal values  
> will include the dash so it was needed to help with parsing the parts by the  
> MDA Verp Bounce Processor.
>

Not rlocal but rdomain.

An E-mail address can still be local@[aaa.bbb.ccc.ddd], pedantically.

> interested in adding an optional hashed or another encoded version, but also  
> in detecting them.  I did a grep of all the "C: MAIL FROM" for the past  
> three day, and there appears to be a wide range of VERP-like encoding  
> formats.  I can see the VERP-01 formats as well, but I see many with hashes,  
> guids, etc.

The VERP-01 formats you're seeing most likely preceded my draft. This kind  
of bounce processing was already fairly common <1999.

To this day Courier still treats it as a private extension, and advertises  
XVERP instead of VERP, in EHLO. I am not aware of any other actual  
implementation. There may be one, but I haven't heard of it.

---

From: Sam Varshavchik <[email-masked]>
Date: Fri, 13 Dec 2019 19:36:14 -0500

Keith Moore writes:

> especially Bad Idea to extend SMTP to try to facilitate that. If you want to  
> know whose mail bounced, sure, add a small token to the return address when  
> sending the mail.   Associate that small token with the original recipient  
> address in a database, so when you get the NDN you can look up that token  
> and see whose mail bounced.   But don't try to make it invertible.   You're  
> only making the service less reliable by trying to do so, and also tempting  
> people to try to decode information which should be opaque to them.

No argument from me. Agreeing to all of the above, but having said that:

The VERP-style encoding of bounce addresses has been in use for at least 20  
years now. I wasn't the one who came up with it. I don't really know when it  
originated. Qmail's ezmlm used it, according to Wikipedia this dates it back  
to 1997 at the latest.

If this ever caused any interoperability problems somewhere, I haven't heard  
of it. I had to deal with more issues related to EAI (in IMAP space mostly)  
than with VERP (namely, no issues at all).

I'm just pointing this out. I'm not married to my current implementation,  
but it worked for me.

---

From: Viruthagiri Thirumavalavan <[email-masked]>
Date: Sat, 14 Dec 2019 22:11:10 +0530

Thanks Keith.

-- 
Best Regards,

---

From: Viruthagiri Thirumavalavan <[email-masked]>
Date: Sat, 14 Dec 2019 22:12:46 +0530

> originated.

http://cr.yp.to/proto/verp.txt

-- 
Best Regards,

---

From: Sam Varshavchik <[email-masked]>
Date: Sat, 14 Dec 2019 11:53:11 -0500

Viruthagiri Thirumavalavan writes:

> <URL:http://cr.yp.to/proto/verp.txt>http://cr.yp.to/proto/verp.txt 

I did mention Bernstein's ezmlm, where I picked this up originally. It's  
likely that he was the first one to come up with this, circa 1997. Certain  
aspects of Qmail which, I'm pretty sure had no equivalents in other popular  
MTAs of that time, made this approach a natural fit; so that's where this  
likely came from, but I could not be sure.

---

From: Viruthagiri Thirumavalavan <[email-masked]>
Date: Sat, 14 Dec 2019 23:13:13 +0530

I thought Daniel J. Bernstein is the one who came up with the idea.  But It
seems like he came up with only the term VERP according to wiki.

Thanks for clarifying.

On Sat, Dec 14, 2019 at 10:23 PM Sam Varshavchik <[email-masked]>
wrote:

> >    The VERP-style encoding of bounce addresses has been in use for at
> least
> >    20
> >    years now. I wasn't the one who came up with it. I don't really know
> when
> >    it
> >    originated.
> >
> >
> >

-- 
Best Regards,

---

From: "Valdis Klētnieks" <[email-masked]>
Date: Sat, 14 Dec 2019 13:14:10 -0500

> seems like he came up with only the term VERP according to wiki.

Eric Thomas/LSoft's Listserv software started doing the same thing with a
different encoding scheme around then as well, but I don't have any info
on who was "first".

---

From: Viruthagiri Thirumavalavan <[email-masked]>
Date: Sun, 15 Dec 2019 03:49:34 +0530

Thanks for the names. We should give them credits if this proposal get
passed.