Subject: Optimising mail to many recipients

From: Jacob Palme <[email-masked]>
Date: Mon, 22 Mar 1999 21:37:00 +0100

Suppose you need to send a large e-mail message, maybe with
a one megabyte attachment, to ten recipients on another
continent. It might then be much more efficient usage of
the long-distance cables, if you connected to one SMTP
server on the other continent, and sent one copy of the
message, asking this server to forward it to the ten
recipients.

Until about a year ago, this could be done using almost any
SMTP server. But I do not think it was very commonly used.
Is the optimisation not worth the effort? Or are those who
can do this optimisation not people who would benefit from
it?

About a year ago, most SMTP servers all around the world
changed their policy, so that they only accepted messages
to or from their local users. The reason for this change
was to make spamming more difficult.

Does the drums standard say anything on this issue?
------------------------------------------------------------------------
Jacob Palme <[email-masked]> (Stockholm University and KTH)
for more info see URL: http://www.dsv.su.se/~jpalme

---

From: [email-masked]
Date: Mon, 22 Mar 1999 23:32:18 +0200 (GMT)

> Suppose you need to send a large e-mail message, maybe with
....
> can do this optimisation not people who would benefit from

	It depends.  It isn't very common, but with large-
	scale email lists some kind of "top-level domain
	distribution" seems to be a practice.  Machinery
	to do it varies, of course, and is peer agreenment
	based.

> Does the drums standard say anything on this issue?

	No ?

> Jacob Palme <[email-masked]> (Stockholm University and KTH)

/Matti Aarnio <[email-masked]>

---

From: Keith Moore <[email-masked]>
Date: Mon, 22 Mar 1999 18:34:35 -0500

> Does the drums standard say anything on this issue?

No.  And it's not within the scope of DRUMS to do so.

---

From: Alan Barrett <[email-masked]>
Date: Wed, 24 Mar 1999 11:05:48 +0200 (SAST)

> message, asking this server to forward it to the ten

This is probably outside the scope of DRUMS, but I have sometimes
thought that, if you have a message for multiple recipients, it would be
a good idea to search for common MX records and use those to reduce the
number of copies that you have to send.

For example, say you have a message addressed to users at A.X.TLD,
B.X.TLD and Y.TLD, and you look up all their MX records and get the
following results:

    A.X.TLD.	MX  0 A.X.TLD.
    A.X.TLD.	MX 10 MAILSERVER.X.TLD.
    A.X.TLD.	MX 20 MAIL.BIGISP.TLD.

    B.X.TLD.	MX  0 B.X.TLD.
    B.X.TLD.	MX 10 MAILSERVER.X.TLD.
    B.X.TLD.	MX 20 MAIL.BIGISP.TLD.

    Y.TLD.	MX  0 FIREWALL.Y.TLD.
    Y.TLD.	MX 50 MAIL.BIGISP.TLD.

Now, normal MX processing rules say that you should send three copies
(to A.X.TLD, B.X.TLD and FIREWALL.Y.TLD, assuming that all those hosts
are alive).  But it might sometimes be more efficient to send two
copies (to MAILSERVER.X.TLD and FIREWALL.Y.TLD), or just one copy (to
MAIL.BIGISP.TLD).  How do you decide which of those would be "best"?

--apb (Alan Barrett)

---

From: Keith Moore <[email-masked]>
Date: Thu, 25 Mar 1999 01:22:00 -0500

> a good idea to search for common MX records

I wonder, in practice, how often this is likely to help much?
(especially given the delay required to look up all MX records
before attempting to deliver any of the mail)

---

From: Bart Schaefer <[email-masked]>
Date: Thu, 25 Mar 1999 18:53:48 -0800 (PST)

(Meta-comment: should we continue this discussion on some other mailing
list?)

> before attempting to deliver any of the mail)

By a remarkable coincidence, a colleague of mine (Bob Glickstein) was
doing some research on exactly this topic last week.  His results:

|From: <[email-masked]>
|Subject: MX experiment
|Date: Sat, 13 Mar 1999 16:28:14 -0800 (PST)
|
|Hi everyone. I wrote a short Perl script that takes a list of e-mail
|addresses and MX-expands them, then reports on the number of resolvable
|addresses, distinct domains, distinct MX hosts, and primary MX hosts.
|
|I fed my program a list of 3602 distinct e-mail addresses culled from
|my personal e-mail archive. Of those, 2944 were resolvable in a total
|of 1717 distinct domains. (The others were invalid addresses.) There
|were 2908 MX hosts total serving those domains, of which 1532 were
|distinct primary hosts. So 1532 SMTP connections could suffice to
|deliver those 2944 messages, a savings of 49% over the naive approach
|of one connection per message, and a savings of 11% over the slightly
|less naive approach of one connection per domain.
|
|The savings could be even higher if we were willing to deliver
|to non-primary MX hosts. For example, addresses in the domain
|uunet.uu.net have the primary MX host mail.uu.net, while addresses
|in telco.com have mail.uu.net as a secondary MX host. Their primary
|is drawbridge.telco.com, and if we always try primary MX's first,
|that's two separate SMTP connections to deliver to uunet.uu.net and
|telcom.com. But if we optimize differently, we can handle both domains
|with one SMTP connection (to mail.uu.net). From my test data, it
|looks like this extra optimization means those 2944 messages could be
|delivered with 1384 SMTP connections---53% fewer than the naive way,
|19% fewer than the less naive way, and 10% fewer than my first clever
|way above.