Subject: [Asrg] Getting SMTP "Bounce email" part of RFC changed... forget binary attachments

From: Jim Witte <[email-masked]>
Date: Thu, 1 Jan 1970 03:50:53 -0500

from Vol 523
>   Despite the recent phenomenal increase in spam, there is still
> massive opposition to *doing* anything about the problem.  I remain,
> as always, amazed. [..] There are plenty of people willing to do 
> something about it but
> talking isn't doing, is it? This place can never solve or even stem
> the problem. It's a research group, research. Not design or

Hi,

   I've been lurking on this list for a couple of days, trying to figure 
out of this is the best place to bring this idea up (someone 
recommended when I posted to the news.admin.net-abuse.email group).

   I have in the past gotten an awful lot of "Mail undeliverable" 
message from the daemon, with attached virus code in raw hex.  I have 
been told that some of the worms will send out email that looks like 
such a mail daemon error.  This could be, but I got the impression here 
that some worm had sent out copies of itself to a bunch of addresses, 
with my address in a spoofed From: header, and that for those addresses 
that had been cancelled, or did not exist (I've noticed that many 
worms/spammers appear to send junk to several "variants" of a 
particular address).  Then the mail (ALL of it, including the attached 
worm code in raw hex) is bounced back to me (the spoofed sender).

   This would not be TOO annoying (any more than any other worm/spam 
is), except that the attachments are usually anywhere from 30-150KB in 
size, and I'm still on a dial-up line (to say nothing of the bandwidth 
these bounces use as they travel through the Internet).

   Furthermore, the attachments (even if they were legitimate 
attachments) are useless, since they are bounced by the daemon in raw 
hex format, and thus are not interpretable to most email readers.  If a 
legitimate attachment got bounced, the original sender should logically 
have a copy of it still around.  So why is the raw hex even left in the 
bounce message?  To copy it there seems to me to server no purpose, 
other than to provide more incentive for people like me to get DSL 
(maybe a good thing, but that's another matter entirely), and clog the 
internet trunks (which is already happening, and may or may not matter, 
especially after the fiber glut of the late '90s)

   Does anyone else think this makes sense and that the RFC for mailer 
daemons should therefore be changed (or a new RFC proposed, or however 
the process goes, I'm new to this, as you can tell..)

Thank you,
Jim Witte
[email-masked]
Indiana University Cog Sci/Comp Sci

_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: Yakov Shafranovich <[email-masked]>
Date: Mon, 02 Feb 2004 09:19:01 -0500

[Subject changed as per posting guidelines,  (was Re: [Asrg] Getting 
SMTP "Bounce email" part of RFC changed... forget binary attachments). Mod.]

Jim Witte wrote:
> 
>   This would not be TOO annoying (any more than any other worm/spam is), 
> except that the attachments are usually anywhere from 30-150KB in size, 
> and I'm still on a dial-up line (to say nothing of the bandwidth these 
> bounces use as they travel through the Internet).
> 

Many email clients have an option to download the first XXX Kbs of a 
message, before downloading the rest.

> the process goes, I'm new to this, as you can tell..)

The SMTP protocol does not distinguish between attachements and 
non-attachements, since it usually does not parse the body of the message.

Yakov
-------
Yakov Shafranovich / asrg <at> shaftek.org
SolidMatrix Technologies, Inc. / research <at> solidmatrix.com
"Some lies are easier to believe than the truth" (Dune)
-------

---

From: "Peter J. Holzer" <[email-masked]>
Date: Mon, 2 Feb 2004 15:30:44 +0100

[Changing the subject - btw, is there a list of the recommended subject
prefixes somewhere? For those of us who post rather infrequently to this
list it would be handy]

> worm code in raw hex) is bounced back to me (the spoofed sender).

Ignoring the question what "raw hex" might be and the fact that there
are worms which disguise themselves as DSNs, yes, real bounced viruses
and spam are annoying enough.

> the process goes, I'm new to this, as you can tell..)

There is no need to change an RFC, since, AFAICS, no RFC says that the
complete message must be included in the bounce message:

RFC 2821:

   If an SMTP server has accepted the task of relaying the mail and
   later finds that the destination is incorrect or that the mail cannot
   be delivered for some other reason, then it MUST construct an
   "undeliverable mail" notification message and send it to the
   originator of the undeliverable mail (as indicated by the reverse-
   path).  Formats specified for non-delivery reports by other standards
   (see, for example, [24, 25]) SHOULD be used if possible.

RFC 3464:

   (d) If the original message or a portion of the message is to be
       returned to the sender, it appears as the third component of the
       multipart/report.

So RFC 2821 only mandates that some non-delivery notification must be
sent, but does not the information included in or the format. Even the
totally useless bounce I got recently "The message sent from <undisclosed>
to <undisclosed> could not be delivered" is conforming to RFC 2821. It
does however strongly recommend the format in RFC 3464
(multipart/report), which includes *the original message or a portion* of
it as an *optional* part. 

So an MTA conforming to both RFCs could (and probably should) only
return a part (maybe the first text/plain part) of the original message.
However, RFC 3464 also says:

   (e) It must preserve enough information to allow the maintainer of a
       remote MTA to understand (and if possible, reproduce) the
       conditions that caused a delivery failure at that MTA.

and the third part should be of type message/rfc822 (though the RFC
doesn't explicitely say so), so some care must be taken when shortening
messages.

Your "raw hex" bounces almost certainly do not conform to RFC 3464, btw.
	
	hp

-- 
   _  | Peter J. Holzer    | In this vale
|_|_) | Sysadmin WSR       | Of toil and sin
| |   | [email-masked]         | Your head grows bald
__/   | http://www.hjp.at/ | But not your chin.           -- Burma Shave

---

From: "Alan DeKok" <[email-masked]>
Date: Mon, 02 Feb 2004 11:43:29 -0500

> non-attachements, since it usually does not parse the body of the message.

  That is, there is nothing within SMTP to allow a recipient MTA to
be informed as to:

  a) size of the body
  b) hints as to the content of the body

  Right now, the recipient has to accept the entire message to obtain
this information.

  If SMTP required the originator to publish the size and number of
messages they were trying to deliver, then that would provide
significant information to the recipient, which would allow him to
make better decisions about the messages.

  e.g. I don't know who you are, and you want to send me a 1M message?
       sorry, you get 10 kbps.

  e.g. You said you wanted to send me a 1k message, and you've just
       hit the 10k mark.  I think I'll hang up now, and blacklist you.

  These sort of protocol additions have *zero* impact on trusted and
honest users of SMTP.  They can have large impact on the ability to
deal with malicious unknown senders.

  But... SMTP is perfect as-is.  Why would we want to change it?

  Alan DeKok.

---

From: "Peter J. Holzer" <[email-masked]>
Date: Mon, 2 Feb 2004 19:26:39 +0100

>   That is, there is nothing within SMTP to allow a recipient MTA to

There is the SMTP SIZE extension (RFC 1870), which is in widespread use.

>   b) hints as to the content of the body

Right. That would be nice.

>        hit the 10k mark.  I think I'll hang up now, and blacklist you.

The size extension (like all extensions) is optional. However, similar
policies could currently be implemented: 

You can throttle the speed to 10 kbps on all unknown senders which don't
support the size extension (this won't hurt unknown senders which send
small messages much).

You can impose a small message size limit on unknown senders and either 
hangup and blacklist them or reject the message after receiving it (and
blacklist them).

> honest users of SMTP.

If the size extension is made mandatory for clients, many MTAs will have
to be upgraded or replaced. This is not zero impact. We already have a
voluntary mechanism for the client to disclose this information.

	hp

---

From: "Alan DeKok" <[email-masked]>
Date: Mon, 02 Feb 2004 13:50:55 -0500

> There is the SMTP SIZE extension (RFC 1870), which is in widespread use.

  Arg.  That's what I get for not reading ~3,5k RFC's.

> to be upgraded or replaced. This is not zero impact.

  It's a one-time cost to upgrade their systems.  They may get this
feature for free, as part of their normal upgrade process.  We already
know that *any* changes to SMTP use, implementation, or deployment
already have costs.  So I'm less concerned about the deployment costs,
as long as they're small.

  Unlike other solutions which require ongoing maintenance and user
interaction (e.g. DK user key distribution), requiring publication of
sender policy is a process which is transparent to all participants
above the MTA level.  And even the MTA administrators don't have to do
more than set policy, and then maintain it.

> We already have a voluntary mechanism for the client to disclose
> this information.

  Which is currently unhelpful... because many "trusted" systems don't
use it.  So the information it provides is unreliable, in that it
doesn't help distinguish bad systems from good ones.  

  As a general principle, most systems could handle spam if they
published a policy for accepting spam (# of messages, size, arrival
rate, bandwidth, etc.), and then the spammers *followed* that policy.
Right not, there's no way for the recipient to publish their policy,
and there are few ways for the originator to publish what they intend
to send.

  There's the following draft which is interesting:

http://www.potaroo.net/ietf/ids/draft-shveidel-mediasize-04.txt

  It's part of the "lemonade" group, and would appear to be relevant.
But the authors intention for it is to allow MTA's to better handle
multi-media mail, not to deal with spam.

  Syntactical objections to it are at:

https://www1.ietf.org/mail-archive/working-groups/lemonade/current/msg00285.html

  Alan DeKok.

---

From: "Peter J. Holzer" <[email-masked]>
Date: Mon, 2 Feb 2004 22:14:27 +0100

>   Arg.  That's what I get for not reading ~3,5k RFC's.

:-)

> as long as they're small.

No disagreement here. I was only objecting to "zero impact". There will
be costs, not only in money and labour to upgrade the software, but also
in rejected legitimate mails, when some MTAs start to reject messages
without a size estimate while others still haven't implemented it. 

I cannot estimate how large that would be: SIZE is already widely, but
not universally deployed.

> rate, bandwidth, etc.), and then the spammers *followed* that policy.

I don't think that's very realistic. Firstly, people want zero spam, and
unlimited "interesting" mails. My users complain if they get a single 1k
spam message per day and they complain if they don't get those 67 40MB
powerpoint presentations that someone sent them in an hour. There just
isn't a correlation between # messages, arrival rate, bandwidth, etc.
and "spam". Secondly, spammers don't admit they send spam, and a single
spammer often masquerades as many senders.

> http://www.potaroo.net/ietf/ids/draft-shveidel-mediasize-04.txt

Indeed. That may make some impact on worms, etc. (However, MTAs MUST
check the real content-types to make it effective: An executable
masquerading as a 4 second audio-File would otherwise slip through
almost everywhere). It won't have much impact on UCE, which is almost
exclusively text (plain or html).

	hp

---

From: Yakov Shafranovich <[email-masked]>
Date: Mon, 02 Feb 2004 16:23:00 -0500

> and there are few ways for the originator to publish what they intend

There has been some talk about a generic policy expression mechanisms 
for MTAs. The question is how do you enforce it?

Yakov
-------

---

From: John Levine <[email-masked]>
Date: 3 Feb 2004 02:25:54 -0000

>rate, bandwidth, etc.), and then the spammers *followed* that policy.

Given that spammers have made it pretty clear that they will stoop to
anything to try to shove their junk into our MTAs, I don't see how
publishing a policy could be of any use unless it was soemthing like
my old NO UCE banner with lawsuits to back it up.  Anything more
complex than a single bit for spam/no spam is overimplemented.

Regards,
John Levine, [email-masked], Primary Perpetrator of "The Internet for Dummies",
Information Superhighwayman wanna-be, http://iecc.com/johnl, Sewer Commissioner
"More Wiener schnitzel, please", said Tom, revealingly.

---

From: "Alan DeKok" <[email-masked]>
Date: Tue, 03 Feb 2004 10:43:48 -0500

> complex than a single bit for spam/no spam is overimplemented.

  This comes back to the question of why doors have locks, when the
locks don't prevent criminals from entering.

  Publishing a policy as to number, size, bandwidth, etc. for incoming
mail allows a recipient to better interact with any cooperating
originator to better manage network resources.

  e.g. Rather than bouncing mail when the disk is full, tell the
       originator "try again in a day"

  More information means better decisions.  Much of this kind of
information is currently *implicit* in SMTP.  It means that in order
to cooperate with the recipient, the originator must jump through all
sorts of hoops to try to discover the recipients policy.  That work is
unnecessary, and is wasteful of resources.

  So how does this apply to spammers?  As you said, they are
non-cooperating originators.  Once the recipient publishes a policy,
and an originator doesn't cooperate, then the recipient has reasonable
grounds for changing the way he deals with that originator.  Getting
such information in the mail envelope stage is *much* preferable to
getting it post-facto, after applying a content filter which may or
may not work.

  In other words, the spammers are non-cooperative in the SMTP
conversation, completely independent of the message body.  That fact
alone means that SMTP envelope methods to address spam will ALWAYS be
more effective than content-based filtering.

  And we are seeing this now.  Many spammers are sending messages
like:

---
  Here's the URL you requested:

  http....
--

  There is so little content to that message that in order to prevent
similar non-spam messages from being caught in a filter, the filter
must err on the side of false negatives... and let spam through.  This
solution is unacceptable in many ways.

  Now you might say that recipients are already publishing such
policies.  But they're not, really.  They may be *enforcing* such
policies, but there is no standard way to publish them.  This lack of
a standard also means that it's more difficult for the local
administrator to decide what the policies are.  The decisions usually
end up being:

  - MTA software says turn on feature FOO to gith spam
    Ok... we turn it on.

  Question: What's the local site policy?
  Answer: The administrator doesn't know.

  So having a method to publish those policies means that the
administrators will create them, rather than just using magic stuff
that other people told them works.  Magic wands are nice for wizards,
but they're inappropriate in a technical environment.

  Alan DeKok.