Subject: Re: [Asrg] [1] Why SPAM is worse in SMTP than in other protocols

From: "[email-masked]" <[email-masked]>
Date: Thu, 18 Dec 2003 08:20:40 +1100

In response to Dave Crocker's invitation to examine human communication in the context of understanding spam, Yakov wrote

"In human communications we have a sender, a receiver, the message and a 
messaging system. The sender creates the message, hands it off to the 
messaging system, and the system delivers it to the receiver.If the delivery is unsuccessful, depending on the system used, the 
sender is notified of the failure. Different messaging systems can also 
deliver the message to multiple receivers, and combine messages from 
multiple senders into one message before being delivered to receivers.
Examples of messaging systems include: direct speech (face to face), 
telephone, fax, television, radio, email, instant messaging, postal 
mail, telegraph, etc. All of these have potential for abuse which varies 
from system to system."

Let's add another distinction which might be useful in examining this (traditionally the difference between broadcast or one-way media and communicative or two-way media). This starts then to get to intention, so following on from Yakov's comments:

Sometimes the sender intends to engage in a dialogue with the recipients, or request information from them. In such cases the sender has an intent to receive a reply. In other cases, the sender merely wishes to disseminate information. In this case a reply may be totally unwanted - the intention is for the message to initiate some action or increased knowledge on the part of the recipient. An advertisement in print or broadcast media is a case in point here. So is the bosses "Message to all staff".  However two-way communications media such as fax, phone and email can also be used with varying degrees of effect to broadcast messages (junk fax, telemarketing, spam) etc.

Hope that adds to our understanding.

Ian Peter

_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: "Alan DeKok" <[email-masked]>
Date: Fri, 19 Dec 2003 15:17:51 -0500

"[email-masked]" <[email-masked]> wrote:
> Sometimes the sender intends to engage in a dialogue with the
> recipients, or request information from them. In such cases the
> sender has an intent to receive a reply. In other cases, the sender
> merely wishes to disseminate information.

  Which is (IMHO) the vast majority of spam.  Very little spam is
intended to establish an ongoing SMTP conversation.  The vast majority
is intended to distribute information through SMTP, about URL's, 1-800
numbers, or other non-SMTP response mechanisms.

  That, alone, says that the vast majority of spam is abusive of SMTP,
as SMTP is not a one-way message delivery protocol.

  If we wanted to have a one-way message delivery protocol, it would
look very different from SMTP.  Therefore, changing SMTP to make it
less useful for high-volume one-way "messaging" is probably not a bad
thing.

  Alan DeKok.

---

From: "Za'mbori, Zolta'n" <[email-masked]>
Date: Sat, 20 Dec 2003 12:44:15 +0100

> as SMTP is not a one-way message delivery protocol.

IMHO SMTP is a "move the message closer to the recipient mailbox" 
protocol. On the SMTP level there are no such thing like one-time 
one-way or two-way. (MTA is comparable to a router. And non-firewall 
routers does not know about sessions.)

And something about the subject:

SMTP compared to other protocols is more attackable because there are no 
rules assigned to email addresses while there are rules assigned to 
ports. (For example nobody can use a DNS service as an NTP service.)

z2

---

From: "Alan DeKok" <[email-masked]>
Date: Sat, 20 Dec 2003 09:53:35 -0500

> routers does not know about sessions.)

  Of course.  But the *intent* behind SMTP was to enable two-way human
communication.  If the intent was to publish information only in one
direction, then the protocol would look very different.

  e.g. UDP "notification" packets, containing URL's where the
information may be found.

  The entire complexity of SMTP is there to support two-way
communication.

> ports. (For example nobody can use a DNS service as an NTP service.)

  Ports are not protocols.  Someone can easily run DNS on the port
assigned to NTP.  This is a big problem for an IDS.  If TCP had a
"protocol" field, in addition to "src/dst port" fields, then a number
of network problems, and application insecurities could have been
avoided.

  Alan DeKok.

---

From: Markus Stumpf <[email-masked]>
Date: Sun, 21 Dec 2003 01:53:49 +0100

>   Of course.  But the *intent* behind SMTP was to enable two-way human

Says who?
Nearly all protocols on the Internet I know of have the itent to enable
two-way human communication. Even router protocols like BGP do.
HTTP, POP3, DNS, LDAP, FTP they all do, just like SMTP does and SMTP does
it in no way more special than any of these, except for protocols like IRC
or VoIP that are specially designed for direct two-way human interaction.

The intent with SMTP was to deliver an "electronic message" from one point
to another. As WAN links were crowded and unstable the design clearly allowed
for relaying and "backup MX" to route messages if the target host was
not directly accessible. Another name for this is a store-and-forward
system.

> information may be found.

At the time SMTP was designed URLs didn't even exist. UDP was really
unreliable and lines were as flaky as hell.

> > ports. (For example nobody can use a DNS service as an NTP service.)

Surely I can use a DNS service as an NTP service.
I could even use DNS for SMTP. All necessary information can be coded
in TXT records. All you need is some protocol agreement between sender
and receiver how to transfer the SMTP protocol via DNS.

I can use SMTP to transfer all kinds of data like executable
programs, XML files, movies, sound files, ... streaming protocols could
be a bit harder to handle but not impossible.
There are services like ftp-mailer that allow FTP Archive access via
SMTP.
Just like there are services like HTTP that can be used to transport
(encoded) SMTP protocol or a lot of other protocols.

> of network problems, and application insecurities could have been

No. Nothing stops me from abusing the HTTP Protocol like in
    POST http://some.mail.server:25/			for SMTP
    POST http://some.telnet.server:23/			for telnet
    POST http://some.www.server:80/cgi-bin/telnet.emu	for telnet
As SMTP, telnet, HTTP operate at another protocol level they don't know
(and need not know) about any fields in TCP/UDP or IP packets.

The idea behind protocol stacks should show you that ports or "protocol
fields" don't say *anything* about the data contained, at best they
give information about the encoding and protocol used to transport the
data and thus it is useless. You cannot pevent users from accessing the
data of a FTP archive by blocking the FTP ports. If the data is also
available via SMTP or HTTP you have gained nothing.

    \Maex

-- 
SpaceNet AG            | Joseph-Dollinger-Bogen 14 | Fon: +49 (89) 32356-0
Research & Development |       D-80807 Muenchen    | Fax: +49 (89) 32356-299
"The security, stability and reliability of a computer system is reciprocally
 proportional to the amount of vacuity between the ears of the admin"

---

From: "Za'mbori, Zolta'n" <[email-masked]>
Date: Sun, 21 Dec 2003 13:44:46 +0100

Hi Alan,

>>ports. (For example nobody can use a DNS service as an NTP service.)

Sorry about the uninformative text.

I mean making some email handling rules predictable will do the 
spam-volume to be smaller.

For example it is predictable by everybody based on the port address of 
a running service that the service software will read a specific 
protocol data unit or not; will accept the content for processing or 
not. It is predictable how transport level and content level information 
will be filtered out or processed.

Back to SMTP. It is unpredictable by spammers that a mailbox address is 
send-only or not, handshake is neccessary or not, 'sequence numbers' are 
necessary or not, session-IDs or secret tokens are necessary or not, the 
address accept emails only from predefined senders or not, a content 
filter will filter out emails based on the content or not, etc.

The result of this unpredictability (lack of 'rules' assigned to email 
addresses): spammers will send spam to any email address what they have 
because they dont have informations how to do self-filtering. (As 
opposed to TCP where almost nobody send out random packets to random 
host:port addresses, and IP-spoofing is almost impossible.)

For example spammers will not stop using Yahoo addresses as sender 
addresses because spammers will not know based on the receiver email 
address that the receiver filter out emails with the help of YDK or not.

z2

---

From: "Alan DeKok" <[email-masked]>
Date: Sun, 21 Dec 2003 10:54:55 -0500

> >   Of course.  But the *intent* behind SMTP was to enable two-way human

  Says the protocol.

> two-way human communication. Even router protocols like BGP do.

  Now you're being cute.  Isn't that nice.

> it in no way more special than any of these,

  All those protocols are just like SMTP, so that's why we have so
many protocols, right?  They all do the same thing!

> unreliable and lines were as flaky as hell.

  And nothing on the Net has changed since then.  So if, as I said, we
WERE to design such a notification protocol, we would not (of course)
use anything we've learned in the past two decades.

> > If TCP had a
...
> (and need not know) about any fields in TCP/UDP or IP packets.

  You've taken my statement exactly backwards.  Congratulations!

  The point ( to re-iterate) was that TCP would contain information
about the protocol it was carrying.  I said nothing about SMTP knowing
about fields in TCP or IP packets.  I have no clue how you totally
inverted the meaning of my statements.

  When you have nested protocols, each protocol SHOULD contain
information about any sub-protocol it contains.  The failure to do so
leads to huge problems when examining network traffic.

  I should know.  My day job is writing software for multi-protocol,
multi-field, gigabit packet classification, that other people put into
network devices.  Issues like TCP ports being loosely coupled to the
protocol in the TCP data are a *huge* problem for anyone wanting to
examine network traffic.

> data and thus it is useless.

  When your protocol stack gets IP packets with proto TCP, and the IP
data contains something other than TCP, your stack still does
something useful with it?  That protocol field has no bearing on how
you treat the encapsulated data?

  Nonsense.  Have you ever done any work with networks, or protocols?

  Alan DeKok.

---

From: "Alan DeKok" <[email-masked]>
Date: Sun, 21 Dec 2003 11:10:55 -0500

"Za'mbori, Zolta'n" <[email-masked]> wrote:
> not.

  No.  I can run HTTP servers on any port.  When used in conjunction
with something like RPC, you have *no* way to use TCP ports to
classify a packet on your network as RPC or HTTP.  The TCP ports don't
tell you.  The *only* thing you can do is to examine the application
data, compare it to RPC and HTTP, to see which one it matches.

  The problem is that the intersection of RPC and HTTP isn't empty.
So there are some packets which you can't classify just by looking at
the packet.  You've got to go poke the port yourself, to see what
application server is running.

> because they dont have informations how to do self-filtering.

  No.  Spammers already have sufficient information to do
self-filtering.  There are domains which didn't have an MX for years,
and when one was put into DNS, it was immediately spammed.  There are
systems which reject traffic to all SMTP destinations, and spammers
still try those destinations years later.

  Spammers have plenty of negative feedback, as information.  But
simply NACK'ing a message is insufficient.  What is needed is negative
feedback as *punishment*.  e.g.  Greylisting is network-based negative
feedback that works by slowing down the rate at which spam is sent.

  Recipient MTA's often do *nothing* when they're attacked by
spammers.  This creates a de-facto positive feedback system, giving
the spammers every incentive to increase their rates.

  Alan DeKok.

---

From: Yakov Shafranovich <[email-masked]>
Date: Sun, 21 Dec 2003 10:32:19 -0500

Za'mbori, Zolta'n wrote:

> Hi Alan,
> 
>>> SMTP compared to other protocols is more attackable because there are 
>>> no rules assigned to email addresses while there are rules assigned 
>>> to ports. (For example nobody can use a DNS service as an NTP service.)
> 
> 

I am not sure how would help us, perhaps you can elaborate. Are you 
suggesting that we should let spammers know about how their spam is 
being filtered? Wouldn't that let them adjust their spam messages so 
they can go through better?

Yakov
-------
Yakov Shafranovich / asrg <at> shaftek.org
SolidMatrix Technologies, Inc. / research <at> solidmatrix.com
  "Why are both drug addicts and computer aficionados both called 
users?" (Clifford Stoll)
-------

---

From: Markus Stumpf <[email-masked]>
Date: Sun, 21 Dec 2003 21:59:38 +0100

>   Says the protocol.

Blah.

> many protocols, right?  They all do the same thing!

Yes they are all just like SMTP with regard to
*> But the *intent* behind SMTP was to enable two-way human
*> communication.
Read what I write.

>   And nothing on the Net has changed since then.

Changes are irrelevant. You said it was the design intent with SMTP
otherwise they would have used URLs. This is nonsense as URLs didn't
exist when they designed SMTP.

> So if, as I said, we

No, you didn't talk about future protocols or future design, you talk
about the intent of the design of SMTP.

> you treat the encapsulated data?

What happens depends on the protocol stack that the sender and the
receiver use. But whether this will be "real" TCP data or something only
decodable by sender/receiver is absolutely irrelevant to the IP stack.
Totally damages TCP data will be transported on IP level from sender to
receiver without and problems and this is good as it is. So setting
a protocol identifier in the underlying protocol may be a nice idea,
but the underlying protocol has to ignore it. It's not its business.
If this is a problem for IDS than this is the problem of the IDS, as it
tries to interpret data that may not be encoded for it to decode. This
is a problem that has always been present for listeners that listen to
data they're not supposed to receive.

>   Nonsense.  Have you ever done any work with networks, or protocols?

You're always so easy with your "nonsense". And it's always easy for you
to try to discredit others by asking them "are you on crack or what?".
Maybe you should simply think before writing all those bullshit.
You just can't stand it if others have a different opinion that doesn't
match your - of course - unfailing point of (limited) view.

When e.g. the ethernet stack gets a frame it doesn't care what the contents
is. The ethernet stack takes the packet, drops the envelope and hands
it off to the next stack. This looks at the content and dispatches it
e.g. to the IP stack handler. This drops the IP envelope and hands it
off to the next handler which decides whether to e.g. dispatch it to
TCP/UDP/ICMP. It is an explicit design goal of protocol stacks that the
higher level protocol doesn't have to know anything about the lower
level protocols to peer communicate with its partner and the lower
level protocol doesn't have to know anything about the
contents/data/encoding of the higher level protocol to transport it.

If the sender and receiver agree on using an UDP ident for TCP stack and
an TCP ident for UDP stack they will communicate perfectly, as will the
underlying and overlying protocols.
So if sender and receiver agree to use the http ports for smtp is the
same thing as if they would agree to use the HTTP ident for SMTP and put
it in the underlying protocol.
Your IDS would choke, just like it would if they'd use port 80 for SMTP.

	\Maex

---

From: "Za'mbori, Zolta'n" <[email-masked]>
Date: Sun, 21 Dec 2003 22:15:41 +0100

Yakov Shafranovich wrote:
> Za'mbori, Zolta'n wrote:
>>
>> If a smart spammer (software) see that the actual spam is filterable 
>> by the domain the software will skip the whole domain. And not sended 
>> spam will make the spam volume smaller.
>>
> 
> Wouldn't that cause the domains not skipped to get more spam?

You are true. The volume will not change. Just the direction.

Sorry about wasting the time.

z2

---

From: Yakov Shafranovich <[email-masked]>
Date: Sun, 21 Dec 2003 18:10:17 -0500

...
> match your - of course - unfailing point of (limited) view.

Lets cool it a bit.

...

Lets step back for a second. We are comparing the SMTP protocol with 
other protocols in order to determine which problems the SMTP protocol 
has similar to other protocols, and see which solutions applied in other 
protocols can help us here.

The point of contention here seems to be whether SMTP is intended to be 
an end-to-end protocol (like a network layer protocol of the OSI model), 
or a hop-to-hop protocol (like a link layer protocol). Obviously SMTP is 
in reality is running above these layers on application layer, but we'll 
ignore that for a second.

Now lets see section 2.1 of RFC 2821:

"  In other words, message transfer can occur in a single connection
    between the original SMTP-sender and the final SMTP-recipient, or can
    occur in a series of hops through intermediary systems.  In either
    case, a formal handoff of responsibility for the message occurs: the
    protocol requires that a server accept responsibility for either
    delivering a message or properly reporting the failure to do so.
"

It seems from here that the SMTP protocol can either operate as an 
end-to-end protocol or a hop-by-hop protocol. The RFC states that a 
server must accept responsibility for the message if it is not the final 
destination, which implies that each individual hop is independent of 
each other. On the other hand, in many cases only one SMTP transaction 
takes place between the originating ISP's MTA and the receiver ISP's MTA.

So what comes out from this, is that SMTP operates on a hop-by-hop 
basis, which in many cases is end-to-end since there is only a single 
hop to be done.

Now the question is, what can we, as a RG, learn from this? How is this 
behavior helpful to stopping spam? What comparisons can we draw from 
this behavior to other protocols?

Yakov
-------
Yakov Shafranovich / asrg <at> shaftek.org
SolidMatrix Technologies, Inc. / research <at> solidmatrix.com
"Some lies are easier to believe than the truth" (Dune)
-------

---

From: "Jon Kyme" <[email-masked]>
Date: Mon, 22 Dec 2003 10:17:46 +0000

> this behavior to other protocols?

We learn that the system involves at least 1 intermediate between MUAs and
MUAr, works asynchonously, in a store-and-forward kinda way. These
intermediate agents act (for the most part) to support the intention of the
message originator.
Intermediates try to provide resources to satisfy the originator, while
being paid-for by the recipient (often).

Other systems may consider the *message* (the data made available) as being
a significant resource - we consider the significant resources as being:
1. the *attention* of the recipient
2. transport and storage requirements

Following this reasoning, we can consider the MTS as a system for giving
the sender access to (human) recipient *attention* - For making recipient
resources available to the sender. The messages it transports are just part
of the mechanism for brokering attention.

SMTP is, from this viewpoint, much more like a form of RPC than anything
else I can think of - but asynchronous, which will allow the sender to
overwhelm the recipient. Clearly, what's weak is the access control (and
flow control).
Recipient resources should be available only subject to recipient policy.

That's as far as we need go with this. We can see that we need mechanisms
for expressing and enforcing recipient policy in our MTS.

--

---

From: "Za'mbori, Zolta'n" <[email-masked]>
Date: Mon, 22 Dec 2003 14:01:55 +0100

> for expressing and enforcing recipient policy in our MTS.

Is it true that a recipient policy is equivalent with a description of 
forward licencing of recipient resources?

z2

---

From: "Jon Kyme" <[email-masked]>
Date: Mon, 22 Dec 2003 15:14:40 +0000

> forward licencing of recipient resources?

That's not a term I'm familiar with. If it means what I guess it means,
that might be a good way of looking at it. In the case of prior explicit
consent I suppose I'm offering a sender license to use my resources. I
guess this is precisely what a consent token does. Without explicit
consent, license is conditional, or reserved.

"license" smells of contract law - IANAL. I dare say a lawyer might have a
view on this.

--

---

From: "Alan DeKok" <[email-masked]>
Date: Mon, 22 Dec 2003 13:31:42 -0500

> protocols can help us here.

  Exactly.  That's the scope of the document I wrote.

> or a hop-to-hop protocol (like a link layer protocol).

  Almost.  We went down a rat-hole of human messaging systems, because
of my contention as to the design intent of SMTP: one to one
communication.  If SMTP was intented to be many to many, with high
latencies, it would look like NNTP.  If it was intented to be many to
many, with low latencies, it would look like IRC.

  That's all I'm going to say on that topic.

> Now the question is, what can we, as a RG, learn from this?

  The LMAP discussion document (based on Hadmut's comments in RMX),
says that the single hop use of SMTP is a large part of the reason why
spam is so wide-spread.  There are 100's of millions of senders, each
sending to only 100's of thousands of recipient MTA's.

> How is this behavior helpful to stopping spam?

  If that imbalance in the network was addressed, spam would become
significantly more manageable.  It wouldn't stop entirely, but having
a blacklist of 100K MTA's is signficantly easier than having
blacklists of millions of IP's.

  PHB's suggestion that MTA's have "recommendations" in DNS would
apply here.  LMAP would say "yes, example.com meant to originate SMTP
from this IP", and a related protocol would say "example.net thinks
I'm an OK MTA, too."  You'd still have to query example.net, so see if
it agrees, but that's fairly simple.

  This process builds a web of trust, where MTA's simple don't bother
to listen to spammers.

> What comparisons can we draw from this behavior to other protocols?

  Many other protocols are only single-hop.  Others are nearly always
multi-hop.  There are fewer which are a combination of the two, like
SMTP.

  If SMTP was one or the other, the spam problem would probably be
easier to solve.  It's the combination of factors that makes SMTP
worse than other protocols.

  Alan DeKok.

---

From: "Alan DeKok" <[email-masked]>
Date: Mon, 22 Dec 2003 13:44:15 -0500

"Jon Kyme" <[email-masked]> wrote:
> In the case of prior explicit consent I suppose I'm offering a
> sender license to use my resources. I guess this is precisely what a
> consent token does. Without explicit consent, license is
> conditional, or reserved.

  Exactly.  In many cases, "license" is a synonym for "permission".

> "license" smells of contract law - IANAL. I dare say a lawyer might have a

  Many contracts involve the granting of license, if certain
conditions are met.  You can have a contract without a license, and a
license without a contract.  The software "click to agree" things are
contacts, disguised as licenses.

 In terms of SMTP, publishing recipient policy may be equivalent to
giving license to people to use the recipients resources within that
policy.  It does make it clear that senders NOT following the policy
are forbidden from using the recipients resources.

  Alan DeKok.

---

From: Yakov Shafranovich <[email-masked]>
Date: Mon, 22 Dec 2003 14:38:09 -0500

> "Jon Kyme" <[email-masked]> wrote:
..
> contracts, disguised as licenses.
> 

If I remember correctly, "click through" licenses are not easily
enforceable without UCITA, which was only passed in two states in the
US. In any case, it is important to keep in mind that licenses and
contracts of this kind operate because of laws that are in place to
enforce them.

> are forbidden from using the recipients resources.

This approach works in conjunction with legal enforcement and proper
laws, which is what the "no solicit" extension by Carl Malamud was
intended to be. This is important to keep in mind.

The hard question would be how to define policy elements that people can
agree on, which has been the sticking point until now. Even the elements
defined in Carl's draft, people had issues with. Some kind of a registry
would probably be needed.

Yakov
-------
Yakov Shafranovich / asrg <at> shaftek.org
SolidMatrix Technologies, Inc. / research <at> solidmatrix.com
"Among all our enemies / The ones to be most feared are often the 
smallest" (Jean de la Fontaine)
-------

---

From: Troy Rollo <[email-masked]>
Date: Tue, 23 Dec 2003 07:16:29 +1100

>   Exactly.  In many cases, "license" is a synonym for "permission".

In law, "license" means "permission". Having a license literally means you 
have been given permission.

Software licenses are commonly also attempting to be contracts. Whether they 
actually become contracts depends on a lot of factors, including how you 
purchased the software, what you have done since receiving it, whether you 
have relied on it  to do something you would not otherwise have been able to 
do, and which state and country you are in.

As for the subject of this thread, it seems to me that the comparison between 
protocols per se is misguided and although I haven't read the whole thread, 
I'm surprised the topic has lasted this long. Spam (not SPAM, the meat 
product) is worse in SMTP because it's the protocol that allows the spammer 
to put the message in the face of the target. Change the protocol without 
changing the very nature of email, and spammers will still find ways to 
exploit it.

Take, for example, the MS network popup messaging protocol (whose name I don't 
recall). It also put the message in the face of the targets. It remained 
unabused for a long time because it's a low profile protocol that spammers 
simply hadn't considered. When they realised it was there, they started 
exploiting it rapidly. As it happened, the protocol wasn't very important, so 
it was either jettisonned entirely or blocked at routers in most places.

-- 
--
Troy Rollo				Chairman, CAUBE.AU
[email-masked]			Executive Director, iCAUCE

---

From: "Alan DeKok" <[email-masked]>
Date: Mon, 22 Dec 2003 15:38:56 -0500

Troy Rollo <[email-masked]> wrote:
> As for the subject of this thread, it seems to me that the
> comparison between protocols per se is misguided and although I
> haven't read the whole thread, I'm surprised the topic has lasted
> this long. Spam (not SPAM, the meat product) is worse in SMTP
> because it's the protocol that allows the spammer to put the message
> in the face of the target.

  HTTP puts messages in front of people, too, but the users have to
request it.  IRC also puts messages in front of people, but spammers
get booted off of the IRC network.  NNTP does it too, but things like
"cancel bots" get rid of a lot of the spam, before it's delivered.
(To say nothing of the Usenet Death Penalty.)

  None of these approaches work in SMTP.

> Take, for example, the MS network popup messaging protocol (whose
> name I don't recall). It also put the message in the face of the
> targets.

  And as you noticed, the abuse stopped because everyone decided they
didn't need the protocol.  We can't do that with SMTP.

  Please read the document.  It tries to explain all this in more
detail than I've included here.

  Alan DeKok.

---

From: "Alan DeKok" <[email-masked]>
Date: Mon, 22 Dec 2003 17:57:10 -0500

> would probably be needed.

  The policies can be opaque tokens in a registry, in which case
spammers can claim they don't know about the registry, and therefore
can't understand the policy.

  Or, the policy can be text, in which case there are international
language issues.

  SMTP is pseudo-English text.  I don't see why any policies published
in SMTP or related protocols can't be pseudo-English text, too.

  Alan DeKok.

---

From: Yakov Shafranovich <[email-masked]>
Date: Mon, 22 Dec 2003 18:58:48 -0500

> in SMTP or related protocols can't be pseudo-English text, too.

I see the problem not in defining the format of policy tokens, but 
rather defitions of policies. For example, if your policy is not to 
accept UBE or UCE, how do you define UBE or UCE? Since defitions are 
very hard to arrive at, as we have seen, this would not be an easy task.

Yakov
-------

---

From: "Eric Dean" <[email-masked]>
Date: Mon, 22 Dec 2003 19:28:15 -0500

Some dumbass spam company that I apparently do business with accidently
forwarded me their entire year-to-date spam history.  I sanitized it to
protect the innocent, such as removing the "Subject", open relay
providers, and client names, but nevertheless found the stats to at
least be of interest to this group.

Enjoy.

---

From: [email-masked] (John R. Levine)
Date: 23 Dec 2003 04:51:39 -0000

> Spam (not SPAM, the meat product) is worse in SMTP because it's the
> protocol that allows the spammer to put the message in the face of
> the target. Change the protocol without changing the very nature of
> email, and spammers will still find ways to exploit it.

It's not just that.  NNTP puts messages in the face of the target just
as much as SMTP does, but the spam problem on usenet, at least in the
big 8 groups, is negligible.  Why?  I'm not entirely sure.  The fact
that every site sees all the traffic makes bulk-based cancelbots
possible, but they're technically not all that hard to circumvent as
hipcrime's occasional forays into nanae have shown.  Does the
possibility of UDPs make usenet nodes behave more responsibly?  Is it
just that there's so many more e-mail users that spammers don't bother
with usenet any more?

I don't use instant message or chat much, but it is my impression that
other than virusbots like fizzer, IRC doesn't have much of a spam
problem.  Why not?  I'd think that it wouldn't be hard to write
spamware that sprinkled messages all over IRC?  Why don't we see much
instant message spam (at least relative to e-mail spam)?  Is it that
the popular IM systems are all closed with a central control point?

Regards,
John Levine, [email-masked], Taughannock Networks, Trumansburg NY
http://www.taugh.com

---

From: "Jon Kyme" <[email-masked]>
Date: Tue, 23 Dec 2003 09:31:58 +0000

>the popular IM systems are all closed with a central control point?

I'd guess it's that IM is (nearly) synchronous and so pretty much
automatically educates users in the art of sender (-ve/+ve) listing. Is it
true that sender forgery isn't do-able in IM ?

Anyway, here's why this thread is probably futile. It's not clear that we
can *quickly* conclude that a channel (protocol) is relatively free of spam
because of any identified characteristic of the *channel*. It may depend on
a characteristic of the users (and the spammers). Or the whole complex.

That said, we already know the key flaws in the current MTS. Weak "trust"
model, lack of "authentication", lack of means for distributing and
enforcing recipient policy. The very flexibility of the MTS, which has made
it so valuable for the transmission of arbitrary content in all of the
possible n-to-m modes, has made it valuable for abusers, as has the wide
deployment / high take-up. This is all well understood.

I'm sure that there are people in the group who could give us a new
protocol which might enable a MTS with fewer of the failings and many of
the strengths of the current system. It may be that this should be
considered a separate project from the task of developing and evaluating
"armour-plating" for the current MTS. Particularly given the very different
timescales that will apply in these two areas.

--

---

From: "Jon Kyme" <[email-masked]>
Date: Tue, 23 Dec 2003 09:37:55 +0000

> Why don't we see much

Nice little article by David McGuire here:
http://www.washingtonpost.com/ac2/wp-dyn/A36039-2003Nov13

--

---

From: "Za'mbori, Zolta'n" <[email-masked]>
Date: Tue, 23 Dec 2003 16:00:03 +0100

> this behavior to other protocols?

IMHO there is an email network built over the IP network. The email 
network is comparable to the IP network.

Routers are hop-by-hop devices on the IP network while MTAs are 
hop-by-hop devices on the email network.

IP numbers and mailbox addresses can not be owned they must licencing. 
IP numbers and mailbox addresses are (like) licence IDs.

IP routers can enforce licencing (can stop IP spoofing) because the 
communication paths related to the forward licensing structure of the IP 
numbers.

MTAs can not enforce licencing (email address forgery).

A host do forward licencing of IP number(s) to its services. A TCP 
service do forward licencing of port access to clients during the 
handshake while UDP does not. The port number and the sequence number 
are (like) licence IDs.

TCP can enforce the licencing because the sequence number is related to 
the sender and the licence number must be attached to each message. 
Additionaly sequence numbers are unpredictable by third parties.

 > How is this behavior helpful to stopping spam?

I dont know about spam-like traffic on the TCP network except port 
scanning, SYN flood and IP spoofing. They are comparable to dictionary 
attack, C/R attack and email address forgery on the email network.

Port scanning can be handled by log analysis, SYN flood attacks can be 
handled by cookies, and IP spoofing can be handled by exchanging 
unpredictable sequence numbers during a handshake procedure.

So organizing communication paths between MTAs closer to the forward 
licencing tree (as in the case on the IP network) and/or forward 
licencing mailbox resources to senders during handshake procedures will 
help enforcing sender licecing on the sender side and/or at the 
receiving side.

IMHO enforcing licences may significantly decrase delivery of (today) spam.

And back to the subject:

Spam is exist in SMTP because the communication paths of the email 
network does not related to an enforcable forward licencing structure 
and because senders can reserving resource from the receiver without 
licence.

z2

---

From: [email-masked] (John R. Levine)
Date: 23 Dec 2003 15:49:53 -0000

>That said, we already know the key flaws in the current MTS. Weak
>"trust" model, lack of "authentication", lack of means for
>distributing and enforcing recipient policy. ...

The real reason there's e-mail spam is that e-mail by its nature the
costs are mostly borne by the recipient.  Everything else flows from
that.  There's no way to fix that without a top to bottom redesign and
maybe not even then.  (By fix I mean move the actual location of the
costs, not layering on an e-postage scheme that's supposed to
artificially compensate recipients and charge senders.)

There aren't any large-scale communication system in which all of the
participants are authenticated and I don't see that any of us know how
to build one.  Paper mail isn't authenticated at all, scribble any
return address you want on an envelope, and they'll deliver it.  The
phone system is probably as close as we'll get, but even there the
authentication is just sufficient to know who to bill for a call, not
to identify the caller, and we all know that tracing the source of a
phone call is not always easy.

All of the scams we see in spam started in other places.  "Make money
fast" started in paper mail -- I once got an actual paper Dave Rhodes
letter.  The 419 scam was done by mail and fax before it moved to
e-mail and in its original guise as the Spanish Prisoner, by quill
pens and sealing wax.  They moved to e-mail because it's cheap, not
because it's unauthenticated.

Even if you could build a gigantic authentication system for e-mail,
it'd only work to the extent that people played by the rules.  Look at
the banking and credit card system, which process a vast number of
daily transactions (but still a couple of orders of magnitude short of
e-mail) in which all of the participants are supposed to be known.
There's constant problems with bogus and bounced checks, invalid and
stolen credit cards, and other fake transactions.  Despite the
enormous scale, transactions still cost about a dime apiece, way too
expensive for e-mail.  Any authentication scheme we build will be
attacked in every way that the bad guys can invent, requiring whole
new layers of rules and sanctions, and I don't see any reason to think
that the problems from such a system would be less bad than the
problems we have now.

I'm not saying that solving the spam problem is hopeless, but I am
saying that lack of authentication isn't the problem, too much mail to
people who don't want it is the problem.  Authentication can help
track a lot of the mail so that non-technical sanctions can be applied
to the senders, but it's only one tool in the toolbox.

Regards,

---

From: "Alan DeKok" <[email-masked]>
Date: Tue, 23 Dec 2003 12:09:44 -0500

[email-masked] (John R. Levine) wrote:
> maybe not even then.

  Greylisting moves some of the cost to the sender, by slowing down
the rate at which messages are sent.

> to build one.

  I agree.  And I'm not sure if we *want* such a system.  Holding
senders accountable is a good idea, but that doesn't mean they need to
be individually authenticated.

> All of the scams we see in spam started in other places.  ...  They
> moved to e-mail because it's cheap, not because it's
> unauthenticated.

  Cheap also means fast turn-around times.  It takes time to set up
scams with pen and paper, and you can't reach many people.  The
Internet is a god-send for scammers: reach millions of people with
near-zero cost.

  Therefore, the obvious ways to attack scammers is to reduce the
number of people they can reach, and to increase their costs.
Authentication won't help here.

  Alan DeKok.

---

From: "Jon Kyme" <[email-masked]>
Date: Tue, 23 Dec 2003 17:58:06 +0000

>artificially compensate recipients and charge senders.)

is this ASRG policy now ?
:-)

--

---

From: [email-masked] (John R. Levine)
Date: 23 Dec 2003 18:27:40 -0000

>>that.  There's no way to fix that without a top to bottom redesign
>>and maybe not even then.  (By fix I mean move the actual location of
>>the costs, not layering on an e-postage scheme that's supposed to

>is this ASRG policy now ?

Sure, I suppose so.  We're working on a new improved charter which
isn't quite done, but I can promise that it doesn't contain the word
"consent".

For my take on e-postage, see my paper at http://www.taugh.com.  It
explains why even if we could technically implement an e-postage
system, which we can't, it wouldn't work anyway.

Regards,

---

From: Philip Miller <[email-masked]>
Date: Tue, 23 Dec 2003 13:46:45 -0500

> isn't quite done, but I can promise that it doesn't contain the word

What is this about, and why has none of it been mentioned on the list 
before? I'd imagine that a complete rewrite of the group's charter is 
probably something that should be a public process.

Philip Miller

---

From: Yakov Shafranovich <[email-masked]>
Date: Tue, 23 Dec 2003 15:22:59 -0500

[Subject changed. Mod.]

> true that sender forgery isn't do-able in IM ?

With the current "big" centralized systems such as MSN, Yahoo, AIM and
ICQ that is true. However, with XMPP (Jabber), IM becomes distributed
among servers, just like email, with an ability for a malicious IM
server or an "IM relay" to be introduced into the system which can forge
IMs and inject spam IMs into the network.

If you take a look at the current XMPP draft, it explictly talks about
methods to detect malicious and forged servers. It recommends using SASL
and TLS for inter-server communications, and a DNS method kind of like
LMAP. There is also a "server callback" method, kind of like C/R for
servers, which was used before and is present for backwards
compatability. The draft is here:

http://www.ietf.org/internet-drafts/draft-ietf-xmpp-core-20.txt

It also includes the following interesting snippet, which basically says
that any IM server can deny access to any other IM server:

"  Because service provisioning is a matter of policy, it is OPTIONAL
    for any given domain to communicate with other domains, and
    server-to-server communications MAY be disabled by the administrator
    of any given deployment.  If a particular domain enables inter-domain
    communications, it SHOULD enable high security.
"

Yakov

P.S. As an interesting historical footnote, the original SMTP document, 
RFC 821, defines a bunch of other commands in addition to MAIL FROM, 
such as SEND FROM, SOML FROM, and SAML, which send messages directly to 
the user's terminal, whichs sounds awfully like IM. Just imagine if SMTP 
was used for IM...

-------
Yakov Shafranovich / asrg <at> shaftek.org
SolidMatrix Technologies, Inc. / research <at> solidmatrix.com
"Be liberal in what you accept, and conservative in what you send" (Jon 
Postel)
-------

---

From: Yakov Shafranovich <[email-masked]>
Date: Tue, 23 Dec 2003 16:43:40 -0500

[Subject changed.]

> artificially compensate recipients and charge senders.)

An important point to be kept in mind that many of the costs of e-mail 
are not tangible money costs like postage or e-postage. For example, a 
lot of email spam is being sent via other people's systems such as open
relays and hacked computers. That means that those resources which have
a certain monetary value are being exploited by spammers for free. Same
goes for forged domains - if they use their own domains, domains costs
money and increase their costs slightly. Even junk mail in the postal
system costs more money aside from postage - it costs money to print,
seal and deliver the envelopes at the post office.

An additional point here is that a lot of the costs are computer 
resources - it costs money to run and buy filters (IETF's mail server 
just crashed recently because of additional processing power needed for 
SpamAssasin). It also costs money to administer anti-spam systems, and 
configure them, it costs money to use some RBLs and outsourcing the 
stuff to BrightMail and Postini costs money. On the other hand, the 
sender does not need to do anything at all - just connect on port 25 and 
  pump the junk in. That's why proposals shifting some of the burdens on 
the sender such as C/R, Yahoo's DK, LMAP, callbacks, GreyListing, 
digital certificates, etc. are good, but each come with their own 
disadvantages and some increase costs for both the sender AND receiver 
(C/R for ex.).

Yakov
-------
Yakov Shafranovich / asrg <at> shaftek.org
SolidMatrix Technologies, Inc. / research <at> solidmatrix.com
"All that is gold does not glitter" (LOTR)
-------

---

From: David Maxwell <[email-masked]>
Date: Wed, 24 Dec 2003 14:01:07 -0500

> artificially compensate recipients and charge senders.)

You're making the issue sound black and white (or black and black, with
the "maybe not even then").

There are certainly technical solutions that would significantly improve
some user's experience of Internet email with regard to receiving less
spam.

> to the senders, but it's only one tool in the toolbox.

Authentication can also allow one to apply technical sanctions. "Lack of
authentication" may not be the problem, but that doesn't mean that
"authentication" can't be part of the solution.

-- 
David Maxwell, [email-masked][email-masked] --> Unless you have a solution
when you tell them things like that, most people collapse into a gibbering, 
unthinking mass.  This is the same reason why you probably don't tell your 
boss about everything you read on BugTraq!    - Signal 11

---

From: David Maxwell <[email-masked]>
Date: Wed, 24 Dec 2003 14:03:05 -0500

> isn't quite done, but I can promise that it doesn't contain the word

Does that mean that the equivalent intent will be shown, without using
the "C" word - or that the intent of the research group will change?

I found Consent to be a very useful concept in the analysis of spam.
What are the reasons for it dissapearing from the charter?

-- 
David Maxwell, [email-masked][email-masked] --> The only difference I see
between voodoo and marketing research is that voodoo sometimes works! 
						- Leonard Stern