Subject: RE: Point of Order: Incomplete, flawed response to MARID WG Chart er

From: "Daryl Odnert" <[email-masked]>
Date: Thu, 19 Aug 2004 09:12:18 -0700

I agree with Roy Badami's response.  "Bad" email is already rejected
in large quantities today by many MTAs using anti-spam, anti-virus,
and anti-DoS-attack techniques.  Sender ID does not attempt to address
this problem, nor does it make this problem significantly worse.

Daryl Odnert
Tumbleweed Communications
Redwood City, California

---

From: "John Glube" <[email-masked]>
Date: Thu, 19 Aug 2004 15:32:45 -0400

Daryl,

* Sender-ID does not call for receiving MTAs to do a
malformed SMTP Mail From check.

However, we know (see the US Bank spoof scenario as an
example) doing such a check would prevent spammers from
sending directly from IP address to the receivers MX.

* Sender-ID does not call for receiving MTAs to do an SMTP
Mail From check in the absence of PRA.

However, we know as has been pointed out by doing so, this
aids in the prevention of one form of 'backscatter' as
pointed out by Chris and helps to minimize the risk of
false positives, as has been pointed out by Meng.

I must be honest and say I am totally confused. 

For almost 3 weeks now, some of us have been raising the
issue of having Sender-ID include an SMTP Mail From check
based on the SPF protocol to pick up and deal with these
obvious exploits. 

Yet the answer has been negative. 

That is until most recently. 

Harry has now responded by suggesting a BCP as opposed to
amending the protocol drafts.

I am most grateful for Harry's response. 

The proposal only deals with one part of the issue, the
malformed SMTP Mail From at the data stage. It does not
call for doing a spoof check using SMTP MAIL FROM in the
absence of PRA at the data stage, so reducing the risk of
false positives as pointed out by Meng and avoiding one
element of back scatter as pointed out by Chris.

However, I am sure the proposal can easily include both
without causing any disruption or delay, while giving
favourable consideration to the solution to the version
string issue as proposed by Scott.

At the same time Andy has now indicated it is appropriate
for individuals to come forward with other proposals, which
means a set of drafts for SPF can be put on the table and
dealt with once we finish dealing with Sender-ID.

Yes, I appreciate Chris's stance may have offended some and
caused others to get upset. However, others have been
equally offended and upset by the refusal to budge on an
issue which to many was clear.

But now that we all understand what the engineering
concerns are, how about we get on with the show and work on
a BCP for Sender-ID, while letting others come forward with
SPF protocol drafts to complete the needed work to support
the BCP?

John

cc. Chris, Harry

John Glube
Toronto, Canada

The FTC Calls For Sender Authentication
http://www.learnsteps4profit.com/dne.html

---
Outgoing mail is certified Virus Free.
Checked by AVG anti-virus system (http://www.grisoft.com).
Version: 6.0.737 / Virus Database: 491 - Release Date: 11/08/2004

---

From: Roy Badami <[email-masked]>
Date: Fri, 20 Aug 2004 00:56:41 +0100

>>>>> "John" == John Glube <[email-masked]> writes:

    John> * Sender-ID does not call for receiving MTAs to do an SMTP
    John> Mail From check in the absence of PRA.

    John> However, we know as has been pointed out by doing so, this
    John> aids in the prevention of one form of 'backscatter' as
    John> pointed out by Chris and helps to minimize the risk of false
    John> positives, as has been pointed out by Meng.

I'm still very confused.

I don't see how giving an SMTP 5xx to mail in response to a forged
MAIL FROM helps prevent backscatter.  Giving a 5xx response to a
forged MAIL FROM pretty much guarantees backscatter (the upstream MTA
will bounce the message to the forged MAIL FROM).

The only way to prevent backscatter would be to accept and silently
discard the message, and that would involve throwing away the
long-established principle of reliable mail delivery, permanently
harming the Internet infrastructure for short term gain.

There seems to be this myth circulating that SPF prevents backscatter
because it checks the MAIL FROM.  This just doesn't hold water: the
effect of SPF is to cause all forged messages to bounce; it increases
backscatter.

This issue is a red herring.  In a world where MAIL FROM is often
forged, _all_ 5xx rejections (or indeed 4xx rejections) result in
backscatter.  There is nothing we (here, in this WG) can do about
this.  Other people are working on the problem; let's move on.

	-roy

---

From: <[email-masked]>
Date: Thu, 19 Aug 2004 20:27:06 -0400

If the sending MTA is a virus with its own SMTP engine, then the 5xx or 4xx DOES prevent
backscatter, because the virus does nothing with the error.

Similarly, if the sending MTA is a the direct connection from a spammer, backscatter prevented.

It is my experience that most viruses with built in MTA engines do direct connect rather then try to
relay through third party.  If they do relay, they can be tracked and shut down manually (or by
heuristics/volume rules on the relay)

I am not sure what the normal is for spammer direct connects (from either spammers server, or
spammers drone, either way doesn't matter).

But I am sure the majority of bad bounces my users get are the result of forged mail-from's of spam
and virus infections.  Now that I have implemented DNSBL's the spam has gone down a lot, but the
virus bounces have not.

Ultimately, if the SENDING MTA is the actual source of the forged Mail-from, then spf with a 5xx or
4xx code does stop backscatter.

Terry Fielder
Manager Software Development and Deployment
Great Gulf Homes / Ashton Woods Homes
[email-masked]
Fax: (416) 441-9085

> -----Original Message-----
> From: [email-masked]
> [mailto:[email-masked]]On Behalf Of Roy Badami
> Sent: Thursday, August 19, 2004 7:57 PM
> To: John Glube
> Cc: 'Daryl Odnert'; 'Chris Haynes'; 'Harry Katz'; 'IETF MARID WG'
> Subject: RE: Point of Order: Incomplete, flawed response to MARID WG
> Charter
>
>
>

---

From: Roy Badami <[email-masked]>
Date: Fri, 20 Aug 2004 01:41:43 +0100

>>>>> "terry" ==   <[email-masked]> writes:

    terry> If the sending MTA is a virus with its own SMTP engine,
    terry> then the 5xx or 4xx DOES prevent backscatter, because the
    terry> virus does nothing with the error.

That's true, but that's equally true of SPF, Sender ID, or any other
cause of 5xx errors (and IME one of the biggest causes is "550 User
Unknown" responses from old addresses)

When there _is_ an upstream MTA, 5xx errors will cause backscatter;
when there isn't they won't.  SPF is no different from anything else
in that respect.

The only way that SPF can solve the backscatter problem is if _every_
MTA implements SPF checks, so that there's _never_ an upstream MTA to
generate a bounce to a forged MAIL FROM.  IMHO you need a pretty high
SPF adoption before you will start to see any significant gains in
backscatter reduction, and this will just take too long.

This seems like a red herring to me.

Backscatter is a big problem now, and it's going to get worse.  I work
for a company that's been on the Internet since 1992 (and had e-mail
access since at least 1988).  What we see is that the _vast_ majority
of delivery attempts are to non-existent users.  Sure, a lot of them
are just speculative, but we see a *lot* of delivery attempts to
e-mail addresses that have long since been disabled, many of which
haven't been valid addresses for the best part of a decade.

The backscatter problem needs to be solved quickly.  There are
promising solutions, and there are many people working on the problem.

Sure, none of them are completely without pain, but I'm confident that
the backscatter problem will be a non-issue long before we reach the
level of SPF adoption that would be necessary to make a serious dent
in it.

Let's move on...

      -roy

---

From: <[email-masked]>
Date: Thu, 19 Aug 2004 20:55:02 -0400

I agree to move on, but 1 quick note re adoption, those that implement spf will suffer significantly
less backscatter, because backscatter from the wrong mta will be rejected.

And you may be right spf etc may cause non-spf implementors to receive more backscatter.

But all this will likely serve to expedite the adoption of spf, as more people want to be like the
other guy who has little/no backscatter.

NOTE:
Spamassassin clobbers my spam quite well.  My users now only suffer from backscatter in the form of
forged mail-from.  If Sender-ID doesn't use spf or something to reject at the mail-from level, I am
not going to waste a milli-second even considering adopting Sender-ID.

Terry Fielder

> From: Roy Badami [mailto:[email-masked]]
> Sent: Thursday, August 19, 2004 8:42 PM
> To: [email-masked]
> Cc: 'Roy Badami'; 'John Glube'; 'Daryl Odnert'; 'Chris Haynes'; 'Harry
> Katz'; 'IETF MARID WG'

---

From: Roy Badami <[email-masked]>
Date: Fri, 20 Aug 2004 02:17:29 +0100

>>>>> "terry" ==   <[email-masked]> writes:

    terry> I agree to move on, but 1 quick note re adoption, those
    terry> that implement spf will suffer significantly less
    terry> backscatter, because backscatter from the wrong mta will be
    terry> rejected.

I think you're mistaken.  Backscatter generally comes from legitimate
MTAs (as you point out, spammers/viruses themselves don't bother to
generate bounces).

When a forged message transits a legitimate MTA, and that MTA is
unable to deliver (receives a 5xx for whatever reason) it generates a
bounce.  This bounce is from <>, so SPF will check the HELO.  Being a
legitimate MTA, it includes its legitimate name in the HELO string.
The bounce will pass an SPF check.

I say again:

SPF does not prevent backscatter (at least not unless every MTA on the
planet adopts it).

The backscatter issue does not consitute an argument against
proceeding with the work on Sender ID and CSV (the WG's current work
items) though both will result in backscatter to some extent.

    -roy

---

From: <[email-masked]>
Date: Thu, 19 Aug 2004 21:45:11 -0400

> Sent: Thursday, August 19, 2004 9:17 PM

SO?!  I only care about if certain portions of the internet adopt, specifically those who know my
users/have my users addresses in their addressbook.  (Read "stop those virus bounces from infected
people who know my people")
> items) though both will result in backscatter to some extent.

OK.  I just reread the charter:
http://www.ietf.org/html.charters/marid-charter.html

And if MTA autorization:
-increases backspatter (you said it, not me)
-doesn't stop spam (may reduce only 1 type of spam source, spammers will adapt)
-only makes a dent in 1 kind of phishing (can't touch social engineering types)

I have been trying to keep up with the group, but perhaps I fell behind: Please help me out: What is
"MTA Authorization" aka "Sender ID" good for then?

Terry Fielder

---

From: Douglas Otis <[email-masked]>
Date: Thu, 19 Aug 2004 18:51:58 -0700

> other guy who has little/no backscatter.

This Back-door issue will improve commensurate with adoption of a
solution, as spammers are not often the direct source of this traffic,
but instead comes from sites at the end of a relay that would otherwise
not wish to send this traffic.  Few solutions have an impact that
exceeds adoption.  That a solution does not TOTALLY prevent a problem
without 100% adoption and therefore is a reason to dismiss such for
consideration seems absurd.

To say this is an unrelated problem, or one to be solved differently,
seems to overlook the relationship of the information needed to solve
each issue separately.  This problem can be related to the information
of the mail channel, or to solutions for the forwarding problem created
by Sender-ID, as example.  Of course, the reaction to not being allowed
in the Front-door, there is always the Back-door being the Return-Path
now being ignored.  Both doors need defended as closing one, will
increase the traffic through the other.

-Doug

---

From: Roy Badami <[email-masked]>
Date: Fri, 20 Aug 2004 03:05:46 +0100

>>>>> "terry" ==   <[email-masked]> writes:

    terry> SO?!  I only care about if certain portions of the internet
    terry> adopt, specifically those who know my users/have my users
    terry> addresses in their addressbook.  (Read "stop those virus
    terry> bounces from infected people who know my people")

Well, if every virus (or spammer) who has the e-mail address of one of
your users is sending mail out through an ISP that implements SPF
checks on their outbound mail relays, then yes, that would make a
significant dent in amount of backscatter that your users receive.

I'm ignoring spammers who send mail out through open relays; they're
unlikely to implement SPF checks, but you can hope to block many of
them through DNSBLs...

    terry> I have been trying to keep up with the group, but perhaps I
    terry> fell behind: Please help me out: What is "MTA
    terry> Authorization" aka "Sender ID" good for then?

It's a tool; it's not a panacea.

Authentication doesn't stop spam, but it enables technologies that
have promise in attacking the spam problem (accreditation and
reputation systems).

It may not be a panacea against phishing, but it is a useful tool in the war.

It's also a useful tool to stop people sending mail pretending to be
me.  Many corporates care about the fact that people are receiving
spam and viruses claiming to be from them, and many non-technical
users actually believe that the company sent them the spam or virus.

These are early days for sender authentication schemes.  Maybe they'll
be useful, maybe they won't.  Personally, I think they will be, though
they won't be the anti-spam panacea that some people seem to believe.

As I see it, the purpose of this WG is to give us a MARID standard to
deploy and test.  Until we have that, know one will know for sure how
well MARID will (or won't) fare in the real world.

     -roy

---

From: Roy Badami <[email-masked]>
Date: Fri, 20 Aug 2004 03:46:55 +0100

>>>>> "Douglas" == Douglas Otis <[email-masked]> writes:

This will be my last post on this topic.  I think I've made my points
as clearly as I can; beyond that we'll just have to agree to differ.

I think this seems to be intended as a reply to me, rather than a
reply to Terry, so I'll reply to it.

    Douglas> This Back-door issue will improve commensurate with
    Douglas> adoption of a solution, as spammers are not often the
    Douglas> direct source of this traffic, but instead comes from
    Douglas> sites at the end of a relay that would otherwise not wish
    Douglas> to send this traffic.

I'm not really sure I know what you mean by 'back door', 'front door'
and 'at the end of a relay'.

Although mail may transit many MTAs, most mail typically goes directly
from a final outbound MTA in the originating administrative domain to
an initial inbound MTA at the receiving adminsitrative domain.  Either
the outbound administrative domain implements checks on outgoing mail,
or unwanted/invalid mail will be rejected by the MTA in the
recipient's adminitrative domain, causing the outbound MTA to generate
a bounce.

This property is common to _all_ solutions that mandate or suggest
rejecting unwanted messages at the SMTP level.  To my knowledge, all
proposals on the table allow SMTP rejection, and as far as I can see
their behaviour with respect to backscatter will be very similar.

    Douglas> Few solutions have an impact that exceeds adoption.  That
    Douglas> a solution does not TOTALLY prevent a problem without
    Douglas> 100% adoption and therefore is a reason to dismiss such
    Douglas> for consideration seems absurd.

I'm not arguing that.  I'm arguing that even a MARID solution that
validates MAIL FROM (such as SPF) is unlikely to have _any_
significant affect on the backscatter problem until it reaches _very_
high levels of adoption.  More to the point, it needs adoption on ISPs
outbound relays, and at the moment interest seems to be more in
validating _incoming_ mail.

    Douglas> To say this is an unrelated problem, or one to be solved
    Douglas> differently, seems to overlook the relationship of the
    Douglas> information needed to solve each issue separately.

Well, I believe largely it is.  IME the main cause of backscatter is
bounces due to spam or viruses sent to addresses that no longer exist
(resulting in 550 User Unknown).

In many cases the inbound relay in the receiving administrative domain
rejects the message with 550, resulting in the outbound relay of the
origninating administrative domain generating a bounce.

However, in a large number of cases the bounce is actually generated
by the receiving administrative domain _after_ the message has been
accepted by the initial inbound relay.

There are two main reasons why this happens (and whether you like it
or not, this isn't going to change any time soon).

The first is that the architecture of some widely used MTAs doesn't
allow them to perform SMTP-time checks for unknown users.  qmail and
Lotus Notes spring to mind.  qmail is widely used by ISPs, and Lotus
Notes has significant adoption in large corporates.

The second is the design of large mail clusters is ISPs or complex
mail routing environments in large corporates.  In many cases the
initial inbound relay doesn't know that the validity of the address,
but simply forwards the message to other internal MTAs based on the
domain (or subdomain).  Hence, when a 550 User Unknown is generated,
it's the receiving domain that's forced to generate a bounce
(regardless of the choice of MTA software).

So yes, I think that the backscatter problem is largely unrelated to
the work we are doing here.  Some solutions may make the problem
marginally worse; some may even make things very marginally better
(though I'm dubious).

The only things that will make things substantially better are either
mandatory checks on outbound mail from all (or the majority) or ISPs,
or solutions where the bounce is validated by the recipient (such as
BATV, SES, Message-ID tracking schemes, etc).

I don't think the former will happen quickly enough to prevent
widespread adoption of the latter.

      -roy

---

From: Dean Anderson <[email-masked]>
Date: Fri, 20 Aug 2004 00:33:00 -0400 (EDT)

>     terry> backscatter, because backscatter from the wrong mta will be

Regards to Terry point: Backscatter from the MTA would not be rejected
since it is a message _to_ the backscatter victim _from_ the MTA. Even if
both MTAs participate in SPF, the SPF check should find the MTA's mailer
delivery system address to be authorized to that MTA.

> The bounce will pass an SPF check.

Exactly right.  It is probably from postmaster, but could be from <>.

Every user everywhere is the customer of an ISP or a 
company with an MTA.  _All_ spam is sent by customers of someone.  So 
_all_ abuse has a relay that will just relay email. An inside "closed" 
relay is "open" to anyone inside the network.   

If we assume that anti-trust laws will require ISPs to permit other ISPs
to provide services to unbundled email, then SPF will have no effect at
all: Ever.  If that isn't the case, then _every_ forged email results in
backscatter:  _All_, _100%_

SPF and many other schemes assumes that the abusers won't alter their
habits.  The goal of the abuser is to abuse.  Of course they are going to
alter their sending habits.  This has happened again and again.

---

From: Dean Anderson <[email-masked]>
Date: Fri, 20 Aug 2004 00:47:30 -0400 (EDT)

> unlikely to implement SPF checks, but you can hope to block many of

This isn't right. Not to /open/ the open relay question, but see
http://www.av8.net/openrelay.html

I realize that many people disagree with that, but courts and lawyers have 
agreed with us. So, I won't argue this if you disagree.  But if you didn't 
know, it may be interesting.

For abusers, there is no difference between an ISP's closed relays and
open relays, unless they are self-described anti-spammers who think that
by abusing the relay they will get it closed or something.  Closed relays 
are "open" to those using the network's IP Address space.   All abusers 
everywhere have one or more relays available to them, because they are 
"customers" right up until they aren't.

There is one difference between open and closed relays since the open
relay blacklists closed down: its not so easy to find open relays to
abuse.  Though we and other ISPs found some time ago that blocking the
blacklist scanning stopped open relay abuse.  But networks can't hide 
their closed relays from their customers.

		--Dean

---

From: Graham Murray <[email-masked]>
Date: Fri, 20 Aug 2004 14:11:19 +0100

> when there isn't they won't.  SPF is no different from anything else

If the upstream MTA (and any MTAs upstream of it) also implements SPF
(or whatever other protocol) then the 5xx error will be sent to the
originator and there would be no back scatter as the mail would be
rejected at the first relay.

---

From: Frank Ellermann <[email-masked]>
Date: Fri, 20 Aug 2004 18:40:10 +0200

Roy Badami wrote:

> The only way that SPF can solve the backscatter problem is
> if _every_ MTA implements SPF checks, so that there's _never_
> an upstream MTA to generate a bounce to a forged MAIL FROM.

No, that's not the idea.  At the moment spammers still forge
my addresses @xyzzy.  As soon as some big providers implement
SPF (e.g. using SpamAssassin 3.x) the spammers will stop to
forge @xyzzy, because they want to reach all of their unhappy
"customers".  The rest of the world minus say AOL and Hotmail
is not good enough for a professioal spammer.

> this will just take too long

Learning new tricks may take a while for spammers, but not that
long.  A smart SPF solution could automatically block an IP if
it generates a SPF FAIL.

> Backscatter is a big problem now

Indeed.  And PRA does not address it.

> The backscatter problem needs to be solved quickly.

Classic SPF solves it.  Bye, Frank

---

From: Roy Badami <[email-masked]>
Date: Fri, 20 Aug 2004 18:01:28 +0100

>>>>> "Frank" == Frank Ellermann <[email-masked]> writes:

    Frank> As soon as some big providers implement SPF (e.g. using
    Frank> SpamAssassin 3.x) the spammers will stop to forge @xyzzy,
    Frank> because they want to reach all of their unhappy
    Frank> "customers".

The smart spammers will.  Though whether there will still be enough
dumb spammers out to cause lots of backscatter is less clear to me.

     -roy