Subject: Re: Best practices to avoid virus and spam
From: "Hector Santos" <[email-masked]>
To: Keld Jørn Simonsen <[email-masked]>
Cc: "IETF-SMTP" <[email-masked]>
Date: Sat, 14 Feb 2004 17:19:39 -0500
List-Id: <ietf-smtp.imc.org>

----- Original Message ----- 
From: "Keld Jørn Simonsen" <[email-masked]>
To: "Keld Jørn Simonsen" <[email-masked]>
Cc: "Keith Moore" <[email-masked]>; <[email-masked]>
Sent: Saturday, February 14, 2004 10:20 AM
Subject: Re: Best practices to avoid virus and spam

> To me, the model is a bit like what the economists call game theory,
> that if you sacrifice some of your own ressources initially, you can
> actually get a better result overall for the society, and actually also
> for yourself.

hmmmmm,  interesting way of putting it. :-)

Exactly!  By sacrificing some RCPT lookup time, your mail system as a whole
AKA "society" is improved.  In your suggestion, you are putting all the
pressure point into one spot, which is doom to fail one way or another
because you took away the capabilities of the each different entity.    By
spreading the "duties" of the entire process, you optimize the end results.

Yes, it seems your end results are different than most, certainly different
than mine.  Even if the end-results are the same,  how one get there is also
quite important.  I would prefer a system written for efficiency and
throughput using compliancy as a basis.  Why would I want to increase
transaction time to one that is 3-4 times larger than it needs to be?
Specially, when the end result is the better or the same.

Anyway,  I am not quite sure what is being proposed here as it sounds more
like an implementation issue.  I don't believe this will help the SMTP
process.  Although, I am in full agreement that the SMTP process needs to
take a deeper look at user mail processing, a taboo in traditional designs.

In fact, I believe there is a current bill in Canada that enforces ISP to
have a "Spam/Virus Mail Filter" installed.   That means to me, that from a
product standpoint, the market will want a "product" or "system of products"
that offers all the features they need to have.  Today, it may be considered
"mal-practice" (or just plain stupid)  to have a mail system that doesn't
offer any kind of SPAM/VIRUS filtering/protection.

-- 
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

Cc: Keith Moore <[email-masked]>, [email-masked]
From: Keith Moore <[email-masked]>
Subject: Re: Best practices to avoid virus and spam
Date: Sat, 14 Feb 2004 11:16:41 -0500
To: =?ISO-8859-1?Q?Keld_J=F8rn_Simonsen?= <[email-masked]>
Sender: [email-masked]

> OK, found out that you cited the very mail, so I will not send them
> again. But what do you say to my analyses that looking at the mail body
> for virus would *reduce* the traffic, and also the bogus error mail?
> Do you agree?

no.

> 1. What happens if you reject the email on grounds of bad receipient?
> The infector then just goes on to the next potential victim in his 
> list.
> Net result: the bandwidth consumed by sending virus on the internet 
> total stays the
> same.

this is only true if the machine that is trying to propagate the virus 
has such a long list of potential victims that the virus on that 
machine will get found and removed before it exhausts that list.  I 
don't think this is true in general, though it could be true for some.

even if the virus is consuming the same bandwidth either way, there's 
no particular reason that a server should consume its own bandwidth 
just to save someone else from consuming bandwidth.  everyone has it in 
his own interests to not waste bandwidth that he is paying for.

>  And the infector will get back to you eventually, so your specific
> bandwidth will possibly not be reducsd much by this, it would probably
> get reduced by your skill in handling this compared to the average
> performance of the internet.

it's not clear how many viruses retry failed transmissions.  even for 
those that do, for a typical virus it would take several failed retries 
to equal the bandwidth that would be consumed.

the other problem with this comparison is that you are only considering 
virus traffic - but an SMTP server that sucks down messages just to see 
if they are viruses also wastes a lot of bandwidth on messages that 
aren't viruses, and makes error reporting less reliable.

> 2. If you read the virus and just discard it, you will reduce the
> traffic of error messages - as you do not issue one.

no, you will reduce the traffic of error messages issued in response to 
viruses.  you will increase the traffic of error messages overall.

> 3. If you keep the infector busy with sending you stuff that you will
> discard, then he cannot do other evil things at the same time.

it's certainly possible for a virus to manage lots of outgoing 
connections at once.

> To me, the model is a bit like what the economists call game theory,
> that if you sacrifice some of your own ressources initially, you can
> actually get a better result overall for the society, and actually also
> for yourself.

just because this works some of the time does not mean it works in this 
specific case.

Date: Sat, 14 Feb 2004 16:20:55 +0100
From: Keld =?iso-8859-1?Q?J=F8rn?= Simonsen <[email-masked]>
To: Keld =?iso-8859-1?Q?J=F8rn?= Simonsen <[email-masked]>
Cc: Keith Moore <[email-masked]>, [email-masked]
Subject: Re: Best practices to avoid virus and spam
Sender: [email-masked]

On Sat, Feb 14, 2004 at 04:13:36PM +0100, Keld Jørn Simonsen wrote:
> On Fri, Feb 13, 2004 at 02:04:31PM -0500, Keith Moore wrote:
> > My opinion is that we should recommend exactly the opposite - that
> > sites should always reject mail before DATA if they have enough
> > information to do that.  The impact of sucking down all messages just to
> > do a virus is considerable.  And error reporting is much more reliable
> > if the error indication is returned in response to SMTP MAIL or RCPT
> > than if it is returned as a bounced mail message.
> 
> I know that your opinion is quite common, but it seems that my latest
> mails on the issue, with all my wise words:-), have not been sent,
> so I will send them again.

OK, found out that you cited the very mail, so I will not send them
again. But what do you say to my analyses that looking at the mail body
for virus would *reduce* the traffic, and also the bogus error mail?
Do you agree?

To me, the model is a bit like what the economists call game theory,
that if you sacrifice some of your own ressources initially, you can
actually get a better result overall for the society, and actually also
for yourself. 

Best regards
keld

Date: Sat, 14 Feb 2004 16:13:36 +0100
From: Keld =?iso-8859-1?Q?J=F8rn?= Simonsen <[email-masked]>
To: Keith Moore <[email-masked]>
Cc: Keld =?iso-8859-1?Q?J=F8rn?= Simonsen <[email-masked]>, [email-masked]
Subject: Re: Best practices to avoid virus and spam
Sender: [email-masked]

On Fri, Feb 13, 2004 at 02:04:31PM -0500, Keith Moore wrote:
> > > but the SMTP server doesn't have any way of knowing that the mail
> > > was a virus.  and it shouldn't have to suck down the message just
> > > for the purpose of finding out.  
> > 
> > Yes, the MTA would not know upfront whether it is a mail with virus,
> > and it would need to receive all of the mail to find out whether it
> > is. I think we agree on that. Whether it then is advisable to do that
> > can be discussed, my opinion is that we should recommend this.
> 
> My opinion is that we should recommend exactly the opposite - that
> sites should always reject mail before DATA if they have enough
> information to do that.  The impact of sucking down all messages just to
> do a virus is considerable.  And error reporting is much more reliable
> if the error indication is returned in response to SMTP MAIL or RCPT
> than if it is returned as a bounced mail message.

I know that your opinion is quite common, but it seems that my latest
mails on the issue, with all my wise words:-), have not been sent,
so I will send them again.

> There are better ways to discourage viruses than to expect SMTP servers
> to accept, detect, and discard them.

I am willing to discuss all ways on how to do this, I am fed up on them.
My discussion was actually not mainly on how to avoid virus, but how to
avoid bogus error messages on virus detection and related error
messages.

Anyhow I am interested in hearing what you propose to avoid virus.

Best regards
Keld

Date: Fri, 13 Feb 2004 14:04:31 -0500
From: Keith Moore <[email-masked]>
To: Keld =?ISO-8859-1?Q?J=F8rn?= Simonsen <[email-masked]>
Cc: [email-masked], [email-masked], [email-masked]
Subject: Re: Best practices to avoid virus and spam
Sender: [email-masked]

> > but the SMTP server doesn't have any way of knowing that the mail
> > was a virus.  and it shouldn't have to suck down the message just
> > for the purpose of finding out.  
> 
> Yes, the MTA would not know upfront whether it is a mail with virus,
> and it would need to receive all of the mail to find out whether it
> is. I think we agree on that. Whether it then is advisable to do that
> can be discussed, my opinion is that we should recommend this.

My opinion is that we should recommend exactly the opposite - that
sites should always reject mail before DATA if they have enough
information to do that.  The impact of sucking down all messages just to
do a virus is considerable.  And error reporting is much more reliable
if the error indication is returned in response to SMTP MAIL or RCPT
than if it is returned as a bounced mail message.

There are better ways to discourage viruses than to expect SMTP servers
to accept, detect, and discard them.

Date: Fri, 13 Feb 2004 08:24:26 +0100
From: Keld =?iso-8859-1?Q?J=F8rn?= Simonsen <[email-masked]>
To: Keith Moore <[email-masked]>
Cc: Keld =?iso-8859-1?Q?J=F8rn?= Simonsen <[email-masked]>, [email-masked]
Subject: Re: Best practices to avoid virus and spam
Sender: [email-masked]

On Thu, Feb 12, 2004 at 02:18:05PM -0500, Keith Moore wrote:
> > > >1. reports that the body contained virus
> > > >2. reports that the user is unknown
> > > >3. reports that the user's account has been closed.
> > > >4. reports that the user's mailbox is full.
> > > 
> > > I think we'd agree that #1 should not be sent to the "sender" of the
> > > 
> > > virus since it is so often forged.
> > > But reports #2-4 are often useful, and they _should_ be detected
> > > before the server has a chance to look at the content of the
> > > message.
> > 
> > reports 2-4 are very often useful! But these reports should not be
> > generated, if the mail in question was a virus (number 1). 
> 
> but the SMTP server doesn't have any way of knowing that the mail was a 
> virus.  and it shouldn't have to suck down the message just for the 
> purpose of finding out.  

Yes, the MTA would not know upfront whether it is a mail with virus, and
it would need to receive all of the mail to find out whether it is.
I think we agree on that. Whether it then is advisable to do that can be
discussed, my opinion is that we should recommend this. As far as I can
see the bandwidth consumed with email is quite low, compared to other
major services like ftp and http, that most sites will also be doing
(at least http). So bandwidth is not a major problem, at least not for 
companies and organizations also running web services. For the personal
user, over modem connections, I think most of these are using some
mailbox service from their ISP, they use it via some IMPA service, and
then the ISP should do the virus scanning fro them, which many do.
So considerations for modem users are less relevant. Then there are the
private ADSL users (like me) that have their own Linux box running with
their own MTA (I have postfix here and I have a fixed IP for my 256 kbit
ADSL, but I mostly get my email via an IMAP service) - I think with the 
current load of vira, which is high my bandwidth would not really be
affected, still. 

Then the CPU cycles to handle virus inspection may be a problem.
I am not sure how much this would take, but on my oudated 1 GHz box,
it is not much. Then, I mostly look for attachments with specific file
name extensions, and then some specific lines for bogus error mail, so I
may not do it properly; and genuine virus checkers may use many more cpu
cycles.

My point is that receiving the full email and do virus check before
receipient checks, is not "just for finding out" whether there was a
virus, but to generate meaningful error messages on the receipient,
avoiding the bogus error messages. Better have my machine do that check
than me or other humans wading thru the vast amount of bogus email
errors in their inbox. It is a trade of bandwidth and cpu cycles for human
time.

And actually, using your bandwidth to receive the virus, for possible
discard, may even *reduce* the bandwidth use on the internet overall, and
eventually also for yourself. My analysis:

1. What happens if you reject the email on grounds of bad receipient?
The infector then just goes on to the next potential victim in his list. 
Net result: the bandwidth consumed by sending virus on the internet total stays the
same. And the infector will get back to you eventually, so your specific
bandwidth will possibly not be reducsd much by this, it would probably
get reduced by your skill in handling this compared to the average
performance of the internet.

2. If you read the virus and just discard it, you will reduce the
traffic of error messages - as you do not issue one. Some of this
traffic goes directly to the infector, but some goes to those that has 
got it somehow, and is relaying it, and there some real humans are 
saved from receiving bogus email - email that is not currently handled
with antivirus nor antispam software.

3. If you keep the infector busy with sending you stuff that you will
discard, then he cannot do other evil things at the same time. 
That would probably reduce number of infected sites, and thus the amount
of infectors. 

Net result: a reduction overall of the bandwidth used.
Probably som more use of cpu, but then network bandwidth is a more
precious ressource than the CPU, IMHO.

BTW, how does the virus propagate via non-infected systems?
One should think that just rejecting based on reciepient error would
go to the infector and stay there, but maybe the infector then relays 
this error message to the purged sender? Or is it open relays that are
being used? (good we have RBL services that tell which machines are
openr:-(  And then relayed to machines that do not use these RBL
services? 

> > I also have my reservations to whether a sensibe RBL for infected
> > machines can be built. How to eliminate machines that are not
> > infected, but propagates virus via some open email lists? 
> 
> I don't think RBLs can ever eliminate all virus mail, or even most
> of it. 

I am not sure either, but would like to try it out.
Ie build my own RBL service. I am looking at the spamikaze
software for this.

> I do think that a means of reliably identifying messages 
> that should never send mail can help reduce virus propagation.

That is another way to address the problem, and the methods are not
mutually exclusive, so we could do both. BTW, I did not intend to advocate specific
ways of doing RBL in an RFC on best practices to avoid bogus error mail,
which is the subject we are discussing here.
> 
> 
> > > >My analysis is that for a number of servers the CPU load stemming
> > > >from this kind of bogus mail is not a problem.
> > > 
> > > I'm not sure what you mean by "analysis" here.
> > 
> > Maybe "experience" is closer to what I mean, but I think that many MTA
> > systems have enough muscle to do some work to deliver better quality
> > of error responses. 
> 
> yes, they do.  in part this is because these days, they have to be designed
> to handle a generous amount of extra load in order to not melt down due to
> occasional viruses, spam, or DoS attacks.  whether this is a desirable
> thing, or whether it's something we can reasonably expect, is a different 
> question.

Yes, and in these days we are under constant virus attacks. 
I do not expect the world to be more friendly in the future, it is
a war between the good and the evil. My impression is that currently
it is tolerable, and if the good people do things a little better,
like collectively just ignoring virus mail, the world can become an even
better place to live in.

> --
> He not busy being born, is busy dying.		- Bob Dylan

I like that citation:-)

Keld

From: "Hector Santos" <[email-masked]>
To: =?iso-8859-1?Q?Keld_J=F8rn_Simonsen?= <[email-masked]>
Cc: <[email-masked]>
Subject: Re: Best practices to avoid virus and spam
Date: Thu, 12 Feb 2004 15:02:55 -0500
Organization: Santronics Software, Inc
Sender: [email-masked]

Hi Keld,

----- Original Message ----- 
From: "Keld Jørn Simonsen" <[email-masked]>
To: "Hector Santos" <[email-masked]>
Cc: "Keld Jørn Simonsen" <[email-masked]>; <[email-masked]>
Sent: Thursday, February 12, 2004 4:22 AM
Subject: Re: Best practices to avoid virus and spam

> > I presumed you were suggested for the RBL sites to include virus/spam
> > infected machines.
>
> Yes, I was.
>
> > Cool,  if one of them did it.  What I pointing out I would hate to see a
new
> > speciflc RBL site "specialized" only in listing virus infected sites.
> >
> > I already do three RBL site lookups.  I don't wish to add a forth.
Thats
> > all :-)
>
> I am not sure if some of them would add this service. I think best chances
are
> that a specialized, or group of specialized RBL sites would come up with
> this service.  Anyway another DNS lookup is IMHO not much to do, compared
to
> all the other checks done.

True, and when done with a good learning performance algorythm, it works
really well. :-)

I think the idea for RBL site admins warrants some consideration.  However,
the problem I see in regards to SORBIG generation like email viruses,  the
potential exist to "deflect" blame somewhere else.  i.e.,  some of users are
infected,  should your ISP site be blacklisted?

Of course, this can happen with any RBL or even without a RBL report.  A
site can detect that your site is sending an unsual amount of bounces with
spoofed addresses and automatically block you.  So whether it is RBL or
system related, it is still something that can happen.  But with the MyDOOM
type emails creating so many bounces, it can happen more greater when
paranoia is high.

Not speaking for Keith, but I think this is the "bad practice" aspect often
attributed to RBL systems.  Its true, but its gotten better and the systems
that are more reputable do a better job of screening "IP reports."

> > Chicken and Egg?    If you are going to reject anyone, why bother with
the
> > overhead?
>
> Because of the improtance of the message sent out. One message is: this
> is bogus mail, the other is: the user is not found. The latter is a
> serious message for the original sender, the former is already known to
the
> offender.

I'm interested in understanding your idea or suggestion better. So please
correct me if I am wrong. Your goal is to basically be able:

            Purge "malicious mail"

from the distribution process regardless of who is being sent to, thus you
feel there is no need to do a recipient check at RCPT?

In other words, you want to be able to make some sense of the mail first
before you decide if the user (or sender) is legitimate or is allowed to
receive it and in this way, you believe that error reporting will be alot
cleaner and more specific to the real every day operations without that
crappy mess created by MyDoom, etc?

Ok, I see this and I agree with the premise that "SMTP" or the system
implementation should take a responsibility in helping to clean up malicious
distribution to its users.

But what I don't understand is why you can't do this now?

Are you saying that your smtp software does not have an option to turn off
RCPT validation?

I am thinking the effect on my customers who turn off "Local User
Validation."  We still have ISPs that host or hub for slip systems.  In this
case, they have no choice but to just accept the mail.  These are also
autheticated or trusted sessions.   But an infected user is also
authenticated so he has the power to relay mail as well.

In this mode of operation (local user validation turned off), if they don't
trap the malicious mail at the DATA state point mail filter process with
their 3rd party AVS software, then it contributes to the MYDOOM bounce
distribution logic.

So what is really important is that your reliance is now heavily weighted on
making sure the AVS is highly effective.  This mode of operation still makes
you prone to new viruses that are yet to be detected or learned by AVS yet.

No doubt, from a scalability and performance standpoint, checking of Local
User Validation first before all the extra overhead checking is much better.
Solves many issues.  I personally don't like this type of operation.  It
helps spammers, not restricts them.

But I do see your point.

I guess what I learned here is maybe is it feasible to explore an provide an
option to delay the RCPT validation until the DATA stage is reached.    We
are not in the Mail Interpretation business, but we provide examples of mail
filter scripts that can be used at DATA.  Maybe an example script where it
checks for everything else before the user validation would be worth the
effort.

By the way, YAHOO validates RCPT at the DATA stage.  I don't know if they do
AVS checking first, but they accept all RCPT and then delay the checking
until DATA is reached.   However, it would naturally make sense to me that
they won't want to exert AVS checking until the basic SMTP related stuff
pass the test first.  It is just too much overhead potential.

-- 
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

Date: Thu, 12 Feb 2004 14:18:05 -0500
From: Keith Moore <[email-masked]>
To: Keld =?ISO-8859-1?Q?J=F8rn?= Simonsen <[email-masked]>
Cc: [email-masked], [email-masked], [email-masked]
Subject: Re: Best practices to avoid virus and spam
Sender: [email-masked]

> > >1. reports that the body contained virus
> > >2. reports that the user is unknown
> > >3. reports that the user's account has been closed.
> > >4. reports that the user's mailbox is full.
> > 
> > I think we'd agree that #1 should not be sent to the "sender" of the
> > 
> > virus since it is so often forged.
> > But reports #2-4 are often useful, and they _should_ be detected
> > before the server has a chance to look at the content of the
> > message.
> 
> reports 2-4 are very often useful! But these reports should not be
> generated, if the mail in question was a virus (number 1). 

but the SMTP server doesn't have any way of knowing that the mail was a 
virus.  and it shouldn't have to suck down the message just for the 
purpose of finding out.  

> I can understand that some rbls are propagating errors.
> I dont think that some rbls today builds their data on 
> spamassasin result, which one should that be?

RCC is one that is willing to accept mail blocked for other reasons -
though I don't see them explicitly soliciting mail identified by
spamassassin.

> I also have my reservations to whether a sensibe RBL for infected
> machines can be built. How to eliminate machines that are not
> infected, but propagates virus via some open email lists? 

I don't think RBLs can ever eliminate all virus mail, or even most
of it.  I do think that a means of reliably identifying messages 
that should never send mail can help reduce virus propagation.

> > >My analysis is that for a number of servers the CPU load stemming
> > >from this kind of bogus mail is not a problem.
> > 
> > I'm not sure what you mean by "analysis" here.
> 
> Maybe "experience" is closer to what I mean, but I think that many MTA
> systems have enough muscle to do some work to deliver better quality
> of error responses. 

yes, they do.  in part this is because these days, they have to be designed
to handle a generous amount of extra load in order to not melt down due to
occasional viruses, spam, or DoS attacks.  whether this is a desirable
thing, or whether it's something we can reasonably expect, is a different 
question.

--
He not busy being born, is busy dying.		- Bob Dylan

Date: Thu, 12 Feb 2004 19:25:30 +0100
From: Keld =?iso-8859-1?Q?J=F8rn?= Simonsen <[email-masked]>
To: Keith Moore <[email-masked]>
Cc: Keld =?iso-8859-1?Q?J=F8rn?= Simonsen <[email-masked]>, [email-masked]
Subject: Re: Best practices to avoid virus and spam
Sender: [email-masked]

On Thu, Feb 12, 2004 at 07:56:59AM -0500, Keith Moore wrote:
> 
> On Feb 12, 2004, at 12:53 AM, Keld Jørn Simonsen wrote:
> 
> >On Wed, Feb 11, 2004 at 10:47:00PM -0500, Keith Moore wrote:
> >>
> >Yes, many things are easier to check upfront than whether this is virus
> >or spam - the latter takes a full scan of the body. And then I think
> >these easy checks should be done as long as they also reliably can
> >classify the mail as forged, such as bogus MX and no PTR and some such.
> 
> neither of these checks can reliably classify the email as forged.

You are right. But there probably are checks that can be done upfront to
find bogus mail, that was my point.

> >My aim is, still, to get rid of bogus error mail.
> >And as far as I see it in my own mailbox, this is mail of the
> >following kinds, *only*:
> >
> >1. reports that the body contained virus
> >2. reports that the user is unknown
> >3. reports that the user's account has been closed.
> >4. reports that the user's mailbox is full.
> 
> I think we'd agree that #1 should not be sent to the "sender" of the 
> virus since it is so often forged.
> But reports #2-4 are often useful, and they _should_ be detected before 
> the server has a chance to look at the content of the message.

reports 2-4 are very often useful! But these reports should not be
generated, if the mail in question was a virus (number 1). 
That is my whole point. 
> 
> yes, I do think that having uniform reporting would be useful.
> 
> >I would be happy not to make a new SMTP error reply code if this can be
> >done better with some other reply means. What would you suggest?
> 
> DSNs and ENHANCEDSTATUSCODES.

OK with me.

> I have yet to see a third party RBL that didn't misrepresent what it 
> was reporting - either in that the information being reported was often 
> stale, or in that there was an attempt to mislead people about the 
> validity of the criteria they were using.  For instance, the RBLs that 
> blocked mail from machines that appeared to relay mail was a dubious 
> criterion because there were valid reasons to relay mail and not all or 
> even most mail from such machines was spam.  (Now with SMTP 
> authentication there are fewer valid reasons).  RBLs that accept 
> reports from other parties spread misinformation.  For instance, if a 
> site uses spamassassin's criteria as a way to decide whether to report 
> to an RBL that a particular IP is sending spam that is spreading 
> unreliable information.  ("propagation of error")

I can understand that some rbls are propagating errors.
I dont think that some rbls today builds their data on 
spamassasin result, which one should that be?

I also have my reservations to whether a sensibe RBL for infected
machines can be built. How to eliminate machines that are not infected,
but propagates virus via some open email lists? I personally get
a lot of virus mail, but surely some of the machines are not
infected, but only relaying. I would like to investigate the
possibility, however.

> >>There might be some cases where the _authoritative_ DNS zone for a 
> >>MAIL
> >>FROM domain or  the _authoritative_ DNS zone for the source IP address
> >>of an SMTP connection says "messages that look like this are not
> >>authorized".  And in some of those cases the SMTP server might be able
> >>to tell that a message "looks like this" from just MAIL FROM and/or
> >>RCPT TO.  In those cases it _might_ make sense for the SMTP server to
> >>return 2xx in response to those commands (if they are otherwise valid)
> >>just so that it can black-hole the message.  Though I think it might
> >>make even more sense to return 4xx in response to those commands - or
> >>just stop responding to that connection - and then refuse to accept 
> >>any
> >>new connections from that source IP for some period of time.  The
> >>rationale here is that the SMTP server doesn't want to waste bandwidth
> >>or cpu cycles by sucking down a message that's going to be discarded
> >>anyway, and it does want to avoid being interrupted by a compromised
> >>machine or spammer's machine if it tries again later.
> >
> >My analysis is that for a number of servers the CPU load stemming from
> >this kind of bogus mail is not a problem.
> 
> I'm not sure what you mean by "analysis" here.

Maybe "experience" is closer to what I mean, but I think that many MTA
systems have enough muscle to do some work to deliver better quality
of error responses. This given the amount of mail received in smaller
businesses, and the muscle normally found in current hardware.

> But my limited 
> experience is that many mail servers spend much more time and bandwidth 
> processing spam than processing useful mail.  (especially if you count 
> the effort spent trying to bounce spam)

Yse that is sad, but with the amount of spam and virus today, this is
foreseable. As said on one of my servers about 99 % is bogus mail.

Best regards
keld

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>
Cc: <[email-masked]>
Subject: Re: SF Support --> was Re: MX Domain Name --> Was Re: Some Ideas I would like to Bounce off ya 
Date: Thu, 12 Feb 2004 13:02:41 -0500
Organization: Santronics Software, Inc
Sender: [email-masked]

----- Original Message ----- 
From: <[email-masked]>
To: "Hector Santos" <[email-masked]>
Cc: <[email-masked]>
Sent: Thursday, February 12, 2004 12:51 PM
Subject: Re: SF Support --> was Re: MX Domain Name --> Was Re: Some Ideas I
would like to Bounce off ya

> But hey.. if you think you have a better idea what's in the DNS than the
> DNS does, you're free to design your product that way.

That wasn't the intent and I made no statement to that effect.

Nonetheless, I really appreciate and thank you for the leeway.

-- 
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

To: Hector Santos <[email-masked]>
Cc: [email-masked]
Subject: Re: SF Support --> was Re: MX Domain Name --> Was Re: Some Ideas I would like to Bounce off ya 
From: [email-masked]
Date: Thu, 12 Feb 2004 12:51:18 -0500
Sender: [email-masked]

--==_Exmh_2087225496P
Content-Type: text/plain; charset=us-ascii

On Thu, 12 Feb 2004 12:24:42 EST, Hector Santos said:
> Oh I think I finally see what is going on here.  You are an administrator!!

So positive of that, are we?

You might want to keep in mind that there are "administrators" on this
list who have forgotten more about SMTP behavior in production than you
have a hope of ever learning.

You might want to keep in mind that there are "administrators" on this
list who have probably created more code than you'll have a hope of creating.

> If it helps, you have no clue to how we develop or share intelligence.
> That's proprietary information.  But its not big secret for developers,
> smart learning and caching.    You can try a lookup for a first time entry
> and if it fails to return SPF info, you have a decision to make.  I've leave
> that for food for thought.

Or you can avoid re-inventing the wheel and let DNS caching, both positive
and negative, do the work for you.  That's exactly why DNS caching was
invented.

But hey.. if you think you have a better idea what's in the DNS than the
DNS does, you're free to design your product that way.

--==_Exmh_2087225496P
Content-Type: application/pgp-signature

--==_Exmh_2087225496P--

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>
Subject: Re: Best practices to avoid virus and spam
Date: Thu, 12 Feb 2004 12:33:43 -0500
Organization: Santronics Software, Inc
Sender: [email-masked]

----- Original Message ----- 
From: "Keith Moore" <[email-masked]>
To: "Hector Santos" <[email-masked]>
Sent: Thursday, February 12, 2004 11:02 AM
Subject: Re: Best practices to avoid virus and spam

> I don't think we're talking about the same thing.

You talked of some "period of time" logic and I just provided some feed back
on "length of period."  Do what you wish with it.

> Then we don't agree.

There is no win-win here or compromise or let go, with you, is there?

> Because there is no way in Hell that third-party
> RBLs should be standardized.  And whether they're here to stay is
> irrelevant.   The job of a standards organization is to describe how to
> do things well, not to encourage bad practices just because they're
> popular.

Ok, thanks for sharing your opinion.  I was not trying to change it.  But
like I said, your statement is not the gospel and other views needed to be
stated.   I said my part.  I'm done with this thread.

--  "Chaos creates change and change creates new opportunities."
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>
Cc: <[email-masked]>
Subject: Re: SF Support --> was Re: MX Domain Name --> Was Re: Some Ideas I would like to Bounce off ya 
Date: Thu, 12 Feb 2004 12:24:42 -0500
Organization: Santronics Software, Inc
Sender: [email-masked]

Oh I think I finally see what is going on here.  You are an administrator!!
I thought this mailing list was for SMTP developers?  If not, then I am
wrong. Sorry. Different mindsets.

If it helps, you have no clue to how we develop or share intelligence.
That's proprietary information.  But its not big secret for developers,
smart learning and caching.    You can try a lookup for a first time entry
and if it fails to return SPF info, you have a decision to make.  I've leave
that for food for thought.

-- 
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

----- Original Message ----- 
From: <[email-masked]>
To: "Hector Santos" <[email-masked]>
Cc: <[email-masked]>
Sent: Thursday, February 12, 2004 12:09 PM
Subject: Re: SF Support --> was Re: MX Domain Name --> Was Re: Some Ideas I
would like to Bounce off ya

On Thu, 12 Feb 2004 11:52:15 EST, Hector Santos said:
> Why are still talking about a "Host.txt" ?
>
> Where did HOSTS.TXT come from?   Not me.

No, but you're doing the same exact thing - trying to keep a local list of
things that really should be in the DNS:

To: Hector Santos <[email-masked]>
Cc: [email-masked]
Subject: Re: SF Support --> was Re: MX Domain Name --> Was Re: Some Ideas I would like to Bounce off ya 
From: [email-masked]
Date: Thu, 12 Feb 2004 12:09:13 -0500
Sender: [email-masked]

--==_Exmh_1952446836P
Content-Type: text/plain; charset=us-ascii

On Thu, 12 Feb 2004 11:52:15 EST, Hector Santos said:
> Why are still talking about a "Host.txt" ?
> 
> Where did HOSTS.TXT come from?   Not me.

No, but you're doing the same exact thing - trying to keep a local list of
things that really should be in the DNS:

> For the final and last time, our implementation requires a LIST of domains
> that will trigger the DMP and SPF lookup.  

You're the one who said your application needed to keep a LIST (your caps).

If you can't understand why this is broken the exact same way that HOSTS.TXT
was broken, then maybe you *SHOULD* keep a local list, and let us know what
insights you have once several hundred thousand sites have deployed SPF.

Hopefully, your LIST already has almost 7,000 entries on it, because that's
what http://spftools.infinitepenguins.net/register.php says is out there.

And that's just sites that bothered putting it in that registry.  There's
undoubtedly a lot more that are in the DNS that are invisible to both the
registry and your LIST.

--==_Exmh_1952446836P
Content-Type: application/pgp-signature

--==_Exmh_1952446836P--

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>
Cc: <[email-masked]>
Subject: Re: SF Support --> was Re: MX Domain Name --> Was Re: Some Ideas I would like to Bounce off ya 
Date: Thu, 12 Feb 2004 11:52:15 -0500
Organization: Santronics Software, Inc
Sender: [email-masked]

Why are still talking about a "Host.txt" ?

Where did HOSTS.TXT come from?   Not me.

Again, its obvious you don't have the technical implementation with this to
make sense of ANYTHING that was said and rather continue to take pot shots
as its the only mode of possibly living for you!.  Geez!  Get a life
already.

-- 
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

----- Original Message ----- 
From: <[email-masked]>
To: "Hector Santos" <[email-masked]>
Cc: <[email-masked]>
Sent: Thursday, February 12, 2004 11:18 AM
Subject: Re: SF Support --> was Re: MX Domain Name --> Was Re: Some Ideas I
would like to Bounce off ya

If you can't understand why the rest of the world moved away from HOSTS.TXT
and started using DNS, and need to keep a local LIST, you probably shouldn't
make any recommendations about how to use DNS.

If however you *do* understand, but work with a bunch of dysfunctional
pea-brains that refuse to see the light and insist on keeping a local LIST
anyhow, you have my condolences.  Good luck on getting an actual working
product out of them.

To: Hector Santos <[email-masked]>
Cc: [email-masked]
Subject: Re: SF Support --> was Re: MX Domain Name --> Was Re: Some Ideas I would like to Bounce off ya 
From: [email-masked]
Date: Thu, 12 Feb 2004 11:18:58 -0500
Sender: [email-masked]

--==_Exmh_1662215976P
Content-Type: text/plain; charset=us-ascii

On Thu, 12 Feb 2004 10:56:45 EST, Hector Santos said:

> For the final and last time, our implementation requires a LIST of domains
> that will trigger the DMP and SPF lookup.

If you can't understand why the rest of the world moved away from HOSTS.TXT
and started using DNS, and need to keep a local LIST, you probably shouldn't
make any recommendations about how to use DNS.

If however you *do* understand, but work with a bunch of dysfunctional
pea-brains that refuse to see the light and insist on keeping a local LIST
anyhow, you have my condolences.  Good luck on getting an actual working
product out of them.

--==_Exmh_1662215976P
Content-Type: application/pgp-signature

--==_Exmh_1662215976P--

Date: Thu, 12 Feb 2004 11:02:36 -0500
From: Keith Moore <[email-masked]>
To: "Hector Santos" <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: Best practices to avoid virus and spam
Sender: [email-masked]

> > > What period of time is this?  And how to you correct yourself? 
> > > How does the system report you that it is fixed itself?
> >
> > My guess is that an appropriate period of time would range from
> > several hours to a day or so.  most mail is retried for a few days,
> > so as long as the problem is fixed then legitimate mail should be
> > able to get through.
> 
> My research as shown at least 30 days was the average.  I can show you
> the analysis if warranted.

I don't think we're talking about the same thing.

> There are many people are incorrectly blacklisted.  Our experience
> is that top people are legitimate, quickly find out and get unlisted
> pretty quickly.    

I don't want to standardize a system that only works well for "top
people".

> I agree some standardization of the RBL sites should be done.  They
> are here to stay.

Then we don't agree.  Because there is no way in Hell that third-party
RBLs should be standardized.  And whether they're here to stay is 
irrelevant.   The job of a standards organization is to describe how to
do things well, not to encourage bad practices just because they're 
popular.

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>
Cc: <[email-masked]>
Subject: SF Support --> was Re: MX Domain Name --> Was Re: Some Ideas I would like to Bounce off ya 
Date: Thu, 12 Feb 2004 10:56:45 -0500
Organization: Santronics Software, Inc
Sender: [email-masked]

Valdis,

I am trying hard to refrain from addressing your personal shots at me.  I
would really sweet of you if you could maybe put some effort in managing
this flaming instinct of yours.

For the final and last time, our implementation requires a LIST of domains
that will trigger the DMP and SPF lookup.    It says nothing about how this
list generated, created or othewise - automatically or manually.  It had
nothing to do with it but only the fact you will need it once you implement
LMAP solutions and learn real operation results from it.   I am happy I was
able to provide to share some insight and maybe some possible implementation
cost savings when you do finally implement it.

-- 
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

----- Original Message ----- 
From: <[email-masked]>
To: "Hector Santos" <[email-masked]>
Cc: <[email-masked]>
Sent: Thursday, February 12, 2004 10:36 AM
Subject: Re: SPF Support -> Re: MX Domain Name --> Was Re: Some Ideas I
would like to Bounce off ya

On Thu, 12 Feb 2004 09:30:29 EST, Hector Santos said:

> In our work,  it became a design requirement to add a "SPF Domain List"
> holding a list of domains to check.  I will now add AOL.COM to that
default
> list.

Then your design is broken, unless you want to keep doing

> Let me check the other top dogs...  msn.com, yahoo.com, microsoft.com,
> hotmail.com, earthlink.com did not have txt records.

*THIS* for every domain that sends you mail.  If you have any sense,
you'd make a cronjob that greps through the mail logs at 2AM and does
a 'dig $FOO txt' for all domains that you got mail from that weren't
already in your list.

Remember that there's a good chance that 75% of your mail *isnt* from the
"top dogs".  And if you're relying on manual checking you might as well
go back to HOSTS.TXT.

To: Hector Santos <[email-masked]>
Cc: [email-masked]
Subject: Re: SPF Support -> Re: MX Domain Name --> Was Re: Some Ideas I would like to Bounce off ya 
From: [email-masked]
Date: Thu, 12 Feb 2004 10:36:53 -0500
Sender: [email-masked]

--==_Exmh_1399229316P
Content-Type: text/plain; charset=us-ascii

On Thu, 12 Feb 2004 09:30:29 EST, Hector Santos said:

> In our work,  it became a design requirement to add a "SPF Domain List"
> holding a list of domains to check.  I will now add AOL.COM to that default
> list.

Then your design is broken, unless you want to keep doing

> Let me check the other top dogs...  msn.com, yahoo.com, microsoft.com,
> hotmail.com, earthlink.com did not have txt records.

*THIS* for every domain that sends you mail.  If you have any sense,
you'd make a cronjob that greps through the mail logs at 2AM and does 
a 'dig $FOO txt' for all domains that you got mail from that weren't
already in your list.

Remember that there's a good chance that 75% of your mail *isnt* from the
"top dogs".  And if you're relying on manual checking you might as well
go back to HOSTS.TXT.

--==_Exmh_1399229316P
Content-Type: application/pgp-signature

--==_Exmh_1399229316P--

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>
Subject: Re: Best practices to avoid virus and spam
Date: Thu, 12 Feb 2004 09:57:52 -0500
Organization: Santronics Software, Inc
Sender: [email-masked]

----- Original Message ----- 
From: "Keith Moore" <[email-masked]>
To: "Hector Santos" <[email-masked]>
Sent: Thursday, February 12, 2004 8:48 AM
Subject: Re: Best practices to avoid virus and spam

>
> > What period of time is this?  And how to you correct yourself?  How
> > does the system report you that it is fixed itself?
>
> My guess is that an appropriate period of time would range from several
> hours to a day or so.  most mail is retried for a few days, so as long
> as the problem is fixed then legitimate mail should be able to get
> through.

My research as shown at least 30 days was the average.  I can show you the
analysis if warranted.

> REALITY is that RBLs block legitimate mail and contribute significantly
> to the failure rate of the mail system.  And no, we shouldn't
> standardize that part of reality.

I agree.  There are many people are incorrectly blacklisted.  Our experience
is that top people are legitimate, quickly find out and get unlisted pretty
quickly.    Some of the better sites will double check before adding the IP
reported by someone.

> What I'm proposing is that we encourage measures like this - that are
> based on information from reliable sources - as alternatives to some of
> the less reliable ad hoc measures that are currently in use.

I agree that RBL is like a "wish it wasn't necessary" type of thing.  I mean
I didn't add it until the SPAM issues was becoming a real problem for
customers and when I did, I cringed at the need to add a  "kludge" which
could not be done with SMTP compliant/policy or method, i.e, more official.

MAPS was the first site I believe, and it was the one we added first.  It
was a bonus that it had an "Official Reputation" or Commercial aspect to it.
It was a bonus in any promotion we would be making.  But it was off by
default.  Didn't want customers saying "Hey, whats all these pings!" to this
site?

When relays.osirusoft.com came along, I believe it offer the best results,
so good, it became the default site for our setup.

However was also so good,  it was a prime target for a dual DoS offensive
attack when SORBIG was released.     But because it was not a "official"
commecial site, with no beholdings to "customers," the site owner took
vengence and rejected all ping request to it.

So sure, why do something we really shouldn't have to be doing?

I agree some standardization of the RBL sites should be done.  They are here
to stay.

-- 
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>
Cc: <[email-masked]>
Subject: SPF Support -> Re: MX Domain Name --> Was Re: Some Ideas I would like to Bounce off ya 
Date: Thu, 12 Feb 2004 09:30:29 -0500
Organization: Santronics Software, Inc
Sender: [email-masked]

----- Original Message ----- 
From: <[email-masked]>
To: "Hector Santos" <[email-masked]>
Cc: <[email-masked]>
Sent: Thursday, February 12, 2004 5:13 AM
Subject: Re: MX Domain Name --> Was Re: Some Ideas I would like to Bounce
off ya

> It certainly looks implemented from here:

Yes, it sure does looks like it!   Can't explain why I didn't catch it or
see it earlier this morning.

Anyway, this is good news!

Ok, lets nix the MX thing for now (subject changed above).

The whole point was about how to best optimize a "DNS" lookup when the end
result is to validate sender machines as possibly the receiver machines as
well to validate the return address.

In our work,  it became a design requirement to add a "SPF Domain List"
holding a list of domains to check.  I will now add AOL.COM to that default
list.

Let me check the other top dogs...  msn.com, yahoo.com, microsoft.com,
hotmail.com, earthlink.com did not have txt records.

-- 
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

Cc: Keith Moore <[email-masked]>, =?ISO-8859-1?Q?Keld_J=F8rn_Simonsen?= <[email-masked]>, <[email-masked]>
From: Keith Moore <[email-masked]>
Subject: Re: Best practices to avoid virus and spam
Date: Thu, 12 Feb 2004 08:48:51 -0500
To: "Hector Santos" <[email-masked]>
Sender: [email-masked]

>>> No need for new protocols, closed networks etc. Maybe a need for some
>>> RBL listing virus/spam infected machines, I don't know.
>>
>> Third-party RBLs are a really, really, really bad idea.  They should 
>> be
>> categorized as Worst Practices.
>
>> ... and then refuse to accept any new connections from that source IP 
>> for
> some
>> period of time.
>
> Is this the reason for your rational RBL being really bad?

No, they have nothing to do with each other.

> Thats the problem there is no reliable way for SMTP at the protocal to
> determine if a system is a spammer.   But even it did and you automatic
> blacklisted it....
>
> What period of time is this?  And how to you correct yourself?  How 
> does the
> system report you that it is fixed itself?

My guess is that an appropriate period of time would range from several 
hours to a day or so.  most mail is retried for a few days, so as long 
as the problem is fixed then legitimate mail should be able to get 
through.

> If anything, I suggest that we take what is REALITY and help 
> standardize it,
> by cleaning it up with reason response codes and also maybe, if 
> possible,
> consolidate the sites.

You say the world REALITY as if it were somehow inherently desirable -  
or at least, not subject to change so we might as well accept it.  Of 
course there are many things that are common in reality which are not 
desirable - like drunk driving and the use of lies to justify killing 
thousands of people with warfare.  (which is far as I can tell is the 
norm rather than the exception)

Our criteria for initial standardization say nothing about reality - 
they do say things like "no known technical omissions".

REALITY is that RBLs block legitimate mail and contribute significantly 
to the failure rate of the mail system.  And no, we shouldn't 
standardize that part of reality.

>  Some systems may want to just block up dialups or
> open relays but not mailing list with no subcribe confirmations.   
> Take at
> look at some of the response tables for the RBL sites.

Again, if a site that owns its IP block says "only these machines can 
send mail from this IP block" (by putting appropriate records in the 
right in-addr.arpa zone) I'd consider that reliable.  If some third 
party makes a similar statement about a source IP address, you have no 
way of knowing whether that is reliable.

> 1) RBL are the BEST thing and ONLY thing available to makes the 
> problem more
> managable.

That reminds me of the tech support guy who realizes that his problems 
get more managable if he leaves his phone off the hook.  Perhaps this 
is the norm, but it's not worthy of standardization.

> Lets face it,  the suggestion of throwing away the idea of RBL is not
> reality or categorizing it a WORST PRACTICE is not going to stop it 
> from
> being used until you offer something that replaces it with the same
> effectiveness.

Perhaps, but we don't need to bless it.  I'm all for encouraging 
reliable ways of detecting or discouraging spam.  Especially those that 
don't impair the flexibility of the mail system.

> I agree.  But Keith until SMTP provides a standard and reliable way to 
> do
> this at the protocol level, RBL will remain to be the top method of 
> stopping
> spam.  They work!

They also make email a lot less reliable.  Of course, you are correct 
to point out that they are popular.

Face it, there is no reliable way to detect spam at the protocol level. 
  There is no reliable way to detect spam even if you look at the email 
message.  Spam is contextual.  In effect, people are demanding that the 
mail system read their minds and _anticipate_ what they will object to. 
  There is no technically sound way to do this.

There are however things we can do to discourage spam without impairing 
reliability.  One is to make mail more traceable - thus making it more 
difficult for spammers to hide.  I think it can also provide a way to 
immediately report to a network when a system on that network is 
propagating a virus.  Another thing that will help is to make it more 
difficult for spammers or viruses to use random systems to propagate 
mail.   We can define a way to allow recipients to specify criteria for 
mail that they don't want to receive.

What I'm proposing is that we encourage measures like this - that are 
based on information from reliable sources - as alternatives to some of 
the less reliable ad hoc measures that are currently in use.

Keith

Cc: Keith Moore <[email-masked]>, [email-masked]
From: Keith Moore <[email-masked]>
Subject: Re: Best practices to avoid virus and spam
Date: Thu, 12 Feb 2004 07:56:59 -0500
To: =?ISO-8859-1?Q?Keld_J=F8rn_Simonsen?= <[email-masked]>
Sender: [email-masked]

On Feb 12, 2004, at 12:53 AM, Keld Jørn Simonsen wrote:

> On Wed, Feb 11, 2004 at 10:47:00PM -0500, Keith Moore wrote:
>>
>>> 1. Always check for virus/spam before checking for valid reciepient, 
>>> or
>>> whether the mailbox is full or some such.
>>
>> Why?  Even if SMTP made it easy to do this (it doesn't), offhand it
>> seems better to perform the easiest and/or more reliable checks first.
>> It's very difficult to be sure that a message is spam - the same
>> message can be spam for one recipient and valid for another.  However
>> the authoritative MX for a domain ought to be able to tell whether an
>> recipient address from that domain is valid.  And to me it makes more
>> sense to reject a message for a reliable reason (like, this recipient
>> address is not valid) than for an unreliable reason (like, we think
>> this is spam).
>
> Yes, many things are easier to check upfront than whether this is virus
> or spam - the latter takes a full scan of the body. And then I think
> these easy checks should be done as long as they also reliably can
> classify the mail as forged, such as bogus MX and no PTR and some such.

neither of these checks can reliably classify the email as forged.

> My aim is, still, to get rid of bogus error mail.
> And as far as I see it in my own mailbox, this is mail of the
> following kinds, *only*:
>
> 1. reports that the body contained virus
> 2. reports that the user is unknown
> 3. reports that the user's account has been closed.
> 4. reports that the user's mailbox is full.

I think we'd agree that #1 should not be sent to the "sender" of the 
virus since it is so often forged.
But reports #2-4 are often useful, and they _should_ be detected before 
the server has a chance to look at the content of the message.

> I see your point about the uncertainty of spam detection, and I would 
> be happy
> enough if we only address the problem of virus. Virus can be determined
> with certainty, (although some may not be found).
>
> I am pretty happy with the current technology on spamdetection with
> spamassasin doing a good job for me.

I'm pretty unhappy with spamassassin myself, because many of its 
criteria are bogus.  It's a useful tool when tuned properly, but I see 
it as promoting bad practices.

>>> 2. Generate a specific error message, maybe we should introduce a
>>> standard error code for this, like 551 - mail rejected as virus or
>>> spam.
>>>
>>> 3. If the mail is virus or spam, then do not send it back to the
>>> sender - as this is most likely a forged address anyway. Discard it
>>> instead.  But if you must, then use the standard error mesaage as
>>> described above.
>>
>> I see no need to define a new SMTP reply code - an enhanced status 
>> code
>> should suffice.  Especially since, as you point out, when the SMTP
>> server does believe that the message is spam, bouncing the message is
>> usually the wrong thing to do anyway.
>
> I would accept not specifying a standardized best practice on error 
> reply
> for virus and spam, and only recommend discarding, but don't you think
> that some may disagree with discarding such mail, and then it would be
> good advice that they at least use som standardized error reply in 
> their
> rejects?

yes, I do think that having uniform reporting would be useful.

> I would be happy not to make a new SMTP error reply code if this can be
> done better with some other reply means. What would you suggest?

DSNs and ENHANCEDSTATUSCODES.

>>> I think with this scheme, we would have avoided alomost all of the
>>> virus/spam and also annoying error traffic .
>>>
>>> No need for new protocols, closed networks etc. Maybe a need for some
>>> RBL listing virus/spam infected machines, I don't know.
>>
>> Third-party RBLs are a really, really, really bad idea.  They should 
>> be
>> categorized as Worst Practices.
>
> I would appreciate if you could explain this in detail to me.

I have yet to see a third party RBL that didn't misrepresent what it 
was reporting - either in that the information being reported was often 
stale, or in that there was an attempt to mislead people about the 
validity of the criteria they were using.  For instance, the RBLs that 
blocked mail from machines that appeared to relay mail was a dubious 
criterion because there were valid reasons to relay mail and not all or 
even most mail from such machines was spam.  (Now with SMTP 
authentication there are fewer valid reasons).  RBLs that accept 
reports from other parties spread misinformation.  For instance, if a 
site uses spamassassin's criteria as a way to decide whether to report 
to an RBL that a particular IP is sending spam that is spreading 
unreliable information.  ("propagation of error")

>> There might be some cases where the _authoritative_ DNS zone for a 
>> MAIL
>> FROM domain or  the _authoritative_ DNS zone for the source IP address
>> of an SMTP connection says "messages that look like this are not
>> authorized".  And in some of those cases the SMTP server might be able
>> to tell that a message "looks like this" from just MAIL FROM and/or
>> RCPT TO.  In those cases it _might_ make sense for the SMTP server to
>> return 2xx in response to those commands (if they are otherwise valid)
>> just so that it can black-hole the message.  Though I think it might
>> make even more sense to return 4xx in response to those commands - or
>> just stop responding to that connection - and then refuse to accept 
>> any
>> new connections from that source IP for some period of time.  The
>> rationale here is that the SMTP server doesn't want to waste bandwidth
>> or cpu cycles by sucking down a message that's going to be discarded
>> anyway, and it does want to avoid being interrupted by a compromised
>> machine or spammer's machine if it tries again later.
>
> My analysis is that for a number of servers the CPU load stemming from
> this kind of bogus mail is not a problem.

I'm not sure what you mean by "analysis" here.  But my limited 
experience is that many mail servers spend much more time and bandwidth 
processing spam than processing useful mail.  (especially if you count 
the effort spent trying to bounce spam)

From: Arnt Gulbrandsen <[email-masked]>
To: Keld =?iso-8859-1?q?J=F8rn?= Simonsen <[email-masked]>
Subject: Re: Best practices to avoid virus and spam
Cc: [email-masked]
Date: Thu, 12 Feb 2004 12:26:30 +0100
Sender: [email-masked]

Keld Jørn Simonsen writes:
> On Thu, Feb 12, 2004 at 11:58:06AM +0100, Arnt Gulbrandsen wrote:
>
>>  Keld Jørn Simonsen writes:
>>  >There are already RBLs that blacklist dynamic IPs. Not that I like it.
>>  >Yes, there will most likely be machines that automatically report
>>  >infected machines.
>>
>>  There already are. That's how the quick-reaction blacklists get 
>>  their speed.
>>
>>  But it's not so bad. Lists such as cbl.abuseat.org and dsbl.org 
>>  don't accept reports from the general public, and they will 
>>  immediately remove an entry, by anyone's request.
>
> Yes, that may solve the problem with DHCP dynamic IP MTAs, Once you 
> have got your instance of a dynamic IP, you could check it at the RBL 
> and get it off the blacklist, which would probably happen instantly. 
> And then you can use your MTA. Would that work?

Even if an address is removed from the database immediately, the DNS 
update and caching effects generally add up to two hours delay. In 
theory it could be more.

Arnt

Date: Thu, 12 Feb 2004 12:18:22 +0100
From: Keld =?iso-8859-1?Q?J=F8rn?= Simonsen <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
Cc: [email-masked]
Subject: Re: Best practices to avoid virus and spam
Sender: [email-masked]

On Thu, Feb 12, 2004 at 11:58:06AM +0100, Arnt Gulbrandsen wrote:
> 
> Keld Jørn Simonsen writes:
> >There are already RBLs that blacklist dynamic IPs. Not that I like it. 
> >Yes, there will most likely be machines that automatically report 
> >infected machines.
> 
> There already are. That's how the quick-reaction blacklists get their speed.
> 
> But it's not so bad. Lists such as cbl.abuseat.org and dsbl.org don't 
> accept reports from the general public, and they will immediately 
> remove an entry, by anyone's request.

Yes, that may solve the problem with DHCP dynamic IP MTAs,
Once you have got your instance of a dynamic IP, you could check
it at the RBL and get it off the blacklist, which would probably
happen instantly. And then you can use your MTA. Would that work?

Best regards
Keld

From: Arnt Gulbrandsen <[email-masked]>
To: [email-masked]
Subject: Re: Best practices to avoid virus and spam
Date: Thu, 12 Feb 2004 11:58:06 +0100
Sender: [email-masked]

Keld Jørn Simonsen writes:
> There are already RBLs that blacklist dynamic IPs. Not that I like it. 
> Yes, there will most likely be machines that automatically report 
> infected machines.

There already are. That's how the quick-reaction blacklists get their speed.

But it's not so bad. Lists such as cbl.abuseat.org and dsbl.org don't 
accept reports from the general public, and they will immediately 
remove an entry, by anyone's request.

Arnt

Date: Thu, 12 Feb 2004 11:46:03 +0100
From: Keld =?iso-8859-1?Q?J=F8rn?= Simonsen <[email-masked]>
To: [email-masked]
Cc: Keld =?iso-8859-1?Q?J=F8rn?= Simonsen <[email-masked]>, Hector Santos <[email-masked]>, [email-masked]
Subject: Re: Best practices to avoid virus and spam
Sender: [email-masked]

On Thu, Feb 12, 2004 at 05:03:53AM -0500, [email-masked] wrote:
> On Thu, 12 Feb 2004 10:22:25 +0100, Keld =?iso-8859-1?Q?J=F8rn?= Simonsen said:
> 
> > > I presumed you were suggested for the RBL sites to include virus/spam
> > > infected machines.
> > 
> > Yes, I was.
> 
> Oh great.  So now in addition to my DHCP address being blocked because
> some previous incumbent was a spammer, I can be blocked because the
> previous owner of the address had a virus. (And you know *very* well that
> some fool is going to "fix" their mail hub's virus scanner to automagically
> feed reports into the RBL without human intervention).
> 
> A great step backward.

>From what? There are already RBLs that blacklist dynamic IPs.
Not that I like it. Yes, there will most likely be machines that automatically
report infected machines.

Keld

To: Hector Santos <[email-masked]>
Cc: [email-masked]
Subject: Re: MX Domain Name --> Was Re: Some Ideas I would like to Bounce off ya 
From: [email-masked]
Date: Thu, 12 Feb 2004 05:13:17 -0500
Sender: [email-masked]

--==_Exmh_1928793229P
Content-Type: text/plain; charset=us-ascii

On Thu, 12 Feb 2004 04:33:42 EST, Hector Santos said:

> I'll wait until you have implemented SPF into your SMTP software or script
> or whatever and then when you see the overhead,  then maybe^H^H^H^H^HMAYBE
> you more enlighten to the situation.

There's a bunch of things in SPF that look to be a total crock performance
wise, in particular the 'include', 'redirect', and 'exp' methods.

It may indeed be possible to recast these as MX entries.  But so far,
you haven't shown anything remotely workable.  Let's look at the worst case
for the AOL entry:

1) We need to do 1 DNS query when they connect to get the TXT record.
If one of the ip4: headers matches, we're done.

2) If none match, we're doing 1 PTR lookup.

That's it.  Two queries.  The only way you can do any better is by getting
it down to a single query - and given the constraints of the MX format,
what you're basically doing is a RBL whitelist (think about it).

> hmmm, lets see if they have it implemented yet....NOPE.   That means I can't
> add it to my server-side list of SPF supported domains yet.   Again Its only
> good for local^H^H^H^H^HLOCAL Domains.

You seem to be very confused about who publishes an SPF record, and for
who's consumption.  You don't need to keep a server-side list, *that's the
reason for the TXT record*.  You check it when you get an inbound connection
that claims to be 'MAIL FROM:<[email-masked]>'.  AOL provides that data for
your benefit.  They probably aren't checking SPF records themselves at
the moment (they ran a weekendlong test recently), mostly due to the
lack of domains supporting it.

But you'd not detect that until *your* site tried to send mail *to* AOL.
At which point they'd just try to look up your SPF record, which either
is or isn't in the DNS.  If you're adding them to a list, you're doing
something very stupid and anti-SPF - only publishing the SPF data to
sites known to check for it is a big dis-incentive for other sites to
start checking.  If you want to publish SPF, just put the damned thing
in the DNS and be done with it.  You have better things to do than
go through the 40 million .COMs and add them each to your server-side
list of things that check for SPF records (unless you're trying to
re-invent the X.400 concept of ADMD/PRMD, in which case I'll quote
Randy Bush and invite you to design your network that way ;)

It certainly looks implemented from here:

% dig aol.com txt

; <<>> DiG 9.2.3 <<>> aol.com txt
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 20101
;; flags: qr aa rd ra; QUERY: 1, ANSWER: 1, AUTHORITY: 4, ADDITIONAL: 4

;; QUESTION SECTION:
;aol.com.                       IN      TXT

;; ANSWER SECTION:
aol.com.                300     IN      TXT     "v=spf1 ip4:152.163.225.0/24 ip4:205.188.139.0/24 ip4:205.188.144.0/24 ip4:205.188.156.0/24 ip4:205.188.157.0/24 ip4:205.188.159.0/24 ip4:64.12.136.0/24 ip4:64.12.137.0/24 ip4:64.12.138.0/24 ptr:mx.aol.com ?all"

;; AUTHORITY SECTION:
aol.com.                2652    IN      NS      dns-07.ns.aol.com.
aol.com.                2652    IN      NS      dns-01.ns.aol.com.
aol.com.                2652    IN      NS      dns-02.ns.aol.com.
aol.com.                2652    IN      NS      dns-06.ns.aol.com.

;; ADDITIONAL SECTION:
dns-07.ns.aol.com.      1386    IN      A       64.12.51.132
dns-01.ns.aol.com.      3237    IN      A       152.163.159.232
dns-02.ns.aol.com.      2722    IN      A       205.188.157.232
dns-06.ns.aol.com.      3190    IN      A       149.174.211.8

http://spf.pobox.com/ lists aol.com as publishing it as well.

--==_Exmh_1928793229P
Content-Type: application/pgp-signature

--==_Exmh_1928793229P--

To: Keld =?iso-8859-1?Q?J=F8rn?= Simonsen <[email-masked]>
Cc: Hector Santos <[email-masked]>, [email-masked]
Subject: Re: Best practices to avoid virus and spam 
From: [email-masked]
Date: Thu, 12 Feb 2004 05:03:53 -0500
Sender: [email-masked]

--==_Exmh_1906651113P
Content-Type: text/plain; charset=us-ascii

On Thu, 12 Feb 2004 10:22:25 +0100, Keld =?iso-8859-1?Q?J=F8rn?= Simonsen said:

> > I presumed you were suggested for the RBL sites to include virus/spam
> > infected machines.
> 
> Yes, I was.

Oh great.  So now in addition to my DHCP address being blocked because
some previous incumbent was a spammer, I can be blocked because the
previous owner of the address had a virus. (And you know *very* well that
some fool is going to "fix" their mail hub's virus scanner to automagically
feed reports into the RBL without human intervention).

A great step backward.

--==_Exmh_1906651113P
Content-Type: application/pgp-signature

--==_Exmh_1906651113P--

From: "Hector Santos" <[email-masked]>
To: "Hector Santos" <[email-masked]>, =?iso-8859-1?Q?Keld_J=F8rn_Simonsen?= <[email-masked]>
Cc: <[email-masked]>
Subject: Re: Best practices to avoid virus and spam
Date: Thu, 12 Feb 2004 04:37:19 -0500
Organization: Santronics Software, Inc
Sender: [email-masked]

----- Original Message ----- 
From: "Hector Santos" <[email-masked]>
To: "Keld Jørn Simonsen" <[email-masked]>

> This is a very HIGH overhead mode of operation.  To prove this, check our
> SMTP statistic for December 2004 and January 2005.

> http://www.winserver.com/sslinfo       (Frame support browser required)
> http://www.winserver.com/public/antispam    (no frame support required)

ughhhhh!  Oops! meant Dec/2003  and Jan/2004.  Time to get to bed!

-- 
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>
Cc: <[email-masked]>
Subject: Re: MX Domain Name --> Was Re: Some Ideas I would like to Bounce off ya 
Date: Thu, 12 Feb 2004 04:33:42 -0500
Organization: Santronics Software, Inc
Sender: [email-masked]

----- Original Message ----- 
From: <[email-masked]>
To: "Hector Santos" <[email-masked]>
Cc: <[email-masked]>
Sent: Thursday, February 12, 2004 3:35 AM
Subject: Re: MX Domain Name --> Was Re: Some Ideas I would like to Bounce
off ya

Two things:

1)  All this is  premature. Obviously not many developers here done much
with SPF.  SO YOU WIN!

I'll wait until you have implemented SPF into your SMTP software or script
or whatever and then when you see the overhead,  then maybe^H^H^H^H^HMAYBE
you more enlighten to the situation.

2) Thanks for the SPF record for AOL.

I can now add it to my internal domain/ip filter list and also research how
that can exposed as an MX record.

hmmm, lets see if they have it implemented yet....NOPE.   That means I can't
add it to my server-side list of SPF supported domains yet.   Again Its only
good for local^H^H^H^H^HLOCAL Domains.

-- 
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

Date: Thu, 12 Feb 2004 10:22:25 +0100
From: Keld =?iso-8859-1?Q?J=F8rn?= Simonsen <[email-masked]>
To: Hector Santos <[email-masked]>
Cc: Keld =?iso-8859-1?Q?J=F8rn?= Simonsen <[email-masked]>, [email-masked]
Subject: Re: Best practices to avoid virus and spam
Sender: [email-masked]

On Thu, Feb 12, 2004 at 03:51:28AM -0500, Hector Santos wrote:
> 
> > DNS has an advntage of rapid dissemination and no specific maintenance
> > at each client site.
> 
> RBL lookups is done via a special zone at a specific DNS site.  RBL zones
> are not distributed, although they might be shared and in some cases
> brought.
> 
> I presumed you were suggested for the RBL sites to include virus/spam
> infected machines.

Yes, I was.

> Cool,  if one of them did it.  What I pointing out I would hate to see a new
> speciflc RBL site "specialized" only in listing virus infected sites.
> 
> I already do three RBL site lookups.  I don't wish to add a forth.  Thats
> all :-)

I am not sure if some of them would add this service. I think best chances are
that a specialized, or group of specialized RBL sites would come up with
this service.  Anyway another DNS lookup is IMHO not much to do, compared to
all the other checks done.

> > I think checking the sender is a good thing upfront, and that receipient
> > validation should be done very *late* in the process. The latter is
> > actually my main point: check everything else first, including the
> > body for virus and spam, and then check the receipient. This to reduce
> > the error generation for invalid reciepient, or other problems such as
> > account closed or mailbox full.
> 
> Chicken and Egg?    If you are going to reject anyone, why bother with the
> overhead?

Because of the improtance of the message sent out. One message is: this
is bogus mail, the other is: the user is not found. The latter is a
serious message for the original sender, the former is already known to the
offender.

> How you are checking for spams/virus?   During the DATA state (receive data,
> call mail filter process, examine and then return response code) or receive
> the message and then do a post analysis?

The first, via diverse filters, eg Milter or postfix' filters.

> > So my recommendation is a prioritzation of mail validation. First check
> > the mail for if it is bogus: wrong sender addresses, no PTR (host
> > unknown), unknown sender, virus in body etc. And then check for the real
> > errors.
> 
> This is a very HIGH overhead mode of operation.  To prove this, check our
> SMTP statistic for December 2004 and January 2005.
> 
> http://www.winserver.com/sslinfo       (Frame support browser required)
> http://www.winserver.com/public/antispam    (no frame support required)
> 
> On December 25, we move all our Validation logic TO AFTER the recipient was
> checked.   The result:
> 
> An average session time of 68 secs (December) was reduced to 22 secs
> (January).
> 
> Now why would I want to go back?

Yes it adds overhead but I have CPU cycles enough of my cheap 2 year old
1 GHz noname intel box. Load 0,1 with 100.000 mails a day and about
6 Mbit/s average ftp and http traffic. Runs Mandrake 9.2 and postfix
with some filters, and then spamd. Pretty vanilla setup.

best regards
Keld

From: "Hector Santos" <[email-masked]>
To: =?iso-8859-1?Q?Keld_J=F8rn_Simonsen?= <[email-masked]>
Cc: <[email-masked]>
Subject: Re: Best practices to avoid virus and spam
Date: Thu, 12 Feb 2004 03:51:28 -0500
Organization: Santronics Software, Inc
Sender: [email-masked]

----- Original Message ----- 
From: "Keld Jørn Simonsen" <[email-masked]>
To: "Hector Santos" <[email-masked]>
Sent: Thursday, February 12, 2004 1:47 AM
Subject: Re: Best practices to avoid virus and spam

> > > Maybe a need for some
> > > RBL listing virus/spam infected machines, I don't know.
> >
> > This is actually a good idea it if can be done with a response code.
Not
> > another RBL look up site :-)
>
> I don't understand what you say here, the RBLs normally work as a lookup
> in the DNS. Are you suggesting another way of distributing the list?

No.

> DNS has an advntage of rapid dissemination and no specific maintenance
> at each client site.

RBL lookups is done via a special zone at a specific DNS site.  RBL zones
are not distributed, although they might be shared and in some cases
brought.

I presumed you were suggested for the RBL sites to include virus/spam
infected machines.

Cool,  if one of them did it.  What I pointing out I would hate to see a new
speciflc RBL site "specialized" only in listing virus infected sites.

I already do three RBL site lookups.  I don't wish to add a forth.  Thats
all :-)

> Arent this what is considered good practice by many MTA administrators?

If it was, we would not such a spam problem :-)

> I think checking the sender is a good thing upfront, and that receipient
> validation should be done very *late* in the process. The latter is
> actually my main point: check everything else first, including the
> body for virus and spam, and then check the receipient. This to reduce
> the error generation for invalid reciepient, or other problems such as
> account closed or mailbox full.

Chicken and Egg?    If you are going to reject anyone, why bother with the
overhead?

One thing what was not very clear in your message.

How you are checking for spams/virus?   During the DATA state (receive data,
call mail filter process, examine and then return response code) or receive
the message and then do a post analysis?

> So my recommendation is a prioritzation of mail validation. First check
> the mail for if it is bogus: wrong sender addresses, no PTR (host
> unknown), unknown sender, virus in body etc. And then check for the real
> errors.

This is a very HIGH overhead mode of operation.  To prove this, check our
SMTP statistic for December 2004 and January 2005.

http://www.winserver.com/sslinfo       (Frame support browser required)
http://www.winserver.com/public/antispam    (no frame support required)

On December 25, we move all our Validation logic TO AFTER the recipient was
checked.   The result:

An average session time of 68 secs (December) was reduced to 22 secs
(January).

Now why would I want to go back?

-- 
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

To: Hector Santos <[email-masked]>
Cc: [email-masked]
Subject: Re: MX Domain Name --> Was Re: Some Ideas I would like to Bounce off ya 
From: [email-masked]
Date: Thu, 12 Feb 2004 03:35:30 -0500
Sender: [email-masked]

--==_Exmh_1520243626P
Content-Type: text/plain; charset=us-ascii

On Wed, 11 Feb 2004 23:36:22 EST, Hector Santos said:

> > Compute the operational cost of having to drop back to TCP
> > (if even feasible) for more responses.  It's quite possible
> > that for vt.edu you'd get at least 400 or so responses back
> > (that's just the ones I *know* about that are Unix/Linux
> > workstations that do their own SMTP handling).
> 
> First, was not suggesting for the addition of 400 records or
> whatever. That would be ludricrous.

Actually, you *were* suggesting it.

You just didn't realize it, because you refused to think it through.

> acbbaf2e.ipt.aol.com
> acbbe0d6.ipt.aol.com
> acbcca06.ipt.aol.com
> acbe9b44.ipt.aol.com
> acbec50d.ipt.aol.com
> acbec811.ipt.aol.com

> How will they do this with SPF?

We're not talking about SPF.  We're talking about stupid MX tricks.

smtp.SENDER.aol.com	MX	acbbaf2e.ipt.aol.com
				acbbe0d6.ipt.aol.com
				acbcca06.ipt.aol.com
				acbe9b44.ipt.aol.com
				acbec50d.ipt.aol.com
				acbec811.ipt.aol.com
(and so on for the 500+ systems you've seen AOL mail from in January)

That's one honking big MX list.  And that's what you proposed.  If it
isn't what you proposed, you need to learn to write what you meant.

But as to how to do it with SPF:

> Do you think they can define this in one SPF line?

aol.com.                300     IN      TXT     "v=spf1 ip4:152.163.225.0/24 ip4:205.188.139.0/24 ip4:205.188.144.0/24 ip4:205.188.156.0/24 ip4:205.188.157.0/24 ip4:205.188.159.0/24 ip4:64.12.136.0/24 ip4:64.12.137.0/24 ip4:64.12.138.0/24 ptr:mx.aol.com ?all"

Apparently, the AOL staff thinks they can do it in one line.

But we're discussing MX records, not SPF using TXT, remember?

> If so, then why can this "information" by translated into a compliant
> sub-domain syntax?

Because the SPF record is short because it allows wildcarding, and
you have to actually enumerate all the members if you're using an MX.

> If you can tell me this is not possible.  Then its a non-starter. But this
> has not been shown yet.

Your proposal *as* *stated* doesn't work for my domain, and it doesn't work
for the aol.com domain.  If you can't understand why a proposal absolutely
positively has to work for aol.com or it's a non-starter, please consider
a different line of business.

> I want you to realize what SPF will mean to you:

If you can't keep SPF and MX straight, please come back when you have a clue.

I've considered SPF.  SPF has its strong points and its weak points.

One of its strong points is that the author at least had enough clue to
realize that MX's are for one thing, and TXT records are for other things,
and not to try to encode TXT records in the MX tree.

> But you will need to add two more TXT records to cover your machine
> domain turing-police.cc.vt.edu (formatted in the same manner above).
> This allow you to use different return path address.

vt.edu	TXT	"v=spf1 ip4:128.173.0.0/16 ip4:198.82.0.0/16 ?all"

There. *Done*.  Period. End Of Discussion.  If SPF is a good idea,
the above single record is *all* that's needed.  It does my domain in
one DNS entry, it does aol.com in one DNS entry.

Unfortunately, I can't comprehend the overwhelming wisdom of the rest
of your note, as I was too unable to follow the wisdom of using masses
of MX records to replace one TXT record, but I will point this out:

> Now remember what the INTENT is:

>   To BLOCK out the spoofers or validate legitmate sending domains

> Well this INTENT is based on the flawed thinking that MOST of your
> spammers are not spoofers.

Odd CAPITALIZATION is often considered a PRIME sign of net.kookery.

--==_Exmh_1520243626P
Content-Type: application/pgp-signature

--==_Exmh_1520243626P--

Date: Thu, 12 Feb 2004 07:47:30 +0100
From: Keld =?iso-8859-1?Q?J=F8rn?= Simonsen <[email-masked]>
To: Hector Santos <[email-masked]>
Cc: Keld =?iso-8859-1?Q?J=F8rn?= Simonsen <[email-masked]>, [email-masked]
Subject: Re: Best practices to avoid virus and spam
Sender: [email-masked]

On Wed, Feb 11, 2004 at 07:00:46AM -0500, Hector Santos wrote:
> 
> > Maybe a need for some
> > RBL listing virus/spam infected machines, I don't know.
> 
> This is actually a good idea it if can be done with a response code.  Not
> another RBL look up site :-)

I don't understand what you say here, the RBLs normally work as a lookup
in the DNS. Are you suggesting another way of distributing the list?
DNS has an advntage of rapid dissemination and no specific maintenance
at each client site.

> > No need for new protocols,................
> 
> Well, we need something to address the uncontrollable abuse of anonymous
> access.  (non-authenticated sessions).  With the current SMTP protocol,  we
> use a suite of methods that offers a near 100% rejection:
> 
> 1) Add an extended Greeting response.  You will instantly stop 40% of your
> dial-up spammers.
> 
> 2) Add a machine domain literal spoof check at HELO/EHLO - 20% rejection
> 
> 3) Internal IP/DIP filter tables (DIP = return path Domain + client IP).
> This helps #5
> 
> 4) RBL Lookup
> 
> I have by default 3 sites, selected for best response times, little down
> time.
> 
> sbl.spamhaus.org
> list.dsbl.org
> bl.spamcop.net
> 
> A special lookup algorithm includes logic to temporarily excludes slow
> response sites and prioritizes the sites by best performance.
> 
> 5) LMAP based solutions, DMP and SPF implemented.

.....

Arent this what is considered good practice by many MTA administrators?
I don't see needs for protocol changes with much of this.

> So in summary, my recommendation is to follow the suggestion in RFC 2821 to
> validate the domain, IP and/or the return path AFTER you get a validated
> RCPT address.  If you do this, you will minimize your spam management
> overhead and maximize the rejection rate.

I think checking the sender is a good thing upfront, and that receipient
validation should be done very *late* in the process. The latter is
actually my main point: check everything else first, including the
body for virus and spam, and then check the receipient. This to reduce
the error generation for invalid reciepient, or other problems such as
account closed or mailbox full.

In that way we can improve the signal/noise level on receipient errors,
which to me may be a very serious errors. I administer many email lists
and I find it a major problem if a reciepient does not get the mail.
I sometimes go to the extreme of doing telephone calls to fix the
problem. Also I occasionally misspell an email address - and then I
would really like to get a notice that this happened.

So my recommendation is a prioritzation of mail validation. First check
the mail for if it is bogus: wrong sender addresses, no PTR (host
unknown), unknown sender, virus in body etc. And then check for the real
errors.

Best regards
keld

Date: Thu, 12 Feb 2004 06:53:26 +0100
From: Keld =?iso-8859-1?Q?J=F8rn?= Simonsen <[email-masked]>
To: Keith Moore <[email-masked]>
Cc: Keld =?iso-8859-1?Q?J=F8rn?= Simonsen <[email-masked]>, [email-masked]
Subject: Re: Best practices to avoid virus and spam
Sender: [email-masked]

On Wed, Feb 11, 2004 at 10:47:00PM -0500, Keith Moore wrote:
> 
> >1. Always check for virus/spam before checking for valid reciepient, or
> >whether the mailbox is full or some such.
> 
> Why?  Even if SMTP made it easy to do this (it doesn't), offhand it 
> seems better to perform the easiest and/or more reliable checks first.  
> It's very difficult to be sure that a message is spam - the same 
> message can be spam for one recipient and valid for another.  However 
> the authoritative MX for a domain ought to be able to tell whether an 
> recipient address from that domain is valid.  And to me it makes more 
> sense to reject a message for a reliable reason (like, this recipient 
> address is not valid) than for an unreliable reason (like, we think 
> this is spam).

Yes, many things are easier to check upfront than whether this is virus
or spam - the latter takes a full scan of the body. And then I think
these easy checks should be done as long as they also reliably can
classify the mail as forged, such as bogus MX and no PTR and some such.

My aim is, still, to get rid of bogus error mail. 
And as far as I see it in my own mailbox, this is mail of the
following kinds, *only*:

1. reports that the body contained virus
2. reports that the user is unknown
3. reports that the user's account has been closed.
4. reports that the user's mailbox is full.

I see your point about the uncertainty of spam detection, and I would be happy
enough if we only address the problem of virus. Virus can be determined
with certainty, (although some may not be found).

I am pretty happy with the current technology on spamdetection with
spamassasin doing a good job for me.

> >2. Generate a specific error message, maybe we should introduce a
> >standard error code for this, like 551 - mail rejected as virus or 
> >spam.
> >
> >3. If the mail is virus or spam, then do not send it back to the
> >sender - as this is most likely a forged address anyway. Discard it
> >instead.  But if you must, then use the standard error mesaage as
> >described above.
> 
> I see no need to define a new SMTP reply code - an enhanced status code 
> should suffice.  Especially since, as you point out, when the SMTP 
> server does believe that the message is spam, bouncing the message is 
> usually the wrong thing to do anyway.

I would accept not specifying a standardized best practice on error reply
for virus and spam, and only recommend discarding, but don't you think
that some may disagree with discarding such mail, and then it would be
good advice that they at least use som standardized error reply in their
rejects?

I would be happy not to make a new SMTP error reply code if this can be
done better with some other reply means. What would you suggest?

> >I think with this scheme, we would have avoided alomost all of the
> >virus/spam and also annoying error traffic .
> >
> >No need for new protocols, closed networks etc. Maybe a need for some
> >RBL listing virus/spam infected machines, I don't know.
> 
> Third-party RBLs are a really, really, really bad idea.  They should be 
> categorized as Worst Practices.

I would appreciate if you could explain this in detail to me.

> There might be some cases where the _authoritative_ DNS zone for a MAIL 
> FROM domain or  the _authoritative_ DNS zone for the source IP address 
> of an SMTP connection says "messages that look like this are not 
> authorized".  And in some of those cases the SMTP server might be able 
> to tell that a message "looks like this" from just MAIL FROM and/or 
> RCPT TO.  In those cases it _might_ make sense for the SMTP server to 
> return 2xx in response to those commands (if they are otherwise valid) 
> just so that it can black-hole the message.  Though I think it might 
> make even more sense to return 4xx in response to those commands - or 
> just stop responding to that connection - and then refuse to accept any 
> new connections from that source IP for some period of time.  The 
> rationale here is that the SMTP server doesn't want to waste bandwidth 
> or cpu cycles by sucking down a message that's going to be discarded 
> anyway, and it does want to avoid being interrupted by a compromised 
> machine or spammer's machine if it tries again later.

My analysis is that for a number of servers the CPU load stemming from
this kind of bogus mail is not a problem.

But adding to a best practice specification that you could block the IP
for a limited period would be fine to me. I think sendmail patches for
this behaviour already exist.

best regards
keld

From: "Hector Santos" <[email-masked]>
To: =?iso-8859-1?Q?Keld_J=F8rn_Simonsen?= <[email-masked]>
Cc: <[email-masked]>
Subject: Re: Best practices to avoid virus and spam
Date: Thu, 12 Feb 2004 00:11:44 -0500
Organization: Santronics Software, Inc
Sender: [email-masked]

----- Original Message ----- 
From: "Keith Moore" <[email-masked]>
To: "Keld Jørn Simonsen" <[email-masked]>
Cc: "Keith Moore" <[email-masked]>; <[email-masked]>
Sent: Wednesday, February 11, 2004 10:47 PM
Subject: Re: Best practices to avoid virus and spam

> > No need for new protocols, closed networks etc. Maybe a need for some
> > RBL listing virus/spam infected machines, I don't know.
>
> Third-party RBLs are a really, really, really bad idea.  They should be
> categorized as Worst Practices.

> ... and then refuse to accept any new connections from that source IP for
some
> period of time.

Is this the reason for your rational RBL being really bad?

Thats the problem there is no reliable way for SMTP at the protocal to
determine if a system is a spammer.   But even it did and you automatic
blacklisted it....

What period of time is this?  And how to you correct yourself?  How does the
system report you that it is fixed itself?

If anything, I suggest that we take what is REALITY and help standardize it,
by cleaning it up with reason response codes and also maybe, if possible,
consolidate the sites.   Some systems may want to just block up dialups or
open relays but not mailing list with no subcribe confirmations.   Take at
look at some of the response tables for the RBL sites.

Here is my take.  Two sides of the coins:

1) RBL are the BEST thing and ONLY thing available to makes the problem more
managable.  Systems that don't use RBL have big spam abuse problems.  You
might be able to manage them in other ways, but for the majority, it is a
god-send.  The fact that the GROWTH of these systems have sky-rocketed is
proof enough that it lot of people think it is a GOOD IDEA to solve the
problems that currently could not be addressed at SMTP.

2) Unfortunately, spammers LEARN from RBL databases which systems are open
relay/proxies.   Studies have shown on average a reported site will correct
the system within 30 days.   This is the double-edge sword of any central
registry system.

Lets face it,  the suggestion of throwing away the idea of RBL is not
reality or categorizing it a WORST PRACTICE is not going to stop it from
being used until you offer something that replaces it with the same
effectiveness.

> The rationale here is that the SMTP server doesn't want to waste bandwidth
> or cpu cycles by sucking down a message that's going to be discarded
> anyway, and it does want to avoid being interrupted by a compromised
> machine or spammer's machine if it tries again later.

I agree.  But Keith until SMTP provides a standard and reliable way to do
this at the protocol level, RBL will remain to be the top method of stopping
spam.  They work! Knocking out atleast 60% of your connections. That is why
they exist in GREAT popularity.

In addition, the new wave is the networking of "intelligence"   What one
system learns can benefit others in the network.

I am just providing input so that this statement of yours is not taken
seriously as gospel.

-- 
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>
Cc: <[email-masked]>
Subject: Re: MX Domain Name --> Was Re: Some Ideas I would like to Bounce off ya 
Date: Wed, 11 Feb 2004 23:36:22 -0500
Organization: Santronics Software, Inc
Sender: [email-masked]

Ok, now, that wasn't so hard, was it?  Thank you. :-)

----- Original Message ----- 
From: <[email-masked]>
To: "Hector Santos" <[email-masked]>
Cc: <[email-masked]>; <[email-masked]>
Sent: Tuesday, February 10, 2004 10:46 PM
Subject: Re: MX Domain Name --> Was Re: Some Ideas I would like to Bounce
off ya

On Tue, 10 Feb 2004 17:21:24 EST, Hector Santos said:

>>         smtp.SENDER.vt.edu        ip: 128.173.54.129

> Compute how many of these you can get back in a single
> 512-byte DNS response (hint - go back and look at
> WHY AOL round-robins their stuff the way they do).

Right.  But I do think yet this is a reason to nix the discussion or idea
yet. Go to my
original message and zoom to the bottom "Potential Problems" where I
discussed this the topic of large expanded list.

Question:

How do you think AOL will describe their SPF records?

> Compute the operational cost of having to drop back to TCP
> (if even feasible) for more responses.  It's quite possible
> that for vt.edu you'd get at least 400 or so responses back
> (that's just the ones I *know* about that are Unix/Linux
> workstations that do their own SMTP handling).

First, was not suggesting for the addition of 400 records or
whatever. That would be ludricrous.

Let me simplify this by summarize what I am seeking:

Is it possible to expose or translate into a specially crafted yet 100%
compliant MX domain name format the total possible SPF record
information which describes the lookup rules for cross checking sender
machine(s)?

> This is a non-starter unless you find a way to do the query such
> that it returns one packet or less of data (for multiple reasons).

I think that is what my original question attempted to poise.

Look, the issue is this:

LMAP based proposal implementations will be VERY VERY expensive.  It is
quite obvious now that the technical analysis of its implications have
not been fully thought out and explored.   Just take a look at the SPF
source code!!!  Most developers will need an API just to implement it
correctly!  It is not for the faint hearted.  The DNS workload will
be awful on the server-side and across the network!  DoS attacks are
MORE a reality than you think.

My proposal raises the idea whether it is FEASIBLE under the following
fundamental premise of:

    100% current MX implementation COMPLIANCE and COMPATIBILITY

to:

Expose the same or similar information attempted to be advertised by
LMAP proposals into a special crafted MX sub-domain naming nomenclature
in order to eliminate for multiple TXT lookups and followup lookups as
reguired by SPF like proposals.

The idea is not as foreign as you might think.

AOL uses a sub-domain format to identify the IP of the sending machine.
I don't know how consistent this is, but it definitely be used as
cross-check.   Here are a few examples:

acbbaf2e.ipt.aol.com
acbbe0d6.ipt.aol.com
acbcca06.ipt.aol.com
acbe9b44.ipt.aol.com
acbec50d.ipt.aol.com
acbec811.ipt.aol.com

The first sub-domain is a hex representation of the connecting IP. I got
atleast 500 or so of these systems in the month of January.

Yes, these are not MX records.  But the idea is the same.

Yes, the AOL large type systems will have to define a way to
advertise their sending machines.

How will they do this with SPF?

Do you think they can define this in one SPF line?

If so, then why can this "information" by translated into a compliant
sub-domain syntax?

If you can tell me this is not possible.  Then its a non-starter. But this
has not been shown yet.

I want you to realize what SPF will mean to you:

For example, for your system, assuming class C,  DMP allows for masking:

  *.in-addr._smtp-client.vt.edu              TXT   DMP=DENY   ; global
default
  *.54.173.128.in-addr._smtp-client.vt.edu   TXT   DMP=ALLOW  ; allowed
senders

But you will need to add two more TXT records to cover your machine
domain turing-police.cc.vt.edu (formatted in the same manner above).
This allow you to use different return path address.

For SPF, it would be TXT records (not 100% sure for your setup):

  _spf.vt.edu   TXT  "v=spf1 a:turing-police.cc.vt.edu"
  vt.edu        TXT  "v=spf1 a:turing-police.cc.vt.edu"

or with IP4:

  _spf.vt.edu   TXT  "v=spf1 ip4:128.173.54/16 -all"
  vt.edu        TXT  "v=spf1 ip4:128.173.54/16 -all"

This basically says that when a SPF TXT lookup for the return path
domain vt.edu is made, the v=spf1 record will say that only this
turing-police.cc.vt.edu or that set of Ip addresses are valid sender
machines.

I want you to realize what this will will mean in practical terms based
on real first-hand experience in real operations.

It means 2 or more text records lookups.  Its not even a standard yet
and we already have BACKWARD compatibility issues with two different types
of
lookups. Why not just stick with _SPF sub-domain?   Poor. simply poor.

Now remember what the INTENT is:

   To BLOCK out the spoofers or validate legitmate sending domains

Well this INTENT is based on the flawed thinking that MOST of your
spammers are not spoofers.

The fact or reality is that atleast a good bit of your spams are from
spoofers with spoofs or bad domains.

The result?

   The majority of your LMAP lookups will fail putting a major
   dent on non-authority and authority DNS lookups.

Now since the RETURN-PATH is still an IMPORTANT identity of the system,
which requirements or concepts that the domain exist, a MX lookup "may"
be able to provide ALL the information required.

We do a suite of test:

    RBL  A
    DMP  TXT with possible recursive alternative domains lookups
    SPF  TXT with follow up A, MX or PTR lookups
    CBV  MX

RBL is the best current method to stop spammers. Thats a non-starter
to attempt or think this will ever go away or be replaced by something
else.  No proposal will replace this.

With DMP/SPF, we had to make it an option to just check LOCAL domains
spoofs because the overhead was TOO HIGH in external domain checks.

MX is used for the CBV.  Others may use a C/R or other return-path
validations
methods.

So if it is possible to expose the sender machines information in MX defined
host
or group of host sub-domain names,  not only do you get the same results of
DMP/SPF but you also GET to do able to do a return address validatation.

I hope this better explains it.

What I want to know it is makes sense or possible to put expose this
information
on the MX records. That is all I am asking.   Saying MX is for RECEIVERS
only
is not an answer.  Saying AOL presents a problem, hence a non-starter, is
not
a complete reason yet to nix the idea.

Sure, there will ALSO be the "extended systems" where it is difficult, but
if atleast 80% of the systems can support this, well, that could mean it
is feasible or warrant more investigation.

All I am saying is that LMAP is a mistake! It adds a COMPLICATED layer
of technology with a high overhead which addresses what ULTIMATELY will
be a short term problem.

We added it. I speak with emperical results behind me.  I can see the
total picture in action and the bottom line is:

        boy, it would be sweet if ALL the redundant lookups can be
        minimized!

Its likes going to the grocery store 5 times to pick up 1 item at a time
knowing all along that the 5th item may be ultimately necessary and would
of made the first 4 trips unnecessary!

Why not pick it up all on one trip? or better yet, pick up item #5
first and save money not buying items 1 to 4?

-- 
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

Cc: Keith Moore <[email-masked]>, [email-masked]
From: Keith Moore <[email-masked]>
Subject: Re: Best practices to avoid virus and spam
Date: Wed, 11 Feb 2004 22:47:00 -0500
To: =?ISO-8859-1?Q?Keld_J=F8rn_Simonsen?= <[email-masked]>
Sender: [email-masked]

> 1. Always check for virus/spam before checking for valid reciepient, or
> whether the mailbox is full or some such.

Why?  Even if SMTP made it easy to do this (it doesn't), offhand it 
seems better to perform the easiest and/or more reliable checks first.  
It's very difficult to be sure that a message is spam - the same 
message can be spam for one recipient and valid for another.  However 
the authoritative MX for a domain ought to be able to tell whether an 
recipient address from that domain is valid.  And to me it makes more 
sense to reject a message for a reliable reason (like, this recipient 
address is not valid) than for an unreliable reason (like, we think 
this is spam).

> 2. Generate a specific error message, maybe we should introduce a
> standard error code for this, like 551 - mail rejected as virus or 
> spam.
>
> 3. If the mail is virus or spam, then do not send it back to the
> sender - as this is most likely a forged address anyway. Discard it
> instead.  But if you must, then use the standard error mesaage as
> described above.

I see no need to define a new SMTP reply code - an enhanced status code 
should suffice.  Especially since, as you point out, when the SMTP 
server does believe that the message is spam, bouncing the message is 
usually the wrong thing to do anyway.

> I think with this scheme, we would have avoided alomost all of the
> virus/spam and also annoying error traffic .
>
> No need for new protocols, closed networks etc. Maybe a need for some
> RBL listing virus/spam infected machines, I don't know.

Third-party RBLs are a really, really, really bad idea.  They should be 
categorized as Worst Practices.

There might be some cases where the _authoritative_ DNS zone for a MAIL 
FROM domain or  the _authoritative_ DNS zone for the source IP address 
of an SMTP connection says "messages that look like this are not 
authorized".  And in some of those cases the SMTP server might be able 
to tell that a message "looks like this" from just MAIL FROM and/or 
RCPT TO.  In those cases it _might_ make sense for the SMTP server to 
return 2xx in response to those commands (if they are otherwise valid) 
just so that it can black-hole the message.  Though I think it might 
make even more sense to return 4xx in response to those commands - or 
just stop responding to that connection - and then refuse to accept any 
new connections from that source IP for some period of time.  The 
rationale here is that the SMTP server doesn't want to waste bandwidth 
or cpu cycles by sucking down a message that's going to be discarded 
anyway, and it does want to avoid being interrupted by a compromised 
machine or spammer's machine if it tries again later.

Date: Wed, 11 Feb 2004 13:24:09 -0500
From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: =?ISO-8859-1?Q?Keld_J=F8rn_Simonsen?= <[email-masked]>
CC: [email-masked]
Subject: Re: Best practices to avoid virus and spam
Sender: [email-masked]

Keld Jørn Simonsen wrote:
> On Wed, Feb 11, 2004 at 09:18:42AM -0500, Bruce Lilly wrote:

> For what I describe, it is only necessary to do it at final delivery,
> when this is permissable anyway. My idea is to swop the check for bad
> data with the checks for valid user and mailbox overflow and such. 
> This would then reduce the amount of error messages for 'no such user'
> or 'mailbox full' - as these errors for the bogus mail will be
> eliminated.

Here's the problem: your SMTP receiver gets a connection from a client.
The session is started, MAIL FROM, RCPT TO (which you don't check), and
DATA.  At the end of the message, you check for virus/spam/whatever and
find none, Then you get around to checking the recipients and find that
(e.g.) 3 out of 12 are invalid.  You can't send a 2yz return code for
DATA, because you can't deliver to invalid addresses.  You have to send
a [45]yz response.  And as pointed out in the 2821 section that I referred
you to, that presents a problem for the sender (*which* address(es) were
responsible for the failure?).  If you deliver to the valid recipients
and the sender resends (perhaps guessing which recipients were invalid,
or sending multiple individual copies), those recipients will receive
duplicates.

Specifically in the case of the recent "MyDoom" variants, the worm uses
an internal SMTP client engine, so on detecting an invalid recipient at
the RCPT TO stage and returning a 5yz code immediately, there is no
delivery and no bounce, as there is no intermediate SMTP relay.  The
bounces in those recent cases are caused precisely by systems that
operate as you suggest; i.e. they do not check for invalid recipients
at RCPT TO.

>>The text following a 554 response can be embellished.  Ideally any 5yz
>>response other than the ones specifically mentioned in 2821 should only
>>be used if some suitable ESMTP extension is negotiated.  There may be
>>some old SMTP clients that do not handle any codes other than the
>>specified legal ones correctly.
> 
> 
> Well, it could be handled in another way, as also described in response
> to Hector.

If the sender uses EHLO and agrees to whatever extension corresponds to
additional error codes.  Otherwise you should fall back to the standard
code, viz. 554.

>>>Well, I wanted as final receiving MTA to discard the message, the sender
>>>address is bogus anyway.
>>
>>Not necessarily -- your virus/spam detector might have provided a false
>>positive to legitimate mail with a valid sender envelope address. Once
>>an intermediate MTA has issued a 2yz response to DATA and subsequently
>>encounters an error on transfer to a downstream SMTP server, it is
>>obligated to send a non-delivery message to the specified sender envelope
>>address (if not null).
> 
> 
> Yes, false negatives are always something to consider. But they will
> happen anyway (with current spam technology such as spamassassin).
> A very small error rate is tolerable, at least to me.
> 
> I am not talking about intermediate MTAs, only final delivery.

If there's an intermediate MTA, there will be a bounce (that is a good
thing in the case of legitimate mail!).  It also applies to spam, etc.,
so the joe-job victim will still get the bounce.  Note that below you
advocate use of intermediate relays, which essentially guarantees a
bounce message.

>>That might be true for legitimate mail, but you can't prevent a
>>spammer or virus author from using HELO rather than EHLO.  Servers
>>MUST support HELO (2821 sect. 4.1.1.1).  Not all clients or servers
>>that support some ESMTP extensions support all extensions.
> 
> 
> Yes, spammers may use HELO. But do we care for spammers?
> I don't:-) So we need not treat them well...

You may have missed the point.  You are obliged to support HELO, and
if a spammer or virus author chooses to use plain SMTP, no mechanism
that depends on an extension will be effective.  And as email is a
critical application and there are many SMTP servers in use, not
all of which support ESMTP, you cannot (if you wish to be compliant
with RFCs 821 and 2821) simply refuse to accept HELO.  You could
of course run a non-compliant server at the risk of being isolated
from those sending with plain SMTP.  You could also become a hermit
and take up residence in an isolated cave.

>>That should be clear if a delivery status notification (DSN) is used.
>>If a DSN is not being used, either plain SMTP is being used or the
>>DSN extension is not supported.
> 
> 
> Hmm, I don't think we have seen spammers use DSN yet, but they may do,
> just to increase the chaos they generate.

The spammers don't, but if a bounce is generated, DSN is the preferred
format if the necessary ESMTP extensions are available.  I've seen
both DSN bounces and non-DSN bounces.

> IP-addresses would be my choice of identification.
> I would not guarantee dynamic addresses to be able to run an MTA.
> 
> Removal of the blocking could be done just by some web interface,
> However, it should not be doable by software, only by manual
> intervention.
> 
> Some thoughts should be done to caveats, I am not fully sure if it would
> work, but would like to see it in action. How do those professional 
> firms do it, that sell such blacklists? And there are projects that
> already does this RBL service, although not well maintained.

There have been horror stories of people who have been inappropriately
blacklisted and who have been unable to be removed from the respective
blacklist.  And there are blacklist operators who have no compunction
about blacklisting an entire ISPs address range because one user had a
worm.  This is a very real problem with blacklists.

> True, they are names. But then people should avoid names like ".com".

Why? Because Microsoft's programmers are too lazy or too incompetent
to check file content to determine type? That's not a very compelling
reason to avoid a name. It may be a compelling reason to avoid
Microsoft's products, but that's another matter.

> And yes, they use .zip - to me that means that .zip files are not
> useable anymore on the net. We have to adapt to this hostile world,
> and not sending .com or .zip files is one of the small offers
> we may need to do.

Nonsense. It means that software suppliers need to pay attention to
the rules, which have been clearly stated, with background information,
for well over a decade.  It also means that users need to take
responsibility for their actions.  If you aren't sure who sent a
file, don't try to run it as an executable!  People should learn that
as small children; "don't accept candy from a stranger".  Certainly
I was taught that many decades ago -- one wonders how so many people
managed to have irresponsible parents; but that's also another matter.

> You are right, but we could stop them from arriving in our mailboxes,
> at least in the more knowledgeable users mailboxes (including the users
> that are wise enough to choose the right ISPs). What I describe does
> avoid code red and MyDoom to arrive in *my mailbox* and even others
> bogus error mail that *I* sent them virus could be avoided in my own
> mailbox.  That is, I think this could be the result, but I invite others
> to analyse whether that is true.

It does not.  If you run no Microsoft products, have never sent spam,
and run your own MTA as you suggest, you may still receive bounces
from somebody else's MTA  because some clueless user looked at a
virus-infected message in the preview pane of an unpatched version of
Microsoft Outlook.  You have no control over the configuration of
somebody else's SMTP server, and you can't force some clueless user
to keep up with a constant stream of band-aid patches from Microsoft
(which in any event provide no protection against the next day's
exploits (http://www.eeye.com/html/Research/Upcoming/index.html)).

> True. But there are also other users that do update their AVS on a daily basis.
> And you get what you deserve if you dont protect yourself against virus.

The problem is that users with no susceptibility experience the collateral
damage from Microsoft's weapons of mass email destruction, such as via
the bounces that you're complaining about.  You can't force any of
your suggestions on others; of what you've suggested, about the best
you can do is to try to convince SMTP receiver operators to return 554
at the end of DATA if a virus is detected.  And there are dangers with
false positives.

> One idea is that the customer on a dynamic IP always should use the MTA
> of the ISP - who then does proper scanning of incoming mail for virus
> and spam. If the dynamic IP customers don't, then good luck!

Some ISPs require that, and enforce it by blocking traffic to port 25
except from their internal servers.  I'm not aware of any that do that
who also scan for virus/spam.  That can be annoying to legitimate users
who occasionally need to work around problems with the ISPs servers or
DNS cache.  It doesn't slow down miscreants, who can easily find an open
relay or web proxy on a different port.

That also enforces a different model for SMTP transfer from the original;
effectively the ISP is an SMTP relay (hopefully not an open one).

>>Not all SMTP clients are MTAs (RFC 2821 section 2.3.8). MUAs can be SMTP
>>clients, and an MUA on a machine with a dynamic address (like this one)
>>is not at all unusual.
> 
> 
> Yes, dynamic IP SMTP agents are not unusual. I once did that too, but
> now I got a fixed address ADSL for my home machine.
> Anyway what do you propose for such animals?

SMTP AUTH seems to work, using the ISP's ESMTP relay (which supports AUTH)
with a MUA client that also supports SMTP AUTH.  However, it is not practical
for MTA-MTA transfers, and an SMTP receiver can't readily tell if a client
is an MUA or an MTA.

> What about your ISP, do they not provide you with a mail relay service?
> Most ISPs do.

My ISPs do. Neither blocks outbound port 25 (which I occasionally have to
use to work around problems).  One requires SMTP AUTH, another appears to
use a different authentication mechanism.  As noted above, use of such an
intermediate relay can be a contributing factor to the bounce problem.
I occasionally have to connect to the Internet via a network that does
block outbound port 25 traffic, and that is a nuisance.

Date: Wed, 11 Feb 2004 17:07:31 +0100
From: Keld =?iso-8859-1?Q?J=F8rn?= Simonsen <[email-masked]>
To: [email-masked]
Subject: Re: Best practices to avoid virus and spam
Sender: [email-masked]

On Wed, Feb 11, 2004 at 09:18:42AM -0500, Bruce Lilly wrote:
> 
> Keld Jørn Simonsen wrote:
> > On Wed, Feb 11, 2004 at 04:57:19AM -0500, Bruce Lilly wrote:
> 
> >>Your first suggestion conflicts with RFC 2821.
> > 
> > 
> > I think it would be OK to check it with the original receipt, or at the
> > final receipt. So "always" was not directed towards any intermediate MTAs.
> > Would it then still be in conflict with RFC 2821? Which section?
> 
> The conflict regards delaying the error return for invalid recipient
> until after DATA (RFC 2821 section 3.3).  See the paragraph near the
> bottom of page 18 beginning with "However, in practice".

OK, I will (did not do it yet)

> The paragraph which follows that one makes it clear that an SMTP
> server isn't supposed to grope about in the data (message), but is
> simply supposed to transfer it based on envelope information. It
> is probably OK for a server performing final delivery to do so. It
> is more difficult for a plain SMTP server to determine if it is the
> first MTA to handle a message, though a server using the submission
> protocol (RFC 2476) could make such an assumption.

For what I describe, it is only necessary to do it at final delivery,
when this is permissable anyway. My idea is to swop the check for bad
data with the checks for valid user and mailbox overflow and such. 
This would then reduce the amount of error messages for 'no such user'
or 'mailbox full' - as these errors for the bogus mail will be
eliminated.

> > I did not mean to especially use 551, but a new code such as 555.
> > My idea was to get rid of all these different error messages
> > that I have tried to list in my filters. I have collected more than 300 different 
> > messages essentially for the same thing. I think there should be a new
> > response error code just for this:
> > 555 - we do not accept virus or spam here.
> > 554 is too vague for the reason.
> 
> The text following a 554 response can be embellished.  Ideally any 5yz
> response other than the ones specifically mentioned in 2821 should only
> be used if some suitable ESMTP extension is negotiated.  There may be
> some old SMTP clients that do not handle any codes other than the
> specified legal ones correctly.

Well, it could be handled in another way, as also described in response
to Hector.

> >>>3. If the mail is virus or spam, then do not send it back to the
> >>>sender - as this is most likely a forged address anyway. Discard it
> >>>instead.  But if you must, then use the standard error mesaage as
> >>>described above. 
> >>
> >>If SMTP were still universally used as intended (i.e. sender's MUA
> >>connects to recipients SMTP server), that might work.  However, in
> >>practice there is usually some intermediate SMTP or submission
> >>protocol server.  If rejection takes place during final delivery,
> >>an intermediate server (now acting as SMTP client to final delivery
> >>SMTP server) has no choice but to send some sort of non-delivery
> >>message (RFC 2821 section 4.4).
> > 
> > 
> > Well, I wanted as final receiving MTA to discard the message, the sender
> > address is bogus anyway.
> 
> Not necessarily -- your virus/spam detector might have provided a false
> positive to legitimate mail with a valid sender envelope address. Once
> an intermediate MTA has issued a 2yz response to DATA and subsequently
> encounters an error on transfer to a downstream SMTP server, it is
> obligated to send a non-delivery message to the specified sender envelope
> address (if not null).

Yes, false negatives are always something to consider. But they will
happen anyway (with current spam technology such as spamassassin).
A very small error rate is tolerable, at least to me.

I am not talking about intermediate MTAs, only final delivery.

> >>So in many cases, there will still be a non-delivery message, and
> >>in the absence of some means to verify authorization to use the
> >>specified sender envelope address, such messages will continue to
> >>cause collateral damage.  [E]SMTP AUTH provides authentication, but
> >>as it is an extension, it can do nothing for plain SMTP transactions.
> > 
> > 
> > Plain SMTP transactions are rare, almost all transactions that 
> > I see on my mail server are ESMTP. SMTP is under 1 promille.
> 
> That might be true for legitimate mail, but you can't prevent a
> spammer or virus author from using HELO rather than EHLO.  Servers
> MUST support HELO (2821 sect. 4.1.1.1).  Not all clients or servers
> that support some ESMTP extensions support all extensions.

Yes, spammers may use HELO. But do we care for spammers?
I don't:-) So we need not treat them well...
> 
> > I understand that almost the same amount of error mail will be
> > generated. The idea, however, is that the receiver of the error
> > mail will have a much better understanding of which kind of error mail
> > it is
> 
> That should be clear if a delivery status notification (DSN) is used.
> If a DSN is not being used, either plain SMTP is being used or the
> DSN extension is not supported.

Hmm, I don't think we have seen spammers use DSN yet, but they may do,
just to increase the chaos they generate. Then however, if we have
determined that the mail is spam or virus, I think we should not honour
any additional requirements the sender has asked us to do. The mail is
malicious, and we cannot trust anything in it - the best thing
we can do is to discard it.

> > Another way to handle some of this is to have RBLs of infected
> > machines, and then reject/discard the mail upfront.
> 
> How do you propose to identify such machines? IP address won't work
> (DHCP), and domain names won't work (easily forged).  How do you
> propose avoiding yet more collateral damage? How do you propose
> removing a machine from an RBL when it has been cleaned?

IP-addresses would be my choice of identification.
I would not guarantee dynamic addresses to be able to run an MTA.

Removal of the blocking could be done just by some web interface,
However, it should not be doable by software, only by manual
intervention.

Some thoughts should be done to caveats, I am not fully sure if it would
work, but would like to see it in action. How do those professional 
firms do it, that sell such blacklists? And there are projects that
already does this RBL service, although not well maintained.

> > I use general rejection of .exe .pif .scr .com etc extentions,
> > this really catches most of the virus. And I really cannot see
> > a valid reason for sending these attachment types by mail,
> > which cannot be done in another more secure way.
> 
> Those are not "types", they are names.  A text file discussing a
> particular domain or the company it represents might very well
> have a name ending in ".com".  Moreover, the virus writers are already
> ahead of you; they are now packaging their payloads as compressed
> archives (e.g. test.zip).

True, they are names. But then people should avoid names like ".com".
And yes, they use .zip - to me that means that .zip files are not
useable anymore on the net. We have to adapt to this hostile world,
and not sending .com or .zip files is one of the small offers
we may need to do. That is a very minor cost in my eyes.

> > I think that the updated antivirus info generally gets widely
> > distributed in about a day and a half. So yes, the virus have about that
> > time to disseminate. Still we see MyDoom and other vira floating around
> > a long time after that outbreak period. This could be stopped.
> 
> It won't stop as long as there are machines with gross fundamental
> security flaws and incompetent users. Code Red is still active...

You are right, but we could stop them from arriving in our mailboxes,
at least in the more knowledgeable users mailboxes (including the users
that are wise enough to choose the right ISPs). What I describe does
avoid code red and MyDoom to arrive in *my mailbox* and even others
bogus error mail that *I* sent them virus could be avoided in my own
mailbox.  That is, I think this could be the result, but I invite others
to analyse whether that is true.

> > This would give a much shorter
> > time, as antivirus info is something that is distributed and fetched by
> > the individual users on a daily basis
> 
> Which users?  There are users who have no AV software installed and
> there are others who haven't updated in literally years.

True. But there are also other users that do update their AVS on a daily basis.
And you get what you deserve if you dont protect yourself against virus.

> > RBLs are also cheaper to employ as the mail can be discarded upfront,
> 
> But they unacceptably cause legitimate mail to be discarded.

Yes, that is a risk, that we should minimize. 

> > Yes, here RBLs would also be a bonus.
> > Anyway just discarding mails with attachments of said types would
> > get you almost all of the virus.
> 
> You cannot determine a "type" by a "name"; they are two unrelated
> things.

In principle you are right, but in practice it works fine.

> > Yes, you may have a point there, I have not looked into that specific issue
> > of how stable the ip addresses of infected machines are.
> > Anyway that would then reflect on the ISPs that they have spammers
> > or virus infected machines among their customers, and then it is the
> > ISP's problem to avoid that. 
> 
> Most ISPs do have customers with infected machines. How do you propose
> that they prevent that?

One idea is that the customer on a dynamic IP always should use the MTA
of the ISP - who then does proper scanning of incoming mail for virus
and spam. If the dynamic IP customers don't, then good luck! For static IP
customers, well they better not be sending spam/virus - but if they do,
they would only hurt themselves.

> > If you have a machine with a fixed ip-address, however, then there
> > would not be any problem for that MTA. And MTAs on dynamic IP is
> > questionable anyhow.
> 
> Not all SMTP clients are MTAs (RFC 2821 section 2.3.8). MUAs can be SMTP
> clients, and an MUA on a machine with a dynamic address (like this one)
> is not at all unusual.

Yes, dynamic IP SMTP agents are not unusual. I once did that too, but
now I got a fixed address ADSL for my home machine.
Anyway what do you propose for such animals?
What about your ISP, do they not provide you with a mail relay service?
Most ISPs do.

Best regards
keld

Date: Wed, 11 Feb 2004 16:14:10 +0100
From: Keld =?iso-8859-1?Q?J=F8rn?= Simonsen <[email-masked]>
To: Hector Santos <[email-masked]>
Cc: Keld =?iso-8859-1?Q?J=F8rn?= Simonsen <[email-masked]>, [email-masked]
Subject: Re: Best practices to avoid virus and spam
Sender: [email-masked]

On Wed, Feb 11, 2004 at 09:31:59AM -0500, Hector Santos wrote:
> 
> ----- Original Message ----- 
> From: "Keld Jørn Simonsen" <[email-masked]>
> To: <[email-masked]>
> Cc: "Keld Jørn Simonsen" <[email-masked]>
> Sent: Wednesday, February 11, 2004 6:12 AM
> Subject: Re: Best practices to avoid virus and spam
> 
> >> Bruce Lilly said:
> 
> If this is a final destination, then I don't see how you are conflict with
> RFC 2821.

Good

> However, if you don't perform a check, how do you know its a final
> destination?

I don't relay, so this is the final destination.

> So by not checking the RCPT, you are in conflict with 2821 iff you are not
> and trusted session. In which case, you are an open relay.

I would like to check RCPT later on.

> Based on your suggestions, I can only presume the client is authenticated or
> trusted in which case, the SMTP tradition is to accept all mail with no need
> to check for RCPT.   Your SMTP router logic (not SMTP sender logic) will
> then determine if the mail is local or remote.
> 
> In this case, you promote BOUNCES for invalid final destinations.  If you
> want to get rid of the bounces, you should validate the local user recipient
> immediately.    If this was a routed message, well, again, you should not be
> accepting the mail this unless the client is authenticated.

yes, probably

> > My idea is to check for whether the mail is a genuine post, or whether
> > it is a bogus mail. And only treat is as a genuine mail if we could not
> > classify it as a bogus mail. Then we can test it for all the normal
> > error conditions.
> 
> It is perfectly ok to have a DATA mail filter that provides successful 250
> response or refusal 45x or 55x DATA responses.  But technically,  this
> should only be done for final destination.   If you are doing this without
> checking for final destination, then you may be conflictive with
> passthru/relay mail concepts.

As said, I do not allow relaying.

> > Well, I wanted as final receiving MTA to discard the message, the sender
> > address is bogus anyway.
> 
> You need to do more with protocol level checks than rely on mail filters.
> You will not catch it all, and most certainly you are contributing the
> bounce problem.  What if you can't detect a virus email but the  recipient
> (local or remote) was bad? A bounce is generated and now you are part of the
> SORBIG generation viruses.

(I believe the name is SOBIG.)

Checks for valid receipts must be done, but after the check for whether
the mail is infected with virus, or spam.

> You really need to do more upfront checking.

Why? My idea is to reduce the error messages to genuine errors,
and filter the virus and spam before doing the genuine mail tests.

> > I understand that almost the same amount of error mail will be
> > generated. The idea, however, is that the receiver of the error
> > mail will have a much better understanding of which kind of error mail
> > it is, and for those receivers that are confident that they do not
> > send out virus or spam, they can just ignore the bogus error mail.
> 
> Yes, it will help both receivers and the senders to see a more descriptive
> dynamic rejection responses.  When we added our DATA mail filter, it was
> completely apparent for the need to have the mail filter be able to pass an
> description rejection response.  But it was not important if was 555, just
> whether it was 45x or 55x.  45x told the sender "Hey, ok to try again"  55x
> said "don't even bother to try again chump"

55x codes may need investigation, and then correction of errors at the
sender side. The idea with 555 was that the nominal sender (which was
forged) can do away with the mail without investigating it. 
The other 55x codes should probably be investigated.
> 
> I think the point here is that SMTP is not meant to be in the mail
> intepretation business.  If you have a dynamic DATA mail filter, well, that
> integration needs be flexible enough so that your filter can pass back a
> response code and/or description response.

That may be, that it should not be a 55x code. 
Then it could be a best practice with a recommended error string, so
that we do not get those myriads of different error strings for
rejection of a mail because of virus or spam content.
> 
> If a 555 is proposed for SMTP, I suggest that it not specific to "reject due
> to Virus/Spam" but make it two.
> 
>         555  Mail rejection to do system policy mail integrity violation
>         556  Mail rejection to do system policy no-solicitating (SPAM)
> violation
> 
> Also, see Carl Malamud's No-solicitation draft, it uses a 550 for rejection

Yes, several possibilities exist.
> 
> >  Yes, the virus and spam mail would be delivered to the MTA's.
> > But not to the MUAs. And that is the main problem. I don't care too much
> > about what my machine does, as long as the load is beyond 0,1 and I dont
> > have to waste my personal time on looking at the mails. And my mail
> > server (which is also an ftp and http-server and other things)
> > happily chunks away on 100.000 emails a day, of which 99.800
> > are bogus mail, which get rejected or discarded. No sweat.
> 
> You have very valid point which I brought up here just recently.  The
> "cencensus" (at atleast 2-3 more vocal ones) is that it is a "user problem."

We are all users.

> I don't agree.  But that point it is too late.
> 
> However, with your DATA approach,  it is still something that can only be
> done on BEHALF on the final destination user with his permission.

Yes, thats understood.

> Nonetheless,  you are assuming you will always be able detect the virus at
> the DATA stage.  What if you don't?   What if you were too late in getting
> an AVS update with the new monster email virus propagating the SMTP bounce
> system?

This will happen. But I have reduced my problem considerably. Very.
Based on my current experiences, it would be diminished by over 95 %.

> What it basically meam you need to more at the protocol level.  Like you
> said, the majority of your unsolicited mail is spoofed.  Therefore, if you
> can STOP it at the protocol level, you have just fixed a major problem!

Probably, but it may be a much greater violation of SMTP than what I
am proposing. And what I am proposing would be effective enough, IMHO.

> > RBLs are also cheaper to employ as the mail can be discarded upfront,
> > while antivirus/antispam requires scanning of the data, which costs more
> > CPU, and network bandwidth.
> 
> And just imagine, if you rejected the RCPT user, you don't even need to do
> an RBL, DMP, SPF, CBV or DATA filter overhead process <g>

But I generated a bogus error report - "no such user", that can end up in
the forged senders inbox. Sometimes the forged sender is me :-(

Best regards
Keld

From: "Hector Santos" <[email-masked]>
To: =?iso-8859-1?Q?Keld_J=F8rn_Simonsen?= <[email-masked]>, <[email-masked]>
Subject: Re: Best practices to avoid virus and spam
Date: Wed, 11 Feb 2004 09:31:59 -0500
Organization: Santronics Software, Inc
Sender: [email-masked]

----- Original Message ----- 
From: "Keld Jørn Simonsen" <[email-masked]>
To: <[email-masked]>
Cc: "Keld Jørn Simonsen" <[email-masked]>
Sent: Wednesday, February 11, 2004 6:12 AM
Subject: Re: Best practices to avoid virus and spam

>> Bruce Lilly said:

> > 1. Always check for virus/spam before checking for valid reciepient, or
>> > whether the mailbox is full or some such.
>
>> Checking recipient can be done at the time of RCPT TO.  Checking for
>>
>> Your first suggestion conflicts with RFC 2821.

> I think it would be OK to check it with the original receipt, or at the
> final receipt. So "always" was not directed towards any intermediate MTAs.
> Would it then still be in conflict with RFC 2821? Which section?

If this is a final destination, then I don't see how you are conflict with
RFC 2821.

However, if you don't perform a check, how do you know its a final
destination?

So by not checking the RCPT, you are in conflict with 2821 iff you are not
and trusted session. In which case, you are an open relay.

Based on your suggestions, I can only presume the client is authenticated or
trusted in which case, the SMTP tradition is to accept all mail with no need
to check for RCPT.   Your SMTP router logic (not SMTP sender logic) will
then determine if the mail is local or remote.

In this case, you promote BOUNCES for invalid final destinations.  If you
want to get rid of the bounces, you should validate the local user recipient
immediately.    If this was a routed message, well, again, you should not be
accepting the mail this unless the client is authenticated.

> My idea is to check for whether the mail is a genuine post, or whether
> it is a bogus mail. And only treat is as a genuine mail if we could not
> classify it as a bogus mail. Then we can test it for all the normal
> error conditions.

It is perfectly ok to have a DATA mail filter that provides successful 250
response or refusal 45x or 55x DATA responses.  But technically,  this
should only be done for final destination.   If you are doing this without
checking for final destination, then you may be conflictive with
passthru/relay mail concepts.

>> > 2. Generate a specific error message, maybe we should introduce a
>> > standard error code for this, like 551 - mail rejected as virus or
spam.
>>
>> So here's another conflict (though returning 554 would be acceptable).

> I did not mean to especially use 551, but a new code such as 555.
> My idea was to get rid of all these different error messages
> that I have tried to list in my filters.

You could use extended smtp codes to do this.

> Well, I wanted as final receiving MTA to discard the message, the sender
> address is bogus anyway.

You need to do more with protocol level checks than rely on mail filters.
You will not catch it all, and most certainly you are contributing the
bounce problem.  What if you can't detect a virus email but the  recipient
(local or remote) was bad? A bounce is generated and now you are part of the
SORBIG generation viruses.

You really need to do more upfront checking.

> I understand that almost the same amount of error mail will be
> generated. The idea, however, is that the receiver of the error
> mail will have a much better understanding of which kind of error mail
> it is, and for those receivers that are confident that they do not
> send out virus or spam, they can just ignore the bogus error mail.

Yes, it will help both receivers and the senders to see a more descriptive
dynamic rejection responses.  When we added our DATA mail filter, it was
completely apparent for the need to have the mail filter be able to pass an
description rejection response.  But it was not important if was 555, just
whether it was 45x or 55x.  45x told the sender "Hey, ok to try again"  55x
said "don't even bother to try again chump"

I think the point here is that SMTP is not meant to be in the mail
intepretation business.  If you have a dynamic DATA mail filter, well, that
integration needs be flexible enough so that your filter can pass back a
response code and/or description response.

If a 555 is proposed for SMTP, I suggest that it not specific to "reject due
to Virus/Spam" but make it two.

        555  Mail rejection to do system policy mail integrity violation
        556  Mail rejection to do system policy no-solicitating (SPAM)
violation

Also, see Carl Malamud's No-solicitation draft, it uses a 550 for rejection

> The bogus error mail is - at least for me personnally - the origins of
> almost all unwanted email today.

About 60-80%!  <g>

>  Yes, the virus and spam mail would be delivered to the MTA's.
> But not to the MUAs. And that is the main problem. I don't care too much
> about what my machine does, as long as the load is beyond 0,1 and I dont
> have to waste my personal time on looking at the mails. And my mail
> server (which is also an ftp and http-server and other things)
> happily chunks away on 100.000 emails a day, of which 99.800
> are bogus mail, which get rejected or discarded. No sweat.

You have very valid point which I brought up here just recently.  The
"cencensus" (at atleast 2-3 more vocal ones) is that it is a "user problem."

I don't agree.  But that point it is too late.

However, with your DATA approach,  it is still something that can only be
done on BEHALF on the final destination user with his permission.

My approach is to concentrate more on the PROTOCOL level with a high focus
on "smtp compliance" removing all ideas of "mail intepretation" by the SMTP
server.   If you follow some of the suggestions I provided to you,  you will
get rid of a major percentage of the problem.

> Still what I am after is limiting the emails that
> end up in the user email inboxes (like my own), and I think what I
> suggest would get a long way to reach that goal.

True. We proved it! :-)   But you need to keep in mind it has to be for
final destination.  The merit of whether you should trap passthru mail, is I
think, a new topic of discussion, but I think it is too dangerous to play
with.

Nonetheless,  you are assuming you will always be able detect the virus at
the DATA stage.  What if you don't?   What if you were too late in getting
an AVS update with the new monster email virus propagating the SMTP bounce
system?

What it basically meam you need to more at the protocol level.  Like you
said, the majority of your unsolicited mail is spoofed.  Therefore, if you
can STOP it at the protocol level, you have just fixed a major problem!

> RBLs are also cheaper to employ as the mail can be discarded upfront,
> while antivirus/antispam requires scanning of the data, which costs more
> CPU, and network bandwidth.

And just imagine, if you rejected the RCPT user, you don't even need to do
an RBL, DMP, SPF, CBV or DATA filter overhead process <g>

> Anyway just discarding mails with attachments of said types would
> get you almost all of the virus.

A major problem with AVS software with SORBIG.  They learned and MYDOOM was
better handle now.   SORBIG was OUT OF CONTROL!

> If you have a machine with a fixed ip-address, however, then there
> would not be any problem for that MTA. And MTAs on dynamic IP is
> questionable anyhow.

Add the extended greeting you will knock out atleast 40% of those suckers!

See our antispam statistics at http://www.winserver.com/sslinfo

A near zero no-spam problem and 100% no mydoom problem.

-- 
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

Date: Wed, 11 Feb 2004 09:18:42 -0500
From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: =?ISO-8859-1?Q?Keld_J=F8rn_Simonsen?= <[email-masked]>
CC: [email-masked]
Subject: Re: Best practices to avoid virus and spam
Sender: [email-masked]

Keld Jørn Simonsen wrote:
> On Wed, Feb 11, 2004 at 04:57:19AM -0500, Bruce Lilly wrote:

>>Your first suggestion conflicts with RFC 2821.
> 
> 
> I think it would be OK to check it with the original receipt, or at the
> final receipt. So "always" was not directed towards any intermediate MTAs.
> Would it then still be in conflict with RFC 2821? Which section?

The conflict regards delaying the error return for invalid recipient
until after DATA (RFC 2821 section 3.3).  See the paragraph near the
bottom of page 18 beginning with "However, in practice".

The paragraph which follows that one makes it clear that an SMTP
server isn't supposed to grope about in the data (message), but is
simply supposed to transfer it based on envelope information. It
is probably OK for a server performing final delivery to do so. It
is more difficult for a plain SMTP server to determine if it is the
first MTA to handle a message, though a server using the submission
protocol (RFC 2476) could make such an assumption.

>>551 is reserved for "user not local". 550 is the error code for rejection
>>for policy reasons, but is not listed as valid after DATA.  554 (transaction
>>failed) is valid after DATA.
>>
>>So here's another conflict (though returning 554 would be acceptable).
> 
> 
> I did not mean to especially use 551, but a new code such as 555.
> My idea was to get rid of all these different error messages
> that I have tried to list in my filters. I have collected more than 300 different 
> messages essentially for the same thing. I think there should be a new
> response error code just for this:
> 555 - we do not accept virus or spam here.
> 554 is too vague for the reason.

The text following a 554 response can be embellished.  Ideally any 5yz
response other than the ones specifically mentioned in 2821 should only
be used if some suitable ESMTP extension is negotiated.  There may be
some old SMTP clients that do not handle any codes other than the
specified legal ones correctly.

>>>3. If the mail is virus or spam, then do not send it back to the
>>>sender - as this is most likely a forged address anyway. Discard it
>>>instead.  But if you must, then use the standard error mesaage as
>>>described above. 
>>
>>If SMTP were still universally used as intended (i.e. sender's MUA
>>connects to recipients SMTP server), that might work.  However, in
>>practice there is usually some intermediate SMTP or submission
>>protocol server.  If rejection takes place during final delivery,
>>an intermediate server (now acting as SMTP client to final delivery
>>SMTP server) has no choice but to send some sort of non-delivery
>>message (RFC 2821 section 4.4).
> 
> 
> Well, I wanted as final receiving MTA to discard the message, the sender
> address is bogus anyway.

Not necessarily -- your virus/spam detector might have provided a false
positive to legitimate mail with a valid sender envelope address. Once
an intermediate MTA has issued a 2yz response to DATA and subsequently
encounters an error on transfer to a downstream SMTP server, it is
obligated to send a non-delivery message to the specified sender envelope
address (if not null).

>>So in many cases, there will still be a non-delivery message, and
>>in the absence of some means to verify authorization to use the
>>specified sender envelope address, such messages will continue to
>>cause collateral damage.  [E]SMTP AUTH provides authentication, but
>>as it is an extension, it can do nothing for plain SMTP transactions.
> 
> 
> Plain SMTP transactions are rare, almost all transactions that 
> I see on my mail server are ESMTP. SMTP is under 1 promille.

That might be true for legitimate mail, but you can't prevent a
spammer or virus author from using HELO rather than EHLO.  Servers
MUST support HELO (2821 sect. 4.1.1.1).  Not all clients or servers
that support some ESMTP extensions support all extensions.

> I understand that almost the same amount of error mail will be
> generated. The idea, however, is that the receiver of the error
> mail will have a much better understanding of which kind of error mail
> it is

That should be clear if a delivery status notification (DSN) is used.
If a DSN is not being used, either plain SMTP is being used or the
DSN extension is not supported.

> Another way to handle some of this is to have RBLs of infected
> machines, and then reject/discard the mail upfront.

How do you propose to identify such machines? IP address won't work
(DHCP), and domain names won't work (easily forged).  How do you
propose avoiding yet more collateral damage? How do you propose
removing a machine from an RBL when it has been cleaned?

> I use general rejection of .exe .pif .scr .com etc extentions,
> this really catches most of the virus. And I really cannot see
> a valid reason for sending these attachment types by mail,
> which cannot be done in another more secure way.

Those are not "types", they are names.  A text file discussing a
particular domain or the company it represents might very well
have a name ending in ".com".  Moreover, the virus writers are already
ahead of you; they are now packaging their payloads as compressed
archives (e.g. test.zip).

> I think that the updated antivirus info generally gets widely
> distributed in about a day and a half. So yes, the virus have about that
> time to disseminate. Still we see MyDoom and other vira floating around
> a long time after that outbreak period. This could be stopped.

It won't stop as long as there are machines with gross fundamental
security flaws and incompetent users. Code Red is still active...

> Another way to limit the outbreak (or incumbination?) period is to 
> apply RBLs for the infected machines.

See above.

> This would give a much shorter
> time, as antivirus info is something that is distributed and fetched by
> the individual users on a daily basis

Which users?  There are users who have no AV software installed and
there are others who haven't updated in literally years.

> RBLs are also cheaper to employ as the mail can be discarded upfront,

But they unacceptably cause legitimate mail to be discarded.

> Yes, here RBLs would also be a bonus.
> Anyway just discarding mails with attachments of said types would
> get you almost all of the virus.

You cannot determine a "type" by a "name"; they are two unrelated
things.

> Yes, you may have a point there, I have not looked into that specific issue
> of how stable the ip addresses of infected machines are.
> Anyway that would then reflect on the ISPs that they have spammers
> or virus infected machines among their customers, and then it is the
> ISP's problem to avoid that. 

Most ISPs do have customers with infected machines. How do you propose
that they prevent that?

> If you have a machine with a fixed ip-address, however, then there
> would not be any problem for that MTA. And MTAs on dynamic IP is
> questionable anyhow.

Not all SMTP clients are MTAs (RFC 2821 section 2.3.8). MUAs can be SMTP
clients, and an MUA on a machine with a dynamic address (like this one)
is not at all unusual.

From: "Hector Santos" <[email-masked]>
To: =?iso-8859-1?Q?Keld_J=F8rn_Simonsen?= <[email-masked]>, <[email-masked]>
Subject: Re: Best practices to avoid virus and spam
Date: Wed, 11 Feb 2004 07:00:46 -0500
Organization: Santronics Software, Inc
Sender: [email-masked]

----- Original Message ----- 
From: "Keld Jørn Simonsen" <[email-masked]>
To: <[email-masked]>
Sent: Wednesday, February 11, 2004 1:36 AM
Subject: Best practices to avoid virus and spam

>
> I have got three pieces of advice:
>
> 1. Always check for virus/spam before checking for valid reciepient, or
> whether the mailbox is full or some such.

This requires the DATA state to be reached.  Adds additional bandwidth.

In my research, spammers LOVE systems that accept the RCTP whether its
validated or not or delayed until the DATA state is reached (i.e.,
YAHOO.COM).

I have emperical statistical information that shows an major increase in
spam when we turned off Local User Validation and added the logic to a DATA
mail filter.  Message refused because the recipient was not validated.  The
result,  increase in spam attacks.    The DATA stage mail filter was the
first thing we ever did to address the spam problem.  It set the stage for
the additional work of a suite of anti-spam techniques that we finally
added.  I talk about this suite of methods below.

So IMO best to reject it as soon as possible.

> 2. Generate a specific error message, maybe we should introduce a
> standard error code for this, like 551 - mail rejected as virus or spam.

This is probably ok. But I always debated the idea of giving spammers more
data or information which can help them improve their spam engines.  What we
do is (by default), is issue a:

        55x Message not accepted by filter

Also, in my view,  DATA interpretation is outside the scope of the SMTP
transport system.  So if a data hook is provided to an outside non-smtp
process checking the DATA,  you should allow the hook return the response
code or response.

> 3. If the mail is virus or spam, then do not send it back to the
> sender - as this is most likely a forged address anyway. Discard it
> instead.  But if you must, then use the standard error mesaage as
> described above.

Good point not to resenting bounces with virus attachments.

However, I think that if your system used instance rejection, then there is
no need for a bounce.

In my view,  the SORBIG generation bounce/virus problem mostly begins with
delayed processing systems that accept all messages for delay validation and
processing.  This will not be necessary if the system performed using
dynamic/instance rejection at the protocol level.

The segment of my customers who have the higher incident of this problem who
those who have turned off the "[X] Local User Validation" option which
delays the user validation at the gate, create a bounce message which is
usually enabled because your SMTP server accepted the message and by RFC
2821 requirements, once you do that you must provide a reason for the
non-delivery of the message.

> Maybe a need for some
> RBL listing virus/spam infected machines, I don't know.

This is actually a good idea it if can be done with a response code.  Not
another RBL look up site :-)

There seem to be some effort or discussion in ASRG to standardize the
response codes (127.0.0.x) issued by the many DNSRBL sites.

> I think with this scheme, we would have avoided alomost all of the
> virus/spam and also annoying error traffic .

> No need for new protocols,................

Well, we need something to address the uncontrollable abuse of anonymous
access.  (non-authenticated sessions).  With the current SMTP protocol,  we
use a suite of methods that offers a near 100% rejection:

1) Add an extended Greeting response.  You will instantly stop 40% of your
dial-up spammers.

2) Add a machine domain literal spoof check at HELO/EHLO - 20% rejection

3) Internal IP/DIP filter tables (DIP = return path Domain + client IP).
This helps #5

4) RBL Lookup

I have by default 3 sites, selected for best response times, little down
time.

sbl.spamhaus.org
list.dsbl.org
bl.spamcop.net

A special lookup algorithm includes logic to temporarily excludes slow
response sites and prioritizes the sites by best performance.

5) LMAP based solutions, DMP and SPF implemented.

These proposals are basically "DIP" checkers - validating the return path
domain with the client IP address (sender machine).  At the moment, the more
complex to implement in SMTP SPF method seems to be the more popular for
easier administration and more flexibility.  However, LMAP based solutions
suffer from external domain DNS TXT record lookup delays/overhead.  So we
only use it for LOCAL domains validation only.  It is very very good in this
area.  It stops those spoofers from using your valid domain in HELO/HELO and
also the return path address in the MAIL FROM:

So for example, we allows only 208.247.131.* machines to use our local
domains, i.e., winserver.com.   If someone tried to use this domain from
another client machine IP address, both DMP and SPF will catch this quite
nicely.

Related to this,  for many systems that send mail to us, including this
mailing list, I will add a DIP filter in #3.  For example:

accept if %DIP%        = ietf.org[132.151.6.22]
accept if %DIP%        = mail.imc.org[208.184.76.39]
accept if %DIP%        = imc.org[208.184.76.39]

I find it ironic these IETF related site do not have DMP or SPF support yet
to help the communicate help support these ideas.  <g>

6) If all the above fails,  we do a CBV (Call Back Verifier) which makes an
attempt to see if your MAIL FROM: (return path) address is valid by
performing an SMTP-based call back.   This blocks atleast 80% of the CBV
attempts with very little error.

7) Finally, if it gets to the DATA state, a sysop or 3rd party developed
hook is spawn to perform a mail/data filter check.  The filter returns the
response code to be used.

Now, I should note that accept for #1 and #2, no method that requires
overhead of any kind is performed until the RCPT state and an recipient
address is provided that would be otherwise a deliverable address.

In other words, if RCPT is refused there is NO need to do any check of any
kind because you are not accepting the transaction.

So in summary, my recommendation is to follow the suggestion in RFC 2821 to
validate the domain, IP and/or the return path AFTER you get a validated
RCPT address.  If you do this, you will minimize your spam management
overhead and maximize the rejection rate.

I know you have a high throughput rate,  if possible, my recommendation is
to enable  "Local user Validation" in the operation and you will instantly
remove your bounce problems.   SMTP only requires a bounce if you accept it
in the first place or it fails to send something.  Of couse, this implies
that you have a tight external mail/relay policy.  So if you don't accept
the message, you won't have any post validation bounce/rejection message
overhead.

Hope some of these tips provide some insight with dealing with spam at the
current SMTP level.

-- 
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

Date: Wed, 11 Feb 2004 12:12:37 +0100
From: Keld =?iso-8859-1?Q?J=F8rn?= Simonsen <[email-masked]>
To: [email-masked]
Cc: Keld =?iso-8859-1?Q?J=F8rn?= Simonsen <[email-masked]>
Subject: Re: Best practices to avoid virus and spam
Sender: [email-masked]

On Wed, Feb 11, 2004 at 04:57:19AM -0500, Bruce Lilly wrote:
> Keld Jørn Simonsen wrote:
> 
> > I would advice that we recommend some best practice procedures,
> > hopefully to be implemented in the MTAs software products of the world. 
> > Maybe we should write an RFC on this.
> > 
> > I have got three pieces of advice:
> > 
> > 1. Always check for virus/spam before checking for valid reciepient, or
> > whether the mailbox is full or some such.
> 
> Checking recipient can be done at the time of RCPT TO.  Checking for
> a virus can only be done after the body has been collected (end of
> DATA).  RFC 2821 discusses difficulties that arise if rejection due
> to invalid recipients is postponed until after DATA.  Checking body
> content is only appropriate at initial (submission or first SMTP server)
> or final stages of delivery -- an intermediate SMTP transfer is supposed
> to be transparent to the data, and intermediaries have no business
> making policy decisions for distant senders or recipients [of course,
> there shouldn't be any intermediate relays except in unusual situations
> (N.B. there may be intermediate servers as noted below which are not
> relays, they are responsible for the sender's or recipient's mail domain)].
> 
> Your first suggestion conflicts with RFC 2821.

I think it would be OK to check it with the original receipt, or at the
final receipt. So "always" was not directed towards any intermediate MTAs.
Would it then still be in conflict with RFC 2821? Which section?

My idea is to check for whether the mail is a genuine post, or whether
it is a bogus mail. And only treat is as a genuine mail if we could not
classify it as a bogus mail. Then we can test it for all the normal
error conditions.

> > 2. Generate a specific error message, maybe we should introduce a
> > standard error code for this, like 551 - mail rejected as virus or spam.
> 
> 551 is reserved for "user not local". 550 is the error code for rejection
> for policy reasons, but is not listed as valid after DATA.  554 (transaction
> failed) is valid after DATA.
> 
> So here's another conflict (though returning 554 would be acceptable).

I did not mean to especially use 551, but a new code such as 555.
My idea was to get rid of all these different error messages
that I have tried to list in my filters. I have collected more than 300 different 
messages essentially for the same thing. I think there should be a new
response error code just for this:
555 - we do not accept virus or spam here.
554 is too vague for the reason.
> 
> > 3. If the mail is virus or spam, then do not send it back to the
> > sender - as this is most likely a forged address anyway. Discard it
> > instead.  But if you must, then use the standard error mesaage as
> > described above. 
> 
> If SMTP were still universally used as intended (i.e. sender's MUA
> connects to recipients SMTP server), that might work.  However, in
> practice there is usually some intermediate SMTP or submission
> protocol server.  If rejection takes place during final delivery,
> an intermediate server (now acting as SMTP client to final delivery
> SMTP server) has no choice but to send some sort of non-delivery
> message (RFC 2821 section 4.4).

Well, I wanted as final receiving MTA to discard the message, the sender
address is bogus anyway.

> So in many cases, there will still be a non-delivery message, and
> in the absence of some means to verify authorization to use the
> specified sender envelope address, such messages will continue to
> cause collateral damage.  [E]SMTP AUTH provides authentication, but
> as it is an extension, it can do nothing for plain SMTP transactions.

Plain SMTP transactions are rare, almost all transactions that 
I see on my mail server are ESMTP. SMTP is under 1 promille.

> While there's no conflict, in practice your third suggestion will
> have limited effect.

I understand that almost the same amount of error mail will be
generated. The idea, however, is that the receiver of the error
mail will have a much better understanding of which kind of error mail
it is, and for those receivers that are confident that they do not
send out virus or spam, they can just ignore the bogus error mail.

The bogus error mail is - at least for me personnally - the origins of
almost all unwanted email today.

> > I think with this scheme, we would have avoided alomost all of the
> > virus/spam and also annoying error traffic .
> 
> Maybe.  But that's already history.  There are plenty of open relays,
> and so long as there is at least one, any future virus/worm could
> use that relay, and the payload would be delivered (either to the
> intended recipient or to the forged return address).  Likewise for
> spam. Closing open relays is therefore a prerequisite to successful
> implementation of any scheme such as you have proposed.

Yes, the virus and spam mail would be delivered to the MTA's.
But not to the MUAs. And that is the main problem. I don't care too much
about what my machine does, as long as the load is beyond 0,1 and I dont
have to waste my personal time on looking at the mails. And my mail
server (which is also an ftp and http-server and other things)
happily chunks away on 100.000 emails a day, of which 99.800
are bogus mail, which get rejected or discarded. No sweat.

It does not matter really about the open relays, there will always be
some, and of cause these should be reported, and appropiate
RBLs be employed. Still what I am after is limiting the emails that
end up in the user email inboxes (like my own), and I think what I
suggest would get a long way to reach that goal.

> > No need for new protocols, closed networks etc. Maybe a need for some
> > RBL listing virus/spam infected machines, I don't know.
> 
> Aside from the need to close open relays and the need for verification
> that the specified sender envelope address is appropriate, there are
> other issues that make your scheme impractical.  Scanning body content
> can only take place at the end of DATA during acceptance (or rejection)
> for delivery.  The minimum timeout for a response at end of
> DATA is 10 minutes. Assuming all clients waited for at least that
> long (some do not), that timeout imposes an upper limit on the
> amount of time available for processing to determine whether or
> not there is a basis for rejection on policy grounds.  If a large
> number of simultaneous connections are being handled by an SMTP
> receiver, that may present an unacceptable processing burden.

That is true, it will take more CPU time to process.
But the idea is that the resulting amount of mail will take less human
time to process. This is a tradeoff, and as long as I have the CPU
cycles, I know myself how I want to have it done. CPU is rather cheap
these days.

Another way to handle some of this is to have RBLs of infected
machines, and then reject/discard the mail upfront.
> 
> >From a practical point of view, anti-virus signatures generally only
> come out after a virus has propagated fairly widely, so for some
> time after a virus is unleashed it will be delivered undetected even
> by those who keep their AV software up-to-date.  The same considerations
> (must take place after DATA is complete at final delivery SMTP
> receiver, requires processing overhead, arms-race considerations)
> apply to spam.

I use general rejection of .exe .pif .scr .com etc extentions,
this really catches most of the virus. And I really cannot see
a valid reason for sending these attachment types by mail,
which cannot be done in another more secure way.

I think that the updated antivirus info generally gets widely
distributed in about a day and a half. So yes, the virus have about that
time to disseminate. Still we see MyDoom and other vira floating around
a long time after that outbreak period. This could be stopped.

Another way to limit the outbreak (or incumbination?) period is to 
apply RBLs for the infected machines. This would give a much shorter
time, as antivirus info is something that is distributed and fetched by
the individual users on a daily basis, while RBLs are dynamic and effective
almost immediately (save zone copies) via the DNS. 

RBLs are also cheaper to employ as the mail can be discarded upfront,
while antivirus/antispam requires scanning of the data, which costs more
CPU, and network bandwidth.
> 
> In the case of virus content, it's difficult to justify requiring a
> site with no Microsoft platforms to install, maintain, and operate
> anti-virus software due to poor design of Microsoft products and/or
> inadequately trained or incompetent users of same.

Yes, here RBLs would also be a bonus.
Anyway just discarding mails with attachments of said types would
get you almost all of the virus.

> One generally
> doesn't allow toddlers to operate plastic tricycles on busy highways;
> likewise those not competent to practice minimal security and those
> whose systems fail to meet some minimum security standards shouldn't
> be allowed on the "information superhighway" (apologies for
> resurrecting that tired cliché).  An RBL is unlikely to work; the
> existence of ISPs that use a pool of IP addresses for dialup and
> broadband subscribers via DHCP means that IP addresses cannot be
> used to identify infected machines -- one minute a particular IP
> address might be assigned to a virus-infected machine running an
> insecure operating system and operated by a clueless user, and the
> next minute it might be assigned to a firewalled system operated
> by a security expert.  If there's an answer to this aspect of the
> spam/virus problem short of universal IPv6 deployment and static IP
> addresses everywhere, it will probably have to be a policy/licensing
> (analogous to a driver's license) matter rather than a purely
> technical solution.

Yes, you may have a point there, I have not looked into that specific issue
of how stable the ip addresses of infected machines are.
Anyway that would then reflect on the ISPs that they have spammers
or virus infected machines among their customers, and then it is the
ISP's problem to avoid that. 

If you have a machine with a fixed ip-address, however, then there
would not be any problem for that MTA. And MTAs on dynamic IP is
questionable anyhow.

best regards
keld

Date: Wed, 11 Feb 2004 04:57:19 -0500
From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: =?ISO-8859-1?Q?Keld_J=F8rn_Simonsen?= <[email-masked]>
CC: [email-masked]
Subject: Re: Best practices to avoid virus and spam
Sender: [email-masked]

Keld Jørn Simonsen wrote:

> I would advice that we recommend some best practice procedures,
> hopefully to be implemented in the MTAs software products of the world. 
> Maybe we should write an RFC on this.
> 
> I have got three pieces of advice:
> 
> 1. Always check for virus/spam before checking for valid reciepient, or
> whether the mailbox is full or some such.

Checking recipient can be done at the time of RCPT TO.  Checking for
a virus can only be done after the body has been collected (end of
DATA).  RFC 2821 discusses difficulties that arise if rejection due
to invalid recipients is postponed until after DATA.  Checking body
content is only appropriate at initial (submission or first SMTP server)
or final stages of delivery -- an intermediate SMTP transfer is supposed
to be transparent to the data, and intermediaries have no business
making policy decisions for distant senders or recipients [of course,
there shouldn't be any intermediate relays except in unusual situations
(N.B. there may be intermediate servers as noted below which are not
relays, they are responsible for the sender's or recipient's mail domain)].

Your first suggestion conflicts with RFC 2821.

> 2. Generate a specific error message, maybe we should introduce a
> standard error code for this, like 551 - mail rejected as virus or spam.

551 is reserved for "user not local". 550 is the error code for rejection
for policy reasons, but is not listed as valid after DATA.  554 (transaction
failed) is valid after DATA.

So here's another conflict (though returning 554 would be acceptable).

> 3. If the mail is virus or spam, then do not send it back to the
> sender - as this is most likely a forged address anyway. Discard it
> instead.  But if you must, then use the standard error mesaage as
> described above. 

If SMTP were still universally used as intended (i.e. sender's MUA
connects to recipients SMTP server), that might work.  However, in
practice there is usually some intermediate SMTP or submission
protocol server.  If rejection takes place during final delivery,
an intermediate server (now acting as SMTP client to final delivery
SMTP server) has no choice but to send some sort of non-delivery
message (RFC 2821 section 4.4).

So in many cases, there will still be a non-delivery message, and
in the absence of some means to verify authorization to use the
specified sender envelope address, such messages will continue to
cause collateral damage.  [E]SMTP AUTH provides authentication, but
as it is an extension, it can do nothing for plain SMTP transactions.

While there's no conflict, in practice your third suggestion will
have limited effect.

> I think with this scheme, we would have avoided alomost all of the
> virus/spam and also annoying error traffic .

Maybe.  But that's already history.  There are plenty of open relays,
and so long as there is at least one, any future virus/worm could
use that relay, and the payload would be delivered (either to the
intended recipient or to the forged return address).  Likewise for
spam. Closing open relays is therefore a prerequisite to successful
implementation of any scheme such as you have proposed.

> No need for new protocols, closed networks etc. Maybe a need for some
> RBL listing virus/spam infected machines, I don't know.

Aside from the need to close open relays and the need for verification
that the specified sender envelope address is appropriate, there are
other issues that make your scheme impractical.  Scanning body content
can only take place at the end of DATA during acceptance (or rejection)
for delivery.  The minimum timeout for a response at end of
DATA is 10 minutes. Assuming all clients waited for at least that
long (some do not), that timeout imposes an upper limit on the
amount of time available for processing to determine whether or
not there is a basis for rejection on policy grounds.  If a large
number of simultaneous connections are being handled by an SMTP
receiver, that may present an unacceptable processing burden.

>From a practical point of view, anti-virus signatures generally only
come out after a virus has propagated fairly widely, so for some
time after a virus is unleashed it will be delivered undetected even
by those who keep their AV software up-to-date.  The same considerations
(must take place after DATA is complete at final delivery SMTP
receiver, requires processing overhead, arms-race considerations)
apply to spam.

In the case of virus content, it's difficult to justify requiring a
site with no Microsoft platforms to install, maintain, and operate
anti-virus software due to poor design of Microsoft products and/or
inadequately trained or incompetent users of same.  One generally
doesn't allow toddlers to operate plastic tricycles on busy highways;
likewise those not competent to practice minimal security and those
whose systems fail to meet some minimum security standards shouldn't
be allowed on the "information superhighway" (apologies for
resurrecting that tired cliché).  An RBL is unlikely to work; the
existence of ISPs that use a pool of IP addresses for dialup and
broadband subscribers via DHCP means that IP addresses cannot be
used to identify infected machines -- one minute a particular IP
address might be assigned to a virus-infected machine running an
insecure operating system and operated by a clueless user, and the
next minute it might be assigned to a firewalled system operated
by a security expert.  If there's an answer to this aspect of the
spam/virus problem short of universal IPv6 deployment and static IP
addresses everywhere, it will probably have to be a policy/licensing
(analogous to a driver's license) matter rather than a purely
technical solution.

Date: Wed, 11 Feb 2004 07:36:05 +0100
From: Keld =?iso-8859-1?Q?J=F8rn?= Simonsen <[email-masked]>
To: [email-masked]
Subject: Best practices to avoid virus and spam
Sender: [email-masked]

Hi!

Some thoughts on how to avoid viruses and spam in these days of MyDoom.

I think the current situation is quite manageable, with the RBL,
the virus checking and spam-checking software. I am running a minor
mail host, but it seems to be popular with virus-senders and spammers,
I get about 50.000 to 100.000 mais a day and more than 99 % are
virus or spam. But current software takes most of it. The only thing I would
like to get rid of, is all the error reporting mails, saying that I have
sent out virus infected mail, or that I have sent mail to somebody whose
address is unknown or their mailbox is full.

I have by the way collected strings for postfix filters to discard
messages reporting bogus virus, available at
http://www.dkuug.dk/keld/virus/

Actually there is some good chances of getting rid of the bogus error
reports. We are dealing with the good guys here, so they may be able to
take proper advice.

I would advice that we recommend some best practice procedures,
hopefully to be implemented in the MTAs software products of the world. 
Maybe we should write an RFC on this.

I have got three pieces of advice:

1. Always check for virus/spam before checking for valid reciepient, or
whether the mailbox is full or some such.

2. Generate a specific error message, maybe we should introduce a
standard error code for this, like 551 - mail rejected as virus or spam.

3. If the mail is virus or spam, then do not send it back to the
sender - as this is most likely a forged address anyway. Discard it
instead.  But if you must, then use the standard error mesaage as
described above. 

I think with this scheme, we would have avoided alomost all of the
virus/spam and also annoying error traffic .

No need for new protocols, closed networks etc. Maybe a need for some
RBL listing virus/spam infected machines, I don't know.

Best regards
Keld

Date: Wed, 11 Feb 2004 00:32:07 -0500
From: Bruce Lilly <[email-masked]>
Reply-To: IETF-SMTP <[email-masked]>
Organization: Bruce Lilly
To: Hector Santos <[email-masked]>
CC: IETF-SMTP <[email-masked]>
Subject: Re: Some Ideas I would like to Bounce off ya
Sender: [email-masked]

Hector Santos wrote:

>                     Using the MX record lookup to provide additional
> information
> 
> "Why the need for additional Information or what sort of information?"
> 
> To address sender machine access concerns.

MX records are for SMTP receivers, not senders.  Forget it.

To: Hector Santos <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: MX Domain Name --> Was Re: Some Ideas I would like to Bounce off ya 
From: [email-masked]
Date: Tue, 10 Feb 2004 22:46:34 -0500
Sender: [email-masked]

--==_Exmh_1733216122P
Content-Type: text/plain; charset=us-ascii

On Tue, 10 Feb 2004 17:21:24 EST, Hector Santos said:

>         smtp.SENDER.vt.edu        ip: 128.173.54.129

Compute how many of these you can get back in a single 512-byte DNS
response (hint - go back and look at WHY AOL round-robins their stuff
the way they do).

Compute the operational cost of having to drop back to TCP (if even feasible)
for more responses.  It's quite possible that for vt.edu you'd get at least 400
or so responses back (that's just the ones I *know* about that are Unix/Linux
workstations that do their own SMTP handling).

This is a non-starter unless you find a way to do the query such that it
returns one packet or less of data (for multiple reasons).  At least the SPF
draft lets us describe ours 2 /16s in one or two TXT records (not that I'm
totally thrilled with Meng's syntax either, but at least it has enough sense to
realize that wildcards (*.foo) and ranges (/16, /20) are a better bet than full
enumeration followed by searching through all the results.

Also, it would almost certainly end up making split-view DNS a requirement for
many sites that don't have it now.  For instance, currently the entire world,
both on campus and off, has one consistent view of our A, MX, and PTR records
(with the exception of 5 machines that are actually *part* *of* our mail hub).
With your proposal, we'd either have to hand back a list of 400+ MX entries for
each piece of e-mail, or run split-view DNS just so our on-campus view didn't
leak to the outside world.

And given the number of PTR queries that arrive at the root nameservers for
1918 addresses, with 1918 *source* addresses on the packets, I doubt that
there's enough production clue out there to make split-view DNS a reality at
the average site.

--==_Exmh_1733216122P
Content-Type: application/pgp-signature

--==_Exmh_1733216122P--

From: "Hector Santos" <[email-masked]>
To: "Keith Moore" <[email-masked]>
Cc: <[email-masked]>, <[email-masked]>
Subject: Re: MX Domain Name --> Was Re: Some Ideas I would like to Bounce off ya
Date: Tue, 10 Feb 2004 19:31:44 -0500
Organization: Santronics Software, Inc
Sender: [email-masked]

What is reality is you that you continue to have the least to say with the
consistent unrealistic desire to have the last say.

Here's an idea!

If you don't have any meaningful input,  you should just ignore the message!
That works just as well!   But if you are going to reply please make it
meaningful and quit wasting bandwidth with useless commentaries.

I'm still waiting for your technical rebuff.  Where is it?

-- 
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

----- Original Message ----- 
From: "Keith Moore" <[email-masked]>
To: "Hector Santos" <[email-masked]>
Cc: <[email-masked]>; <[email-masked]>; <[email-masked]>
Sent: Tuesday, February 10, 2004 5:57 PM
Subject: Re: MX Domain Name --> Was Re: Some Ideas I would like to Bounce
off ya

>
> > > Valdis was exactly right.    And I don't know why he or anyone else
> > > should waste his time responding to every one of your proposals in
> > > detail.
> >
> > Like I said, if he or you can "Baffle me with Brilliance," then
> > GREAT!,
>
> For some reason, you seem to be of the opinion that it's worth other
> people spending significant amounts of time to impress you or to gain
> your confidence.   That doesn't seem consistent with reality.
>
>

Date: Tue, 10 Feb 2004 17:57:39 -0500
From: Keith Moore <[email-masked]>
To: "Hector Santos" <[email-masked]>
Cc: [email-masked], [email-masked], [email-masked]
Subject: Re: MX Domain Name --> Was Re: Some Ideas I would like to Bounce off ya
Sender: [email-masked]

> > Valdis was exactly right.    And I don't know why he or anyone else
> > should waste his time responding to every one of your proposals in
> > detail.
> 
> Like I said, if he or you can "Baffle me with Brilliance," then
> GREAT!,

For some reason, you seem to be of the opinion that it's worth other
people spending significant amounts of time to impress you or to gain
your confidence.   That doesn't seem consistent with reality.

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>
Cc: "IETF-SMTP" <[email-masked]>
Subject: Re: Some Ideas I would like to Bounce off ya 
Date: Tue, 10 Feb 2004 17:21:36 -0500
Organization: Santronics Software, Inc
Sender: [email-masked]

----- Original Message ----- 
From: <[email-masked]>
To: "Hector Santos" <[email-masked]>
Cc: "IETF-SMTP" <[email-masked]>
Sent: Tuesday, February 10, 2004 1:15 PM
Subject: Re: Some Ideas I would like to Bounce off ya

>> By proposal a special  MX domain name format which includes this
>> information, and even possibly using the preference with a hi/lo 2 word
>> concept.  Lo word having the preference value, the hi word having bit map
>> flags.

> Looks like a proposal for a new MX format.  This won't fly, propose a
> new RR instead.

Is the preference part throwing you off?    Lets forget about that.   That
isn't important at all.

If you can forget about that,  I'm would be the difference of having:

        smtp.vt.edu

or simply someting like:

        smtp.rcvd-send.vt.edu

that has no effect on any current MX structure, logic to lookup (with or
without A record), symantics or otherwise.

If you are telling me that the naming "convention" is something that many
based on certain methodologies and thus, there might be some resistance to
this,  then fine, what would be a great reason to stop right there.

See my message to Keith.

-- 
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>
Cc: <[email-masked]>, <[email-masked]>
Subject: MX Domain Name --> Was Re: Some Ideas I would like to Bounce off ya
Date: Tue, 10 Feb 2004 17:21:24 -0500
Organization: Santronics Software, Inc
Sender: [email-masked]

----- Original Message ----- 
From: "Keith Moore" <[email-masked]>
To: "Hector Santos" <[email-masked]>
Cc: <[email-masked]>; <[email-masked]>; <[email-masked]>
Sent: Tuesday, February 10, 2004 1:25 PM
Subject: Re: Some Ideas I would like to Bounce off ya

>
> Valdis was exactly right.    And I don't know why he or anyone else should
waste
> his time responding to every one of your proposals in detail.

Like I said, if he or you can "Baffle me with Brilliance," then GREAT!,
Neither of you have, nor provided reason yet to reject it but to suggest
"Hey stupid, I know better. Go away."     It is obviously you don't the get
into the details issue of what is being proposed for SMTP developers because
you aren't a developer. So things like this don't seem bother you,.

But if you care to see WHY this is a problem and why a MX solution may help,
here is why:

The details of a LOG is shown below:   Here is the summary:

Valdis attempts to sent mail to me with the ip, EHLO and MAIL FROM
information:

        connection ip:   128.173.54.129
        machine domain:  turing-police.cc.vt.edu
        return path: [email-masked]

With these basic information,   a DMP and SPF lookup would of failed.   Two
TXT lookups would of been performed for DMP:

        129.64.173.128.in-addr._smtp-client.vt.edu
        129.64.173.128.in-addr._smtp-client.turing-police.cc.vt.edu

For SPF, a 4 more TXT lookups:

        _spf.vt.edu
        vt.edu
        _spf.turing-police.cc.vt.edu
        .turing-police.cc.vt.edu

In all 6 lookups would fail.    It is TURNED off because of the high
overhead expense of looking up other domains other than your own.

In order for valdis (and you too) to operate in the future world of LMAP, he
will will have to add atleast 1 TXT record for SPF support only and atleast
4-6 more TXT records for DMP support to reflect that IP address and that
machine domain name in his DNS setup.

However, if you havn't walked away yet,  the LOG below shows how a CBV was
performed to validate his return path where a MX lookup was performed and a
smtp-base validation was performed.

Now, all I was proposing that there MAY BE no difference in current MX
compatibility (hoping this is where you can prove it wrong) if he simply did
this:

Instead of having the host domain name

        smtp.vt.edu

have instead:

        smtp.RECEIVER-NO-SPAM.vt.edu     ip: 198.82.161.8
        smtp.SENDER.vt.edu        ip: 128.173.54.129

(please don't forcus on the actual tags)

because now we have a MX lookup that addresses all concepts such as

    o validating the sender machine as intended by the LMAP proposals,

For those who might to go further, they can double check to see if the a
host is available for return path notifications if necessary with 1 single
MX lookup.

More importantly, it should break anything,  except maybe with the above, an
attempt might be made to send mail to the SENDER machine.   So maybe if this
idea has any merit, could it be done with a single record domain name such
as:

        smtp.RCVD-SENDER_128.173.54.129.vt.edu     ip: 198.82.161.8

This would not CHANGE any thing other than changing the name of domain.

Do you know if any particular reason why a MX domain name concept can not
work here?

-- Detail send authentication log of Valdis sending mail to me --

20040210 11:48:13 -------------------------------------
20040210 11:48:13 version    : 1.54 / 1.52
20040210 11:48:13 calltype   : SMTP
20040210 11:48:13 callerip   :
20040210 11:48:13 lanip      : 208.247.131.9
20040210 11:48:13 state      : rcpt
20040210 11:48:13 cip        : 128.173.54.129
20040210 11:48:13 cdn        : turing-police.cc.vt.edu
20040210 11:48:13 from       : <[email-masked]>
20040210 11:48:13 rcpt       : <[email-masked]>
20040210 11:48:13 ruid       : 228761
20040210 11:48:13 srvdom     : winserver.com
20040210 11:48:13 srvip      : 208.247.131.9
20040210 11:48:13 srvsock    : 484
20040210 11:48:13 sapfilter  : pass
20040210 11:48:13 sapdmp     : skipped
20040210 11:48:13 sapspf     : skipped
20040210 11:48:13 saprbl     : testing 129.54.173.128.sbl.spamhaus.org
20040210 11:48:14 saprbl     : testing 129.54.173.128.list.dsbl.org
20040210 11:48:15 saprbl     : testing 129.54.173.128.bl.spamcop.net
20040210 11:48:16 saprbl     : pass
20040210 11:48:17 sapcbv     : total mx records: 1
20040210 11:48:17 try mx     : smtp.vt.edu ip: 198.82.161.8
20040210 11:48:17 # connecting to 198.82.161.8
20040210 11:48:17 S: 220 zidane.cc.vt.edu ESMTP Mirapoint 3.4.3-CR; Tue, 10
Feb 2004 11:46:44 -0500 (EST)
20040210 11:48:17 C: NOOP WCSAP v1.54 Wildcat! Sender Authentication
Protocol http://www.santronics.com
20040210 11:48:17 S: 250 OK
20040210 11:48:17 C: HELO mail.winserver.com
20040210 11:48:17 S: 250 zidane.cc.vt.edu Hello ntbbs.winserver.com
[208.247.131.9], resetting message state
20040210 11:48:17 C: MAIL FROM: <>
20040210 11:48:17 S: 250 <>... Sender ok
20040210 11:48:17 C: RCPT TO: <[email-masked]>
20040210 11:48:17 S: 250 <[email-masked]>... Recipient ok
20040210 11:48:17 C: RCPT TO: <[email-masked]>
20040210 11:48:17 S: 550 <[email-masked]>...
Relaying denied
20040210 11:48:17 C: QUIT
20040210 11:48:17 sapcbv     : 250
20040210 11:48:17 result     : accept (-1)
20040210 11:48:17 wcsap finish (4547 msecs)

-- 
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

Date: Tue, 10 Feb 2004 13:25:09 -0500
From: Keith Moore <[email-masked]>
To: "Hector Santos" <[email-masked]>
Cc: [email-masked], [email-masked], [email-masked]
Subject: Re: Some Ideas I would like to Bounce off ya
Sender: [email-masked]

Valdis was exactly right.    And I don't know why he or anyone else should waste
his time responding to every one of your proposals in detail.

> Did you read the entire message or did you stop at the very first "Gotcha"
> and say "Ah Ha!"
> 
> The reasons you cited here are way off and wrong.  If you would like to try
> again,  please take the time to read the entire message and feel free to
> find all the "real" gotchas.  I'm looking forward to it.

To: Hector Santos <[email-masked]>
Cc: IETF-SMTP <[email-masked]>
Subject: Re: Some Ideas I would like to Bounce off ya 
From: [email-masked]
Date: Tue, 10 Feb 2004 13:15:30 -0500
Sender: [email-masked]

--==_Exmh_-1065927940P
Content-Type: text/plain; charset=us-ascii

On Tue, 10 Feb 2004 12:45:16 EST, Hector Santos said:
> Valdis,
> 
> Did you read the entire message or did you stop at the very first "Gotcha"
> and say "Ah Ha!"

Which part of "Thou shalt not screw with the format of MX records" do you not
understand?

> By proposal a special  MX domain name format which includes this
> information, and even possibly using the preference with a hi/lo 2 word
> concept.  Lo word having the preference value, the hi word having bit map
> flags.

Looks like a proposal for a new MX format.  This won't fly, propose a
new RR instead.

I'm sorry, but when the third paragraph already involves what appears to be
a complete and total non-starter, and the next 6 or 7 paragraphs explain
how this is better than the TXT method that is being used precisely because
paragraph 3 *is* a non-starter, explain why I should go read another 20
paragraphs in the hope that even starting with a totally broken idea, there's
something salvagable there?

I don't see any new and novel "If this information was conveyed somehow, we
could then do XYZ".  Maybe there's one lurking in there, but all I can see is
"we have stuff we're thinkig of putting in a TXT record because it's not a good
idea to put it in the MX record.  Let's put it in the MX instead".

--==_Exmh_-1065927940P
Content-Type: application/pgp-signature

--==_Exmh_-1065927940P--

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>
Cc: "IETF-SMTP" <[email-masked]>
Subject: Re: Some Ideas I would like to Bounce off ya 
Date: Tue, 10 Feb 2004 12:45:16 -0500
Organization: Santronics Software, Inc
Sender: [email-masked]

Valdis,

Did you read the entire message or did you stop at the very first "Gotcha"
and say "Ah Ha!"

The reasons you cited here are way off and wrong.  If you would like to try
again,  please take the time to read the entire message and feel free to
find all the "real" gotchas.  I'm looking forward to it.

-- 
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

----- Original Message ----- 
From: <[email-masked]>
To: "Hector Santos" <[email-masked]>
Cc: "IETF-SMTP" <[email-masked]>
Sent: Tuesday, February 10, 2004 11:46 AM
Subject: Re: Some Ideas I would like to Bounce off ya

On Tue, 10 Feb 2004 04:21:17 EST, Hector Santos
<[email-masked]>  said:

> By proposal a special  MX domain name format which includes this
> information, and even possibly using the preference with a hi/lo 2 word
> concept.  Lo word having the preference value, the hi word having bit map
> flags.

Gaak.

Don't go there. There's a *reason* why all those proposals use either
a specially structured TXT or invent a new RR.

MX'es are a standardized DNS query format, with very specific structure
and semantics.  Hacking them up would basically screw every single one of
the millions of deployed DNS resolvers - which would be *totally* within
their rights to say "I did a MX query, got back a format that doesn't look
like the MX standard, so it's obviously broken and to be discarded. Whoops,
'550 Host Unknown'.

Feel free to be the first to deploy your modified MX in a situation where
falling back to an A record doesn't work.  There's an MX for 'vt.edu',
but no A record for same.  Won't fly here.  You can't find the MX for us,
you can't e-mail us.  And with a modified MX, nobody will find a valid one.

I have to admit, it *would* solve the spam problem. ;)

To: Hector Santos <[email-masked]>
Cc: IETF-SMTP <[email-masked]>
Subject: Re: Some Ideas I would like to Bounce off ya 
From: [email-masked]
Date: Tue, 10 Feb 2004 11:46:36 -0500
Sender: [email-masked]

--==_Exmh_-1135767736P
Content-Type: text/plain; charset=us-ascii

On Tue, 10 Feb 2004 04:21:17 EST, Hector Santos <[email-masked]>  said:

> By proposal a special  MX domain name format which includes this
> information, and even possibly using the preference with a hi/lo 2 word
> concept.  Lo word having the preference value, the hi word having bit map
> flags.

Gaak.

Don't go there. There's a *reason* why all those proposals use either
a specially structured TXT or invent a new RR.

MX'es are a standardized DNS query format, with very specific structure
and semantics.  Hacking them up would basically screw every single one of
the millions of deployed DNS resolvers - which would be *totally* within
their rights to say "I did a MX query, got back a format that doesn't look
like the MX standard, so it's obviously broken and to be discarded. Whoops,
'550 Host Unknown'.

Feel free to be the first to deploy your modified MX in a situation where
falling back to an A record doesn't work.  There's an MX for 'vt.edu',
but no A record for same.  Won't fly here.  You can't find the MX for us,
you can't e-mail us.  And with a modified MX, nobody will find a valid one.

I have to admit, it *would* solve the spam problem. ;)

--==_Exmh_-1135767736P
Content-Type: application/pgp-signature

--==_Exmh_-1135767736P--

From: "Hector Santos" <[email-masked]>
To: "IETF-SMTP" <[email-masked]>
Subject: Some Ideas I would like to Bounce off ya
Date: Tue, 10 Feb 2004 04:21:17 -0500
Organization: Santronics Software, Inc
Sender: [email-masked]

Some Ideas I would like to bounce off ya.  You're welcome to comment.

These ideas are based on the following premises:

- Adding more value to addressing important 'issues' faced today in SMTP
- Maximum compatibility
- Minimum changes

                    Using the MX record lookup to provide additional
information

"Why the need for additional Information or what sort of information?"

To address sender machine access concerns.  A few new proposals are being
drafted that promote additional lookups to address machine access questions.
In addition, to help in the no-solicitation or "no-spam wanted here"
proposals as well.  Basically to provide server attribute information that
can help address the key issues we have today (or better said, most systems
in the industry are concern with).

"Why the MX record?"

Since all SMTP servers require support for MX lookup,  I think it would a
way to avoid the additional overhead issues related to TXT lookups promoted
by these new DNS based sender-machine drafts.   It will be a major overhead
reduction by using the MX record instead of the TXT record.

"How?"

By proposal a special  MX domain name format which includes this
information, and even possibly using the preference with a hi/lo 2 word
concept.  Lo word having the preference value, the hi word having bit map
flags.

"No paper hat! I didn't mean that, How will this be  a major overhead
reduction?"

Oops Sorry! (slap across the head)

I am bringing this up in the first place having first hand experience of how
these new TXT based DNS lookup features work in the real world - basically
high expensive DNS lookup overhead when looking up non-compliant domains
which if we are going to use these as a basis to address the spoofing domain
problem means over 75% of the time - the DNS lookup will fail.

The major benefit the drafts have is to secure your own local domains.  Once
you use them to check other systems - your performance goes down the drain.
Sure, that could be a deployment issue but as someone here recently
highlighted about his trip to the MIT Anti-Spam Conference:

             "If everyone used idea XYZ, we solve the problem."

Well, DUH!   I guess that includes the spammers implementing the ideas too!

"Ok silly, wouldn't this be asking a lot of administrators to change their
MX records?

Don't you think the DNS based TXT lookup proposals will be asking more!?  At
least with MX, it is something they will be familiar with and if kept
simple,  a change can be done in 1 minute.

However, the real icing on the cake is that developers will have a VERY
simply way to implement a MX logic when then the TXT lookup logic.  I guess
you guys have not yet seen the technical implementation detail requirements
for SPF have you?  HA!

"But why the MX record again or how would a new MX format be used?"

At face value,  in the same way the DNS-based TXT lookups are used.  Instead
you are not looking up a TXT record but instead a MX record to obtain the
same information.   In additional, by using a MX record, it falls more
inline with SMTP compatibility in regards to Return Path validations.   If
at some point of a normal SMTP transaction, a bounce or DSN is required, the
MX will need to be looked up based on the domain part of the return path.
You can kill 2 birds with one stone!

"Ok, even if I am not totally convinced, what information would be the MX
record and how can we use it?"

Well, there I don't have all the ideas, maybe those who see some "value or
merit" here can jump in with some MX domain name format ideas, before they
below the idea away.  But I  can  give you some examples based on our
current work with DMP and SPF support in our product  which again is WHY I
am bringing this up in the first place - first hand experience shows a
tremendous overhead issue with non-compliant domain lookup.

When the return path is provided, a MX lookup is performed:

With no change to the format,  one can possible used the IP addresses
returned to compare with the connection IP address.   This addresses the
simplest scenario of a SMTP host to final destination transaction.  The
non-server MUA (end-user sending to host) is still an issue which I think
remains an issue with the TXT based lookup drafts.

With a change format,  it could be something as simple as these (note,
verbosity is intended for illustrative purposes only.  Brighter people can
come with a better naming scheme):

      mx1.nospam.winserver.com
      mx1.sendonly.winserver.com
      mx1.rcvdonly.winserver.com
      mx1.sendrcvd.winserver.com

First let me get this nospam tag out of the way since it is pretty straight
forward.  A MX with nospam basically puts a "sign" out there (DNS) with the
hope spammers may use this to avoid trying to spam your system.   They
benefit too by having "no spam" attribute in the MX record because a) its
easy to add to software that already has a MX lookup logic, and b) they can
now avoid connection overhead which they can shift to spamming other systems
that don't advertise their desire to not spam them.    This can have a major
network overhead reduction benefit.

Back to the sender validation. In my opinion, the following two major points
with any of the new wave "sender" management foundations being contemplated
are important:

    o True Result Information is the only 100% valid result.
    o False/Unknown results provides no decision or low confidence.

The same principles apply does in DMP/SPF.  If the domain contains DNS txt
record information, you can perform DMP/SPF logic, otherwise "no decision"
can be made and you move on.

Both are based on using the connection IP (CIP) and the return path domain
(RPD) as part of the lookup process.   DMP is a simpler to deploy but more
expensive.  SPF is harder to deploy and *CAN* provide the same results with
when compared at equal level. SPF offers more logic than DMP is designed to
do.  This is why we implemented both.

Now, these proposals only validate the sender machine. They do not address
the sender or return path.   We address this with CBV (Callback Verifier).

All together, they offer great spoofing protection.

But in the end, it all boils down to logic to pulling DOMAIN information
from DNS  at the protocol level:

    o One or more TXT lookups (DMP/SPF) to check sender machine
    o One MX lookup to initiate the CBV process

where both have a common return path domain lookup logic (DMP/SPF secondary
lookup is based on the machine domain name presented at HELO/EHLO)

In my view, what makes it all easy to see is that is a DNS lookup is
required based on the return path domain, it might as well be based on a
lookup with a high expectancy to be there - the MX record.

So logically, if the MX record can also provide the same information that
DMP/SPF wish to expose about a system,  it will offer a great benefit in the
following areas:

- Lower overhead/bandwidth for servers and clients and network

- Lower implementation/change requirements by Software developers

- Lower learning curve for DNS administrators

- Promotes a standard method for MX domain naming.

- Higher compatibility with SMTP

which to me, yields higher acceptance and faster deployment.

Look folks.  This has nothing to do with trying to kill efforts by ARSG.
After all, we did implement DMP and SPF,  but these proposals will have a
problem of acceptance because it require significant change to your
software.   So it make more sense within the development industry to see
"what best" works with minimal changes.   I don't think 1 solution can be
used so maybe multiple offering will be required.   At least that is the
approach we took at this early stage adding "SMTP Sender Management"
concepts to our software.   But in the end,  one will prevail and I think
that will be based on a) easy of implementation and b) solve the main issue.

While others would say "We waited until the endorsed method was selected and
based on that SPF seems to be the winner,"  I would say what I said above -
DMP is easier to add and gives you basically the same results as SPF - but
more importantly,  you achieve the same results with much more benefits
across the board by just using what is already there - the MX record.

In any case, in my belief, ultimately I see it will be important to get
"server" information before connections are made as a important development.
Wither its TXT records or new records, etc, whatever it is, it should be 1
or as few as lookup as possible to get everything that is required.

Potential Problems using the MX record?

Large farms - separate receivers farms vs. separate sender farms.

Issue #1

one of the enhancements made to our DNS resolver, assisted with some input
gathered from discussions here,  is to establish the complete expanded list
of MX host.  Before, we just used the MX records returned with no A record
expansion.   The final optimized, sort list *may* include logic for delay
host IP lookup.

In this case,  a potential for some overhead reduction gains can be lost
with the requirement to have the IP resolves with additional DNS A record
lookups.

However, in practice, I see this only happen with large ISP with mucho MX
records and from what I see they rotate a fix set with resolved IPs, enough
to satisfy a transaction, i.e, AOL may yield 15 records with the first 10
resolved.

Issue #2

An send only machine with a return path to a receive machine.  The
connection IP is that of the sending machine.

One solution is to include the MX record:

      mx1.sendonly.winserver.com

so that the IP check can be made against the connection IP and the SMTP send
mail logic to be slightly alters to exclude this MX record from his DNS MX
resolver.    To help support SMTP servers that are not excluding the record,
these sendonly MX records will have high preference values that when sorted
will be among those that are located at the bottom a HOST list.   The
"rcvdonly" or "both" MX records resolved should be at the TOP of a sorted
host list.  for example:

      mx1.rcvdonly.winserver.com     preference: 10
      mx2.rcvdonly.winserver.com    preference: 10
      mx3.rcvdonly.winserver.com    preference: 10
      mx4.rcvdonly.winserver.com    preference: 10
      mx5.rcvdonly.winserver.com    preference: 10
      mx1.sendonly.winserver.com    preference: 10000
      mx2.sendonly.winserver.com    preference: 10000
      mx3.sendonly.winserver.com    preference: 10000
      mx4.sendonly.winserver.com    preference: 10000
      mx5.sendonly.winserver.com    preference: 10000

Anyway, comments?

Date: Fri, 06 Feb 2004 12:51:01 -0600
From: "Eric A. Hall" <[email-masked]>
To: Keith Moore <[email-masked]>
CC: John C Klensin <[email-masked]>, [email-masked], [email-masked], [email-masked]
Subject: Re: Using SUBMIT without bothering the user (was Re: MyDoom, Sorbig - Actions taken?)
Sender: [email-masked]

On 2/6/2004 12:24 PM, Keith Moore wrote:

> but I'll look forward to reading the draft and seeing what it has to say 
> about the subject.

http://www.ietf.org/internet-drafts/draft-hall-email-srv-00.txt

it's about to expire, actually

-- 
Eric A. Hall                                        http://www.ehsco.com/
Internet Core Protocols          http://www.oreilly.com/catalog/coreprot/

Date: Fri, 06 Feb 2004 12:50:06 -0600
From: "Eric A. Hall" <[email-masked]>
To: John C Klensin <[email-masked]>
CC: Keith Moore <[email-masked]>, Arnt Gulbrandsen <[email-masked]>, [email-masked], [email-masked]
Subject: Re: Using SUBMIT without bothering the user (was Re: MyDoom, Sorbig - Actions taken?)
Sender: [email-masked]

On 2/6/2004 11:20 AM, John C Klensin wrote:

> On a devil's advocate basis, let me respond to your DHCP comment 
> above by saying "it all depends on how DHCP is used and in which 
> environments... and that it may be lots more appropriate for 
> some than for others".  Specifically,

<snip>

> The real problem with Submit (and many other potential SRV and 
> DHCP targets/values), it seems to me, is that someone has to 
> make a policy decision about the semantics of the server or 
> service that is being searched for.

That's exactly correct. That's also precisely why there are so many
configuration services available, and why they all have different favored
deployment scenarios. The market HAS spoken -- they want ALL of the
different configuration services! Therefore, application services which
want to provide autoconfiguration that works in every scenario need to
support the whole array. Of course, in the usual case, the application
services support none of them explicitly.

The SRV draft essentially aims for the specific usage scenario which I
personally believe to be the most common: the user has an email address,
and they want to fetch and send email associated with that address. That
scenario reflects people on ISP accounts, people in typical job/work
environments, and so forth. It does not consider all of the other
scenarios. For the covered usages, keying off the domain part is the
appropriate mechanism, but also implicies that the user can obtain this
data while roaming on different subnets within an organization or remote
networks (assuming that the infrastructure allows it, which isn't the
concern of the draft).

If anybody wants to define algorthims and whatnot for other mechanisms
(DHCP, LDAP, ServLoc, whatever), feel free to define them.

-- 
Eric A. Hall                                        http://www.ehsco.com/
Internet Core Protocols          http://www.oreilly.com/catalog/coreprot/

Date: Fri, 6 Feb 2004 13:24:09 -0500
From: Keith Moore <[email-masked]>
To: John C Klensin <[email-masked]>
Cc: [email-masked], [email-masked], [email-masked], [email-masked], [email-masked]
Subject: Re: Using SUBMIT without bothering the user (was Re: MyDoom, Sorbig - Actions taken?)
Sender: [email-masked]

> The real problem with Submit (and many other potential SRV and 
> DHCP targets/values), it seems to me, is that someone has to 
> make a policy decision about the semantics of the server or 
> service that is being searched for. 

precisely.  and when that "someone" has to be the user it's hard to see why
making that policy decision is less work than simply configuring a server.

but I'll look forward to reading the draft and seeing what it has to say 
about the subject.

--
He not busy being born, is busy dying.		- Bob Dylan

Date: Fri, 06 Feb 2004 12:20:22 -0500
From: John C Klensin <[email-masked]>
To: Keith Moore <[email-masked]>, "Eric A. Hall" <[email-masked]>
cc: Arnt Gulbrandsen <[email-masked]>, [email-masked], [email-masked]
Subject: Re: Using SUBMIT without bothering the user (was Re: MyDoom, Sorbig - Actions taken?)
Sender: [email-masked]

--On Thursday, 05 February, 2004 20:41 -0500 Keith Moore 
<[email-masked]> wrote:

>
> Mumble.  configuration is a Hard Problem.  With very few
> exceptions most of the solutions I've seen to do configuration
> end up making the situation more complex, less fault-tolerant,
> and harder to configure correctly - and it's usually because
> they key off of the wrong thing for the particular quantity
> they're trying to configure.  For example, DHCP is really only
> good for things that are inherently tied to the network
> location, which isn't much besides address, netmask, and
> default route.  (and it botches address configuration by
> making IP addresses ephemeral).

Keith,

You are putting your finger on exactly the things that make me 
uncomfortable with the Email SRV draft.  But most of them are 
also issues we explore in the draft.

On a devil's advocate basis, let me respond to your DHCP comment 
above by saying "it all depends on how DHCP is used and in which 
environments... and that it may be lots more appropriate for 
some than for others".  Specifically,

	* there are issues of defaults and overrides of
	information obtained elsewhere that are matters of
	implementation and configuration, not of the DHCP specs.
	E.g., if my host has an address (or an SMTP server, or a
	printer list) already, should it bother asking DHCP?
	And, if it does ask (whether always or in the hope of
	getting some values which it doesn't have already),
	should the DHCP values override or supplement the
	pre-existing ones, or should they be ignored?
	
	* suppose there is information I can obtain through
	either DHCP or SRV or some other mechanism.   Should I
	ask all of them, and in which order?  And, if I get back
	inconsistent results... (see previous paragraph).
	
	* sometimes, but not always, "network location" (or
	"network topology") are actually pretty well connected
	to administrative/ policy domains.  The appropriateness
	of that association may depend on local issues that have
	little or nothing to do with anything that can be said
	--technically or morally-- Internet-wide and, sometimes,
	with how other protocols are configured.  Using the
	Submit case as an example, if access to the LAN
	supported by a DHCP server is closely controlled, and
	that LAN contains an SMTP server that is configured to
	act as a relay for anything originating from that LAN,
	then having DHCP dispense the name or address of that
	server may be completely appropriate (although having
	every SMTP client attached to the LAN may not be).
	
	* and, of course, whether the addresses DHCP dispenses
	are ephemeral or not is also a matter of configuration.
	With some configurations, it is a tool for centralized
	management of addresses within a (physical or logical)
	LAN, but those addresses are no less stable than if they
	were configured into the hosts on that LAN one at a
	time.   It just makes renumbering with, e.g., a change
	in ISPs, a lot easier.

The real problem with Submit (and many other potential SRV and 
DHCP targets/values), it seems to me, is that someone has to 
make a policy decision about the semantics of the server or 
service that is being searched for.  When there is only one 
choice of semantics, the answer of which protocol to use and how 
to use it typically becomes pretty simple.  When there are 
several --and Submit is clearly an example of that, with "based 
on local network" versus "based on my usual address" being only 
two of the choices-- then things get complicated.  And it is not 
clear whether "configure in the desired semantics and the right 
protocol search order(s)" is going to be easier for that poor 
user than "just configure in the right host".

     john

Cc: Keith Moore <[email-masked]>, Arnt Gulbrandsen <[email-masked]>, [email-masked], [email-masked]
From: Keith Moore <[email-masked]>
Subject: Re: Using SUBMIT without bothering the user (was Re: MyDoom, Sorbig - Actions taken?)
Date: Thu, 5 Feb 2004 20:41:55 -0500
To: "Eric A. Hall" <[email-masked]>
Sender: [email-masked]

> As John pointed out, draft-hall-email-srv-00.txt uses the domain in the
> email address as the principle key for these lookups. In answer to the
> point you raised, the address bindings are per-identity, not global.
> 'Normal' users typically have MUAs where each email address is bound 
> to a
> different identity profile, which may use different submission and/or
> retrieval servers. Also, the draft specifically states that these 
> lookups
> are intended to be the starting point, not the end of the process. 
> Folks
> can use it to bootstrap a configuration process, can use it to perform 
> a
> relatively complete configuration process, or can skip it if they want.
> The spec tries to standardize an algorithm and the necessary elements 
> so
> that different MUAs do it consistently, not to enforce a procedure.
>
> One other point is that the draft also defines a retrieval algorithm 
> and
> SRV mechanism, and is not limited to submissions.

Mumble.  configuration is a Hard Problem.  With very few exceptions 
most of the solutions I've seen to do configuration end up making the 
situation more complex, less fault-tolerant, and harder to configure 
correctly - and it's usually because they key off of the wrong thing 
for the particular quantity they're trying to configure.  For example, 
DHCP is really only good for things that are inherently tied to the 
network location, which isn't much besides address, netmask, and 
default route.  (and it botches address configuration by making IP 
addresses ephemeral).

Date: Thu, 05 Feb 2004 19:45:58 -0600
From: "Eric A. Hall" <[email-masked]>
To: Keith Moore <[email-masked]>
CC: Arnt Gulbrandsen <[email-masked]>, [email-masked], [email-masked]
Subject: Re: Using SUBMIT without bothering the user (was Re: MyDoom, Sorbig - Actions taken?)
Sender: [email-masked]

On 2/5/2004 7:41 PM, Keith Moore wrote:

> configuration is a Hard Problem.

No kidding. When it's defined broadly enough, it's impossible since the
requirements are contradictory.

> DHCP is really only good for things that are inherently tied to the 
> network location, which isn't much besides address, netmask, and 
> default route.  (and it botches address configuration by making IP 
> addresses ephemeral).

DHCP is a fine choice for what it provides in the scope that it actually
functions within. That's a lot of caveats right there.

Most of this is discussed in the draft.

-- 
Eric A. Hall                                        http://www.ehsco.com/
Internet Core Protocols          http://www.oreilly.com/catalog/coreprot/

Date: Thu, 05 Feb 2004 19:30:17 -0600
From: "Eric A. Hall" <[email-masked]>
To: Keith Moore <[email-masked]>
CC: Arnt Gulbrandsen <[email-masked]>, [email-masked], [email-masked]
Subject: Re: Using SUBMIT without bothering the user (was Re: MyDoom, Sorbig - Actions taken?)
Sender: [email-masked]

On 2/5/2004 10:54 AM, Keith Moore wrote:

>> Can one, for example, do some SRV DNS lookup?
> 
> What would form the lookup key?  It doesn't have anything inherent to
> do with the user's From address (which after all can vary from one
> message to another), nor does it have anything inherent to do with the
> user's ISP (I use 4 different ISPs at different times, but I want to
> keep the same email address and submission server no matter what ISP
> I'm using).

As John pointed out, draft-hall-email-srv-00.txt uses the domain in the
email address as the principle key for these lookups. In answer to the
point you raised, the address bindings are per-identity, not global.
'Normal' users typically have MUAs where each email address is bound to a
different identity profile, which may use different submission and/or
retrieval servers. Also, the draft specifically states that these lookups
are intended to be the starting point, not the end of the process. Folks
can use it to bootstrap a configuration process, can use it to perform a
relatively complete configuration process, or can skip it if they want.
The spec tries to standardize an algorithm and the necessary elements so
that different MUAs do it consistently, not to enforce a procedure.

One other point is that the draft also defines a retrieval algorithm and
SRV mechanism, and is not limited to submissions.

-- 
Eric A. Hall                                        http://www.ehsco.com/
Internet Core Protocols          http://www.oreilly.com/catalog/coreprot/

From: "Dan Wing" <[email-masked]>
To: <[email-masked]>
Subject: RE: MyDoom, Sorbig - Actions taken?
Date: Thu, 5 Feb 2004 17:12:29 -0800
Importance: Normal
Sender: [email-masked]

> > For example, all business mail egressing Cisco's network
> > should have a MAIL FROM with @cisco.com,
> 
> Please don't misinterpret the intent of the following question; I'm
> just checking to see how unusual Cisco really is.  Does Cisco
> provide any sort of network access for visitors (consultants,
> suppliers, major customers, etc.)?  If so, does Cisco require such
> visitors to use cisco.com in MAIL FROM of outbound email (which
> may be headed for a non-Cisco SMTP server)?

Being a networking company, and employing thousands of geeks,
we are unique, of course.

There is no enforcement of the MAIL FROM address on outgoing 
email.  But a more 'normal' company (let's say, Boeing or Ford 
or McDonald's) could very reasonably have a more restrictive
policy and ensure that all egressing mail is from their 
domain(s).

-d

Date: Thu, 05 Feb 2004 19:01:13 -0500
From: Bruce Lilly <[email-masked]>
Reply-To: [email-masked]
Organization: Bruce Lilly
To: Dan Wing <[email-masked]>
CC: [email-masked]
Subject: Re: MyDoom, Sorbig - Actions taken?
Sender: [email-masked]

Dan Wing wrote:

> It's possible, and reasonable, for an SMTP server that accepts
> mail submission to require a username/password and apply policy
> based on that username/password.  This is available today with the
> AUTH extension to SMTP.

Yep. This message is an example.

> Of course today nearly all SMTP servers that accept mail
> submission merely validate the legitimacy of the sender based
> on the sender being in the ISP's (or enterprise's) IP address
> space.

That wouldn't work for this message; it's hitting my ISP's outbound
SMTP server from a different address space.  This machine being a
laptop, that is frequently the case.

> In either case, though, the SMTP server that accepts mail submission
> can fairly trivially validate the MAIL FROM address, and even the
> From: address, based on the IP address (or username/password) of
> the user.

Not necessarily; an email address and IP address are not always
related.  A shared secret is another matter.

> For example, all business mail egressing Cisco's network
> should have a MAIL FROM with @cisco.com,

Please don't misinterpret the intent of the following question; I'm
just checking to see how unusual Cisco really is.  Does Cisco
provide any sort of network access for visitors (consultants,
suppliers, major customers, etc.)?  If so, does Cisco require such
visitors to use cisco.com in MAIL FROM of outbound email (which
may be headed for a non-Cisco SMTP server)?

Date: Thu, 5 Feb 2004 16:15:15 -0500
From: Keith Moore <[email-masked]>
To: "Hector Santos" <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: MyDoom, Sorbig - Actions taken?
Sender: [email-masked]

> Just do me one favor.  If you wish to show me up, please do so
> professionally,  technically and verbose in you reasoning. I have no
> problem being baffled with brilliance.

Tell you what.  I'll try to speak in professional and technical terms. 

I will not promise to be verbose.  There are times when this gives the
question or topic more attention than it deserves, and there are times
when no amount of explanation will serve.  Sometimes the best response to
a question is a terse one.

And in return, if you don't understand an answer that I give, or don't
like it, or it doesn't comport with your model of the problem - just
accept that for whatever information it might convey to you.  Or not.

--
He not busy being born, is busy dying.		- Bob Dylan

From: "Hector Santos" <[email-masked]>
To: "Keith Moore" <[email-masked]>
Cc: <[email-masked]>, <[email-masked]>
Subject: Re: MyDoom, Sorbig - Actions taken?
Date: Thu, 5 Feb 2004 15:52:56 -0500
Organization: Santronics Software, Inc
Sender: [email-masked]

----- Original Message ----- 
From: "Keith Moore" <[email-masked]>
To: "Hector Santos" <[email-masked]>

> If you've worked in this field for 25 years and still don't understand
> layering, maybe you should try working on something simpler.

hahahahaha!   did you watch David Letterman yesterday?  The skit on cliches?
"Maybe they will find Bin-Laden on Mars?"

For the sake of the mailing list, I will not stoop any further.

Just do me one favor.  If you wish to show me up, please do so
professionally,  technically and verbose in you reasoning. I have no problem
being baffled with brilliance.

See ya,

-- 
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

To: Hector Santos <[email-masked]>
Cc: [email-masked], Keith Moore <[email-masked]>
Subject: Re: MyDoom, Sorbig - Actions taken? 
From: [email-masked]
Date: Thu, 05 Feb 2004 14:34:46 -0500
Sender: [email-masked]

--==_Exmh_-1122030772P
Content-Type: text/plain; charset=us-ascii

On Thu, 05 Feb 2004 12:40:59 EST, Hector Santos said:

> anything and you are far better than me.   I am sure you also have more
> experience in a "File System" and "Hosting System" too.  Do you have a web
> site to check out your feature list?  Maybe I can steal a few tips for our
> new versions.

1342 Representation of Non-ASCII Text in Internet Message Headers. K.
     Moore. June 1992. (Format: TXT=15845 bytes) (Obsoleted by RFC1522)
     (Status: PROPOSED STANDARD)
1427 SMTP Service Extension for Message Size Declaration. J. Klensin,
     WG Chair, N. Freed, Ed., K. Moore. February 1993. (Format: TXT=17856
     bytes) (Obsoleted by RFC1653) (Status: PROPOSED STANDARD)
1522 MIME (Multipurpose Internet Mail Extensions) Part Two: Message
     Header Extensions for Non-ASCII Text. K. Moore. September 1993.
     (Format: TXT=22502 bytes) (Obsoletes RFC1342) (Obsoleted by RFC2045,
     RFC2046, RFC2047, RFC2048, RFC2049) (Status: DRAFT STANDARD
1653 SMTP Service Extension for Message Size Declaration. J. Klensin,
     N. Freed, K. Moore. July 1994. (Format: TXT=17883 bytes) (Obsoletes
     RFC1427) (Obsoleted by RFC1870) (Status: DRAFT STANDARD)
1870 SMTP Service Extension for Message Size Declaration. J. Klensin,
     N. Freed, K. Moore. November 1995. (Format: TXT=18226 bytes)
     (Obsoletes RFC1653) (Also STD0010) (Status: STANDARD)
1891 SMTP Service Extension for Delivery Status Notifications. K.
     Moore. January 1996. (Format: TXT=65192 bytes) (Obsoleted by RFC3461)
     (Status: PROPOSED STANDARD)
1894 An Extensible Message Format for Delivery Status Notifications.
     K. Moore, G. Vaudreuil. January 1996. (Format: TXT=77462 bytes)
     (Obsoleted by RFC3464) (Updated by RFC2852) (Status: PROPOSED
     STANDARD)
2017 Definition of the URL MIME External-Body Access-Type. N. Freed,
     K. Moore, A. Cargille. October 1996. (Format: TXT=9000 bytes)
     (Status: PROPOSED STANDARD)
2047 MIME (Multipurpose Internet Mail Extensions) Part Three: Message
     Header Extensions for Non-ASCII Text. K. Moore. November 1996.
     (Format: TXT=33262 bytes) (Obsoletes RFC1521, RFC1522, RFC1590)
     (Updated by RFC2184, RFC2231) (Status: DRAFT STANDARD)
2183 Communicating Presentation Information in Internet Messages: The
     Content-Disposition Header Field. R. Troost, S. Dorner, K. Moore,
     Ed.. August 1997. (Format: TXT=23150 bytes) (Updates RFC1806)
     (Updated by RFC2184, RFC2231) (Status: PROPOSED STANDARD)
2184 MIME Parameter Value and Encoded Word Extensions: Character Sets,
     Languages, and Continuations. N. Freed, K. Moore. August 1997.
     (Format: TXT=17635 bytes) (Obsoleted by RFC2231) (Updates RFC2045,
     RFC2047, RFC2183) (Status: PROPOSED STANDARD)
2231 MIME Parameter Value and Encoded Word Extensions: Character Sets,
     Languages, and Continuations. N. Freed, K. Moore. November 1997.
     (Format: TXT=19280 bytes) (Obsoletes RFC2184) (Updates RFC2045,
     RFC2047, RFC2183) (Status: PROPOSED STANDARD)

and lots of text in the rest of the 2045-2049 series and others.

Now what's this about a feature list?

--==_Exmh_-1122030772P
Content-Type: application/pgp-signature

--==_Exmh_-1122030772P--

Date: Thu, 5 Feb 2004 14:23:54 -0500
From: Keith Moore <[email-masked]>
To: "[email-masked]" <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: Using SUBMIT without bothering the user (was Re: MyDoom, Sorbig - Actions taken?)
Sender: [email-masked]

> > > Which reminds me. How is a MUA supposed to find its submit server?
> > 
> > The same way an MUA finds an SMTP server to submit mail.
> 
> I think above refers to that SMTP and POP3/IMAP server names are 
> preconfigured in the mail client setup for each user.

Well, that's one way to do it.  And as a way of setting defaults for
ISPs that distribute software to their customers and for companies that
distribute software to their employees, it's probably better than the
alternatives of either (a) tying the submission server to the domain
in the sender's From address or (b) tying the submission server to the
ISP the sender is currently connected to.

My guess is that one of the consequences of CANSPAM is going to be that
companies become a lot more insistent that personal email not be sent 
from company email addresses or through company mail servers - which
either means that they're going to forbid all personal email from
company desktops and laptops (like forbidding all personal phone calls -
nice in theory but somewhat onerous and often impractical), or they're
going to need ways that employees can keep personal mail separate from
company mail while still being able to use company workstations and
perhaps software to read and send occasional pieces of personal mail. 

--
He not busy being born, is busy dying.		- Bob Dylan

Date: Thu, 05 Feb 2004 10:16:19 -0800 (PST)
From: [email-masked]
Subject: Re: Using SUBMIT without bothering the user (was Re: MyDoom, Sorbig - Actions taken?)
To: Keith Moore <[email-masked]>
Cc: Arnt Gulbrandsen <[email-masked]>, [email-masked], [email-masked], [email-masked]
Sender: [email-masked]

> > > It doesn't have anything inherent to do with the user's From address

> > The way the wind seems to be blowing, SPF will change that during the
> > next year or two. For better or worse.

> I hope you're wrong.  Because if there's anything worse than spammers, it's
> people who try to impose half-baked solutions to spam on the rest of ous.

I doubt very very much he's wrong. We may not end up with SPF exactly, but I
think it is inevitable that one of these critters is going to widely deploy and
compromise various existing facilities.

In fact my worst fear could easily be realized, which is that multiple,
incompatible schems of this sort will deploy. Figuring out how to keep
forwarding working with one of these is bad enough, making it work with more
than one of these things, especially if the requirements they impose conflict
in some way, doesn't bear thinking about.

This is why there's going to be a BOF (MARID) on this in Korea - we REALLY need
to get a handle on the situation.

				Ned

Date: Thu, 5 Feb 2004 13:34:59 -0500
From: Keith Moore <[email-masked]>
To: "Hector Santos" <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: MyDoom, Sorbig - Actions taken?
Sender: [email-masked]

> If its more than 25
> years covering design, development and marketing with every form of
> mail software from MUA to mid ware to hosting, then by gawd, I don't
> know anything and you are far better than me.

If you've worked in this field for 25 years and still don't understand
layering, maybe you should try working on something simpler.

> I ask the QUESTION hopefully to get an intelligent discussion going

You seem to think that an intelligent discussion is one that reinforces
your presumptions, and that being told that your presumptions are wrong
is an inherent sign of a lack of intelligence.  Maybe you should re-
evaluate your idea of what intelligence is. 

If you want to build an SMTP server that does filtering - knock yourself
out.   If you want to standardize such a practice, you need to explain
why your approach will actually solve the problem when applied on a
large scale, and do so with less complexity/cost than other approaches,
and why it makes sense in the long term rather than just as a short-term
hack.

Why, after all, is this SMTP's problem?  Why isn't it the OS vendor's
responsibility to build systems that are not easily compromised?  Why
isn't it the application vendor's responsibility to build application
data formats that cannot convey viruses?  Why isn't it the user's
responsibility to choose OSes and apps that are safe, or to avoid 
launching apps on questionable data that was received from over the net?

It might be that things work best when there is a balance of
responsibility, when each party makes a reasonable effort to keep from
contributing to the problem.  It's reasonable for operating systems to
have "permissions" including "execute permission" that is not set by
default whenever a file is created.  It's reasonable for applications to
expect that they will be exposed to data created by malicious parties
and to be constructed to avoid causing harm to the host system and data 
when that happens.  And it's reasonable to expect users to take some
care in what OS and apps they install on their system and what they
do with data that they receive from questionable sources.  It's probably
reasonable to expect SMTP to keep better track of where messages came 
from and who sent them (for some meaning of "where" and "who").

What is probably not reasonable is to expect an unrelated part of the
system - like SMTP - to be smarter about applications and their data
formats than the parts of the system that deal directly with these 
things.

> Keith, with all due respect,  you seem to have a fundamental flawed
> premise about new IETF participants having little experience and
> young,   I am not making that up.  You said it yourself in a 2001
> article:
> 
> http://lists.w3.org/Archives/Public/ietf-discuss/2001Nov/0097.html

Well, you're taking that message out of context.  But if that shoe fits,
wear it.

> Maybe you should reread the response to your message:
> 
> http://lists.w3.org/Archives/Public/ietf-discuss/2001Nov/0098.html

For every clueful newbie who knows what he is talking about, knows the
limitations of his own experience, and just doesn't speak the same
language as more experienced IETFers, there are ten newbies who think
that the whole Internet works just like the tiny portion that they've
seen, and that other's experience is either irrelevant or a sign of
arrogance.  

Every organization or group of people, to be successful, has to figure
out how to filter out people who work against it.  Some of them have
formal entrance barriers, some of them have other means.

People who have been around here for awhile have learned to figure out
which newbies have the potential to make constructive contributions
(because they can learn from the experience of others, integrate it with
their own, and express the result in terms that make sense to others)
and which ones aren't likely to be able to do anything but get in the
way.  

> If I had to say it myself,  the response pretty much says it all.
> 
> All this started with a simple pertinent question! 

And you got a pertinent answer.  You just didn't want to recognize it as
such.

--
He not busy being born, is busy dying.		- Bob Dylan

Date: Thu, 05 Feb 2004 13:18:25 -0500
From: John C Klensin <[email-masked]>
To: [email-masked], [email-masked]
Subject: Re: Using SUBMIT without bothering the user (was Re: MyDoom, Sorbig - Actions taken?)
Sender: [email-masked]

--On Thursday, 05 February, 2004 10:44 -0800 [email-masked] 
wrote:

>> > Which reminds me. How is a MUA supposed to find its submit
>> > server?
>>
>> The same way an MUA finds an SMTP server to submit mail.
>
> I think above refers to that SMTP and POP3/IMAP server names
> are  preconfigured in the mail client setup for each user.

Yes.

> That does not however means we can not provide default values
> that lot  less advanced users could find useful and passing
> default values to user  for certain protocols is not really
> SUBMIT protocol only question (i.e.  not really for
> ietf-smtp). The protocol that is often used to pass on
> default parameters for host configuration is DHCP when it
> provides  computers with ip address, dns server name, etc.

In theory, DHCP could be used to pass that information.  If I 
recall, there is a key in there for SMTP server, but not a 
separate one for a Submit server (which might be different). 
But I know of no MUAs that pay any attention to DHCP-provided 
information and few or no operating systems that make 
miscellaneous DHCP key-value pairs available to applications in 
any event.

> if
> you look at IETF  work, I beliebve "Service Location Protocol"
> (see RFC2608) could be the  new generation version ofthat. I'm
> not sure if anybody actually ever  attempted to use that for
> configuring default SUBMIT server name - that  would require
> extension to client mail program obviously, but I dont see
> why it would not work if the functionality was written.

There is an Internet-Draft (draft-hall-email-srv-00.txt) that 
Eric Hall and I have been working on intermittently that 
specifies a way to use SRV for that purpose.   The difficulties 
are just the ones that earlier responses in this thread have 
identified, e.g., figuring out what to use as the lookup key. 
I've been somewhat dissatisfied with the proposal as a result 
and, in particular, with the differences between:

	* I want to find a Submit server on the subnet to which
	I'm attached, but want (or need) to use an Reverse-path
	address associated with that location.
	
	* I want to find a Submit server on the subnet to which
	I'm attached, but need to find one that will let me use
	my preferred Reverse-path (which may require telling the
	SRV process what that address is).
	
	* I want to find a Submit server associated with my
	corporate/ enterprise/ personal LAN, even though I may
	not be physically attached to it but may, instead, be
	connecting into it via an application-layer, IP-layer,
	or below-IP-layer tunnel/ VPN  (It is possible that
	those are actually three or four different cases.)
	
	* I want to find a Submit server associated with my
	corporate/ enterprise/ personal LAN, but may plan to
	access and authenticate myself to it using SMTP AUTH
	rather than any technology that depends on network
	addresses topology.

The I-D covers roughly one of those cases, and I will probably 
continue to find it unsatisfying until most of them are 
addressed in some way.

> Perhaps if some  enterprise mail package vendor would
> implement it and pitch it along to  their fortune500 clients
> as great new "feature", word would spread and  eventually this
> might get implemented by others...

Flying pigs may need to be inserted here by reference.  Most of 
the LAN and enterprise network planners for those Fortune 500 
clients whom I've run across are much happier with a client (or 
associated Operating System) that they can custom-configure for 
the server(s) they want used and that is then _really_ hard to 
change and reconfigure.   When one discusses service location 
models with them for anything less passive --and more subject to 
problematic spoofing-- than "find an appropriate printer inside 
my firewall", their reaction is mostly "this sounds like another 
opportunity for someone to attack my network and users".

But hope springs eternal.

    john

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>, <[email-masked]>
Subject: Re: MyDoom, Sorbig - Actions taken? 
Date: Thu, 5 Feb 2004 13:10:05 -0500
Organization: Santronics Software, Inc
Sender: [email-masked]

----- Original Message ----- 
From: "Brad Clements" <[email-masked]>
To: <[email-masked]>
Sent: Thursday, February 05, 2004 10:57 AM
Subject: Re: MyDoom, Sorbig - Actions taken?

>
> Wouldn't it be nice if the same sort of thing could be done with email
messages?
> Imagine all the viruses that would be blocked because they could only send
mail on
> behalf of the "originating location's mail domain".

IP Analytics might become a plan.  That might be more true than we care to
realize.

> Of course, it can't really be done due to the current design of SMTP. Yes,
there are
> some proposals that allow recipients to try to verify the allowability of
an IP address
> to send mail on behalf "claimed sender addresses", but I consider these to
be band-
> aid solutions.

These SORBIG generation virus are nasty is that they enhanced their
distribution process by using stolen valid return paths and return on a
secondary wave of distribution based on bounces.   Some of the proposals
will address some of the distribution but not all.

Brad,  you know the products we have - so you know we have the experience.
:-)

The issue and why I ask the question.

YES, traditionally SMTP is a tranparent process but the market pressures are
changing this.  I wanted to know what some are doing.   We have many
customers HURTING and they are BLAMING the SMTP software for creating the
bounces!   Yes,  you and I know whats going on.  But they don't.  We are no
position, Keith might, to argue with them :-)  So as I always DO by nature
is LOOK for solutions - what can be done?  The status quo is not acceptable
any more.

> I think this is a good discussion to have. That is, can the email system
be changed to
> reduce propagation of these types of viruses? I didn't see the initial
posting in this
> thread, so I cannot comment on any early proposals that may have made.

> ...

> So, perhaps mail-ng is the place to discuss sweeping reforms to messaging
as we
> know it, and this forum is a place to discuss shorter term solutions that
are reliable,
> effective and possible.

Nah,  Keith already decided what thats going to be.  Its another BBS!   :-)

Lets keep it here - SMTP, thats the reality.  Thats where the real change
will occur with extensions I hope and solid working proposals.

The question I ask here is to see how other vendors are addressing the
problem.  Are they using AVS api?  What experiences do they have with any
ASRG proposals?   DMP has help tremendously to protect your sender machine
from the group of spoofs return paths using your address.

Check out our SMTP statistics filtering breakdown at
http://www.winserver.com/sslinfo

> I must say I'm rather disappointed with the tone of this thread, the
innuendo and
> chest thumping on all sides.  I think list subscribers are here because
they have
> made contributions to "the technology of messaging" in the past, currently
are doing
> so, or will do so in the future.
>
> So comments like "you're clueless go away" and "I'm great look at all the
stuff I
> wrote" seem out of character.

I agree.  Unfortunately,  it is not hard for a new IETF list comer to see
what's going on here. It is "in character" as a norm with a few here and
unfortunately,  the "good old gang" mentality is prevailing.    For them, a
new comer means no experience and this unfortunate mentality has stalled
real progress.

-- 
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

Date: Thu, 5 Feb 2004 13:02:33 -0500
From: Keith Moore <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
Cc: [email-masked], [email-masked], [email-masked]
Subject: Re: Using SUBMIT without bothering the user (was Re: MyDoom, Sorbig - Actions taken?)
Sender: [email-masked]

> Keith Moore writes:
> >> Can one, for example, do some SRV DNS lookup?
> >
> > What would form the lookup key?
> 
> Exactly.
> 
> > It doesn't have anything inherent to do with the user's From address
> 
> The way the wind seems to be blowing, SPF will change that during the 
> next year or two. For better or worse.

I hope you're wrong.  Because if there's anything worse than spammers, it's
people who try to impose half-baked solutions to spam on the rest of ous.

Keith

Date: Thu, 5 Feb 2004 10:44:58 -0800 (PST)
From: "[email-masked]" <[email-masked]>
To: [email-masked]
Subject: Re: Using SUBMIT without bothering the user (was Re: MyDoom, Sorbig - Actions taken?)
Sender: [email-masked]

> > Which reminds me. How is a MUA supposed to find its submit server? 
> 
> The same way an MUA finds an SMTP server to submit mail.

I think above refers to that SMTP and POP3/IMAP server names are 
preconfigured in the mail client setup for each user.

That does not however means we can not provide default values that lot 
less advanced users could find usefull and passing default values to user 
for certain protocols is not really SUBMIT protocol only question (i.e. 
not really for ietf-smtp). The protocol that is often used to pass on 
default parameters for host configuration is DHCP when it provides 
computers with ip address, dns server name, etc. if you look at IETF 
work, I beliebve "Service Location Protocol" (see RFC2608) could be the 
new generation version ofthat. I'm not sure if anybody actually ever 
attempted to use that for configuring default SUBMIT server name - that 
would require extension to client mail program obviously, but I dont see 
why it would not work if the functionality was written. Perhaps if some 
enterprise mail package vendor would implement it and pitch it along to 
their fortune500 clients as great new "feature", word would spread and 
eventually this might get implemented by others...

-- 
William Leibzon
Elan Networks
[email-masked]

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>
Cc: "Keith Moore" <[email-masked]>
Subject: Re: MyDoom, Sorbig - Actions taken? 
Date: Thu, 5 Feb 2004 12:40:59 -0500
Organization: Santronics Software, Inc
Sender: [email-masked]

----- Original Message ----- 
From: "Keith Moore" <[email-masked]>
To: "Hector Santos" <[email-masked]>
Cc: "Keith Moore" <[email-masked]>; <[email-masked]>;
<[email-masked]>
Sent: Thursday, February 05, 2004 9:49 AM
Subject: Re: MyDoom, Sorbig - Actions taken?

> That's because you don't know what you are talking about (either about
> my focus or about mail system design).

Oh man!  What a 2 year old!  Keith,  I'm sure you more mail system product
design, development and market experience than I do.   If its more than 25
years covering design, development and marketing with every form of mail
software from MUA to mid ware to hosting, then by gawd, I don't know
anything and you are far better than me.   I am sure you also have more
experience in a "File System" and "Hosting System" too.  Do you have a web
site to check out your feature list?  Maybe I can steal a few tips for our
new versions.

I ask the QUESTION hopefully to get an intelligent discussion going, but I
guess that was too much to ask of you without pulling teeth!   if your focus
(any only answer) is user needs was a way to address MAJOR industry
transport problem - Hey, knock yourself out!   But tell us SMTP developers
that it can't address it at SMTP  - its flawed thinking.

Of course "us" not you, SMTP developers now it is a transparent process, but
"us" SMTP developers know better as the market helps mold the process.

Keith, with all due respect,  you seem to have a fundamental flawed premise
about new IETF participants having little experience and young,   I am not
making that up.  You said it yourself in a 2001 article:

http://lists.w3.org/Archives/Public/ietf-discuss/2001Nov/0097.html

I found it during research to find out why the IETF is so resistance to
solving problems and also see the mindset of the what seem to the key closed
mind participants.   Maybe you should reread the response to your message:

http://lists.w3.org/Archives/Public/ietf-discuss/2001Nov/0098.html

If I had to say it myself,  the response pretty much says it all.

All this started with a simple pertinent question!  Yeah, I know. You don't
think it was pertinent.

My apologies to OTHERS on this. What a complete waste.

-- 
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

To: Dan Wing <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: MyDoom, Sorbig - Actions taken? 
From: [email-masked]
Date: Thu, 05 Feb 2004 12:43:40 -0500
Sender: [email-masked]

--==_Exmh_483730796P
Content-Type: text/plain; charset=us-ascii

On Thu, 05 Feb 2004 09:19:38 PST, Dan Wing <[email-masked]>  said:

> > At the least, all the "hey you have a virus" automated replies
> > will go to a somewhat more appropriate location ;-)
> 
> Unfortunately, the AV companies have no doubt found there is
> marketing value in sending notices these far and wide.

Well, the bunch with the "Your computer is broadcasting an IP address"
pop-ups were told by a judge to cut it out. :)

--==_Exmh_483730796P
Content-Type: application/pgp-signature

--==_Exmh_483730796P--

From: "Hector Santos" <[email-masked]>
To: "Abhijit Menon-Sen" <[email-masked]>
Cc: <[email-masked]>
Subject: Re: MyDoom, Sorbig - Actions taken?
Date: Thu, 5 Feb 2004 12:23:39 -0500
Organization: Santronics Software, Inc
Sender: [email-masked]

----- Original Message ----- 
From: "Abhijit Menon-Sen" <[email-masked]>
To: "Hector Santos" <[email-masked]>
Cc: <[email-masked]>
Sent: Thursday, February 05, 2004 10:14 AM
Subject: Re: MyDoom, Sorbig - Actions taken?

> At 2004-02-05 09:19:05 -0500, [email-masked] wrote:
> >
> > Now, if you are are suggesting that maybe SMTP "should" perform MIME
> > processing to maybe WARN/HELP the user, then we are in tune with what
> > I hoping to get going here.
>
> Dear Mr. Santos,
>
> Just for clarification -- are you suggesting that SMTP servers perform
> MIME processing to identify potentially evil content, so that users do
> not have to deal with it?

I am not advocating it should be part of the process. It is implementation
issue.

The original question was WHAT others are doing at SMTP? if at all.

> Furthermore, are you suggesting that this is necessary because the MUA
> cannot be relied upon to protect the user adequately, once the message
> has been received?
>

Would you allow a virus to enter the system (user desktop) if you had to
potential to stop it?

I would appreciate the opportunity to clarify this.  After all,  Keith (boy,
what a character) seems to feel I don't know what I am talking about nor
experience in mail system design when in fact, I have more than him.
Obviously, he has a philosophy based on not having a product in the market
place. On the other hand, I have products and market pressures I have to
deal with.  But that's neither here or there and it is sad that he had to
stoop to that level when all wanted to know is what others are doing in this
area.

Of course, we know SMTP is  transparent process.   But is it really?

I expected 3 type of answers:

1) In regards to return paths/sender machines, they are doing XYZ and/or ABC
or nothing. I didn't expect much in this area.

2) SMTP is inherently a transparent process.  Mail Filter done by 3rd party
Integration.

3) SMTP is being redesigned to build-in the mail filter using new growing
AVS API  systems.

I clearly didn't expect the obvious - the MUA/USER!

The latter is occurring more and more due to market pressures.  Now we could
no longer rely on 3rd party solutions, but also vendor partnerships or
building it in.   Microsoft will be releasing soon an AVS API built-in to
the OS.  You can also buy them today.  Those customers with system wide AVS
software who use delay processing with our server are better off.   We don't
have to worry about them.  But that is not the norm. The pressure is to do
more with SMTP.  I know that is an implementation issue.

My original question was how other people dealing with this problem.

Passing it on to users is not acceptable solution.  Not by me. By customer
demands.

-- 
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

From: "Dan Wing" <[email-masked]>
To: <[email-masked]>, <[email-masked]>
Subject: RE: MyDoom, Sorbig - Actions taken?
Date: Thu, 5 Feb 2004 09:19:38 -0800
Importance: Normal
Sender: [email-masked]

> > > Wouldn't it be nice if the same sort of thing could be done with email
> > > messages?
> >
> > no, it wouldn't be nice to be forced to always submit mail from the same
> > location.
> >
> > IP addresses are inherently tied to network locations.  email
> > addresses are not.
>
> That's right. That's why I said this isn't really possible with
> SMTP. After all, being able to send from anywhere as anyone is
> allowed AND used frequently today.

It's possible, and reasonable, for an SMTP server that accepts
mail submission to require a username/password and apply policy
based on that username/password.  This is available today with the
AUTH extension to SMTP.

Of course today nearly all SMTP servers that accept mail
submission merely validate the legitimacy of the sender based
on the sender being in the ISP's (or enterprise's) IP address
space.

In either case, though, the SMTP server that accepts mail submission
can fairly trivially validate the MAIL FROM address, and even the
From: address, based on the IP address (or username/password) of
the user.  For example, all business mail egressing Cisco's network
should have a MAIL FROM with @cisco.com, and should be virus
checked and perhaps also be spam checked.  And all mail egressing
AOL's network should have a MAIL FROM with @aol.com.

So I don't agree that "it isn't possible with SMTP".  It actually
is possible with SMTP.

> I think I didn't make my point clear. I was suggesting a
> conceptual vision, which would rely on a briefly mentioned topic
> on the mail-ng list as "sender authentication/verification".
>
> If you can verify the sender then it could become possible to
> control message "injection" into the "mail system".  And one of
> those controls could be "is this sender allowed to send from this
> 'location'?". I'll handily leave "location" undefined.
>
> This not really possible with SMTP as it stands now anyway.

Agreed -- some changes are necessary to achieve the goal you are
describing.

> > > Imagine all the viruses that would be blocked because they could only
> > > send mail on behalf of the "originating location's mail domain".
> >
> > the viruses wouldn't be blocked, they'd just have different
> > From addresses.
>
>
> True. Perhaps I should have said "now we can track down the
> person who has a machine that's infected".

If the identity is tied to only a domain, the best you can do is
tie the email to that domain.  At that point you can contact the
domain and inform them of the problem.  It's then their problem
to track down the virus- (or spam-) infected system.

> At the least, all the "hey you have a virus" automated replies
> will go to a somewhat more appropriate location ;-)

Unfortunately, the AV companies have no doubt found there is
marketing value in sending notices these far and wide.

-d

From: Arnt Gulbrandsen <[email-masked]>
To: Keith Moore <[email-masked]>
Subject: Re: Using SUBMIT without bothering the user (was Re: MyDoom, Sorbig - Actions taken?)
Cc: [email-masked], [email-masked]
Date: Thu, 5 Feb 2004 18:14:25 +0100
Sender: [email-masked]

Keith Moore writes:
>> Can one, for example, do some SRV DNS lookup?
>
> What would form the lookup key?

Exactly.

> It doesn't have anything inherent to do with the user's From address

The way the wind seems to be blowing, SPF will change that during the 
next year or two. For better or worse.

Arnt

Date: Thu, 5 Feb 2004 11:54:51 -0500
From: Keith Moore <[email-masked]>
To: Arnt Gulbrandsen <[email-masked]>
Cc: [email-masked], [email-masked], [email-masked]
Subject: Re: Using SUBMIT without bothering the user (was Re: MyDoom, Sorbig - Actions taken?)
Sender: [email-masked]

> Which reminds me. How is a MUA supposed to find its submit server? 

The same way an MUA finds an SMTP server to submit mail.

> Can one, for example, do some SRV DNS lookup?

What would form the lookup key?  It doesn't have anything inherent to do
with the user's From address (which after all can vary from one message
to another), nor does it have anything inherent to do with the user's
ISP (I use 4 different ISPs at different times, but I want to keep the 
same email address and submission server no matter what ISP I'm using).

--
He not busy being born, is busy dying.		- Bob Dylan

From: "Brad Clements" <[email-masked]>
Organization: MurkWorks, Incorporated.
To: [email-masked]
Date: Thu, 05 Feb 2004 11:49:23 -0500
Subject: Re: MyDoom, Sorbig - Actions taken?
Reply-to: [email-masked]
Content-description: Mail message body
Sender: [email-masked]

On 5 Feb 2004 at 11:37, Keith Moore wrote:

> > Wouldn't it be nice if the same sort of thing could be done with email
> > messages?  
> 
> no, it wouldn't be nice to be forced to always submit mail from the same
> location.
> 
> IP addresses are inherently tied to network locations.  email addresses are
> not.

That's right. That's why I said this isn't really possible with SMTP. After all, being able 
to send from anywhere as anyone is allowed AND used frequently today.

I think I didn't make my point clear. I was suggesting a conceptual vision, which 
would rely on a briefly mentioned topic on the mail-ng list as "sender 
authentication/verification".  

If you can verify the sender then it could become possible to control message 
"injection" into the "mail system". And one of those controls could be "is this sender 
allowed to send from this 'location'?". I'll handily leave "location" undefined.

This not really possible with SMTP as it stands now anyway. 

> > Imagine all the viruses that would be blocked because they could only
> > send mail on behalf of the "originating location's mail domain".
> 
> the viruses wouldn't be blocked, they'd just have different From addresses.

True. Perhaps I should have said "now we can track down the person who has a 
machine that's infected". 

At the least, all the "hey you have a virus" automated replies will go to a somewhat 
more appropriate location ;-)

-- 
Brad Clements,                [email-masked]   (315)268-1000
http://www.murkworks.com                          (315)268-9812 Fax
http://www.wecanstopspam.org/                   AOL-IM: BKClements

From: Arnt Gulbrandsen <[email-masked]>
To: Keith Moore <[email-masked]>
Subject: Using SUBMIT without bothering the user (was Re: MyDoom, Sorbig - Actions taken?)
Cc: [email-masked], [email-masked]
Date: Thu, 5 Feb 2004 17:50:52 +0100
Sender: [email-masked]

Keith Moore writes:
> no, it wouldn't be nice to be forced to always submit mail from the 
> same location.
>
> IP addresses are inherently tied to network locations. email addresses 
> are not.

Which reminds me. How is a MUA supposed to find its submit server? RFC 
2476 is silent on that point. "Ask the user" strikes me as a copout.

Can one, for example, do some SRV DNS lookup?

Arnt

Date: Thu, 5 Feb 2004 11:37:25 -0500
From: Keith Moore <[email-masked]>
To: [email-masked]
Cc: [email-masked], [email-masked]
Subject: Re: MyDoom, Sorbig - Actions taken?
Sender: [email-masked]

> I'd like to draw a parallel between "outgoing messages" and "outgoing IP packets".
> 
> Here, I'm using the term "outgoing" in a very broad sense.
> 
> To reduce the amount of IP packet spoofing, router software manufacturers and 
> network admins have been encouraged to check "outgoing" packets and filter out 
> packets containing "spoofed" addresses. This can be easily done in routers and 
> could/should/has (?) reduced or eliminated spoofed packets traversing the Internet.
> 
> Wouldn't it be nice if the same sort of thing could be done with email messages?  

no, it wouldn't be nice to be forced to always submit mail from the same location.

IP addresses are inherently tied to network locations.  email addresses are not.

> Imagine all the viruses that would be blocked because they could only send mail on 
> behalf of the "originating location's mail domain".

the viruses wouldn't be blocked, they'd just have different From addresses.

From: "Dan Wing" <[email-masked]>
To: <[email-masked]>, <[email-masked]>
Subject: RE: MyDoom, Sorbig - Actions taken? 
Date: Thu, 5 Feb 2004 08:32:51 -0800
Importance: Normal
Sender: [email-masked]

> Wouldn't it be nice if the same sort of thing could be done with 
> email messages?  Imagine all the viruses that would be blocked 
> because they could only send mail on behalf of the "originating 
> location's mail domain".

I'm trying to imagine, really hard.  With such protections in
place, the viruses would have to use the sender's From: domain
to spread.  That isn't too onerous for the virus, and with a 
large enough domain (enough unique localparts) it provides too
little information to identify the infected user's PC.

> Of course, it can't really be done due to the current design of 
> SMTP.

Some ISPs, notably Earthlink, block outgoing traffic on port
25 for exactly this reason -- to apply policy on the outgoing 
mail.  They could apply any policy they want.  I have no idea
what sort of policy/policies that are currently applied to the
mail routed through their mailers, but I do know that a few years
ago they wouldn't permit using a MAIL FROM that wasn't 
@earthlink.net.

-d

> Yes, there are some proposals that allow recipients to try to 
> verify the allowability of an IP address to send mail on 
> behalf "claimed sender addresses", but I consider these to be 
> band-aid solutions.
> 
> And most importantly, controlling "outgoing" messages has nothing 
> to do with the payload of those messages. So, MIME issues, for 
> example, aren't in the picture  because the transport mechanism 
> is transparent.
>
> I think this is a good discussion to have. That is, can the email 
> system be changed to 
> reduce propagation of these types of viruses? I didn't see the 
> initial posting in this 
> thread, so I cannot comment on any early proposals that may have made.
> 
> I do think that "propagation control" is more likely to be 
> successful if its payload transparent.  I also think that given 
> the current design of SMTP, no easy, complete and reliable solution 
> for SMTP will be forthcoming.  
> 
> So, perhaps mail-ng is the place to discuss sweeping reforms to 
> messaging as we 
> know it, and this forum is a place to discuss shorter term 
> solutions that are reliable, 
> effective and possible.
> 
> I must say I'm rather disappointed with the tone of this thread, 
> the innuendo and 
> chest thumping on all sides.  I think list subscribers are here 
> because they have 
> made contributions to "the technology of messaging" in the past, 
> currently are doing 
> so, or will do so in the future.
> 
> So comments like "you're clueless go away" and "I'm great look at 
> all the stuff I 
> wrote" seem out of character.
> 
> Thanks for reading this far.
> 
> -- 
> Brad Clements,                [email-masked]   (315)268-1000
> http://www.murkworks.com                          (315)268-9812 Fax
> http://www.wecanstopspam.org/                   AOL-IM: BKClements
> 

To: Hector Santos <[email-masked]>
Cc: [email-masked]
Subject: Re: MyDoom, Sorbig - Actions taken? 
From: [email-masked]
Date: Thu, 05 Feb 2004 11:17:28 -0500
Sender: [email-masked]

--==_Exmh_440257232P
Content-Type: text/plain; charset=us-ascii

On Thu, 05 Feb 2004 02:27:43 EST, Hector Santos said:

> Yes, it is too bad that many designers of today have not taken ethical
> engineering courses like I was required.   Yes, it too bad  there was alot
> of pressure to have "automation" to be more important over security
> concerns.   But this has NOTHING to do with how the SOBIG-generation virus
> are exploiting everything that is weak about SMTP.

If it didn't get run, it wouldn't be exploiting *anything* about SMTP.

Why does it get run? Breakage at the MUA level.  Failing to understand
that means you're doomed to keep shimming your SMTP to "fix" the problem.
If you like, I'll send you the logfiles of the 3 million or so MyDoom-A
that managed to blast through our Listserv machine, which *NONE* of your
checks would actually have stopped (by sheer blind luck, coming INTO
the list the EHLO and From: and To: and MAIL FROM: and RCPT TO: were all
correctly set for a posting from a "local" user.

> While one can question of the quality assurance of the tire design, one can
> also question the quality assurance of the materials,  the manufacting,  the
> testing and the risk assessment analysis used to suggest that "risk" is a
> "user problem"  not a "vendor"  problem.     The fact is the WORLD did not
> consider to lower speed limits because it would be a RIDICULOUS suggestion
> conflictive with all other industries - car racing/chasing for one.   It is
> so dumb, it is not even worth any more further thought.

So changing the roads because the tires are bad is stupid,
but changing the SMTP because the MUA is bad isn't.

What's wrong with that picture?

--==_Exmh_440257232P
Content-Type: application/pgp-signature

--==_Exmh_440257232P--

From: "Brad Clements" <[email-masked]>
Organization: MurkWorks, Incorporated.
To: [email-masked]
Date: Thu, 05 Feb 2004 10:57:36 -0500
Subject: Re: MyDoom, Sorbig - Actions taken? 
Reply-to: [email-masked]
Content-description: Mail message body
Sender: [email-masked]

I'd like to draw a parallel between "outgoing messages" and "outgoing IP packets".

Here, I'm using the term "outgoing" in a very broad sense.

To reduce the amount of IP packet spoofing, router software manufacturers and 
network admins have been encouraged to check "outgoing" packets and filter out 
packets containing "spoofed" addresses. This can be easily done in routers and 
could/should/has (?) reduced or eliminated spoofed packets traversing the Internet.

Wouldn't it be nice if the same sort of thing could be done with email messages?  
Imagine all the viruses that would be blocked because they could only send mail on 
behalf of the "originating location's mail domain".

Of course, it can't really be done due to the current design of SMTP. Yes, there are 
some proposals that allow recipients to try to verify the allowability of an IP address 
to send mail on behalf "claimed sender addresses", but I consider these to be band-
aid solutions.

And most importantly, controlling "outgoing" messages has nothing to do with the 
payload of those messages. So, MIME issues, for example, aren't in the picture 
because the transport mechanism is transparent.

I think this is a good discussion to have. That is, can the email system be changed to 
reduce propagation of these types of viruses? I didn't see the initial posting in this 
thread, so I cannot comment on any early proposals that may have made.

I do think that "propagation control" is more likely to be successful if its payload 
transparent. I also think that given the current design of SMTP, no easy, complete 
and reliable solution for SMTP will be forthcoming.  

So, perhaps mail-ng is the place to discuss sweeping reforms to messaging as we 
know it, and this forum is a place to discuss shorter term solutions that are reliable, 
effective and possible.

I must say I'm rather disappointed with the tone of this thread, the innuendo and 
chest thumping on all sides.  I think list subscribers are here because they have 
made contributions to "the technology of messaging" in the past, currently are doing 
so, or will do so in the future.

So comments like "you're clueless go away" and "I'm great look at all the stuff I 
wrote" seem out of character.

Thanks for reading this far.

-- 
Brad Clements,                [email-masked]   (315)268-1000
http://www.murkworks.com                          (315)268-9812 Fax
http://www.wecanstopspam.org/                   AOL-IM: BKClements

Date: Thu, 5 Feb 2004 20:44:02 +0530
From: Abhijit Menon-Sen <[email-masked]>
To: Hector Santos <[email-masked]>
Cc: [email-masked]
Subject: Re: MyDoom, Sorbig - Actions taken?
Sender: [email-masked]

At 2004-02-05 09:19:05 -0500, [email-masked] wrote:
>
> Now, if you are are suggesting that maybe SMTP "should" perform MIME
> processing to maybe WARN/HELP the user, then we are in tune with what
> I hoping to get going here.

Dear Mr. Santos,

Just for clarification -- are you suggesting that SMTP servers perform
MIME processing to identify potentially evil content, so that users do
not have to deal with it?

Furthermore, are you suggesting that this is necessary because the MUA
cannot be relied upon to protect the user adequately, once the message
has been received?

Thank you.

> --
> Hector Santos, Santronics Software, Inc.
> http://www.santronics.com
> Author of Silver Xpress (offline Mail System with MUA client)
> Author of Gold Xpress (SX + comm)
> Author of Platinum Xpress (integrated mail/file frontend system with MUA
> client)
> Author of IFTP (intelligent scripting FTP client)
> Owner and chief designer of Wildcat! RPC Client/Server Intranet/Dialup
> Hosting System
> Author of Wildcat wcBASIC/SDK RPC API/SDK for c/c++, delphi, java, basic,
> foxpro

--
Abhijit Menon-Sen
Humble Acolyte

Cc: Keith Moore <[email-masked]>, <[email-masked]>, <[email-masked]>
From: Keith Moore <[email-masked]>
Subject: Re: MyDoom, Sorbig - Actions taken? 
Date: Thu, 5 Feb 2004 09:49:21 -0500
To: "Hector Santos" <[email-masked]>
Sender: [email-masked]

> I am getting a little peeve with your continued disrespect.

<yawn>.  And by your repeated verbose attempts to pass off ignorance as 
sophistication you're demonstrating _what_ exactly?

> Going back to the original question,  would this be YOUR method (using 
> MIME)
> to help solve or address your email virus problems being dump your way?

If vendors followed the spec, the only way that recipients' computers 
would be vulnerable to viruses would be if the recipients manually 
saved the attachment, and then manually launched it using a separate 
program.  It wouldn't completely keep viruses from propagating (there 
are some people stupid enough to do anything) but it would be 
sufficient to keep them from propagating quickly or widely.

> Rather to being so condescending, why don't you explain how layering 
> when
> adhered by the MUA will solve the problem of abusive transportation of
> viruses.

If the MIME specifications were followed, there would be no incentive 
to send email viruses in the first place, since there would be little 
chance of their propagating widely.  Miscreants would use other means 
to attack systems.

>    You are bent of this being an USER/MUA problem maybe that's
> because you are currently focus on "User needs."   Well,  I disagree.

That's because you don't know what you are talking about (either about 
my focus or about mail system design).

The MUA is the part of the system that represents the user's interests 
(as in, not having his computer attacked).  The MUA is the part of the 
system that knows about message content (as in, knowing the meanings of 
attachment types and what to do with them).  The MUA is the part of the 
system that has to adapt to, and learn how to handle, new kinds of 
content.  The MTS doesn't know about any of these things, and for good 
reason.  The MTS doesn't know about the characteristics of, or 
configuration of, end systems.  Content that is perfectly legitimate to 
one recipient is potentially harmful to another.  The MTS has no way of 
knowing the difference.

To expect an MTA to be better at filtering viruses than an MUA is to 
expect a part of the system that doesn't act on behalf of a recipient 
to represent the recipient's interests better than the recipient's MUA. 
  It expects the MTA to know almost as much about content as the MUA 
even though the MTA's job is to be transparent.  It expects MTAs to 
adapt as quickly to changes in content used by recipients as the 
recipient's MUAs.  It makes it harder to transmit new kinds of content 
over the Internet because it will require the upgrade of all MTAs in 
the path, and it therefore makes Internet mail less flexible and less 
useful.  A mail architecture that did that would expect ISPs to spend 
large sums of money to compensate for Microsoft's negligence.

Now, there's something to be said for security in depth, and having a 
virus filter near the recipient, say attached to the recipient's 
inbound SMTP server and message store, as a backup to accidental bugs 
in the recipient's MUA, probably does make some sense.  And if it's 
placed there then it can still be responsive to recipient's needs (as 
in, "yes, I really do need the message with the three .exe attachments 
delivered intact; I was expecting them and the message was digitally 
signed by the vendor who sent them to me.").  But you don't need 
support in SMTP to do that.

> You still have not address the question when MIME is not used for 
> attachment transportation via SMTP.

You solve that problem in the same way as for MIME attachments; it's 
just not in scope for the MIME spec to say that.

Keith

From: "Hector Santos" <[email-masked]>
To: "Arnt Gulbrandsen" <[email-masked]>
Cc: <[email-masked]>
Subject: Re: MyDoom, Sorbig - Actions taken?
Date: Thu, 5 Feb 2004 09:19:05 -0500
Organization: Santronics Software, Inc
Sender: [email-masked]

----- Original Message ----- 
From: "Arnt Gulbrandsen" <[email-masked]>
To: "Hector Santos" <[email-masked]>
Cc: <[email-masked]>
Sent: Thursday, February 05, 2004 8:24 AM
Subject: Re: MyDoom, Sorbig - Actions taken?

> Hector Santos writes:
> > Can you elaborate what exactly they are not "adhering" to?
>
> If I send you a body-part with content-type image/jpeg, that's what it
> is. Some MUAs do this:
>
>   [clip]

Hello?  It was really a rhetorical question!  You are not saying anything
new to me.  Its too late by this POINT!

> Another evil problem is to misparse boundary lines. If a MUA gets it
> wrong, then a message can pass through the firewall, pass through the
> virus checker, and by exploiting the MUA bug still cause much damage.
>

This has nothing to do with my question is how can SMTP address this issue
BEFORE, and I said again, BEFORE the MUA gets the message.

Why?

Because when the USER gets the message, its too late.  Anything can happen,
such as the user being fooled which is what you, Keith and Valdis are
saying!

Well DUH!

Now, if you are are suggesting that maybe SMTP "should" perform MIME
processing to maybe WARN/HELP the user, then we are in tune with what I
hoping to get going here.  Instead, I getting comments that are OUT of the
world!

> In the future, PLEASE try a google search rather than asking such
> questions on mail-ng.

MAIL-NG?   The last I looked this isn't MAIL-NG!  You are really bent on
slamming me no matter what.

Whats so hard about maybe trying to answer the original questions?

Maybe you should stop passing the buck to end-users MUA software?  Its too
late by that point!

Maybe you should google santronics.com (http://www.santronics.com) and GET A
FEEL about what I do and the software product lines that ranging across the
entire spectrum from MUA wares to middle ware, gating, frontends,  hosting
products, etc, before being so disrespectful?

-- 
Hector Santos, Santronics Software, Inc.
http://www.santronics.com
Author of Silver Xpress (offline Mail System with MUA client)
Author of Gold Xpress (SX + comm)
Author of Platinum Xpress (integrated mail/file frontend system with MUA
client)
Author of IFTP (intelligent scripting FTP client)
Owner and chief designer of Wildcat! RPC Client/Server Intranet/Dialup
Hosting System
Author of Wildcat wcBASIC/SDK RPC API/SDK for c/c++, delphi, java, basic,
foxpro

From: Arnt Gulbrandsen <[email-masked]>
To: Hector Santos <[email-masked]>
Subject: Re: MyDoom, Sorbig - Actions taken?
Cc: [email-masked]
Date: Thu, 5 Feb 2004 14:24:12 +0100
Sender: [email-masked]

Hector Santos writes:
> Can you elaborate what exactly they are not "adhering" to?

If I send you a body-part with content-type image/jpeg, that's what it 
is. Some MUAs do this:

  1. Tell the user there's a JPEG attachment and offer to show it.
  2. When the user issues the "show attachment" command:
  3a. Notice that the first six bytes (after base64 decoding) are "GIF87a"
  3b. Choose to treat the body-part as image/gif instead of image/jpeg
  3c. Show the attachment as a GIF file.

Sounds harmless to you? But if you do it you're deceiving your user. 
Deception leads to misunderstandings and sometimes worse. What if the 
body-part started with "MZ" or "%!" instead of "GIF87a"?

Another evil problem is to misparse boundary lines. If a MUA gets it 
wrong, then a message can pass through the firewall, pass through the 
virus checker, and by exploiting the MUA bug still cause much damage.

In the future, PLEASE try a google search rather than asking such 
questions on mail-ng.

Arnt

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>
Cc: <[email-masked]>, "Keith Moore" <[email-masked]>
Subject: Re: MyDoom, Sorbig - Actions taken? 
Date: Thu, 5 Feb 2004 02:27:43 -0500
Organization: Santronics Software, Inc
Sender: [email-masked]

----- Original Message ----- 
From: <[email-masked]>
To: "Hector Santos" <[email-masked]>
Sent: Thursday, February 05, 2004 1:42 AM
Subject: Re: MyDoom, Sorbig - Actions taken?

>> On Thu, 05 Feb 2004 01:29:37 EST, Hector Santos said:

>> Rather to being so condescending, why don't you explain how layering when
>> adhered by the MUA will solve the problem of abusive transportation of
>> viruses.   You are bent of this being an USER/MUA problem maybe that's
>> because you are currently focus on "User needs."   Well,  I disagree.
You
>> still have not address the question when MIME is not used for attachment
>> transportation via SMTP.

> First off, the 'begin 644 foo.tar' went out of style a decade ago.

It has nothing to do whether you believe it went out of style or not.   That
isn't your option.  However, the OPTION is still available at the MUA level
in many applications and there are MANY legacy applications (which seem to
apply better with the mindset of people in IETF) that still suffer from
using the "out of styte" method.

> Secondly, even when it was in use, there were MUAs that would
automatically
> extract a uuencode - but none that I know of was ever silly enough to
> extract-and-run.

This has nothing to do with Extract and Run.   Nothing.  That applies to ALL
malicious virus that reach users. Give a baby a lolly pop and without you
doing anything or teaching it, he will eventually lick it!   The problem is
giving to him in the first place.  How can you prevent all the lolly pops
being thrown in his face?

Yes, it is too bad that many designers of today have not taken ethical
engineering courses like I was required.   Yes, it too bad  there was alot
of pressure to have "automation" to be more important over security
concerns.   But this has NOTHING to do with how the SOBIG-generation virus
are exploiting everything that is weak about SMTP.

> And you know what?  If the MUAs aren't supporting extract-and-run,
> either MIME, or UUENCODE, or even encoded as a Lisp S-expr, then it
> *doesn't matter* what the SMTP layer is doing.

Well, I guess this explains why we will continue to have problems with SMTP
being abused.

> Go back and ponder.  Firestone. Speed limits.  That's what you're asking
for.

No am I not.  In your (naive) UNDERSTANDING of the problem, you are.  Not
me.

> When you realize that tire problems have to be fixed *at the tire*,
> then maybe you'll be able to contribute.

Valdis,  it was a ridiculous analogy in your last message as it still is
now.  I could easily blew this analogy away.

While one can question of the quality assurance of the tire design, one can
also question the quality assurance of the materials,  the manufacting,  the
testing and the risk assessment analysis used to suggest that "risk" is a
"user problem"  not a "vendor"  problem.     The fact is the WORLD did not
consider to lower speed limits because it would be a RIDICULOUS suggestion
conflictive with all other industries - car racing/chasing for one.   It is
so dumb, it is not even worth any more further thought.

What I am reading instead is both embarassing and insulting and frankly, I
have contributed to solving the problem and it was not putting the burden
entirely on the user.  What have you done with your SMTP product offerings
to customers?

If you would like to explain how your smtp product (if you have one) or your
employer/company has addressed the SORBIG generation viruses in terms on how
it handles bounces it generates further burdening downlinks,  I will be
interested to hear you share this.   If this is proprietary information,
then say so, I can perfectily understand this too.

I bet your employer SMTP servers are stripping attachments and/or delaying
deliver until analyzed automatically or manually by some IS support staff.
This is a process that begins SMTP, not at the MUA.

Oh what's the point.  The status quo remains.

Thanks for your comment.

-- 
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

To: Hector Santos <[email-masked]>
Cc: [email-masked], Keith Moore <[email-masked]>
Subject: Re: MyDoom, Sorbig - Actions taken? 
From: [email-masked]
Date: Thu, 05 Feb 2004 01:42:45 -0500
Sender: [email-masked]

--==_Exmh_-472295803P
Content-Type: text/plain; charset=us-ascii

On Thu, 05 Feb 2004 01:29:37 EST, Hector Santos said:

> Rather to being so condescending, why don't you explain how layering when
> adhered by the MUA will solve the problem of abusive transportation of
> viruses.   You are bent of this being an USER/MUA problem maybe that's
> because you are currently focus on "User needs."   Well,  I disagree.  You
> still have not address the question when MIME is not used for attachment
> transportation via SMTP.

First off, the 'begin 644 foo.tar' went out of style a decade ago.

Secondly, even when it was in use, there were MUAs that would automatically
extract a uuencode - but none that I know of was ever silly enough to
extract-and-run.

And you know what?  If the MUAs aren't supporting extract-and-run,
either MIME, or UUENCODE, or even encoded as a Lisp S-expr, then it
*doesn't matter* what the SMTP layer is doing.

Go back and ponder.  Firestone. Speed limits.  That's what you're asking for.
When you realize that tire problems have to be fixed *at the tire*,
then maybe you'll be able to contribute.

--==_Exmh_-472295803P
Content-Type: application/pgp-signature

--==_Exmh_-472295803P--

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>
Cc: "Keith Moore" <[email-masked]>, <[email-masked]>
Subject: Re: MyDoom, Sorbig - Actions taken? 
Date: Thu, 5 Feb 2004 01:29:37 -0500
Organization: Santronics Software, Inc
Sender: [email-masked]

I am getting a little peeve with your continued disrespect.  Do not believe
for one minute you "better than me" by any means. You don't want to go
there. I ask you to cool it.

Going back to the original question,  would this be YOUR method (using MIME)
to help solve or address your email virus problems being dump your way?

Rather to being so condescending, why don't you explain how layering when
adhered by the MUA will solve the problem of abusive transportation of
viruses.   You are bent of this being an USER/MUA problem maybe that's
because you are currently focus on "User needs."   Well,  I disagree.  You
still have not address the question when MIME is not used for attachment
transportation via SMTP.

-- 
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

----- Original Message ----- 
From: "Keith Moore" <[email-masked]>
To: "Hector Santos" <[email-masked]>
Cc: "Keith Moore" <[email-masked]>; <[email-masked]>;
<[email-masked]>
Sent: Thursday, February 05, 2004 12:53 AM
Subject: Re: MyDoom, Sorbig - Actions taken?

> >  If people here (who seem to have a
> > strong will over what is done or said) really honestly believe there
> > is SMTP
> > can't help, and the adhering to MIME is the only anwser,  there will
> > never
> > be a solution.
>
> apparently you don't understand the concept of layering.
>
> feel free to rejoin the conversation once you figure it out.
>
>

Cc: Keith Moore <[email-masked]>, <[email-masked]>, <[email-masked]>
From: Keith Moore <[email-masked]>
Subject: Re: MyDoom, Sorbig - Actions taken? 
Date: Thu, 5 Feb 2004 00:53:08 -0500
To: "Hector Santos" <[email-masked]>
Sender: [email-masked]

>  If people here (who seem to have a
> strong will over what is done or said) really honestly believe there 
> is SMTP
> can't help, and the adhering to MIME is the only anwser,  there will 
> never
> be a solution.

apparently you don't understand the concept of layering.

feel free to rejoin the conversation once you figure it out.

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>
Cc: <[email-masked]>, <[email-masked]>
Subject: Re: MyDoom, Sorbig - Actions taken? 
Date: Thu, 5 Feb 2004 00:26:04 -0500
Organization: Santronics Software, Inc
Sender: [email-masked]

hmmmmmm,

I understand 100% the attachments issue,  the  "passing executables."   And
I too believe it was UNETHICAL for the "microsofts" of the world to
introduce OLE automation in the name of "feature integration" as well as the
old unix wienies of the past to allow for automa of "data" execution, sort
of like the espilon of APL or LISP where interpreted DATA turns into CODE,
etc.

But that is all besides the point.  If people here (who seem to have a
strong will over what is done or said) really honestly believe there is SMTP
can't help, and the adhering to MIME is the only anwser,  there will never
be a solution.

It is not only about the USER or the SOFTWARE on his end.   UUENCODING was
around for nearly 20+ years, certainly many years before MIME came along.
So it not only about MIME.  Thats just a way to pass the attachments with
much more definition of course, that can help identification.  But isn't the
issue.

Part of the problem and THUS the reason, the Microsoft and the SCO have
$500,000 bounties on the culprits, is the transport system allows for it to
propagate via an exploitation of SMTP's weaknesses.

-- 
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

----- Original Message ----- 
From: <[email-masked]>
To: "Hector Santos" <[email-masked]>
Cc: <[email-masked]>; <[email-masked]>
Sent: Wednesday, February 04, 2004 11:27 PM
Subject: Re: MyDoom, Sorbig - Actions taken?

On Wed, 04 Feb 2004 22:39:53 EST, Hector Santos
<[email-masked]>  said:

> I guess the lack of response means people don't feel this is a problem
SMTP
> could help address?

What would help this (and a lot of other security issues) a lot more is
if certain vendors actually paid attention to all the warnings about
transporting active content in e-mail ever since RFC1341 came out like
12 years ago.

Your proposal doesn't really do any good - consider that it's quite
possible for a virus to get loose on an Exchange server and ruin several
thousand people's days without ever going anywhere near SMTP (remember
that some are multi-vector, so all it takes is one Exchange user on Kazaa..)

Putting warts onto one protocol because one vendor can't get another
protocol
right is a highly questionable practice at best.  It's the moral equivalent
of reducing the speed limit to 30 on all the interstates because one tire
company had trouble making tires that didn't have the threads fall off at
higher speeds.

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>
Subject: Re: MyDoom, Sorbig - Actions taken?
Date: Thu, 5 Feb 2004 00:02:05 -0500
Organization: Santronics Software, Inc
Sender: [email-masked]

How so Keith?

Can you elaborate what exactly they are not "adhering" to?

What if the attachment was uuencoded instead?

-- 
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

----- Original Message ----- 
From: "Keith Moore" <[email-masked]>
To: "Hector Santos" <[email-masked]>
Cc: "Keith Moore" <[email-masked]>; <[email-masked]>
Sent: Wednesday, February 04, 2004 11:03 PM
Subject: Re: MyDoom, Sorbig - Actions taken?

> > It not just about the user.
> 
> no, it's the "mail reader" _software_ (the mail user agent) failing to 
> adhere to the MIME spec that allows these viruses to propagate so well.
> 
> 
> 
> 

To: Hector Santos <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: MyDoom, Sorbig - Actions taken? 
From: [email-masked]
Date: Wed, 04 Feb 2004 23:27:25 -0500
Sender: [email-masked]

--==_Exmh_-511123531P
Content-Type: text/plain; charset=us-ascii

On Wed, 04 Feb 2004 22:39:53 EST, Hector Santos <[email-masked]>  said:

> I guess the lack of response means people don't feel this is a problem SMTP
> could help address?

What would help this (and a lot of other security issues) a lot more is
if certain vendors actually paid attention to all the warnings about
transporting active content in e-mail ever since RFC1341 came out like
12 years ago.

Your proposal doesn't really do any good - consider that it's quite
possible for a virus to get loose on an Exchange server and ruin several
thousand people's days without ever going anywhere near SMTP (remember
that some are multi-vector, so all it takes is one Exchange user on Kazaa..)

Putting warts onto one protocol because one vendor can't get another protocol
right is a highly questionable practice at best.  It's the moral equivalent
of reducing the speed limit to 30 on all the interstates because one tire
company had trouble making tires that didn't have the threads fall off at
higher speeds.

--==_Exmh_-511123531P
Content-Type: application/pgp-signature

--==_Exmh_-511123531P--

Cc: Keith Moore <[email-masked]>, <[email-masked]>
From: Keith Moore <[email-masked]>
Subject: Re: MyDoom, Sorbig - Actions taken?
Date: Wed, 4 Feb 2004 23:03:55 -0500
To: "Hector Santos" <[email-masked]>
Sender: [email-masked]

> It not just about the user.

no, it's the "mail reader" _software_ (the mail user agent) failing to 
adhere to the MIME spec that allows these viruses to propagate so well.

From: "Hector Santos" <[email-masked]>
To: <[email-masked]>, <[email-masked]>
Subject: Re: MyDoom, Sorbig - Actions taken?
Date: Wed, 4 Feb 2004 22:39:53 -0500
Organization: Santronics Software, Inc
Sender: [email-masked]

Hmmmmmmm,

It not just about the user. While that is true that top level, the key
problem is the distribution.  That is what makes this SOBIG-generation virus
different from any other normal "Do not open Attachment" email you might
get.

It is more than just "Social Engineering."

The virus success is 100% based on the weakness of the SMTP system and the
current behavior of ANTI-SPAM software to assist in the distribution
process.   It is design exploit the SMTP inability to validate return-paths
and/or sender machines

1)  Initial user gets mail or downloads the virus from somewhere.

2) Virus is installed,  gathers user Outlook  Address book.

3) It begins to send out mail to recepients using a mixed Return Path.

4) Distribution Pattern #1  Mail that gets delivered ->  go to step 1.

5) Distribution Pattern #2 - Expired addresses - Mail bounced back to return
paths.

6) Distribution Pattern #3 - Anti-Spam Software - Filtered Mail bounced back
to return paths.

With the SORBIG virus,  some AVS vendors learned to remove attachment in
bounced mail and they also learned to use a NULL address to avoid further
loops in the process.  This is evident in this new virus.

The SPF, DMP proposals has helped my customers eliminate the mis-matched
return-path/sender machine pairs.  CBV also help to reduced those return
paths mixed in the distribution that are no longer valid.  Mail Filters did
the rest.

I guess the lack of response means people don't feel this is a problem SMTP
could help address?

-- 
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

----- Original Message ----- 
From: "Keith Moore" <[email-masked]>
To: "Hector Santos" <[email-masked]>
Cc: <[email-masked]>; <[email-masked]>
Sent: Wednesday, February 04, 2004 5:03 PM
Subject: Re: MyDoom, Sorbig - Actions taken?

>
> > I would be extremely interesting in hearing from SMTP developers here
how
> > they are addressing the latest big problem of the MyDoom virus with
thier
> > customers,etc?  What they have learn to better addressed the problem
with
> > the current specs.
> >
> > Does anyone have the complete "technical logic" of how this email virus
> > exploits SMTP?
>
> the virus doesn't exploit SMTP.  it exploits mail readers that fail to
> follow the MIME specification.
>
> --
> He not busy being born, is busy dying.  - Bob Dylan
>
>

Date: Wed, 4 Feb 2004 17:03:32 -0500
From: Keith Moore <[email-masked]>
To: "Hector Santos" <[email-masked]>
Cc: [email-masked], [email-masked]
Subject: Re: MyDoom, Sorbig - Actions taken?
Sender: [email-masked]

> I would be extremely interesting in hearing from SMTP developers here how
> they are addressing the latest big problem of the MyDoom virus with thier
> customers,etc?  What they have learn to better addressed the problem with
> the current specs.
> 
> Does anyone have the complete "technical logic" of how this email virus
> exploits SMTP?

the virus doesn't exploit SMTP.  it exploits mail readers that fail to 
follow the MIME specification.

--
He not busy being born, is busy dying.		- Bob Dylan

From: "Hector Santos" <[email-masked]>
To: "IETF-SMTP" <[email-masked]>
Subject: MyDoom, Sorbig - Actions taken?
Date: Wed, 4 Feb 2004 09:42:08 -0500
Organization: Santronics Software, Inc
Sender: [email-masked]

I hope I am not off topic here with this.

I would be extremely interesting in hearing from SMTP developers here how
they are addressing the latest big problem of the MyDoom virus with thier
customers,etc?  What they have learn to better addressed the problem with
the current specs.

Does anyone have the complete "technical logic" of how this email virus
exploits SMTP?

I know it relies on sending stolen valid addresses and sending "innocent"
looking bounces, but  I believe there are some other "tricky aspects"  about
it  I think may or may not be addressable by SMTP.

If I'm off topic, please ignore. :-)

-- 
Hector Santos, Santronics Software, Inc.
http://www.santronics.com

Date: Tue, 03 Feb 2004 11:29:12 -0500
From: Tony Hansen <[email-masked]>
To: "Richard O. Hammer" <[email-masked]>
CC: [email-masked]
Subject: Re: when to dot-stuff and dot-strip
Sender: [email-masked]

If you're ONLY using SMTP and POP, then this is a viable optimization. 
But if you throw in IMAP or direct user access to the files, then the 
optimization gets in the way.

	Tony Hansen
	[email-masked]

Richard O. Hammer wrote:
> In an MTA which I am developing, I dot-strip every message body which 
> comes in through SMTP.  Later I dot-stuff every message body which goes 
> out through SMTP or POP3.  (Reference RFC 2821 section 4.5.2 and RFC 
> 1939 section 3 paragraph 4.)
> 
> But I never use the message body between the times when I dot-strip and 
> dot-stuff.  Unless I am mistaken, I could entirely skip the 
> dot-stripping and dot-stuffing, given these functions which I am 
> performing.
> 
> So, if this sort of optimization is worth considering, I think it would 
> entail rethinking the boundaries which surround where message bodies 
> need to be dot-stuffed.  Perhaps dot-stuffing would occur when a message 
> first entered the SMTP system, dot-stripping would occur when a message 
> was handed to some local process where no further Internet transport was 
> expected.

Date: Tue, 03 Feb 2004 09:38:31 -0500
From: "Richard O. Hammer" <[email-masked]>
To: [email-masked]
Subject: when to dot-stuff and dot-strip
Sender: [email-masked]

In an MTA which I am developing, I dot-strip every message body which 
comes in through SMTP.  Later I dot-stuff every message body which 
goes out through SMTP or POP3.  (Reference RFC 2821 section 4.5.2 and 
RFC 1939 section 3 paragraph 4.)

But I never use the message body between the times when I dot-strip 
and dot-stuff.  Unless I am mistaken, I could entirely skip the 
dot-stripping and dot-stuffing, given these functions which I am 
performing.

So, if this sort of optimization is worth considering, I think it 
would entail rethinking the boundaries which surround where message 
bodies need to be dot-stuffed.  Perhaps dot-stuffing would occur when 
a message first entered the SMTP system, dot-stripping would occur 
when a message was handed to some local process where no further 
Internet transport was expected.

Rich Hammer
mailscreen.net
Hillsborough, N.C.

Date: Mon, 02 Feb 2004 15:56:37 -0500
To: John C Klensin <[email-masked]>, [email-masked]
From: Martin Duerst <[email-masked]>
Subject: Re: Comments on and FWD: I-DACTION:draft-klensin-emailaddr-i18n-02.txt
Cc: [email-masked]
Sender: [email-masked]

Hello John,

I have read your draft. Congratulations, I think it is
extremely well written and well argued.

I have the following comments:

- Editorial: occasionally, please make shorter paragraphs.
   It makes things a bit easier to read.

- Name of the extension: Something a little bit more specific
   than just "I18N" would probably be better.

- 3.4.1: Quite a bit of text is spent on the idea to upgrade
   Web browsers to automagically detect and convert special
   (e.g. ACE-like) encoding forms in the context of Webmail.
   This is a total non-starter, and you should just say so.
   What I have not seen discussed, but is much more feasible
   (although it's still much better to not have to do it) is
   to upgrade the webmail software on the server side to
   convert things from ACE or whatever to actual characters
   (or numeric character references if the charset used in
   the Web page has limitations) and back.

- Editorial: I don't think you have used the term ACE (ASCII
   compatible encoding), but it would be more precise in some cases
   (instead of writing something like 'IDN/punicode-like').

- Editorial: In point 2. of 4.1, it's unclear how many alternatives
   there are, or what goes logically together.

- Regarding point 3. in 4.1, and point 7.2, I think allowing
   anything else but UTF-8 should not only be strongly discouraged,
   it should be forbidden. There are some reasons for this:
   - Does the current LHS allow anything else than US-ASCII?
   - Not having a globally consistent way to interpret the
     bytes in there as characters creates all kinds of problems.
   - For security reasons (e.g. to avoid smuggling of syntax-
     relevant US-ASCII characters in overlong encoding), on
     may want to run various checks on UTF-8. This won't
     work with arbitrary byte sequences.

- Editorial: 4.3, first paragraph, overstriking: This indeed has been
   used in the past. Is it still used? Where? I would tone this down
   a bit more, e.g. change "that there is a long history" ->
   "that historically, there has been some use".

- Editorial: 4.3 "it SHOULD first verify that the string is
   valid for a domain name according to IDNA rules": There should
   be a reference to the relevant section/operation/flags in IDNA.

- 6., point 4.: Yes, please bundle UTF-8-HEADERS and your proposal,
   and make 8BITMIME mandatory. This will strongly simplify
   implementation and deployment.

- 7.1: allowing punycode in domain names with UTF-8 LHS seems
   no big harm. Mail should still be delivered in such a case.
   But the question (more for Paul's draft than for yours) is
   whether MUAs should convert this to something readable.
   Probably saying MAY is the right thing here, to mark the
   use of punycode in this case as an occasional option, not
   something that should be used in general.

- 7.2: See above.

- 7.3: First a stupid question: What characters are excluded by
   prohibiting quoted strings or characters? If it's things
   such as spaces, quotes, and so on, excluding them is probably
   not a bad idea.
   Rather than using restrictions, it may be good to give some
   guidelines.
   Some guidelines or restrictions may be needed for right-to-
   left characters/bidirectional issues.

7.4: see above; yes, please require 8BITMIME

7.5: Yes, if this extension and 8BITMIME are in use,
      RFC 2047 encoding should be dropped.

8: Editorial: "accomodate local character sets": Is this sets
    in the strict mathematical sense (in this case I suggest
    using 'character repertoires') or does it include encodings
    (then I would suggest 'character encodings' or maybe 'charset's).
    "some character script other than ASCII": Latin, Cyrillic,...
    are scripts. ASCII isn't a scrip.

8: ideas such as SEPARATOR="...": In my opinion, don't.
    Scripts around the world have imported/accepted punctuation from
    other scripts (in particular Latin) much better than letters.

Regards,   Martin.

At 12:41 04/01/30 -0500, John C Klensin wrote:
>Hi.  This draft is an update on the SMTP approach to mailbox 
>internationalization.
>
>Three things:
>
>         * I am copying the SMTP list on this because everything
>         is getting connected to everything else, as my on-list
>         discussion with Nathaniel about the "trace fields in the
>         envelope" proposal illustrates.  Please, to preserve
>         everyone's sanity, do not start parallel discussions or
>         cross-post -- this one is clearly an IMAA issue, at
>         least for historical reasons.
>
>         * There is one more proposal I-D coming in this series.
>         As a one-line summary, it outlines ("specifies" would be
>         too strong at this stage, but that is the intent) the
>         encapsulation model that this draft suggests for
>         downgrading.
>
>         * Very high-level summary of changes from the -01
>         version: I have become convinced that, if we are going
>         to have an internationalized structure for email --not
>         merely a collection of kludges and workarounds-- we are
>         going to need to make some rather basic changes.  They
>         include SMTP extensions for i18n addresses and alternate
>         addresses and UTF-8 headers as well as UTF-8 headers, in
>         some form, themselves.  I've picked up on Paul's UTF-8
>         header proposal because it seems sensible and nothing
>         else is on the table, but the proposal announced below
>         eliminates the need for alternate address headers and
>         all of the mucking around in the message payload by MTAs
>         they imply.   And I'm convinced that we should have
>         _one_ header to cover this rather than a collection.
>         I.e., the model is that one is internationalized or not,
>         rather than one in which internationalization is done
>         piecemeal and incrementally, with all of the profiling,
>         multiplicative cases, and long-term cruft that would
>         imply.
>
>Please do not react to what you think is in this proposal without reading 
>it or, especially based only on what you think it is about from reading 
>the above.   That is a waste of your time and that of everyone on the 
>list, as previous rounds of such discussions amply demonstrate.
>
>best,
>    john

From: "Hector Santos" <[email-masked]>
To: "Carl Malamud" <[email-masked]>, <[email-masked]>
Subject: Re: revised draft-malamud-no-soliciting-05
Date: Sun, 1 Feb 2004 12:16:57 -0500
Organization: Santronics Software, Inc
Sender: [email-masked]

----- Original Message ----- 
From: "Carl Malamud" <[email-masked]>
To: <[email-masked]>
Sent: Saturday, January 31, 2004 1:37 PM
Subject: revised draft-malamud-no-soliciting-05

>
> Hi -
>
> Based on comments received, I've put a new draft at:
>
> http://trusted.resource.org/no-solicit/
>
> It has also been resubmitted to the i-d queue.  Thanks
> for everybody for your comments.  Please feel free to
> send additional comments to this list, to me, to the
> iesg, or to the forum of your choice.
>
> Regards,
>
> Carl
>
>
>

A few minor comments.  Please take these from the viewpoint of an
experienced developer, technical document writer, reviewer, etc, as if I was
going to use this specification for the first time to implement, i.e.,  it
was endorsed,  I found it on the net and I was going to use the spec for the
first time.  Mostly about wording.

| 2. The No-Soliciting SMTP Service Extension
|
|   Per [RFC2821], a "NO-SOLICITING" SMTP service extension is defined..

This sentence made me pause for a second wondering if this means the
"NO-SOLICITING" extension is actually defined in RFC2821.  Don't laugh but
just to make sure I actually went ahead and double checked. <g>   It also
made me think that maybe it might mean it was something planned for a new
RFC2821.

I think it should be paraphrased, maybe like so:

    Per EHLO specification  [RFC2821],  a new "NO-SOLICITING" EHLO service
extension is defined.

A similar statement in:

| 2.5 Solicitation Mail Header
|
|  Per [RFC2822], a new "Solicitation:" header field is defined which
|   contains one or more solicitation class keywords.

This sounds like the header is defined in the '2822' document or that your
document is augmented with planned changes to the 2822 document.  Again,
don't laugh!  I definitely double checked 2822 to see if there was a new
"Solicitation:" header defined.   Made me think I had missed something in
basic 2822 mail support! <g>

To rephrase this sentence, do you really need to say "Per [RFC2822]?"
Maybe a rewording can be:

    Following RFC2822 for creating and formatting network control headers,
this specification defines a new "Solicitation:"
    header field which contains one or more solicitation class keywords.

| 2.1 The EHLO Exchange

Possibly a note should be made the NO-SOLICITING extension support "does
not" | "should not"  imply current servers a similar no solicitating tag or
message in the greeting is now deprecated.    I see your historical note
about Levine/Hoffman and the greeting tag and bandwidth consideration.   In
my view,  it is only a  (small) consideration now  when both are used.  It
is not when one or the other is used.   So it should be discussed as to what
you recommend as there will be implementations already using a greeting tag.
In other words,  "What should they do? if anything.  What do you recommend?"

In the same vain, unless I missed it,  a comment about minimal support?
for clients and servers?

For Clients, I see 3 things:

       solicitation: header
       detecting EHLO No Soliciting
       using /MAIL FROM

You might clients might want to simply add the solicitation header only or
they might just want to look fo "NO SOLICITING" in the ehlo.

For servers:

       Using EHLO No Soliciting
       Adding a policy to support SOLICIT=
       Adding a policy to support Solicitation:

I hope these comments are useful and not redundant.

--- 
Hector Santos, Santronics Software, Inc.
http://www.santronics.com