Subject: Does marid-submitter-02 really make sense?

From: Hadmut Danisch <[email-masked]>
Date: Thu, 5 Aug 2004 09:40:27 -0700

Hi,

to paraphrase my yesterday's session's question:

Does the SUBMITTER parameter really make sense?

Case 1: The sender is authorized by some domain to 
send messages. The message will have to be accepted and therefore 
to be transmitted. No benefit in checking the parameter before 
transmission. Could do this also after transmission.

Case 2: The sender is not authorized or does not want 
to reveal her identity (e.g. spammer). Sender would be stupid 
if she gave a SUBMITTER parameter not covering her IP address. 
So the sender will certainly do as if she had not yet implemented the 
SUBMITTER, i.e. omit it. Message will be transmitted as well.

So what sense does it make to introduce this new extension? Could have
done the same thing better based on header checks after transmission
of message. An additional header entry is much easier to implement or
configure. 

The only sense I can see is a smooth transition: E.g. allow both -
submitter and header based checks - until Dec-31 2005 and require
everyone to support it by then. But if this is desired, then this is a
completely wrong way.

In such a case the MARID group should not hide the smooth transition
in a SMTP extension. Instead it should mark this all as a "temporary
ad hoc solution to fight spam and crime now", and reengineer a better
solution then. 

Yesterday's answers to my question didn't convince me. On the
contrary. It caused doubts whether this is really a well planned 
solution or rather a workaround that does not really work around. 

Wouldn't it be better to allow a new header entry line instead?

regards
Hadmut

---

From: [email-masked]
Date: Thu, 05 Aug 2004 10:11:33 -0700 (PDT)

> transmission. Could do this also after transmission.

Maybe, maybe not. Having the information early might in some MTAs allow for
throughput optimizations.

> SUBMITTER, i.e. omit it. Message will be transmitted as well.

You're missing cases 3 and 4:

Case 3: The sender should be authorized by some domain but for whatever
reason is not. This extension allows this condition to be detected early
and may avoid transfer of a large message.

Case 4: The sender should not be authorized and uses SUBMITTER anyway.
(Even if you assume spammers never do this directly, which may or may
not be true, there are cases where mail is forced through intermediaries
that might not be prepared to do a validation check but would be able to add a
SUBMITTER field.) This again transfer of the message body to be avoided.
This is clearly a corner case, but it needs to be on the
list.

> of message. An additional header entry is much easier to implement or

There is absolutely no value and considerable harm in using an additional
header field. The only advantage SUBMITTER has is temporal in nature - it
delivers the information sooner. This is lost with a header that's delivered at
the same time as all the others. And in order to be effective the field has to
be compared with the value derived from other other header fields, so the same
amount of work has to be done in either case.

Even worse, when there's a header present containing this information there are
bound to be lazy implementations that will not perform the check. This opens a
security hole. Of course SUBMITTER has a similar hole, but the liklihood
of someone bothering to implement SUBMITTER and not do the header check
seems a lot smaller than the liklihood of someone just using a header
value without the check.

> Wouldn't it be better to allow a new header entry line instead?

Absolutely not. I don't especially care if SUBMITTER happens or not - I can
implement it and I can live without it. But I'm strongly opposed to a header
equivalent of SUBMITTER.

				Ned

---

From: Hadmut Danisch <[email-masked]>
Date: Thu, 5 Aug 2004 11:01:55 -0700

> throughput optimizations.

Really? I don't see significant potential for optimizations. 

And even if you do need this you can rely on the fact that the 
message header is transmitted before the message body. If you want to
optimize nobody keeps you from evaluating the message header before
completion of the message transfer. A header line will be transmitted
only very few bytes later than a SMTP extension. So not an argument.

> and may avoid transfer of a large message.

Not convincing. The main cases will be authorized and unauthorized. 
I doesn't make sense to extend the SMTP protocol just for the case of
misconfiguration. It does not make sense to invent a protocol
extensions with major disadvantes just to prevent some bytes of
transmission in case of misconfiguration.

> This is clearly a corner case, but it needs to be on the

I don't see that there will be a significant number of idiots sending
messages while they know that the receiver is MARID aware and will
discard the message anyway if they could do better without SUBMITTER.
Even a spammer has to pay money and time for transmission. If a
spammer sees a MARID aware MTA and knows that the message will be
discarded it is logical to go to the next victim instead of wasting
time and money. So this is not a relevant case.

> The only advantage SUBMITTER has is temporal in nature - it
> delivers the information sooner.

Not significantly sooner than a header line. Nobody forces you to wait
for completion of transmission before to look at the header. 

If this is the only advantage, then it is not a real advantage at
all. Why introduce a protocol extension without a real advantage?

Hadmut

---

From: [email-masked]
Date: Fri, 06 Aug 2004 00:16:00 -0700 (PDT)

> Really? I don't see significant potential for optimizations.

I see some potential for optimization. Whether or not it is significant I don't
know - that would only emerge with testing, and I haven't implemented
this yet, let alone tested it.

I note, however, that past experience has been that the potential for
optimization with various SMTP extensions has been underestimated fairly
consistently.

> completion of the message transfer.

Sure, but you can't stop the body from being transmitted in this case.

> A header line will be transmitted

I disagree.

> transmission in case of misconfiguration.

I never said it did.

> time and money. So this is not a relevant case.

Which is exactly what I said it was.

> for completion of transmission before to look at the header.

Again, this remains to be seen.

> all. Why introduce a protocol extension without a real advantage?

Again, I am personally ambivalent about SUBMITTER. My purpose in responding to
you was to point out that (a) Your case against it really isn't that cut and
dried, and (b) I am strongly opposed to the alternative header scheme you
proposed.

				Ned

---

From: Shevek <[email-masked]>
Date: Mon, 9 Aug 2004 12:23:12 +0100 (BST)

> throughput optimizations.

You are proposing an optimisation of what will become the rare failure 
case. That's like optimising printing the 'bad password' message. It never 
happens.

I agree with Hadmut. I don't think it's worth changing SMTP to introduce a
new field to optimise what will soon become a fringe case. However, I also
disagree with the introduction of any new header fields.

S.

-- 
Shevek                                    http://www.anarres.org/
I am the Borg.                         http://www.gothnicity.org/

---

From: Margaret Olson <[email-masked]>
Date: Mon, 9 Aug 2004 08:55:20 -0400

> case. That's like optimising printing the 'bad password' message. It 

The advantage to SUBMITTER is more obvious in combination with 
reputation and accreditation. It lets you start the 
accreditation/reputation check early, either before accepting data or 
in parallel with the PRA check in the headers.

Margaret.

---

From: [email-masked]
Date: Mon, 09 Aug 2004 10:23:32 -0700 (PDT)

> case. That's like optimising printing the 'bad password' message. It never

First, I am not proposing anything here. Not only is SUBMITTER not my idea, I
have been quite clear that I am ambivalent about its inclusion in MARID.

Second, the optimization in case 1 isn't a failure case but a success case.
There may be a throughput improvement in some MTAs if the check can be done
sooner. Mind you, I'm not saying there is guaranteed to be such an improvement
- just that it is a possibility. Actual implementation experience would be
needed to tell for sure.

> disagree with the introduction of any new header fields.

My objection to the use of a new header field for this was and continues to be
the primary reason for participating in this thread. Unlike the SMTP extension,
where there may be an upside and there's likely little if any downside, the
header field notion is stamped "bad idea" right out of the box.

				Ned

---

From: Jim Fenton <[email-masked]>
Date: Mon, 09 Aug 2004 23:22:30 -0700

>throughput optimizations.

Some other correspondents (thank you Margaret!) have clarified that the optimization here is in being able to begin an accreditation/reputation check earlier.  But even without Submitter it should be possible to begin this check as soon as the last header is received.  This seems like a rather small optimization.

>and may avoid transfer of a large message.

Again, I think that the savings are small:  this is presumably an isolated case (an "honest mistake", not a bulk mailing) so this would not happen very often.  Even if the bits need to be transferred, the recipient need not store them once a decision has been made not to accept the message.

>This is clearly a corner case, but it needs to be on the

There are actually two cases here:

4a:  The sender should not be authorized and uses SUBMITTER with an address that corresponds to the PRA.  Savings do result, but I would not expect bulk mailers to do this.

4b:  The sender should not be authorized but uses SUBMITTER with an authorized address.  A spammer might do this in order to make sure that, if the message is rejected, the recipient has to do as much work as possible to reject it, or might not go to the trouble of verifying SUBMITTER against the PRA.  This is a reason that, IMO, any SMTP server which implements submitter, MUST verify it against the PRA (not "SHOULD" as in section 4.2 of the current draft).

I consider the optimizations provided by Submitter to not be worth the additional complexity it introduces.  This is particularly true when there is discussion about making it mandatory at some point (not really a "service extension" any more but a change to SMTP itself).  At that point not only MTAs but also firewalls (which often allow only specific ESMTP options, since others are considered "dangerous") will need to change to accommodate this.

-Jim

---

From: Dave Crocker <[email-masked]>
Date: Tue, 17 Aug 2004 08:03:20 -0700

Margaret,

MO> The advantage to SUBMITTER is more obvious in combination with 
MO> reputation and accreditation. It lets you start the 
MO> accreditation/reputation check early, either before accepting data or 
MO> in parallel with the PRA check in the headers.

Maybe this has been mentioned before, but I had missed it. The idea that
it permits a "parallel processing" (or, really, pre-fetching)
optimization simply had not registered with me.

d/
--
 Dave Crocker <dcrocker-at-brandenburg-dot-com>
 Brandenburg InternetWorking <www.brandenburg.com>
 Sunnyvale, CA  USA <tel:+1.408.246.8253>