Subject: Re: [lemonade] SMTP QUICKSTART

From: Tony Finch <[email-masked]>
Date: Thu, 2 Mar 2006 03:21:21 +0000

I've been doing some implementation experiments with OpenSSL. It's fairly
easy to pipeline from both ends before the TLS startup, so you could
do something like this:

	C --> S:	QTLS\r\n
			TLS client hello
	C <-- S:	220 OK\r\r
			TLS server hello etc.

The idea is to treat the QTLS command a bit like BDAT, i.e. it
incorporates trailing binary data. If the SMTP handshaking goes wrong, the
server could swallow the TLS client hello and they could resynchronize.
The "going wrong" could occur if the client pipelines QHLO and QTLS into
the same packet as its TCP SYN, and the server has unexpectedly changed.
The recovery from this should be as simple as the client re-sending a
corrected hello sequence, rather than a disconnection and reconnection. In
the successful case, the start-up exchange will require the same number of
round trips as TLS-on-connect.

There are a few implementation complications: OpenSSL doesn't expose its
client hello parsing code in a way that's convenient to use without also
getting the server's response - though juggling BIO layers could probably
deal with that. However it might be better to implement a purely syntactic
check for the failure case, to avoid running a load of protocol logic that
isn't going to be used.

The end of the TLS handshake is less happy than the start - it's quite
inconvenient to hook in to OpenSSL's negotiation code to piggy-back
application data onto its final packets. There is a tiny bit of support
for doing so at the client end after a session cache hit, but not at the
server end, e.g. for a capability banner. I need to do more investigation
to come up with a work-around or patch.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
GERMAN BIGHT HUMBER THAMES: NORTHWEST 6 OR 7 DECREASING 4 OR 5, BECOMING
VARIABLE 3 OR 4 IN NORTH GERMAN BIGHT. WINTRY SHOWERS. MODERATE OR GOOD.

_______________________________________________
lemonade mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/lemonade

---

From: Dave Cridland <[email-masked]>
Date: Thu, 02 Mar 2006 09:54:16 +0000

> easy to pipeline from both ends before the TLS startup, so you could
Two things on this:

First, I think that even if you can do this on OpenSSL, it's going to 
be really tricky to do on enough existing mobile platforms that it's 
not worth doing. It might be better to do:

C --> S: QTLS\r\n
C --> S: TLS Hello
S --> C: TLS Hello

This is rather weird - a SMTP command which has no response - but the 
specification informally proposed for QTLS suggests that it has no 
possible failure mode - it's 220 or sever the connection. By omitting 
it, then it's relatively easy to take advantage of it. Remember that 
pipelining has two features - not waiting for a response, and sending 
within the same packet - you don't have to use both features at the 
same time.

> round trips as TLS-on-connect.
Okay, I think think isn't going in the right direction. I think the 
scenario of:

C: TCP: SYN Data: None
S: TCP: SYN+ACK Data: Banner with capabilities
C: QTLS, TLS Hello

etc gains you the same round-trips as SMTPS, and the same level of 
robustness as existing STARTTLS, but means the server has to have the 
tricky bits. Plus, I don't think that data in the SYN is a good idea, 
since it makes SYN based attacks very much harder to deal with.

This does skew the overall protocol in favour of server operating 
systems that can be made to provide an API to sending data in SYN+ACK 
packets, of course, but that's simpler than dealing with every 
potential client operating system and TLS implementation.

Dave.
-- 
           You see things; and you say "Why?"
   But I dream things that never were; and I say "Why not?"
    - George Bernard Shaw

---

From: Tony Finch <[email-masked]>
Date: Thu, 2 Mar 2006 13:01:54 +0000

On Thu, 2 Mar 2006, Dave Cridland wrote:
>
> First, I think that even if you can do this on OpenSSL, it's going to be
> really tricky to do on enough existing mobile platforms that it's not worth
> doing. It might be better to do:
>
> C --> S: QTLS\r\n
> C --> S: TLS Hello
> S --> C: TLS Hello
>
> specification informally proposed for QTLS suggests that it has no possible
> failure mode - it's 220 or sever the connection.

There are about two failure modes. The main one occurs if the client
didn't complete the SMTP negotiation correctly, because it spoke first.
I have an updated draft in the works which has a much safer mechanism
for allowing this.

The second is if we use Chris's idea of including the server name in the
QTLS command; the server has to be able to reject commands which specify
an unknown server name. (However since this feature wouldn't be much use
unless it was also supported by POP and IMAP, and since it can be added to
any protocol that uses TLS via an RFC 3546 server name indication in the
TLS client hello, it's probably not worth pursuing.) Similar problems can
arise in servers which initialize their TLS layer lazily, at STARTTLS
time instead of before listing capabilities.

It might be easier to decouple the SMTP and TLS layers if (like BDAT) the
client includes a byte count of the client hello in the QTLS command, so
the TLS layer doesn't have to be invoked to reject the QTLS command.

> Okay, I think think isn't going in the right direction. I think the scenario
> of:
>
> C: TCP: SYN Data: None
> etc gains you the same round-trips as SMTPS, and the same level of robustness
> as existing STARTTLS, but means the server has to have the tricky bits. Plus,
> I don't think that data in the SYN is a good idea, since it makes SYN based
> attacks very much harder to deal with.

Er, sorry, when I said SYN I meant the final ACK of the TCP handshake.
Your suggestion is (a) not supported by any TCP stacks, and (b) vulnerable
to being used as an attack traffic amplifier. I'm continuing to pursue a
spec that allows the client to talk first because it has advantages both
after the TCP handshake and after the TLS handshake.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
RATTRAY HEAD TO BERWICK ON TWEED: NORTH OR NORTHWEST 4 OR 5, OCCASIONALLY 6 AT
FIRST, THEN VEERING NORTH OR NORTHEAST, OCCASIONALLY 6 LATER IN NORTH WEATHER:
SNOW SHOWERS. GOOD OCCASIONALLY POOR. MODERATE OR ROUGH

---

From: Dave Cridland <[email-masked]>
Date: Thu, 02 Mar 2006 16:42:38 +0000

> I have an updated draft in the works which has a much safer 
Okay, but if the client gets the SMTP negotiation out of the way 
first, then it won't speak first. I'm personally very nervous about 
having clients assume things based on previous sessions, because 
breakage is going to happen, be difficult to test, etc.

> TLS client hello, it's probably not worth pursuing.)

Right, so this isn't a problem.

>  Similar problems can
Yes, but we can mandate that a server only advertises QTLS support if 
it has verified that the TLS layer is available and ready. In fact, 
without this, there's little point in QTLS over STARTTLS.

> the TLS layer doesn't have to be invoked to reject the QTLS command.
Nope. The APIs for TLS support on some platforms don't have any 
indication of how many octets the TLS hello is going to be, so this 
simply won't get used where it's needed most.

> Er, sorry, when I said SYN I meant the final ACK of the TCP 

Ah... Yes, then we're back to the client speaking before receiving 
the banner.

> Your suggestion is (a) not supported by any TCP stacks

Yours isn't supported by any TLS implementations. Something has to 
change somewhere, I'm proposing changing the TCP stacks on servers, 
and the servers, rather than the protocol. This is in part because 
for server implementors, the option of using an open source server OS 
is present, whereas client implementors won't have much success in 
persuading people to change the OS on their phone.

> , and (b) vulnerable
> to being used as an attack traffic amplifier.

I understood that any real danager of this was heavily mitigated by 
the typical deployment of SYN flooding protection, and the use of 
firewalling to prevent the obviously bogus SYN+ACK packets aimed at 
broadcast or multicast addresses. In other words, this is no more a 
danger than DNS.

>  I'm continuing to pursue a

Right... The client always talks first after the TLS handshake in 
ESMTP, though. This might be because it needs to reissue EHLO 
(technically mandatory, although one can still be paranoid about 
downgrade attacks and yet elide the second EHLO), or because it 
authenticates.

Dave.

---

From: Tony Finch <[email-masked]>
Date: Fri, 3 Mar 2006 23:20:29 +0000

I've finished writing the updated QUICKSTART spec.

http://www-uxsup.csx.cam.ac.uk/~fanf2/hermes/doc/antiforgery/draft-fanf-smtp-quickstart.txt

This version includes a basic subset which should be easy to implement,
and which brings the MAIL command forward from the client's 9th packet to
its 5th in typical use:

     C: TCP  SYN
     S:      SYN/ACK
     C:      ACK
     S: SMTP <qsmtp-greet>
     C:      QTLS
        TLS  client hello
     S:      server hello, certificate, etc.
     C:      key exchange, change cipher, finish handshake
     S:      change cipher, finish handshake
             <qsmtp-greet>
     C:      QHLO <domain>
             AUTH PLAIN <creds>
             MAIL FROM:<...>
             RCPT TO:<...>
             ...
     S:      220 OK
             235 OK
             250 OK
             250 OK
             ...

Full QUICKSTART allows the client to pipeline commands before the server
greeting and after the TLS handshake in a way that should be safe, albeit
not so trivial to implement. If all goes well, the MAIL command appears in
the client's 3rd packet, which I think is pretty good. Even Dave should be
able to get a message out in two or three seconds :-) If the client has
to recover from a cache miss it should be no slower than the basic form.

     C: TCP  SYN
     S:      SYN/ACK
     C:      ACK
        SMTP QHLO <domain> <qhlo-id>
             STARTTLS
        TLS  client hello
     S: SMTP <qsmtp-greet>
             220 OK
             220 OK
        TLS  server hello, change cipher, finish handshake
     C:      change cipher, finish handshake
        SMTP QHLO <domain> <qhlo-id>
             AUTH PLAIN <creds>
             MAIL FROM:<...>
             RCPT TO:<...>
             ...
     S:      220 OK
             235 OK
             250 OK
             250 OK
             ...

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
WEST NORTHERN SECTION: IN NORTH, VARIABLE 3 OR 4. FAIR. GOOD. IN SOUTH,
EAST OR SOUTHEAST 4 OR 5, OCCASIONALLY 6. MAINLY FAIR. GOOD.

---

From: Randall Gellens <[email-masked]>
Date: Fri, 3 Mar 2006 18:33:49 -0800

At 5:09 PM -0800 2/21/06, Ned Freed wrote:

>   > 
> http://www.cus.cam.ac.uk/~fanf2/hermes/doc/antiforgery/draft-fanf-smtp-tls-on-connect.txt
>
>  I have to say I agreed with the decision not to go with separate 
> ports and with
>  STARTTLS instead. And although this does reduce round trips significantly I'm
>  fairly unenthusiastic it, but less because I agreed with the prior 
> decision and
>  more because  I predict there will be opposition to moving back to the smtps
>  approach from the security folks. And although we've supported smtps in our
>  product for a long time, I don't in practice see the large amounts 
> of lingering
>  use of it you do.

One of the main arguments against alternate-port-TLS is what happens 
if/when we deploy something else that replaces TLS?  Does each 
protocol then need three ports?

I wonder if it would be worth trying an approach that connects on the 
standard port, but immediately tried to negotiate TLS, without 
waiting for the usual greeting and extension list?  As long as 
existing servers send an error when they see the start of a TLS 
negotiation, it might be OK.

>   > 
> http://www.cus.cam.ac.uk/~fanf2/hermes/doc/antiforgery/draft-fanf-smtp-quickstart.txt
>
>  This extension seems quite reasonable to me.

I'm bothered by a few things:
	(1) would there be problems with clients connecting to servers 
and getting unexpected multi-line greetings?  [Ned seems to think 
this would be fine, so maybe we don't worry here.]
	(2) this business of the client using something extended as its 
first command to signal to the server that it is fine seems fragile
	(3) the burden on the client of correctly caching the server's 
extensions at the right time doesn't seem to be clearly described: in 
one place it says the client uses its most recent cache, yet in 
another it says that the client can't use SMTP AUTH if the server 
didn't advertise it prior to TLS; the most recent cache is likely to 
be post-TLS.
	(4) why would a server advertise PIPELINING in the greeting yet 
not in an EHLO response?  Or is there another reason the client can't 
assume PIPELINING if it sees it in the greeting yet issues an EHLO?

If we do proceed with this, I'd suggest that it MUST NOT be used on 
port 25; it MUST be a port 587 mechanism only.  This is because some 
servers have adopted anti-spam techniques that in some cases detect 
extra commands that are sent with the HELO/EHLO command (or otherwise 
look at client use of PIPELINING without having negotiated it).

I'd suggest that, while logging of client IP address is useful, 
reverse DNS information may be misleading and is generally much less 
useful.  They are certainly not equivalent.  It is fairly trivial for 
someone to setup their own PTR entries for IP addresses they own, 
containing any host names they choose; even when no malice is 
intended, the reverse zone is the most likely to be misconfigured.

>   > 
> http://www.cus.cam.ac.uk/~fanf2/hermes/doc/antiforgery/draft-fanf-smtp-rcpthdr.txt

>  (0) How are malformed or unparseable To:/Cc:/Bcc: fields supposed to be
>      handled? There's no place to report such an error in the dialogue and
>      a DSN isn't designed to report such issues.

>  (1) How are problems with specific recipient addresses supposed to 
> be handled?

If we do proceed with this draft, I'd suggest that malformed and 
unparseable address headers, as well as errors for a specific 
address, be handled by a new extended SMTP response code and 
including the bad address (or the entire header if it is bad) in 
square brackets in the text.  This would have to be a response after 
end of data, of course.

It's still a lot tricker for clients to get this right than an 
immediate error after the specific RCPT TO, but it's much, much, much 
better than accepting the message and generating a DSN.

At 5:16 PM +0000 2/22/06, Tony Finch wrote:

>  On Wed, 22 Feb 2006, Dave Cridland wrote:
>
>   > > 
> http://www.cus.cam.ac.uk/~fanf2/hermes/doc/antiforgery/draft-fanf-submission-streamlined.txt
>>
>>  BURL doesn't allow clients to pipeline AUTH - that's just possible
>   > anyway with a server that supports PIPELINING.
>
>  The BURL spec is the only document that gives permission to do it.

This is something we could likely tighten up in profile-bis.

>  Pipelined QUIT is included in an example in RFC 2920. It shouldn't be a
>  problem if the client waits for all the server's responses.

The client would pipeline QUIT and check the responses after the 
connection closes, right?  I suppose we'd want an extension that said 
"if any RCPT is bad then please ignore any subsequent ones because I 
don't really want it sent even though it might seem so but that is 
just because I am pipelining everything".  It would be the moral 
equivalent of the compiler flags to treat all warnings as errors. 
The client would include this new keyword on MAIL FROM (maybe 
ABORTONBADRCPT), the server would generate usual responses to every 
RCPT TO, then if any were bad, it would send a failure after 
end-of-data.

>  You can't reliably auto-detect plaintext SMTP (or IMAP) versus
>  TLS-on-connect on the same port, because there's a mismatch between who
>  talks first. If it was consistently the client that talked first then the
>  server could autodetect, but the SMTP or IMAP server banner prevents this.

But the server can check before it sends the greeting to see if the 
client sent any data and if so if it is a TLS initiation, couldn't 
it?  They way some servers now check if the client sent commands 
without waiting for the EHLO response?
-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly-selected tag: ---------------
Get your facts first, and then you can distort them as much as you please.
--Mark Twain

---

From: Dave Cridland <[email-masked]>
Date: Sun, 05 Mar 2006 09:18:47 +0000

On Sat Mar  4 02:33:49 2006, Randall Gellens wrote:
>>  (1) How are problems with specific recipient addresses supposed 
>> to be handled?
> 
> square brackets in the text.  This would have to be a response 
> after end of data, of course.
> 
> immediate error after the specific RCPT TO, but it's much, much, 
> much better than accepting the message and generating a DSN.
> 
> 
Chris Newman briefly proposed doing it the other way around, that is, 
generating the headers from the envelope.

I've written this up as 
http://dave.cridland.net/draft-cridland-esmtp-headergen-pre.txt with 
XML source at 
http://svn.dave.cridland.net/svn/ietf-drafts/draft-cridland-esmtp-headergen.xml

(This is very rough thus far, just aimed at being a talking point. In 
particular, as I write this, I note I've not yet seperated out 
informative and normative references at all.)

Given the availability of PIPELINING, this seems like a better and 
easier way to go.

> connection closes, right?  I suppose we'd want an extension that 
> said "if any RCPT is bad then please ignore any subsequent ones 
> because I don't really want it sent even though it might seem so 
> but that is just because I am pipelining everything".  It would be 
> the moral equivalent of the compiler flags to treat all warnings as 
> errors. The client would include this new keyword on MAIL FROM 
> (maybe ABORTONBADRCPT), the server would generate usual responses 
> to every RCPT TO, then if any were bad, it would send a failure 
> after end-of-data.
> 
> 
It's actually saying "If any RCPT is bad then I'm going to waste a 
stackload of data transmission, and therefore cost and battery life". 
I think if you want this behaviour, you have to introduce a 
post-envelope sync point - ie, you send the envelope, and examine 
responses to decide whether you wish to continue.

Dave.

---

From: Tony Finch <[email-masked]>
Date: Sun, 5 Mar 2006 15:15:38 +0000

On Fri, 3 Mar 2006, Randall Gellens wrote:
>
> I wonder if it would be worth trying an approach that connects on the standard
> port, but immediately tried to negotiate TLS, without waiting for the usual
> greeting and extension list?  As long as existing servers send an error when
> they see the start of a TLS negotiation, it might be OK.

I don't think the error handling would be particularly elegant - the
client would probably have to drop the connection and re-try from the
beginning, and the server will likely force it to do so. My latest
QUICKSTART draft describes how to make STARTTLS as efficient (in terms of
packet count if not byte count) as TLS-on-connect.

> 	(2) this business of the client using something extended as its first
> command to signal to the server that it is fine seems fragile
> extensions at the right time doesn't seem to be clearly described: [...]

The latest QUICKSTART draft is much more robust. I think I have explained
the cacheing better, though there's still some room for improvement.

> If we do proceed with this, I'd suggest that it MUST NOT be used on port 25;
> it MUST be a port 587 mechanism only.  This is because some servers have
> adopted anti-spam techniques that in some cases detect extra commands that are
> sent with the HELO/EHLO command (or otherwise look at client use of PIPELINING
> without having negotiated it).

I've explained why I think this isn't such s problem as it might seem in
the latest draft's security considerations section. The more robust
signalling helps a lot.

> I'd suggest that, while logging of client IP address is useful, reverse DNS
> information may be misleading and is generally much less useful.

I should have been clearer that I was assuming "paranoid" reverse DNS
checking.

Thanks for looking at the drafts.

Tony.
-- 
f.a.n.finch  <[email-masked]>  http://dotat.at/
SHANNON ROCKALL: NORTHWEST 4 OR 5 BECOMING VARIABLE 3 OR 4 THEN SOUTHEAST 5 TO
7. SHOWERS THEN RAIN. MAINLY GOOD.

---

From: Randall Gellens <[email-masked]>
Date: Mon, 6 Mar 2006 14:43:52 -0800

At 9:18 AM +0000 3/5/06, Dave Cridland wrote:

>  Chris Newman briefly proposed doing it the other way around, that 
> is, generating the headers from the envelope.
>
>  I've written this up as 
> http://dave.cridland.net/draft-cridland-esmtp-headergen-pre.txt 
> with XML source at 
>  (This is very rough thus far, just aimed at being a talking point. 
> In particular, as I write this, I note I've not yet seperated out 

I'd agree that this approach (generating headers from envelope) is 
better than the other way around, since the client can easily 
determine which address is bad, but I'd still question the value.  It 
seems to me that saving round-trips is the key, that as 3G systems 
are deployed, total octet savings isn't as important, while latency 
and round-trips still are.  I could easily be wrong, of course.  But 
I'd like to see better explanations of why the added complexity is 
worth the octet savings.

(By the way, I like that 'foo' is the username for 'Fred Oo' -- clever).

>> the moral equivalent of the compiler flags to treat all warnings 
>> as errors. The client would include this new keyword on MAIL FROM 
> stackload of data transmission, and therefore cost and battery 
> life". I think if you want this behaviour, you have to introduce a 

I suppose it depends on if we want to optimize for the error case or 
the non-error case.  If we expect at least one RCPT to fail, then we 
don't want to send the data until we know.  But if we expect that all 
RCPTs will be fine, then we don't want to introduce a delay waiting 
for the server response if we can pipeline the entire transaction. 
The size of the message is also significant, of course.  If we have a 
small message, then why have any delay?  But if we have a large 
message, maybe it makes sense to wait.

So perhaps there is use for such an extension plus client advice on 
how and when to use it.

-------------- Randomly-selected tag: ---------------
The reliability of machinery is inversely proportional to the
number and significance of any persons watching it.

---

From: Randall Gellens <[email-masked]>
Date: Mon, 6 Mar 2006 15:39:45 -0800

At 12:56 PM +0000 2/23/06, Tony Finch wrote:

>  For example, it's common to have dual-purpose MSA
>  and MX servers which operate in MSA mode after AUTH and MX mode otherwise.
>  If the client pipelines an AUTH and an intended submission, but the auth
>  fails, bad things will happen - recipients will be rejected that would
>  have been OK, an incomplete message will suffer an anti-spam penalty
>  instead of being fixed up, etc.

Another reason to stop running dual-purpose servers on port 25, and 
restrict 25 to message relay, and use port 587 for message 
submission.  It makes the client's intent much more clear.
-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly-selected tag: ---------------
"Contrariwise," continued Tweedledee, "if it was so, it might
be, and if it were so, it would be; but as it isn't, it ain't. That's
logic!"

---

From: Randall Gellens <[email-masked]>
Date: Mon, 6 Mar 2006 15:40:13 -0800

At 12:56 PM +0000 2/23/06, Tony Finch wrote:

>  The BURL spec requires a sync point after the "LAST" BDAT or BURL command,
>  instead of after the message envelope. I think it would be better for BDAT
>  and BURL to not require any synchronization at all: if the client wants a
>  sync, it can use the NOOP command. Note that the main reason a sync point
>  is required after DATA is so that the client and server agree about
>  whether the next utterance from the client is a command or is message
>  data. This isn't a problem for BDAT because the command and data are sent
>  together, with a single response from the server for both.
>
>  This would allow a client to pipeline multiple messages together, which
>  means you can move from using TCP in interactive mode to bulk data mode,
>  and therefore make the best use of the available bandwidth. For example,
>  consider a 1Mb/s link with a 100ms RTT. If the client sends a 5KB message,
>  this takes about 50ms to transmit, so by the time the server's replies
>  arrive the client will be one or two messages further on; if there was a
>  problem, the client can deal with it asynchronously, and it will have done
>  50ms extra useful work. If the client sends a 20KB message, the server's
>  replies will start arriving part-way through message transmission, so if
>  there is a problem the client can usefully RSET it before the next chunk
>  to save time and bandwidth.
>
>  I think it would be cool to allow this kind of high-bandwidth asynchronous
>  message transmission. It is also more friendly from the point of
>  congestion control than having multiple concurrent SMTP connections
>  perpetually in TCP slow start.

Interesting.  Both mobile and desktop clients often send multiple messages.
-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly-selected tag: ---------------
"The meat is rotten, but the booze is holding out."
--alleged computer translation of "The spirit is willing,
  but the flesh is weak."

---

From: Randall Gellens <[email-masked]>
Date: Mon, 6 Mar 2006 15:40:23 -0800

At 1:22 PM +0000 2/23/06, Tony Finch wrote:

>  I think both Ned and Dave mentioned the irritation that can arise from a
>  message and receiving a bounce for some of the recipients. (Note to self:
>  [email-masked] is the wrong address).

It's [email-masked].

>   One advantage of a sync point
>  after the message envelope is that the client can decide not to proceed if
>  a subset of the recipients are rejected; this doesn't work if the envelope
>  and data are pipelined. (It is also limited by how strongly the server can
>  verify addresses.) What do you think of having a NOPARTFAIL parameter to
>  the MAIL command, which means that the client is asking the server to
>  reject the message data if any of the recipients are rejected - "no
>  partial failure".

Perhaps "NOBADRCPT" or my earlier suggestion of "ABORTONBADRCPT".

Maybe "ERRONBADRCPT"?
-------------- Randomly-selected tag: ---------------
When a man's best friend is his dog, that dog has a problem.
                                             --Edward Abbey

---

From: Dave Cridland <[email-masked]>
Date: Mon, 06 Mar 2006 23:45:23 +0000

On Mon Mar  6 22:43:52 2006, Randall Gellens wrote:
>>  (This is very rough thus far, just aimed at being a talking 
>> point. In particular, as I write this, I note I've not yet 
>> seperated out informative and normative references at all.)
>> 
>>  Given the availability of PIPELINING, this seems like a better 
>> and easier way to go.
> 
> determine which address is bad, but I'd still question the value.  
> It seems to me that saving round-trips is the key, that as 3G 
> systems are deployed, total octet savings isn't as important, while 
> latency and round-trips still are.  I could easily be wrong, of 
> course.  But I'd like to see better explanations of why the added 
> complexity is worth the octet savings.
> 
> 
Total octet count is primarily a battery problem, as I understand 
things, and also a layer 8 consideration.

There's other issues that we can look at if the addressing header 
fields of the message are (largely) built from the envelope, however, 
such as revisiting the OMA's requirement for large recipient list 
manipulation. "Server-side" defined aliases and distribution lists 
might also be worth looking at.

(Of course, I already have my aliases and lists defined server-side, 
so this makes good sense for me).

> (By the way, I like that 'foo' is the username for 'Fred Oo' -- 
I didn't think it was as groundbreaking an achievement as Bob Ar, I 
was merely following a trend. :-)

>>  It's actually saying "If any RCPT is bad then I'm going to waste 
>> a stackload of data transmission, and therefore cost and battery 
> I suppose it depends on if we want to optimize for the error case 
> or the non-error case.  If we expect at least one RCPT to fail, 
> then we don't want to send the data until we know.  But if we 
> expect that all RCPTs will be fine, then we don't want to introduce 
> a delay waiting for the server response if we can pipeline the 
> entire transaction. The size of the message is also significant, of 
> course.  If we have a small message, then why have any delay?  But 
> if we have a large message, maybe it makes sense to wait.
> 

True. I think unfortunately it's a trade-off, and I can't see a way 
around that, which is fustrating. It's a good argument for assembling 
on the IMAP server and using BURL, over the alternative of assembling 
on the submit stream (and optionally using POSTADDRESS for FCC). 
Using a single BURL chunk means you're wasting less in the error case.

Dave.

---

From: Ned Freed <[email-masked]>
Date: Mon, 06 Mar 2006 17:08:07 -0800 (PST)

> > informative and normative references at all.)

First, I think FULLNAME/ENCNAME is neither necessary nor sufficient. We should
stick to a single DISPLAYNAME argument and have it take whatever string is
going to end up in the header, encoded as xtext.

The insufficiency is that clients routinely need to generate headers containing
multiple encoded words in a charset other than UTF-8. This proposal doesn't
allow for this case.

Second, you really should allow for the generation of bcc and resent-bcc as
well as all the others. i do like limiting the set of headers to the sensible
ones, however.

> I'd like to see better explanations of why the added complexity is

Full agreement on all points. If we have to do it this is the right way,
however, I remain unconvinced that it is worth it. (And this is coming from
someone whose SMTP server already has the capability to generate headers from
envelope addresses - adding this is probably a few dozen liens of code at the
very most.)

				Ned

---

From: Dave Cridland <[email-masked]>
Date: Tue, 07 Mar 2006 09:09:07 +0000

On Tue Mar  7 01:08:07 2006, Ned Freed wrote:
>> point.
I was trying to push the RFC2047 encoding toward the server, to 
simplify the client, in the hope an IMAP extension might do the same 
for the decoding. The trouble is, the xtext encoding is pretty 
verbose, and I thought the double-encoding of RFC2047+xtext was just 
really ugly. FWIW, I'd be quite happy to push the display-name 
entirely onto the server, by telling it to go look it up, but therein 
lies a specification nightmare.

I'll rework it as a single DISPLAYNAME argument. Is anyone interested 
in group syntax, by the way?

> well as all the others. i do like limiting the set of headers to 
I've never been sure why a submission client would specify a Bcc 
header, it's always seemed to me to be unsafe behaviour, given that 
the client cannot know how Bcc is treated by the submission server, 
so I deliberately missed out the Bcc and Resent-Bcc headers. This 
also saved me from writing anything more in the Security 
Considerations section. :-)

> envelope addresses - adding this is probably a few dozen liens of 

I'll reiterate two points:

1) I wrote up this off-hand suggestion from Chris Newman primarily to 
start discussion. I haven't implemented this either, and the actual 
octet savings are very slim indeed, they're smaller than just 
omitting the addressing header fields. (Roughly equal to omitting the 
envelope, which is where all this started).

2) However, having a reasonable framework for generating addressing 
header fields from the envelope does provide us with a base to 
manipulate addresses and address lists at a single point in the ESMTP 
transaction, rather than twice.

Dave.

---

From: "Robert A. Rosenberg" <[email-masked]>
Date: Tue, 7 Mar 2006 00:28:08 -0500

At 15:40 -0800 on 03/06/2006, Randall Gellens wrote about [lemonade] 

>>   reject the message data if any of the recipients are rejected - "no

If I am sending a multi-addressed message, I would like the ability 
(even if pipelining) to get a response stating if each recipient was 
accepted or not, and have the MUA requeue the message with all the 
rejected addresses while allowing the message to get sent with the 
valid addresses. IOW: Leave it as "failed" as presently occurs if it 
can not get sent BUT remove the accepted addresses so the failed 
version only contains the rejected addresses.

---

From: Randall Gellens <[email-masked]>
Date: Thu, 9 Mar 2006 16:11:39 -0800

At 11:45 PM +0000 3/6/06, Dave Cridland wrote:

>  It's a good argument for assembling on the IMAP server and using 
> BURL, over the alternative of assembling on the submit stream (and 
> optionally using POSTADDRESS for FCC). Using a single BURL chunk 
> means you're wasting less in the error case.

This is a good point and one that I think should be mentioned in 
profile-bis.  It's an advantage of CATENATE that I hadn't considered.
-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly-selected tag: ---------------
They say that 90 per cent of TV is junk.  But 90 per cent
of *everything* is junk.
      --Gene Roddenberry, interview in TV Guide, Apr. 27, 1974
      (quoting Sturgeon's Revelation: "Ninety percent of
      everything is crap.")

---

From: Randall Gellens <[email-masked]>
Date: Thu, 9 Mar 2006 16:22:00 -0800

At 12:28 AM -0500 3/7/06, Robert A. Rosenberg wrote:

>  At 15:40 -0800 on 03/06/2006, Randall Gellens wrote about 
> [lemonade] Re: PIPELINING and failed recipients:

That attribution is misleading, since I was quoting a message from 
Tony Finch, I think.

> (even if pipelining) to get a response stating if each recipient 
> was accepted or not, and have the MUA requeue the message with all 
> the rejected addresses while allowing the message to get sent with 
> the valid addresses. IOW: Leave it as "failed" as presently occurs 
> if it can not get sent BUT remove the accepted addresses so the 
> failed version only contains the rejected addresses.

I am not sure what you are saying.  Are you asking that clients be 
written to behave in a specific way regarding a message where some 
recipients are rejected by the submission server?

Currently, clients can choose which behavior to support: immediate 
send to good recipients, or hold message in abeyance and alert user, 
who can either correct or delete the bad rejected recipients.

No one has suggested anything that would force the client to choose 
one way or the other.

An extension such as "NOBADRCPT" (or "ABORTONBADRCPT" or 
"ERRONBADRCPT") would allow the client to behave in either way.
-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly-selected tag: ---------------
"SFPD Homicide: Our Day Begins when Yours End" --Slogan seen on back of
San Francisco Police Jacket, as reported by Herb Cain in SF Chronicle