Subject: Re: [Asrg] Re: 6. Proposals - Pull System (revisited)

From: David Maxwell <[email-masked]>
Date: Mon, 1 Dec 2003 12:16:54 -0500

On Sat, Nov 29, 2003 at 12:44:33PM +0100, Dag Kihlman wrote:
> I agree with Scott: people seem unwilling to make hard choices. The SMTP
> protocol is old, ideal for spammers and inefficient. Any textbook covering

Could you point out an 'innefficiency' in SMTP more specifically,
please?

That's one fault I don't see SMTP as having - unless you're saying that
all the spam clogging the pipes makes SMTP inefficient... not quite the
normal application of that word.

> communication between tiers have one basic rule: avoid communication as much
> as possible. The ideal solution is to send all at once and if necessary
> receive a receipt containing the data you want or an error message. Any new
> standard ought to be built on this simple principle. Why HELO? Send
> everything and let the receiving server go through the sender's data
> returning the appropriate answer.

That sounds like a contradiction to me: "avoid communication" AND "Send
everything"

Steps like HELO will continue to be useful in protocols for the same
reason the 3 way handshake is critical to TCP. Before you commit
significant resources to a session, you need to sanity check it. Failure
to do so leaves the door open to DoS attacks.

-- 
David Maxwell, [email-masked][email-masked] --> Mastery of UNIX, like
mastery of language, offers real freedom. The price of freedom is always dear,
but there's no substitute. Personally, I'd rather pay for my freedom than live
in a bitmapped, pop-up-happy dungeon like NT. - Thomas Scoville 

_______________________________________________
Asrg mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/asrg

---

From: "Dag Kihlman" <[email-masked]>
Date: Mon, 1 Dec 2003 22:10:52 +0100

David!

I think people just get annoyed by pull talk so I have skipped that... This
message is free from all pull arguments...

As to SMTP inefficiences:
I guess You and I aggree that it is a good thing to keep memory usage down
to have a fast server less vulnerable to DoS attacks. However we do not
count the memory the same way.

You and SMTP want to send small bits of information and catch an error as
soon as possible. The general idea is: why send everything including a huge
manuscript as an attatchment only to learn that the recipient is unknown! I
aggree. It would probably not be efficient.

However when discussing memory usage in server to server communication you
must consider one other factor: the time the the data remains in the memory.
SMTP has something like at least 12 trips over the Internet (for a
successful mail).

The thing which takes time in SMTP are the network trips. I have no figures
but say an ordinary mail (without attatchments) in SMTP takes 1 second to
send and that it uses an avarage of 0,4Kb. Then 1Mb will last for 2600
mails. If all is sent at once a mail could take say 1,5Kb and be completed
in 0,1 seconds. Then 1Mb will last for 6800 mails. I repeat: my figures are
just estimations, but any textbook on server to server communication in
DCOM, CORBA, Web Services etc stress the fact that network trips waste
server resources. Web Services are actually designed for single
request-response solutions (even if sessions, cache and similar technologies
are available). Any mail without an attatchment is so small that I cannot
see why what is true in more complex programs would not be true in SMTP.

I realize attatchements are a problem so I would suggest:
A Without attatchment
1. Send all data but the attatchment + a boolean signaling no attatchemnet
2 Let the server go through the data according to SMTP protocol and return
the first error message. Otherwise return OK

B With attatchment
1 Send all data but the attatchment + a boolean signaling attatchment
2 Let the server go through the data according to SMTP protocol and return
the first error message. Otherwise return OK
3 Send the attatchment
4 Return OK or error message

I.e. I just minimize the network trips and pack the data in larger chunks
and let the server go through the chunk according to SMTP and return an
error as soon as one is encountered. The CPU will not have to handle several
OK messages and several calls.

Again, I cannot understand why what is common knowledge in COM, DCOM, Web
Services and CORBA would not apply to mail servers. It is all a matter of
processing data and handling network trips. If anyone could explain the
difference I will have learnt something essential.

/DK

---

From: David Maxwell <[email-masked]>
Date: Mon, 1 Dec 2003 17:43:30 -0500

> see why what is true in more complex programs would not be true in SMTP.

DCOM and CORBA are most likely to be seen on a trusted network, not on
the open Internet. I don't know that they make an effort to address the
resource consumption issue. 

Web services are different, because the only work the server has to do
is getting the reply out. In the case of email, the server is accepting
responsibility for a larger workload - filtering, and possibly sending
bounces, or delivery to multiple mailboxes.

So, for general protocol design, I'd agree that an 'SMTP replacement'
should probably require fewer round trips than current SMTP. Perhaps
something like:

1: MTA id, Sender, # of recipients, (recipient list, if below some
threshhold), size of content
1b: Server ACK/NAK/??
2: Recipient list if above threshhold
2b: Server ACK/NAK/??
3: Content. (Or token, for Pull implementations)
3b: Server ACK/NAK/??

I have nothing against looking at replacements for SMTP - but I would
suggest that it should occur on a sub-list. It's a longer term project
than tweaking the existing protocol, and I'd prefer that it not bog down
discussions of changes that could be made more rapidly.

-- 
David Maxwell, [email-masked][email-masked] -->
Any sufficiently advanced Common Sense will seem like magic... 
					      - me

---

From: "Dag Kihlman" <[email-masked]>
Date: Tue, 2 Dec 2003 01:25:16 +0100

----- Original Message ----- 
> should probably require fewer round trips than current SMTP. Perhaps
True! The number of roundtrips is a matter of statistics and mathematics. It
would not really be creative to take the textbook as a divine commandment.
DCOM, CORBA, Web Services and COM (in Component Services etc) address the
resource consumption by avoiding roundtrips, but a mail server will have
many more break points, i.e. "Not OK" so the textbooks are not 100%
accurate. The important thing is to convince others that SMTP has too many
rountrips. A more slimmed approach is needed.

> discussions of changes that could be made more rapidly.

I agree. But I think we are too few to discuss this issue and the number of
possible implementations are too many. Possibly the best way is to implement
solutions and test them. My students will implement a Web Service based
solution with no connection to SMTP. I hope to encourage some of them to do
a final thesis on the subject push or pull, hopefully with some
implementations cooperating with SMTP mail servers. Since some of my
colleagues believe in SMTP I think no one need worry that the students will
get only the twisted ideas. ;-)

I would be happy passing any papers on pull to the students... Papers on
push are not cencured but they are abundant so the students will find them
anyway...

/DK

---

From: Yakov Shafranovich <[email-masked]>
Date: Mon, 01 Dec 2003 19:29:20 -0500

> From: "David Maxwell" <[email-masked]>
[..]
> possible implementations are too many.

If sufficient interest can be gathered, we can look into a subgroup.

Yakov
-------
Yakov Shafranovich / asrg <at> shaftek.org
SolidMatrix Technologies, Inc. / research <at> solidmatrix.com
"Power tends to corrupt, and absolute power corrupts absolutely" (Lord 
Acton)
-------

---

From: Bart Schaefer <[email-masked]>
Date: Tue, 2 Dec 2003 01:53:24 +0000

On Dec 2,  1:25am, Dag Kihlman wrote:
}
} The important thing is to convince others that SMTP has too many
} rountrips. A more slimmed approach is needed.

RFC 2920, perhaps?

---

From: "Ken Hirsch" <[email-masked]>
Date: Tue, 2 Dec 2003 01:05:33 -0500

Pure pull systems suffer from polling overhead.  This is already becoming a problem
with RSS and think how few people know what RSS is.  This is a potentially major
scalability problem.

See
http://philringnalda.com/blog/2002/10/joels_rss_problem.php
http://www.sidhe.org/~dan/blog/archives/000149.html
http://www.sidhe.org/~dan/blog/archives/000170.html
http://www.mnot.net/blog/2003/05/03/rss_traffic_characterisation
http://www.teledyn.com/mt/archives/001496.html
http://www.teledyn.com/mt/archives/001499.html
http://www.tbray.org/ongoing/When/200x/2003/08/02/RSSNumbers

The next major version of Windows (Longhorn) is slated to include an RSS reader.  I
think it was on the alphas distributed at PDC and available to MSDN subscribers.

Personally I use a web-based RSS aggregator, which keeps things synched between home
and office.  It also has much lower bandwidth usage than client-side readers.

I use bloglines (http://www.bloglines.com/), but there are some others:
http://mywireservice.com/
http://fyuze.com/
http://www.newsisfree.com/
http://www.feed-me.info/DesktopDefault.aspx