Subject: Mail Server Registries and Foreign Sender Authentication: A Proposal

From: "Randy Smith" <[email-masked]>
Date: Fri, 23 Mar 2007 20:51:55 -0600

Greetings,

I was recently discussing various issues surrounding email with a
coworker and had a couple of ideas for authentication systems that I
would like to get some feedback on. You can read my ideas at
http://perlstalker.blogspot.com/2007/03/mail-server-registries-and-foreign.html.

As I said, I'm looking for feedback. Are these ideas worth pursuing or
am I barking up the wrong tree?

Thank you for your time.

-- 
Randy Smith
http://vuser.org/
http://perlstalker.vuser.org/

---

From: Frank Ellermann <[email-masked]>
Date: Sat, 24 Mar 2007 05:25:01 +0100

> am I barking up the wrong tree?

The tree is right, but it's not here:  This list is dead.

For slightly more active lists see the ASRG or the SMTP
lists.  You might wish to check out DKIM (an RFC approved
recently now waiting for its RFC number), SPF (RFC 4408),
and draft-irtf-asrg-howe-siq-03.txt for proposals in 
similar directions, some pieces of the puzzle already
exist.  They're not based on OpenID, that's rather new.

Frank

---

From: "HECTOR SANTOS" <[email-masked]>
Date: Sun, 25 Mar 2007 16:24:06 -0400

> am I barking up the wrong tree?

In my opinion, you are not barking up the wrong tree, but the 
"inevitable tree."   Its not a new concept, but a concept we have moved 
away from in the name of having an open anonymous system. But we seem to 
be reinventing ourselves to go back to this direction.

In short, you are simply talking about a client/server negotiation 
handshake stage.  In my opinion, maybe not in our "engineering" life 
time, this is will be the ultimate method of client/server operations. 
The anonymous sender will be "thing" of the past.

Look at the old Fidonet technology, 100% based on a similar 
client/server negotiation concepts.

1) Every server MUST be "registered" with a "HOST", in Fidonet, that 
would be your uplink Net or Zone Host.  In the internet, that might be 
your "ISP", the early forms of Fidonet Net/Zone Hosting systems that 
were the backbone of the topology.

2) When installing the software, you applied for a FIDONET address and 
your "ISP" would check your system for 100% FTN compliance before you 
were assigned a Fidonet Address.

3) You were not allowed to be part of the "net" if you did not have a 
100% FTN compliant client and server.   Not compliant? No Fidonet 
address was given to you.

4) There were four basic requirements:

4.a) The software support the legacy FTN1 file transfer protocol. The 
more advanced IEMSI is akin to what we have today with EHLO and extended 
server attributes.

4.b) Although you can apply for a private address, ALL servers must be 
available during Zone Mail Hour (ZMH).  This is akin to having a 100% 
closed system but there were designated times where you allowed 
anonymous mail (from a registered FTN member) to come in.

4.c) ZMH was designed for making sure ROUTED mail is ALWAYS available. 
So even if you didn't allow DIRECT mail, a person can send routed mail 
to you via your HOST which you must ALWAYS accept.

4.d) During ZHM, at a minimum FTN1 must be supported.

There were probably others, but the above were the main things and in 
addition, the most contentious issues during the "Fidonet/Internet 
Migration Days."

The legacy FTN1 protocol was REALLY old and it has major problems in 
packet switching networks.  ZMODEM and other file transfer protocols 
designed for Packet switching networks were part of the newer generation 
  Fidonet software, but were OPTIONAL and not required.  However, new 
developers were not bothering with the older problematic FTN1 protocol 
and using the new protocols which at least 99% of all systems supported.
But the old school FTN net police insisted on FTN1 and rejected 
applications, registrations of any person installing newer and better 
software if it didn't have FTN1 support.  This turned off alot of 
developers and with the Internet humming around the corner, and new FTSC 
(akin to IETF) groups created to get around these policing issues,
Fidonet began on a death spiral.  The one thing the FTSC had control of 
was the NodeList, this would be akin to DNS.  So new Nodelist ZONE files 
were created to establish new networks where the FTSC had no control.

In any case, conceptually, we are talking about the same type of direction.

   - Registration of servers and clients within some "controlled
     and centralized" nodelist/DNS system.

   - It will confronted with the same Legacy issues of supporting
     legitimate clients that might not part of the "controlled network."

     As with Fidonet, this is where you will have the major
     uphill battle. The IETF is dead set against breaking the
     the current open network integrity of the system.  Legacy
     software must be supported.

I happen to agree. Legacy software must be supported, however, I am also 
open to introducing a ZMH like idea where systems are allowed to 
designate "open" and "closed" times for legacy systems.

Based on these experiences, I had started a IETF draft awhile back that 
basically defined "SERVER ATTRIBUTES" records that clients will use to 
obtain server attributes that will cover these client/server negotiation 
parameters.   I happen to believe the idea will be eventually be a major 
part of the system, in some form or another and we are beginning to 
touch based with it with DKIM and its SSP ideas.  Also look at the 
growth of SIP communications.  It is all going (back) in this direction.

--
Hector Santos/CTO
Santronics Software, Inc.

---

From: Frank Ellermann <[email-masked]>
Date: Mon, 26 Mar 2007 06:33:09 +0200

HECTOR SANTOS wrote:

> applications, registrations of any person installing newer and
> better software if it didn't have FTN1 support.

Nobody was forced to use it for sending mail, unless that person
was a *C obliged to test that applicants can receive FTN1 mails.
I needed FTN1 for crash mails (rarely, but still).

> The one thing the FTSC had control of was the NodeList

Nope.  The FTSC was the standards committee, like IETF/IESG/IAB.

The NodeLists were controlled by the *Cs (ultimately the ZCs of
the relevant zones), that's like ICANN.  The FTSC controlled the
definition of the nodelist format (in theory, never changed in
my time), not the content.

Frank

---

From: Douglas Otis <[email-masked]>
Date: Mon, 26 Mar 2007 18:20:20 -0700

> am I barking up the wrong tree?

It would seem OpenID is ideal for controlling a recipient's access to
information being sent using BURL style messages.  OpenID means the
sender would not need to control how the recipient confirms their
identity.  There would need to be a convention established to translate
email-addresses to a URI convention suitable for use with OpenID.

This would protect message content as well as confirm the recipient
actually received their message.  This seems like an ideal mechanism for
various sensitive commerce related transactions.  By pointing to the
message with a URI, there would not be any need to verify the identity
of the message source.  However, the source URI should use the same
conventions as that used for OpenID recipient.

As OpenID really needs a specialized viewer, where one designed to
function as an MUA would not be unreasonable.  OpenID could also help
establish filtering criteria as well.  OpenID is an interesting
mechanism.

-Doug

---

From: "Randy Smith" <[email-masked]>
Date: Wed, 28 Mar 2007 07:36:41 -0600

> be reinventing ourselves to go back to this direction.

The biggest problem, IMO, is not the open system but the anonymous
one. One reason spam works so well is that its so very easy to lie
about who the sender is. I don't think there's any reason why a mail
admin couldn't setup their own registration server but as the
recipient, I need to have some way of knowing that I can trust the
registry, hence the web of trust built around server keys.

> The anonymous sender will be "thing" of the past.
[snip: fidonet history]
> designate "open" and "closed" times for legacy systems.

What I'm thinking of would be an extension to SMTP rather than an
entirely new system. As the main admin for an ISP, the last thing I
want to do is build a second system to handle a new protocol that
hardly anyone uses (at least, until the rest of the Net migrated). I
think the hand shake could be complete wrapped within the SMTP
conversation.

> growth of SIP communications.  It is all going (back) in this direction.

It may be necessary to add a capabilities keyword similar to IMAP or
specify that the EHLO response include a list of supported keywords.

Thank you for taking the time to respond to my ramblings. :-)

---

From: "Randy Smith" <[email-masked]>
Date: Wed, 28 Mar 2007 07:42:51 -0600

> email-addresses to a URI convention suitable for use with OpenID.

Perhaps. Simpler from the recipient's perspective would be to have the
send specify a the URL as part of the SMTP conversation.

> various sensitive commerce related transactions.

I hadn't thought of that but it might be an interesting fringe benefit.

> By pointing to the

Since OpenID is built to allow authentication, among other things,
against 3rd party systems, it seems like an excellent way to allow and
recipient server to authenticate all users who wish to send or deliver
mail with their server.

Thanks for taking the time to respond.

---

From: "william(at)elan.net" <[email-masked]>
Date: Wed, 28 Mar 2007 07:03:40 -0800 (PST)

On Wed, 28 Mar 2007, Randy Smith wrote:

> about who the sender is.

Hence we have SPF and proposals such as DKIM and others.

> I don't think there's any reason why a mail

One of the issues is that SMTP is not fully client-server but rather
store-forward protocol. When you communicate with someone over SMTP
as a client you do not know if its true end-user or not.

> hardly anyone uses (at least, until the rest of the Net migrated).

SMTP is built in such a way that extensions to SMTP that any serious
extensions only you support are to large degree exactly like a new
protocol.

> I think the hand shake could be complete wrapped within the SMTP
> conversation.

We have that in a way - EHLO provides list of capabilities of the server.
keywords issued during RCPT and MAIL FROM provide list of capabilities
of the client. Even with additional EHLO-like keyword that client would
issue the issue is that it could "lie" if it does not want to list certain
capabilities based on what it saw capabilities of the server are - so its
all volunterily in the same way MAIL FROM and RCPT keywords are when
client wants to advertise and turn on certain protocol functions.

> specify that the EHLO response include a list of supported keywords.

EHLO is already basicly list of supported extensions where each
one can be further subdevided (this is rarely used) into support 
sub-extensions/parameters

> Thank you for taking the time to respond to my ramblings. :-)

No problem, that's what we're still here for :)

---

From: Dean Anderson <[email-masked]>
Date: Wed, 28 Mar 2007 22:50:28 -0400 (EDT)

> one. One reason spam works so well is that its so very easy to lie

This is myth, stated since at least 1996, probably earlier. 10+ years of
anti-spam work shows that one does not need to know "who" is sending
mail to effectively complain. One only needs the IP address and a time.  
Their ISP knows who they are, or they don't. The ability to lie is
irrelevant.

That's as good as that identity information gets, too. Privacy laws
would preclude anything else.

The biggest problem is that the real commercial bulk emailers comply
with CAN-SPAM, and thus aren't a problem, while the miscreants send pure
annoyance with no commercial purpose, but it takes a bit of research to
figure that out.  One really needs to find out who the miscreants are.
I've identified a few: they were pretending to be anti-spammers. They
still pretend to be anti-spammers.

> I don't think there's any reason why a mail admin couldn't setup their
> own registration server but as the recipient, I need to have some way
> of knowing that I can trust the registry, hence the web of trust built
> around server keys.

This was tried and didn't work.  'Trust' for email sending, extends to
everyone, and the system is still subject to abuse, albeit
cryptographically signed abuse, and now you have a problem with key
management, and you still need their ISP to translate a real identity to
a crypto key, which they still can't do without a court order or
warrant.  And a new account comes with a different crypto key, and keys
will be stolen by viruses and rooting.  Same problem, many dollars
later. But good if you are selling crypto-email systems and consulting.

I showed in 2003 that information theory PROVES that the miscreant
abusers cannot be stopped by technical means, because a communications
system that is free from abuse [that is 'can't be abused' in contrast to
just 'isn't being abused'] is also free from covert channels, and hence
contrary to a thereom of information theory which states that a
communication system can't be proven free of covert channels. Covert
channels are always possible.  It is always possible to abuse a
communication system.

So, whack-a-mole is as good as it gets.  Actually, that's not a strong
enough statement. This is better: Whack-a-mole is as good as it CAN get,
without upending information theory.  Since information theory is tied
to thermodynamics, and is physically and mathematically sound, that
probably won't happen.

What to do?  Get good a whack-a-mole.  Indeed, we can get better at
that:  standard forms for abuse complaints comes to mind.  Maybe a
standarized web app for submitting such forms, that can be supported by
every ISP complaint/ticket system as a standard interface.

--

It is quite ironic that the people who talk most about reputation
systems have no problem associating _themselves_ with very
disreputatable people and court-proven liars, people pretending to be
anti-spammers, but obviously not.  I suspect there is a story there,
somewhere;  amid the people who pretend to be anti-spammers while
engaging in abuse and lies for their personal gain.  A long time ago, I
said that someday, when the script kiddies get to be adults, and the
statute of limitations expires on their activity, that they will come
forward and spill their beans on who, what, why and when. I think the
"Kevin Mitnicks" of the script-kiddie spamming world will want to tell
their stories. The first generation of script kiddies should be coming
of age and beyond the statute of limitations pretty soon.

		--Dean

-- 
Av8 Internet   Prepared to pay a premium for better service?
www.av8.net         faster, more reliable, better service
617 344 9000

---

From: Jeff Macdonald <[email-masked]>
Date: Wed, 28 Mar 2007 22:52:57 -0400

> recipient server to authenticate all users who wish to send or deliver

Randy, could you use OpenID terms in describing your SMTP extension?
I'm having trouble understanding how this would work from your
description in your blog. Adding PGP seems to add additional overhead
for what OpenID provides (unless I'm totally mis-understanding OpenID).
Here are what I think are some of the relevant terms:

MTA terms:
C:	Sending MTA	- sending message
S:	Receiving MTA	- receiving message

OpenID terms:
Consumer	- wants proof
End User	- wants to prove their identity to Consumer
User Agent	- End user web browser

I is the Identity server

Say there is a new ESMTP keyword, OPENID. Here's a breakdown loosely
following your example:

C->S: connects
S->C: banner

C->S: ehlo
S->C: OPENID is returned along with whatever else

C->S: OPENID <url identifier>
S: 	<becomes a Consumer>
S->I:   <fetches url identifier: Section 3.3 of OpenID spec>
S->C:   250 <identity provider URL: Section 3.5 of OpenID spec>

S->I: associate with identity provider? Section 4.1.x

C->I: go to identity provider? Section 4.2.x

C->S: OPENID CRED <stuff from 4.2.2.3?>
S->C: 250 Ok Credentials are OK

<continue with normal SMTP>

I may of abused SMTP extensions in this example (re: OPENID CRED).

-- 
:: Jeff Macdonald | Principal Engineer, Messaging Technologies
:: e-Dialog | [email-masked]
:: 131 Hartwell Ave. | Lexington, MA 02421 
:: v: 781-372-1922 | f: 781-863-8118 
:: www.e-dialog.com

---

From: Douglas Otis <[email-masked]>
Date: Thu, 29 Mar 2007 11:39:33 -0700

On Mar 28, 2007, at 7:52 PM, Jeff Macdonald wrote:
>> against 3rd party systems, it seems like an excellent way to allow  
>> and recipient server to authenticate all users who wish to send or  
>> deliver mail with their server.
>

I don't think an SMTP extension is required.  Explaining follows:

> description in your blog. Adding PGP seems to add additional  
> overhead for what OpenID provides (unless I'm totally mis- 
> understanding OpenID). Here are what I think are some of the  
> relevant terms:
>
> MTA terms:

There are OpenID extensions proposed to allow the exchange of public  
keys related to an Identity.

http://openid.net/specs/openid-service-key-discovery-1_0-01.html

There are OpenID extensions proposed for email-addresses such as:

http://www.sappenin.com/openid/ext/oet/openid-email-transform- 
extension-1_0.html

http://blog.phpbb.cc/2007/02/04/smtp-service-extension-for-yadis- 
discovery/

Now imagine an extension to DKIM, OpenPGP, or S/MIME where a query  
method is defined for OpenID.

This would allow senders to sign messages and allow recipients to  
verify using extensions proposed for identifying email-addresses  
using OpenID.

This would not require any change to SMTP, but would require a minor  
change to DKIM, OpenPGP, or S/MIME.  But it gets better...

> User Agent	- End user web browser

Add:
Sender	- wants proof of recipient and secure receipt for web  
exchanged information.

It would also be possible to forgo any public-key cryptography  
related to the message by using standardized conventions for where  
the web exchanged information is placed with regard to a specific  
identity.

Perhaps a subdomain of OMail.<email-domain> could work where the  
local-part is normalized within the first path.  Sub-path components  
could function as the message-ID, for example.

When a message is exchanged using something like BURL, the identity  
of the sender can be deduced from the embedded URI.  Nothing else is  
needed as it would be expected this location would be identified  
using SSL certs when related to commerce.  It would be rather  
difficult to forge such messages when the entire message body is  
found via the link.  The web server could then use OpenID to ensure  
only the intended recipient receives the message.  When done using  
HTTPS, email becomes secure without use of encrypted messages.

OpenID is already establishing whitelists to protect blogs.  This  
effort could extend to whitelisting email that is sent using such  
conventions.

> S->C: OPENID is returned along with whatever else

There could be extensions made in the HTTP header structures found in  
OMail subdomain that lists the root names of all "authorized" SMTP  
clients, for example.  I doubt this would be needed once only BURL  
style messages are adopted as an exclusive mode of exchange.  This  
will take awhile.

> S->C:   250 <identity provider URL: Section 3.5 of OpenID spec>

Possible, but this would require a lengthy adoption process.  There  
could be a convention used for the SMTP client host-name that also  
signals use of an authorization field found within the OMail  
subdomain. This convention might look something like:

EHLO:
  OMAIL-<host-name>.<email-domain>.

> I may of abused SMTP extensions in this example (re: OPENID CRED).

An extension could also be used.  Which ever shows the least  
resistance will likely be the best choice.

-Doug