Subject: Re: [imap5] MSNs vs UIDs

From: Arnt Gulbrandsen <[email-masked]>
Date: Wed, 13 Aug 2008 15:28:38 +0200

Does it really matter? To me there's one overwhelming argument: Most 
IMAP4 clients are UID-only. If we want adoption, I think we need a 
_very_ good reason to make IMAP5 UIDs any different from IMAP4 UIDs.

I don't think we would have chosen IMAP4 UIDs if starting from scratch. 
I think I might have liked Mark's UIDL-like suggestion better. But 
there are a lot of IMAP4 UID-only clients in the world.

Arnt
_______________________________________________
imap5 mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/imap5

---

From: Chris Newman <[email-masked]>
Date: Thu, 14 Aug 2008 19:27:36 -0700

--On August 13, 2008 15:28:38 +0200 Arnt Gulbrandsen <[email-masked]> wrote:
> IMAP4 clients are UID-only. If we want adoption, I think we need a _very_
> good reason to make IMAP5 UIDs any different from IMAP4 UIDs.

The problem with IMAP4 UIDs is they give physical existence to mailboxes in 
the data model.  In other words, a "pool of messages" mailstore can 
implement IMAP mailboxes via tags on messages _except_ for the fact that 
IMAP4 UIDs and UIDVAILIDITY need to persist.

I'm convinced that on-disk mailboxes, a necessary evil 20 years ago, are 
just evil today.

Suppose we view IMAP5 as three feature packages:
  IMAP5compat  (make it easy for IMAP4 clients to upgrade)
  IMAP5core    (make it easy for basic mail clients to do basic operations)
  IMAP5sync    (features for managing a local cache, lemonade stuff)

UIDs are the critical feature for IMAP5compat, but I think we should 
consider alternatives for the other feature packages.

		- Chris

---

From: Lyndon Nerenberg <[email-masked]>
Date: Thu, 14 Aug 2008 19:33:16 -0700

> I'm convinced that on-disk mailboxes, a necessary evil 20 years ago, are 

I'm not! But no matter ...
> UIDs are the critical feature for IMAP5compat,

If you still consider messages (including the headers!) immutable, make 
the UID be the SHA-1 hash of the message.

Backwards compatibility must die if we're actually going to fix IMAP4's 
problems.

--lyndon

---

From: <[email-masked]>
Date: Fri, 15 Aug 2008 14:11:30 +0300

Hi Chris,
could you elaborate what you mean by "on-disk mailboxes"?

IMHO, a local store (cache, if you like) on the client is very useful when you are stuck on an airport without any wireless access points around...

If you mean that the client removes all mails from the server to manage a local mailbox instead. Well, I do not consider it evil; I think it's a choice (a dumb one) that a client vendor is allowed to make.

Best regards: Zoltán Ördögh
E-mail: zoltan dot ordogh at nokia dot com
Phone: +358 50 386 0566

-----Original Message-----
From: [email-masked] [mailto:[email-masked]] On Behalf Of ext Chris Newman
Sent: 15 August, 2008 05:28
To: Arnt Gulbrandsen; [email-masked]
Subject: Re: [imap5] MSNs vs UIDs

> _very_ good reason to make IMAP5 UIDs any different from IMAP4 UIDs.

The problem with IMAP4 UIDs is they give physical existence to mailboxes in the data model.  In other words, a "pool of messages" mailstore can implement IMAP mailboxes via tags on messages _except_ for the fact that
IMAP4 UIDs and UIDVAILIDITY need to persist.

I'm convinced that on-disk mailboxes, a necessary evil 20 years ago, are just evil today.

UIDs are the critical feature for IMAP5compat, but I think we should consider alternatives for the other feature packages.

		- Chris

---

From: Mark Crispin <[email-masked]>
Date: Fri, 15 Aug 2008 06:40:09 -0700

> I'm not! But no matter ...

I think that this is all a good idea.  The further off the deep end IMAP5 goes,
the sooner it will die.

> Backwards compatibility must die if we're actually going to fix IMAP4's

Take a look at the POP4 proposal, since that is the inevitable result of doing
the sorts of things discussed on this list.

Once you remove everything from IMAP that made it IMAP, you might as
well call it POP.

_________________________________________________________________
Get thousands of games on your PC, your mobile phone, and the web with Windows®.
http://clk.atdmt.com/MRT/go/108588800/direct/01/
_______________________________________________
imap5 mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/imap5

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Fri, 15 Aug 2008 16:03:08 +0200

Lyndon Nerenberg writes:
> Backwards compatibility must die if we're actually going to fix 
> IMAP4's problems.

IMNSHO IMAP5 can be similar enough that many clients need only minor 
changes, and still fix most of the IMAP4 issues.

What's the benefit of making UIDs be anything other than unsigned integers?

Arnt

---

From: Lyndon Nerenberg <[email-masked]>
Date: Sun, 17 Aug 2008 16:37:56 -0700 (PDT)

> What's the benefit of making UIDs be anything other than unsigned integers?

A UID derived from the SHA-1 hash of the messages is globally unique. It 
gives you content-addressable storage for free, and can simplify cache 
synchronization for offline clients.

--lyndon

   This is AT&T.  Communication is our business, not our mode of working. - Jim Clemans
_______________________________________________
imap5 mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/imap5

---

From: "Bron Gondwana" <[email-masked]>
Date: Mon, 18 Aug 2008 14:17:02 +1000

> synchronization for offline clients.

I definitely second this proposal.

The only downside is that you can't have two identical copies of the same message in
a mailbox.  Yes, I have had that happen (copy to another folder, move back)

There are various alternatives, including GUIDs for the identifier tag.  That's
fine too.

I've been reading this list for a little while now, waiting for a moment to butt in
with this.  I pushed for SHA1 identifiers in Cyrus (we have them now, though not
exposed to the client without my patch that adds FETCH DIGEST.SHA1) and use them
for integrity checking at FastMail.

Regardless of this, my number-one wish list item is a standardised way to get a
cheap digest of the message contents, allowing a client with low bandwidth to
"import" a message store that's not necessarily in sync with the upstream (mail
was moved to a different provider, metadata became corrupted, whatever) and
still recover the data contents of the message without having to push it all
across the link.  A standard method of obtaining a digest allows this.

Also, by standardising on something like sha1, servers will be able to calculate
them at delivery or first access time and cache them - though even with a generic
digest mechanism, a smart server could still cache.  They'll never change!

Bron ( and I still think sha1 is good for this case.  You can use message size as
       a second key, and even if sha1 gets broken with chosen plaintext, it's
       still going to be hard to predict the exact headers that postfix and
       sieve will add before it gets delivered to my mailbox, which is the only
       unauthenticated attack vector on my message pool )
-- 
  Bron Gondwana
  [email-masked]

---

From: Dave Cridland <[email-masked]>
Date: Mon, 18 Aug 2008 09:07:14 +0100

> I definitely second this proposal.

I don't. It's not that content hash based identifiers are bad - quite  
the opposite, they've a few very useful properties - but they're big,  
don't have a concise representation, and are difficult to synchronize  
efficiently.

> There are various alternatives, including GUIDs for the identifier  
That would be even worse - you'd lose all the benefits of a  
content-linked reference.

> across the link.  A standard method of obtaining a digest allows  

I do think this might be useful in some cases, but I think if you go  
this route and discard UIDs, you're optimizing for the worst case.

UIDs provide several properties of a mailbox - the most negative is  
that they're not an intrinsic part of the message, so as Chris  
pointed out, they enforce additional, ideally persistent, data to be  
kept somewhere.

So SHA-1 - bring 'em on. But not as a replacement for UIDs, just an  
addition.

Dave.
-- 
Dave Cridland - mailto:[email-masked] - xmpp:[email-masked]
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade
_______________________________________________
imap5 mailing list
[email-masked]
https://www.ietf.org/mailman/listinfo/imap5