Subject: [imap5] something to remember in light of recent conversations

From: Mark Crispin <[email-masked]>
Date: Tue, 2 Aug 2011 09:31:29 -0700 (PDT)

http://www.xkcd.com/927/

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

---

From: Bron Gondwana <[email-masked]>
Date: Tue, 2 Aug 2011 20:50:31 +0200

> http://www.xkcd.com/927/

I'll see you and raise you:

http://www.dilbert.com/strips/comic/2011-08-02/

More seriously, I do concede that an IMAP "submit" command is purely
a re-invention of something that's covered by existing methods, but I
do believe it can provide a better client experience.  Of course this
is the one of my 4 suggestions for improvements to IMAP that got all
the attention.

But - there's no rational alternative to GUID, DIGEST or MOVE out there.
MOVE is emulated with COPY+EXPUNGE, but this have the disadvantage of
hitting quota limits.  We see that frequently, largely because of:
"delete is Move to Trash".  But that's what users want and expect,
so telling me "it's wrong to use that workflow" is not convincing.

GUID can be mostly emulated with Message-Id, but the reality is that it's
only somewhat-unique among messages in the wild, which isn't good enough
for anything but a heuristic, and DIGEST is only emulatable by either a
hueristic again, or downloading the whole message.

It's interesting to see that gmail created a GUID equivalent in their
extended API, because it's something they use internally.  It maps
nicely to POP too.

Bron.

---

From: Dave Cridland <[email-masked]>
Date: Tue, 02 Aug 2011 21:17:36 +0100

> is the one of my 4 suggestions for improvements to IMAP that got all
I'm not convinced the gain is worth the effort.

> so telling me "it's wrong to use that workflow" is not convincing.
A Trash command, possibly with associated \Deleted magic, might be  
the better solution here.

We have gone through a substantial dance around MOVE recently, I did  
become convinced that some form of MOVE might actually work, if one  
limited one's expectations sufficiently.

But I suspect that for the specific case of "deleting messages", a  
specific command might be warranted.

The problem I have with move-as-delete is basically that it's not the  
metaphor that the system is designed for; it's the moral equivalent  
of deleting files on a UNIX system solely by "mv". Except it's worse,  
because if you did that, you could at least move them back to the  
same place.

That said, I do see the value of being able to locate all one's  
deleted messages.

> extended API, because it's something they use internally.  It maps

GUID and/or a DIGEST - and I think there's a large overlap there -  
should be just a matter of writing the spec out. I don't think  
anyone's violently against those, are they?

Dave.
-- 
Dave Cridland - mailto:[email-masked] - xmpp:[email-masked]
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade

---

From: Bron Gondwana <[email-masked]>
Date: Tue, 2 Aug 2011 22:39:13 +0200

> I'm not convinced the gain is worth the effort.

I'm only partially convinced.  I'm going to have a look at it again when
I get some development time to spend on it.

> the better solution here.

Hmm... perhaps.  Along with SPECIAL-USE so that the Trash folder is
findable easily.  Then again, it doesn't solve the issue of
archiving half your INBOX off to an Archive folder when you're
close to quota.  Copying messages one at a time kinda sucks as a
workaround.

> limited one's expectations sufficiently.

I'm fairly happy to limit expectations to "it works just like
COPY+STORE+EXPUNGE in that order, except it doesn't fail if the
folders are inside the same quotaroot and an intermediate state
would have bumped the quota.  It also doesn't cause overquota
rejection of delivered messages during the action".

> specific command might be warranted.

The general case of moving messages between folders is still a problem.
We get support requests for both.

> The problem I have with move-as-delete is basically that it's not
> the metaphor that the system is designed for; it's the moral
> equivalent of deleting files on a UNIX system solely by "mv". Except
> it's worse, because if you did that, you could at least move them
> back to the same place.

Sorry, how can't you move messages back to where they were deleted
from when they're in Trash?  It's the same metaphor that is used on
Windows, on Mac, and on modern "desktop environments"[tm] on Linux.
It's a very familiar paradigm.

Also, we store a "source folder" flag on every message that's copied
to Trash through our new web interface so that we can implement an
"undo" for deletes.  It's a skanky Base-64 encoded value or something.
We may replace it with an annotation now we have real per-message
annotation support.

> That said, I do see the value of being able to locate all one's

It's more that leaving them in the folder is unsafe with random
clients, far too many call CLOSE or EXPUNGE and wipe out the
deleted messages they didn't create themselves.

> anyone's violently against those, are they?

I hope not :)  There's kind of a devil in the details with DIGEST,
if you start offering arbitrary algorithms on arbitrary parts or
ranges of parts - and how do you tell clients which ones you're
caching and hence are "fast".  GUID is somewhat more simple.

I should get my writing hat on.  I haven't looked at the process for
proposing and getting standards accepted - far easier to flame on
mailing lists.  I should fix myself and do something about that!

Bron.

---

From: Mark Crispin <[email-masked]>
Date: Tue, 2 Aug 2011 13:47:17 -0700 (PDT)

On Tue, 2 Aug 2011, Bron Gondwana wrote:
> from when they're in Trash?

Review the UID assignment rules.

Once a UID is expunged from a mailbox, it can't be reused.  That means
that a message can not reappear with the same UID that it had before.

You can only make a new instance of the message with a new UID.

The whole notion of moving messages to a Trash mailbox is fundamentally
flawed.  It confuses a user interface paradigm with an internal
representation.

-- Mark --

http://panda.com/mrc

---

From: Bron Gondwana <[email-masked]>
Date: Tue, 2 Aug 2011 23:01:01 +0200

> that a message can not reappear with the same UID that it had before.

No shit - you append it with a new UID.  Despite your repeated attempts
to paint me as one, I'm not a blithering moron.

> You can only make a new instance of the message with a new UID.

Indeed.  Though if you have a GUID or DIGEST you can fairly efficiently
detect this case - and along with the case of an accidental move and
move back, which has the same underlying issue.

> flawed.  It confuses a user interface paradigm with an internal

I don't even know how to start responding to this.  The user interface
paradigm needs an internal representation.  There is a user interface
that people have clearly chosen, for whatever reason - wrongheaded or
not - take your pick.  My interest is in coming up with an internal
representation that actually provides the interface users want rather
than telling them to alter themselves to make my life easier.

I believe in having the computer do extra work so that users don't
have to.  You appear to believe the opposite, that users should change
their behaviour to make life easier for the computer.  I can't cross
this divide and join you I'm afraid.

Bron.

---

From: Dave Cridland <[email-masked]>
Date: Tue, 02 Aug 2011 22:02:19 +0100

> close to quota.  Copying messages one at a time kinda sucks as a

Except I'm *very* far from convinced that archival of messages is a  
suitable reaction to getting close to quota.

> rejection of delivered messages during the action".

RIght, but if you're close to quota, this is exactly the case where  
moving messages about isn't going to help much anyway.

> We get support requests for both.
OK. Review the last thread on MOVE. There was a real concerted effort  
to finding a MOVE that would be maximlally implementable. It wasn't  
great, but it seemed better than no MOVE at all for a number of use  
cases. (It was, as I recall, roughly a one-at-a-time-but-serverside).

> It's a very familiar paradigm.
Two reasons:

> We may replace it with an annotation now we have real per-message
(Love the use of the word skanky, BTW)

2) The UID is messed up. This is particularly annoying if you don't  
have (or particularly want) SORT, and use of SORT really slows down  
online low-bandwidth use.

The UID is basically the filename equivalent; so on UNIX you can  
pseudo-delete by "mv foo ~/Trash", and then undelete by "mv  
~/Trash/foo .", whereas on IMAP you cannot. (And yes, I know that's  
not really how trash folders work in Linux; they have to preserver  
the original filename and deleted timestamp in order to handle  
multiple deletes etc).

> deleted messages they didn't create themselves.
OK, so we'd need a flippy switch to turn off EXPUNGEs. Perhaps the  
server can be told that you use "trash aware" clients and needs an  
ENABLE before EXPUNGE works again, or something.

I can forsee support problems, mind, but I think the flag-as-deleted  
model is also worth preserving if we can (I infinitely prefer it).

It's worth noting that the \Deleted flag was state of the art for  
supporting undelete mechanisms of the time; the fact that fashion has  
changed is indeed significant, but I should stress that it was very  
far from an arbitrary decision. Also, doing it this way meant that  
costly mbox file rewriting could be done less, I suspect.

> caching and hence are "fast".  GUID is somewhat more simple.

I was assuming DIGEST and GUID were largely identical, actually, but  
I can go along with this, I've honestly not thought about them much.

> mailing lists.  I should fix myself and do something about that!

http://tools.ietf.org/ has lots of pointers to help, including  
pointers to xml2rfc, a spell checker and nits checker, and a link to  
the submissions tool.

It is generally easier to mount a constructive argument both for and  
against an I-D rather than a sketch in a mail message, so I'd  
encourage you to document even the ideas you know are likely to be  
shot down.

Dave.

---

From: Dave Cridland <[email-masked]>
Date: Tue, 02 Aug 2011 22:08:59 +0100

> this divide and join you I'm afraid.

The question is which computer should be doing the work, and how much  
the UI paradigm needs to impose its model onto the exant model of the  
mailstore.

I think there's a fair argument that "IMAP was there first", so the  
onus is somewhat on the mail client to handle the hard work, rather  
than passing the buck to the server.

That said, I'm generally open to ways of supporting the model from  
the server, if possible and reasonable.

This is basically why I'd be willing to look into (and work upon) a  
"Trash command" which could co-exist with a deleted-flag model.

Dave.

---

From: Mark Crispin <[email-masked]>
Date: Tue, 2 Aug 2011 14:13:14 -0700 (PDT)

On Tue, 2 Aug 2011, Bron Gondwana wrote:
> No shit - you append it with a new UID.

Which is a completely new instance of the message, which every
synchronizing client must redownload.

> Despite your repeated attempts

Then stop acting like one.

> paradigm needs an internal representation.

No it does not.  The user interface software maps from whatever internal
representation is present to the paradigm that it presents to the user.

> have to.

If that is the case, then the computer software, not the protocols, should
do the necessary work.  It does not mean that protocols must be altered so
that computer software is reduced to a null transform.

> You appear to believe the opposite, that users should change

Maybe that is because you persist in making strawman arguments.

-- Mark --

http://panda.com/mrc

---

From: Bron Gondwana <[email-masked]>
Date: Tue, 2 Aug 2011 23:34:42 +0200

> suitable reaction to getting close to quota.

Oh no, it's not a reaction.  It's just your normal monthly/yearly
archiving, except you also happen to be close to quota.

> moving messages about isn't going to help much anyway.

You're not doing it to "help", you might be at 75% of quota and still
months off being close to the limit, but want to move what amounts to
50% of your messages.  If you move it all at once, you hit the quota,
if you move it in little batches, you don't.  Annoying and unintuative.

> OK. Review the last thread on MOVE. There was a real concerted
> effort to finding a MOVE that would be maximlally implementable. It
> wasn't great, but it seemed better than no MOVE at all for a number
> of use cases. (It was, as I recall, roughly a
> one-at-a-time-but-serverside).

The result being "can't fail in a way that leaves huge piles of
steaming mess around if there's an error half way through" I'm
guessing.

It does have to minimally implementable at BOTH ends, of course,
or it will wind up being minimally implemented, which is kind of
pointless.

> 1) You need to remember it, as you note:

Well, yeah.

> online low-bandwidth use.

Again - yes, difficult.

You could always be really obnoxious about it and just bump every
message with a higher "original UID" up, but if someone accidentally
deleted UID 1 and tried to undo you could eat up UIDs a bit fast.

Of course, once you get close to 2^32 you just raise the UIDVALIDITY
by one and renumber back from 1 again, so it's theoretically solvable.

> the original filename and deleted timestamp in order to handle

Yeah, so there are some "impedence mismatches" between what's wanted
and what IMAP provides.  Many clients use SORT now of course, or
cache everything locally so they can get the sorted-by-date view
efficiently and render quickly.

> ENABLE before EXPUNGE works again, or something.

No dice.  First non-aware client to connect and issue an EXPUNGE or
CLOSE will wipe out your carefully assembled house-of-cards.

The only way I can see it working is a 'non-expungable-but-\Deleted'
flag.  We did something like that in our web interface in the past
when expunge was too expensive.  We set 'medeleted' rather than the
real deleted, and all "SORT"s were actually "SEARCH NOT FLAG medeleted".
Talk about your workarounds.

Layer a few of these together, and things get pretty horrible.  Thankfully
"EXPUNGE" in Cyrus now costs exactly as much as a FLAG store until the
garbage collection happens, so clients aren't left implementing their own
different workarounds for expensive server-side expunge.

> model is also worth preserving if we can (I infinitely prefer it).

On what grounds?  Mostly you see clients doing "UID STORE +FLAG \Deleted
$uids" then "UID EXPUNGE $uids" if they're being polite - which is no
different fundamentally than "UID DELETE $uids" other than pushing the
intermediate state cleanup out to the clients.

> supporting undelete mechanisms of the time; the fact that fashion
> has changed is indeed significant, but I should stress that it was
> very far from an arbitrary decision. Also, doing it this way meant
> that costly mbox file rewriting could be done less, I suspect.

Oh sure - it pushed the garbage collection out to the client.  I can
see why the decision was made, and like any tradeoff it has its good
and bad points.  I'm not arguing in any way that it's wrong.  You
certainly need a way for a client to be able to delete fast - indeed
the client needs to be able to batch deletes if some punk is sitting
there bouncing on the D key.  I do that myself.

And it's certainly easier on the server to put the responsibility
for batching the mbox file rewrite on the client.  Mind you, by the
time you're supporting QRESYNC and CONDSTORE while still supporting
RFC3501's requirement that sequence numbers don't change arbitrarily
between batchable non-UID commands - you wind up having to support
some sort of MVCC on the server anyway - at which point you wind
up having to garbage collect enough state that regular EXPUNGE
gets cheaper.

> I can go along with this, I've honestly not thought about them much.

Google's GUID is a random string.

http://code.google.com/apis/gmail/imap/#x-gm-msgid

They also have a Thread ID which applies across folders X-GM-THRID.
We have something very similar called 'CID' (conversations ID) but
we're not providing to anything outside our testing environment yet.

DIGEST on the other hand is something that can be calculated at the
other end as well, so can be used for integrity checking of cached
data, and detecting the same message even if it went out and back in
via a copy between servers or from a client which moved them offline
and forgot the fact, so APPENDed rather than COPY(i)ed.

> pointers to xml2rfc, a spell checker and nits checker, and a link to

Funky.  Thanks.

> encourage you to document even the ideas you know are likely to be

Will do.

Thanks,

Bron.

---

From: Bron Gondwana <[email-masked]>
Date: Tue, 2 Aug 2011 23:37:32 +0200

On Tue, Aug 02, 2011 at 10:08:59PM +0100, Dave Cridland wrote:
> The question is which computer should be doing the work, and how
> much the UI paradigm needs to impose its model onto the exant model
> of the mailstore.
> 

Absolutely.  Except when the "impedence mismatch" (such a good
term for describing the situation) gets too great, something has
to give!  Either clients migrate to a new protocol, or the existing
protocol changes.

> "Trash command" which could co-exist with a deleted-flag model.

Excellent.

Bron.

---

From: Bron Gondwana <[email-masked]>
Date: Wed, 3 Aug 2011 00:19:59 +0200

> synchronizing client must redownload.

True.  Luckily, this is a much rarer case than even sending email.
Undo being slightly more expensive is not the end of the world, and
synchronizing clients with a GUID/DIGEST can redownload much more
cheaply.

> Then stop acting like one.

I disagree with your assertion that wanting to solve problems makes
me a moron.  I didn't come flying in with no experience, I've been
running a mail service for a reasonable amount of time, and also
maintaining a server implementation and doing work on clients.

> representation is present to the paradigm that it presents to the user.

Indeed it does.

> that computer software is reduced to a null transform.

That's fine, so long as the protocols provide a method to map the
user interface's requirements.  The issue is that IMAP provides no
way to identify that a message is "the same" content, except by it
having the same triple (mboxname, uidvalidity, uid).

Except for the same client, on the same connection, there's no way
to see that you have the same message after a COPY.

There's no way to move a message without using extra quota during
the move.

> Maybe that is because you persist in making strawman arguments.

The same things that annoyed me about IMAP+SMTP 10 years ago still
annoy me.  I have more experience now to have a good idea of what
solutions to them are likely to work if you're trying to solve the
problems rather than maintain "protocol purity".

At the point where you're working around deficiencies in an API
rather than providing the facilities in the API, you get
un-necessary complexity.  It's my assertion that IMAP clients have
a large amount of un-necessary complexity to work around, and that
a primary goal of any potential IMAP5 should be reducing that
un-necessary complexity - so that indeed the software is as close
to a null transform as possible.  Layers of transforms and workarounds
stacked over each other does not make a good overall system.

The protocol is only one component on the system, it doesn't deserve
to be raised on a pedestal above all others.  The client needs to
transmit an intention from the user to the mail store, and the more
layers of translation and rewriting that intention goes through, the
less clearly it gets articulated.

Yes, fashions change, and trying to follow all of them is a losing
battle, but never changing to match the reality of how you're
actually being used is also a losing battle.

Bron.

---

From: Mark Crispin <[email-masked]>
Date: Tue, 2 Aug 2011 16:46:38 -0700 (PDT)

On Wed, 3 Aug 2011, Bron Gondwana wrote:
> me a moron.

Wanting to solve problems does not make you a moron.

Demanding that your proferred solution be adopted, to the exclusion of any
other approach, makes you a moron.

Repeating incessantly "it's all so simple, it is just this trivial change,
and the people who say that it is more complicated than that are silly old
farts who can't adapt to change" makes you a moron.

Insisting that you understand all the issues, and accusing those who
disagree of being mindless followers of a "protocol purity" religion,
makes you a moron.

> I didn't come flying in with no experience, I've been

Knowledge is a dangerous thing, and a little knowledge is the most
dangerous of all.  The more you learn in the future, the more that you
will come to realize how much you do not know.

Life is full of seemingly-simple solutions to intractable problems.
Disaster results when such false solutions are deployed.  It takes
experience and wisdom to avoid such mistakes; the latter coming somewhat
after the former.

The classic example is "what would have happened if Y2K had happened a few
years later?"  Y2K was a non-issue largely because the WWII generation was
still around.

> user interface's requirements.

It is not the responsibility, nor the mission, of the protocol to provide
such mapping.  Mapping is the responsibility of the consumer of the
protocol.

> The issue is that IMAP provides no

That would, in turn, point you to the solution if you had the experience
and wisdom.

> to see that you have the same message after a COPY.

RFC 4315.

> There's no way to move a message without using extra quota during

Those are server implementation concepts.

To put it in UNIX terms, there is no requirement in IMAP that COPY be
implemented as a cp rather than an ln.

Even if MOVE were added to IMAP, there is (can be) no requirement that a
server implement MOVE as an rather than a cp+rm.

A protocol specification does not (can not) dictate server implementation
specifics.

Quota itself is a mentality of systems with limited storage that must be
recycled.  It is a completely alien concept in the increasing percentage
of environments in which is no such thing as storage recycling; "expunge"
is merely a "remove from view" but data is retained forever.

Now, with this all said, a GUID would be useful in a new IMAP.  The horse
long ago left that barn for the existing IMAP.  Similarly, it is desirable
to abolish mailboxes in favor of named views to a single store (thus
making the whole COPY/MOVE argument moot); Gmail has the right idea albeit
with a wretchedly bad export to existing IMAP.

> I have more experience now to have a good idea of what

The practice of burning strawmen does not make bogus arguments any more
convincing.

-- Mark --

http://panda.com/mrc

---

From: Gilles LAMIRAL <[email-masked]>
Date: Wed, 03 Aug 2011 06:57:36 +0200

Hello,

> if you move it in little batches, you don't.  Annoying and unintuative.

How poor your tools are! You have an ugly server doing stupid 
IMAP implementation (cp+rm) and stupid quota (while disk space double
evey 18 months) and you also have a stupid IMAP client
that can't do little batches at your place.

So with all that poor and stupid tools you're using on both sides
you then ask the protocol to change itself expecting that both sides
won't stay poor and stupid to implement the new protocol.
They will stay poor and stupid because the issues you describe 
here (move over quota) can be solved right now easily with the 
current protocol.

-- 
Au revoir,                             09 51 84 42 42
Gilles Lamiral. France, Baulon (35580) 06 20 79 76 06

---

From: Bron Gondwana <[email-masked]>
Date: Wed, 3 Aug 2011 08:24:20 +0200

> that can't do little batches at your place.

Argument doesn't hold up when you have large numbers of people paying
a small amount of money to use a service - you need to have quotas of
some sort.

We could possibly change the server so that multiple copies of the
same message didn't count towards quota - they don't use any additional
space on the server (well, not much) because of hardlinks.  This would
indeed be a workaround, if somewhat confusing.

But yes, that could be a way to solve our individual case.

> here (move over quota) can be solved right now easily with the 

Actually, they can't without making the server not charge quota for
the move.  Sure you can do small batches, but you need to guess what
size is "small enough", and it's going to be a bigger IO hit for the
server and bandwidth hit if you guess wrong.

In other words you're working around the fact that the protocol doesn't
allow you to express your intention directly to the server.

Bron.

---

From: Gilles LAMIRAL <[email-masked]>
Date: Wed, 03 Aug 2011 09:07:25 +0200

Hello,

> a small amount of money to use a service - you need to have quotas of

Ok. Just write an imap server implementation allowing a move
within quota. Easy.
 
> indeed be a workaround, if somewhat confusing.

So do it. They are other ways to do it.

> server and bandwidth hit if you guess wrong.

This client change only has to be considered if the server stay poor and stupid.
In that case one message at a time will do the job. And bandwidth increase
for the IMAP commands is peanuts.

> allow you to express your intention directly to the server.

I'm working around the fact you have poor and stupid tools on both sides
you don't master with peanuts bandwith constraints and false marketing 
ones, "my clients pay so few I can't add a 2TB hard disk costing 100 USD".
5 cents for 1 GB is to much? Wait 18 months it will be 2 cents.

---

From: "Bron Gondwana" <[email-masked]>
Date: Wed, 03 Aug 2011 09:43:12 +0200

> other approach, makes you a moron.

I'm happy with another solution, so long as it actually solves
the problems that are hampering the current approaches.

> experience and wisdom to avoid such mistakes; the latter coming somewhat

Life is also full of seemingly complex problems which are only complex
because you're doing it wrong.  For example generating sendmail.cf
 from m4 and then generating THAT from another language because m4
is too complex/ugly as well.  And then putting a firewall in front
of sendmail that does stateful inspection because sendmail is too
riddled with security holes due to its architecture.

Or you could do what quite a few have done since which is build a
modular server with a configuration file syntax that wasn't designed
to make things easier for the computer, but instead to make configuring
the most common servers easier, and more complex ones possible.

> years later?"  Y2K was a non-issue largely because the WWII generation  

Woah.  I take it back, you're not just saying I'm a moron, you're saying
that everyone born after you is a moron.  Well, I guess it doesn't matter
what I try then, because once the WWII generation die off, we're all
royally f^&*ed.

This is a massive tangent from the original discussion, but Y2K was
largely caused by making things simpler for the computer, because
computers didn't have the capacity to waste on extra bytes.  Later in
this same email you argue that quotas are a temporary artifact of
artificial scarcity which will disappear.  I think you're at least
partially right there.

It was also exacerbated by the source code to many systems being no
longer available.  Thankfully, things are a bit better now, with most
programmers using some form of source control.  I was disappointed,
looking back at my degree, how poorly source control was taught.
They spent ages talking about waterfall, but didn't get us using
RCS (yeah, guess what we were taught) or CVS right at the start.
I had to learn to love source control on my own time - because you
can't love CVS.

> such mapping.  Mapping is the responsibility of the consumer of the

Nonsense - clarity of communication is a responsibility of the whole
stack.  If there is no clear way to communicate the intention of the
user to the server, then the server is left to infer the user's
intent by reverse engineering the operations it sees over the
protocol.  If the set of actions that users want to perform doesn't
mesh well with the protocol, then

> That would, in turn, point you to the solution if you had the experience

That's nice.  I'm presuming you mean "mark it \Deleted but don't
expunge it".  That's not a solution in a world where a different
session can complete the delete for you without meaning to.   If
there was no EXPUNGE, only UID EXPUNGE, then I would be happy
to agree, but that ship has sailed.  You can't trust it in the
arbitrary case, and it dirties up the view in existing clients.

>> Except for the same client, on the same connection, there's no way
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
>> to see that you have the same message after a COPY.

Um ...how RFC 4315 provides the information about it being the same message
to anyone other than the client that actually does the copy?  Every other
connected client has no semantic information and no way to get it.

> A protocol specification does not (can not) dictate server implementation

It can dictate things like "A must happen before B" though.  Or at least
"it must not be possible for B to happen without A happening as well",
which means you can either do atomic, or A then B, but not B then A.

> is merely a "remove from view" but data is retained forever.

Such environments still have abuse controls of some sort.  Hanging infinite
uncontrolled storage out over any protocol to untrusted users causes
problems.

> with a wretchedly bad export to existing IMAP.

Let's write that protocol then.

Bron

---

From: "Bron Gondwana" <[email-masked]>
Date: Wed, 03 Aug 2011 09:52:33 +0200

> 5 cents for 1 GB is to much? Wait 18 months it will be 2 cents.

I think we calculated it costs us about 35c per gigabyte of email,
purely for storage.  That's RAID1, metadata on SSD, one onsite live
replica, one offsite live replica, plus a separate backup.

Then there's server costs, maintenance, abuse management, support,
deprecation, etc.

A 2Tb consumer hard disk is so clearly not equivalent to the cost of
running a complete service that I can't believe you would propose it.

Bron.

-- 
Using Opera's revolutionary email client: http://www.opera.com/mail/

---

From: Gilles LAMIRAL <[email-masked]>
Date: Wed, 03 Aug 2011 10:28:12 +0200

Hello,

> replica, one offsite live replica, plus a separate backup.

And how much your clients pay per month for how many GB?

> Then there's server costs, maintenance, abuse management, support,

So continue to give all the cost numbers, I already know we'll all
smile when we do the difference between what cost your current whole
system and what will cost the future whole system with all the imap
change you ask. Peanut dust.

> running a complete service that I can't believe you would propose it.

If I had so poor clients as you mean to deal with I certainly 
would use 2Tb consumer hard disks.

May be you know what Google use as hard disks? 
The poor consumer ones. Really. Follow them,
they do know why they've done that choice, technically
and commercially speaking.

---

From: "Bron Gondwana" <[email-masked]>
Date: Wed, 03 Aug 2011 10:47:57 +0200

> And how much your clients pay per month for how many GB?

More than that.  Like I said, that's JUST storage.

> change you ask. Peanut dust.

I don't precisely see what that has to do with making MOVE not
affected by quota.

> would use 2Tb consumer hard disks.

I pity your clients.  There's a middle ground between "gold plated"
and "dirt cheap no redundancy".  We keep the raw emails on 2Tb consumer
hard disks, mostly, but not on just a single one.

> and commercially speaking.

Same reason we did - it's dirt cheap and performance and reliability
is sufficient if you have additional layers of indexing on faster drives
and redundancy to get around the lower reliability.

None of which is relevant to the argument about whether quotas are
required to avoid rampant abuse, or whether a way to move messages
within the same quotaroot without having to check if you're playing
chicken with the limit is worthwhile.

---

From: Gilles LAMIRAL <[email-masked]>
Date: Wed, 03 Aug 2011 11:29:54 +0200

Hello,

> More than that.  Like I said, that's JUST storage.

What they pay for storage?

> I don't precisely see what that has to do with making MOVE not

Good if you don't see what that has to do with making MOVE not
affected by quota. So do I. Just don't invoke storage cost in
your argumentation and don't invoke extra "server costs, maintenance, 
abuse management, support, deprecation" meaning nothing to you in
the discussion.

> I pity your clients. 

They're laughing because I don't poison them with quotas,
I monitor disk spaces and buy terabytes when needed, before needed.
No one has ever cried by moving messages. They pay for this.

> There's a middle ground between "gold plated"

The argument is exactly the same for 35 cents per GB.
How much your clients pay per month for how many GB for storage?
Stop extra false arguments before replying the question.

---

From: "Bron Gondwana" <[email-masked]>
Date: Wed, 03 Aug 2011 11:46:48 +0200

> What they pay for storage?

No - they pay for an entire service.  The fact that they don't
pay an infinite amount means that storage must be restricted in
some way, because it has a cost.

Some of them pay nothing at all.

> abuse management, support, deprecation" meaning nothing to you in

If you have quotas at all, this becomes an issue.  If you don't
have quotas, sure, no problem.

> No one has ever cried by moving messages. They pay for this.

Good for you.  Do you provide free accounts as well?  How do
you stop abuse?

> Stop extra false arguments before replying the question.

https://www.fastmail.fm/mail/personal.html
https://www.fastmail.fm/mail/businessplans.html

There will be different pricing structures for the Opera branded
services, but I don't think they're fixed yet, and even if they
were I wouldn't be allowed to tell the world before we announced
them officially.

Bron.

---

From: "Bron Gondwana" <[email-masked]>
Date: Wed, 03 Aug 2011 12:18:43 +0200

> https://www.fastmail.fm/mail/businessplans.html

While we're waving our "how much do your clients pay for your
excellent service" wands around - I tried to go to:

https://compte.laposte.net/inscription/etape1.do

but I get a zero-byte response.

So how much do your clients pay, and how much space do you give
them before you start charging more?

---

From: Gilles LAMIRAL <[email-masked]>
Date: Wed, 03 Aug 2011 17:35:59 +0200

Hi,

> them before you start charging more?

Here I'm the client. It's free for 10 GB.

---

From: Jan Kundrát <[email-masked]>
Date: Wed, 03 Aug 2011 10:34:24 +0200

> and commercially speaking.

You're right that quite a few of the massive storage systems available
these days run on 7200 RPM SATA drives, and that some storage vendors
are happy to use the desktop-grade disks in them. What Bron was saying,
though, is that the total cost of a storage system is not only the price
of the disk drives, but instead has to include quote a lot of other
expenditures, both in HW cost, as well as in power/space consumption and
maintenance.

There's no doubt that the "storage is cheap" mantra is true. What is
also true is that the data deduplication business is pretty flourishing
(at least that's what the array vendors tell you), so I guess the
problem of limited capacity is very real for most enterprises.
Dismissing that with "just buy a 2TB disk drive, man" is not really helpful.

Cheers,
Jan
-- 
Trojita, a fast e-mail client -- http://trojita.flaska.net/