Subject: Re: [lemonade] I-D ACTION:draft-ietf-lemonade-vfolder-01.txt

From: Randall Gellens <[email-masked]>
Date: Thu, 8 Jun 2006 15:00:38 -0700

At 1:41 PM +0200 5/31/06, Arnt Gulbrandsen wrote:

>  1. APPEND/COPY.
>
>  In section 1 and 8, APPEND and COPY are explicitly permitted on 
> vfolders. I think that's desirable, but there's a problem, perhaps 
> a nasty one: A client may expect that if it appends/copies a 
> message to a mailbox, that message shows up in that mailbox. If the 
> mailbox happens to be a vfolder and the message doesn't match the 
> search, the message may (apparently) disappear without a trace.
>
>  There are three possibilities: Either the message must be made to 
> fit the search (e.g. by setting the right flags if the search uses 
> flags), or the search must be modified to include that message's 
> UID, or the WG considers it acceptable that the message sometimes 
> goes only to the backing mailbox, not to the vfolder. Personally, I 
> think the first two are horrid and strongly favour the latter, I'm 
> just a little nervous that it might confuse some IMAP clients.
>
>  Section 8 should probably get a sentence or paragraph on this, once 
> there's consensus.

>
>  2. VFOLDER on VFOLDER.
>
>  In section 2, a vfolders may be created with another vfolder as 
> backing. I don't see a use-case for that, and I do see one 
> drawback, namely that it complicates making a vfolder 
> "users.stephane.blah" with "users.stephane.*" as backing. The 
> present draft does not permit multi-mailbox backing, but I don't 
> expect that state of affairs to last very long. People want 
> multi-mailbox backing. More on that below.
>
>  Can anyone think of a use-case or other argument for allowing 
> vfolders on vfolders?

I think perhaps these two issues are related, with the core issue 
being: does a vfolder appear to a client as just another mailbox, or 
is it special in some way?

If a vfolder is distinguishable from a regular mailbox, then perhaps 
it is OK to not use a vfolder as a backing store.  In a non-list 
email exchange, Alexey suggested that if a vfolder is used as a 
backing mailbox, that the server would in reality make the search 
criteria the intersection of those of the new vfolder and the older 
vfolder.

So, if:
	- mailbox TOP is a vfolder using mailbox MIDDLE as a backing store, and
	- mailbox MIDDLE is a vfolder using mailbox BOTTOM as a backing store, and
	- mailbox BOTTOM is a normal mailbox,

Then allowing mailbox MIDDLE to be used as the backing store for 
vfolder TOP becomes just a short-hand for creating vfolder TOP using 
as its search criteria that of vfolder MIDDLE plus new elements.

But after vfolder TOP is created, it would be seen to be using 
mailbox BOTTOM as its backing store.

On the COPY/APPEND issue, if vfolders are distinguishable, then 
perhaps it is OK if a COPY/APPEND might seem to vanish.  With 
UIDPLUS, perhaps the APPENDUID and COPYUID response codes might 
indicate the UID in the backing mailbox?

-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly-selected tag: ---------------
yoin (yoh-EEN; Japanese; noun): experiential reverberation that
continues to move one long after the initial external stimulus
has ceased.

_______________________________________________
lemonade mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/lemonade

---

From: Curtis King <[email-masked]>
Date: Mon, 12 Jun 2006 16:53:34 -0600

On 8-Jun-06, at 4:00 PM, Randall Gellens wrote:

> perhaps it is OK if a COPY/APPEND might seem to vanish.

Having a message disappear which you just copied/appended will be  
very confusing to people. No matter how the mailbox is marked. If the  
client controls the view it can add appended messages to view if needed.

ck

---

From: Randall Gellens <[email-masked]>
Date: Mon, 12 Jun 2006 16:24:40 -0700

At 4:53 PM -0600 6/12/06, Curtis King wrote:

> very confusing to people. No matter how the mailbox is marked.

Confusing to *people* yes.  But the *client* needn't be confused, and 
if the client isn't confused, then the user doesn't need to be 
confused either.

What I was suggesting was that if the client knows it has selected a 
vmailbox, and it does a copy or append, then as long as it gets an OK 
response it can present the message to the user as being part of the 
mailbox.  Especially if it has received the UID of the 
appended/copied message in the command response.

>  If the client controls the view it can add appended messages to 
> view if needed.

A good point, but it would be awkward for the client to need to 
update the vmailbox filter criteria each time it does a copy or 
append.

-------------- Randomly-selected tag: ---------------
If A = B and B = C, then A = C, except where void or prohibited by law.

---

From: Curtis King <[email-masked]>
Date: Mon, 12 Jun 2006 23:33:31 -0600

On 12-Jun-06, at 5:24 PM, Randall Gellens wrote:

> Confusing to *people* yes.  But the *client* needn't be confused,  
> and if the client isn't confused, then the user doesn't need to be  
> confused either.
>
> What I was suggesting was that if the client knows it has selected  
> a vmailbox, and it does a copy or append, then as long as it gets  
> an OK response it can present the message to the user as being part  
> of the mailbox.  Especially if it has received the UID of the  

Ok - Then what does VMAILBOX get me? If I have to merge a view and  
the append uid on the client side, why not do that from the start  
using search?

> update the vmailbox filter criteria each time it does a copy or  

We are suggesting the same thing, I was thinking the client would  
merge the results of the search with uid returned from the copy or  
append. We can do this now using SEARCH and UIDPLUS. VMAILBOX does  
not make this any easier than the standard IMAP commands.

ck

---

From: Randall Gellens <[email-masked]>
Date: Tue, 13 Jun 2006 11:59:21 -0700

At 11:33 PM -0600 6/12/06, Curtis King wrote:

>>>   Having a message disappear which you just copied/appended will 
>>> be very confusing to people. No matter how the mailbox is marked.
>>
>> an OK response it can present the message to the user as being 
>> part of the mailbox.  Especially if it has received the UID of the 

Can you maybe write up a quick proto-draft that proposes this?  I 
think we should go with whichever approach is simpler for the client 
and hopefully also easier for the server to support.  Because time is 
an issue I think the group needs to see the counter-proposal right 
away.

My understanding is that vmailbox has two primary use cases: (1) 
allowing a client to get notifications only for state changes to a 
subset of "interesting" messages, and (2) allowing a client to 
conceive of a mailbox as a subset of "interesting" messages to reduce 
the size of its cache and the amount of traffic exchanged with the 
server.  If this understanding is correct, and your counter-proposal 
supports both uses but has advantages, I'd strongly support us 
adopting it.

We are in a hurry, but we also want the best solution.
-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly-selected tag: ---------------
Arithmetic is being able to count up to twenty without taking
off your shoes.

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Tue, 13 Jun 2006 21:37:29 +0200

Randall Gellens writes, answering Curtis King:
> Can you maybe write up a quick proto-draft that proposes this?

Seconded. Strongly.

> I think we should go with whichever approach is simpler for the client 

I agree.

> supports both uses but has advantages, I'd strongly support us 

FWIW, I have users for whom a goal is to share vfolders among clients, 
zero or more of which aren't going to support lemonade in a hurry. The 
latter may seem necessarily irrelevant to lemonade, but I think it's 
worth keeping in mind: People won't ditch their desktop clients just 
because they get a new mobile device, and for a desktop client, 
implementing lemonade is not going to be a very a high priority.

Arnt

---

From: Dave Cridland <[email-masked]>
Date: Tue, 13 Jun 2006 23:09:14 +0100

>> Can you maybe write up a quick proto-draft that proposes this?
Done - attached is what Curtis and myself thrashed out as something 
that was essentially optimizing what clients can do already.

>> conceive of a mailbox as a subset of "interesting" messages to 
>> reduce the size of its cache and the amount of traffic exchanged 
>> with the server.  If this understanding is correct, and your 
>> counter-proposal supports both uses but has advantages, I'd 
>> strongly support us adopting it.
> 
> 
This is primarily aimed at item (2). I don't perceive getting state 
changes for messages outside the criteria to be a problem, I simply 
don't think mailboxes with sufficient activity for this to be a 
problem exist such that they're likely to be looked at on a mobile 
device.

It can be used to provide a client-side virtual mailbox "user 
experience", if desired. Polymer, my own client, is already doing 
something fairly similar to provide filtered views onto mailboxes - 
if this extension were supported, I could do it much more 
efficiently. Internally, Polymer actually treats these views as a 
mailbox.

There are other uses I've not fully considered - it's possible to 
have multiple "virtual mailboxes" selected concurrently with this 
extension in a single connection, in effect, and it's also possible 
to use the extension for colourizing summary listings, for instance.

The extension as described is intended to be fitted onto both SEARCH 
and SORT, incidentally, and as such, exceeds the capability of the 
current VFOLDER draft in some respects. I'm afraid I've not devoted 
much time into the Extended SORT, but if you look closely, all the 
hooks are there. We haven't considered THREAD, although I'd be 
interested in exploring that, but I don't think we have the time to 
nail it.

> FWIW, I have users for whom a goal is to share vfolders among 
> clients, zero or more of which aren't going to support lemonade in 
> a hurry. The latter may seem necessarily irrelevant to lemonade, 
> but I think it's worth keeping in mind: People won't ditch their 
> desktop clients just because they get a new mobile device, and for 
> a desktop client, implementing lemonade is not going to be a very a 
> high priority.

Yes, there's a considerable number of use-cases for which you do need 
a real virtual mailbox - if you'll pardon the expression.

I'm quite confident that virtual mailboxes are a useful area of 
research, and a useful feature to have. I have been convinced by 
Curtis and others, though, that they're sufficiently complex that 
they are inadvisable to mandate in Lemonade.

Dave.
-- 
Dave Cridland - mailto:[email-masked] - xmpp:[email-masked]
  - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/
  - http://dave.cridland.net/
Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade
_______________________________________________
lemonade mailing list
[email-masked]
https://www1.ietf.org/mailman/listinfo/lemonade

---

From: Randall Gellens <[email-masked]>
Date: Tue, 13 Jun 2006 15:24:57 -0700

At 9:37 PM +0200 6/13/06, Arnt Gulbrandsen wrote:

>  for a desktop client, implementing lemonade is not going to be a 
> very a high priority.

This may depend on what advantages lemonade offers. 
Forward-without-download is not likely to be something most desktop 
clients would be interested in, because they would still need to 
support old servers and they already have a composition and 
submission model.

However, anything that speeds up resynchronizing large mailboxes 
might be beneficial enough that desktop clients implement the new 
extensions.  Compression is another extension that would be helpful, 
especially when users have dial-up.
-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly-selected tag: ---------------
The budget should be balanced.  The treasury should be refilled.
Public debt should be reduced.  The arrogance of public officials
should be controlled.        --Marcus Tullius Cicero (106-43 B.C.)

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Wed, 14 Jun 2006 09:38:48 +0200

I'll answer later today. But first:

> they are inadvisable to mandate in Lemonade.

I don't have a strong opinion on whether they're advisable to mandate in 
lemonade.

Arnt

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Wed, 14 Jun 2006 17:14:05 +0200

> that was essentially optimizing what clients can do already.

Here's how I read it, and what I've thought of:

In the server's RAM, CONTEXTS and VFOLDER seem to be the same. In both 
cases, the server keeps track of modsequences and updates UID sets in 
RAM. For CONTEXTS, it's one UID set, for VFOLDER, it's two UID sets and 
a mailbox name.

In VFOLDER, all that information is stored persistently on the server. 
In CONTEXTS, parts of it is stored on the client, and parts may not be 
stored at all, but instead recomputed.

If the client uses CONDSTORE and stores the result sets persistently, 
performance at session start time is essentially the same for the two. 
If the client doesn't use CONDSTORE, or if it issues SEARCHes for all 
contexts when it SELECTs the backing mailbox, the server groans under 
the load.

For CONTEXTS, there is a nice fallback to working with stock IMAP 
servers. For VFOLDER, there is none.

For CONTEXTS, two clients don't stay in sync, unless the clients share 
state in some way outside the extension. VFOLDER works with vanilla and 
extended IMAP clients.

VFOLDER access is governed by ACL, CONTEXTS avoids all that. When a 
mailbox is deleted or renamed, VFOLDER specifies a certain behaviour 
(which may or may not be considered good behaviour), CONTEXTS makes it 
depend on whether the rename/delete was issued by the context-owning 
client or some other.

That's it, right? Have I missed anything?

If I trust clients to use good IMAP, have decent caching and want to 
work with plain IMAP servers, I prefer CONTEXTS. If I don't, or I want 
the extension to be valuable to plain IMAP clients, I prefer VFOLDER. 
So what's most desirable within the charter of Lemonade?

Arnt

---

From: Randall Gellens <[email-masked]>
Date: Wed, 14 Jun 2006 17:58:47 -0700

In Section 3.4.  Notifications of changes, there should perhaps be a 
mention of the interaction with reconnect.  Specifically, if the 
client requests notification of changes and then preserves the 
session, it will get those notifications when it resumes the session?

n Appendix A you say:
    In lieu of a SELECT, the client performs a SEARCH RETURN (COUNT
    NOTIFY), which provides a count, and requests updates to the results.
    For whichever area is currently in view in the summary window, it can
    then perform a SEARCH RETURN (PARTIAL from:to), which will return the
    required results.

    There is no SELECT from the client, and no possibility of duplicate
    message downloads, and therefore this method of providing virtual
    mailbox

But in RFC 3501, SEARCH is only valid in SELECTED state, so the 
client would have to first SELECT the target mailbox.
-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly-selected tag: ---------------
Secrecy is as essential to Intelligence as vestments and
incense to a Mass, or darkness to a Spiritualist seance, and
must at all costs be maintained, quite irrespective of whether
or not it serves any purpose.             --Malcolm Muggeridge

---

From: Ray Cromwell <[email-masked]>
Date: Wed, 14 Jun 2006 23:21:35 -0700

> problem exist such that they're likely to be looked at on a mobile 
 The original motivation for VFOLDER was an attempt to solve multiple 
use case issues, such as #1 and #2, and handle legacy clients, among 
other things. It is true that separate mechanisms can be designed for 
this (notifications), but I am concerned about duplicating effort and 
complicating the job that the mobile client has to do to provide the 
illusion that they are not separate (to the user) (i.e. messages grouped 
into a "folder" on the client for reasons of client-side cache and 
network traffic, and messages grouped into a "folder" on the client for 
reasons for notifications)

  I had thought that there was consensus that out-of-band notifications 
were important for several reasons, such as satisfying OMA requirements 
and enhancing battery life on mobile clients. The work done on 
notifications so far has assumed the existence of something like VFOLDER 
in its architecture, because the assumption is that users do want to 
control the set of messages and events which generate out-of-band 
notifications. I disagree that mailboxes with sufficient activity for 
this to be a problem don't exist, in fact, a mailbox that gets only a 
few dozen messages and state changes per day would still be a candidate, 
depending on the cost of out-of-band notifications.

  The use case for vfolders and notifications goes something like this:

1) user wants "push-like" mobile experience for some category of messages
2) user tells server which criteria it wants for these mobile messages 
(e.g. create a vfolder that includes only those messages)
3) user tells server which events in this folder it wants to receive 
notifications for

So VFOLDER fulfills dual purposes in use case #2, it allows reduction of 
client cache and network traffic, but it also provided a named grouping 
which could have attached notification semantics (#3).  We haven't yet 
decided how #3 works (sieve?) but part of the motiviation for making 
vfolders behave like ordinary folders was that presumably, whatever 
notification mechanisms the group decided on would work equally well 
with regular folders and with vfolders, so there wouldn't be a need for 
separate mechanisms for each.

So my concern is, how do out-of-band notifications work with the context 
proposal? And secondly, how does inband notifications in the context 
proposal, and IDLE, and the msgevents draft go together? Are we heading 
along the way of having 3 separate in-band mechanisms for delivering 
state changes?

-Ray
(ray.cromwell at oracle dot com via private SMTP server for spam 
catching reasons)

---

From: Curtis King <[email-masked]>
Date: Thu, 15 Jun 2006 00:37:00 -0600

On 14-Jun-06, at 9:14 AM, Arnt Gulbrandsen wrote:

>> Done - attached is what Curtis and myself thrashed out as  
>> something that was essentially optimizing what clients can do  
>> already.
>
> In the server's RAM, CONTEXTS and VFOLDER seem to be the same. In  
> both cases, the server keeps track of modsequences and updates UID  
> sets in RAM. For CONTEXTS, it's one UID set, for VFOLDER, it's two  
> UID sets and a mailbox name.
>
> In VFOLDER, all that information is stored persistently on the  
> server. In CONTEXTS, parts of it is stored on the client, and parts  
> may not be stored at all, but instead recomputed.
>
> If the client uses CONDSTORE and stores the result sets  
> persistently, performance at session start time is essentially the  
> same for the two. If the client doesn't use CONDSTORE, or if it  
> issues SEARCHes for all contexts when it SELECTs the backing  
> mailbox, the server groans under the load.

I would hate to create VMAILBOX on such a server ;) An index is an  
index the cost will be same, except with CONTEXT there will be 1 less  
for each "view".

The performance issue is listing of all the mailboxes and vmailboxes,  
more so with clients than servers. This is one biggest problems with  
some existing clients today, vmailboxes will make the situation worst.

> For CONTEXTS, two clients don't stay in sync, unless the clients  
> share state in some way outside the extension. VFOLDER works with  
> vanilla and extended IMAP clients.

What? Clients stay in sync using existing IMAP mechanisms.

>
> VFOLDER access is governed by ACL, CONTEXTS avoids all that.

It avoids it by using the ACL of the backing mailbox.

> When a mailbox is deleted or renamed, VFOLDER specifies a certain  
> behaviour (which may or may not be considered good behaviour),  
> CONTEXTS makes it depend on whether the rename/delete was issued by  
> the context-owning client or some other.

Incorrect it should be handled the same way as having any other  
mailbox deleted or renamed currently.

> That's it, right? Have I missed anything?

Yes, VMAILBOX while helping with one scaling problem can create an  
other. The number of items returned by LIST and the number mailboxes  
a client might want to STATUS. VMAILBOX doesn't help with improving  
SORT or THREAD update issues. CONTEXT can be extended to-do so which  
will be a great benefit to all clients.

>
> If I trust clients to use good IMAP, have decent caching and want  
> to work with plain IMAP servers, I prefer CONTEXTS. If I don't, or  
> I want the extension to be valuable to plain IMAP clients, I prefer  
> VFOLDER. So what's most desirable within the charter of Lemonade?

Thinking you can solve poor client implementations by adding a  
feature to servers is overly optimisic. As I pointed out VMAILBOX can  
make some "plain IMAP" clients perform worst. We do need to trust  
developers will produce the best servers and clients they can. A  
great protocol doesn't prevent bad implementations.

I prefer instead to give the client developers more tools to make  
better clients with. One problem when developing clients is the most  
you can expect a server to support is RFC3501 until now. LEMONADE  
sets a new standard which clients can expect from most major servers.  
I think most active desktop clients will start to use the tools set  
the by LEMONADE profile.

ck

---

From: Ray Cromwell <[email-masked]>
Date: Thu, 15 Jun 2006 00:52:42 -0700

> will be a great benefit to all clients.
I don't see this being a problem for the client, unless users create 
dozens and dozens of vmailboxes and tells the client that all of them 
are interesting enough to sync. Yes, in a pathological case, CONTEXT 
would be more efficient since presumably the client would issue a single 
uber-search and parse out the results into individual 'vmailboxes' on 
the client side, rather than issue dozens of searches.  But, what you 
see as a bug for a pathological case (vmailboxes show up in LIST)  I see 
as a feature (clients can share vmailboxes easily)

 (What VMAILBOX currently doesn't do without extensions, and what 
CONTEXT could do with more extensions, I think isn't a valid comparison 
at this point. CONTEXT doesn't make it easy for two or more devices or 
users to share a 'virtual' view into a mailbox. Neither spec handles 
multiple mailbox searches which is clearly a highly desirable user 
feature, and CONTEXT doesn't handle, atleast in an obvious way, a 
crucial part of the Lemonade OMA MEM realization for notifications)

Now, some of those problems could be fixed for CONTEXT, and if so, I'd 
have no problem supporting it, I just want to make sure that whatever 
gets adopted satisifies whats already been agreed upon as far as 
Lemonade and OMA realization.

-Ray

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Thu, 15 Jun 2006 13:26:35 +0200

> index the cost will be same, except with CONTEXT there will be 1 less 

I don't see that. The work required to update the index differs substantially.

Take the case where a user read INBOX by mostly partitioning it into 
fifteen groups: Ten specific to a mailing list (SEARCH HEADER "List-ID" 
"lemonade.ietf.org" etc), one which catches other mailing lists, one 
which catches messages with the $Junk flag set, one which catches 
messages to/from the user's  personal address, one which catches work 
mail (based on From/To domain), and one for the rest. Perfectly 
plausible. Just so I have some numbers, I'll say that the inbox 
contains 35,000 messages, of which 150 have been delivered since last 
time I read mail. (If you think 35,000 is a lot, use gmail for a while, 
it'll change your mind.)

There are four costs to consider, at least that I can think of.

1. The amount of persistent storage required on the server. For VFOLDER, 
that's 15 sets of UID pairs, 15 mailbox names, 15 ACLs, etc. For 
CONTEXTS, nothing. Agreed?

2. Ditto on the client(s). For VFOLDER, nothing needs be stored. For 
CONTEXTS, the client must to store the search expression. If the client 
also stores the matching UID set and its highest modseq, that benefits 
the server greatly (but makes little difference to the client).

3. The amount of CPU/whatever required to start a session (SELECT or 
equivalent). For VFOLDER, the server must consider 150 messages every 
time the client SELECTs a VFOLDER. For CONTEXTS, the client affects 
that. In the lowest-server-load case, the client enables just one 
CONTEXT at a time and uses CONDSTORE optimally, so the server needs to 
consider 150 messages each time the client switches context. In the 
worst case, the client enables all 15 contexts after issuing SELECT 
INBOX and does not use CONDSTORE, and the server has to consider each 
of the 35,000 messages 15 times. I suspect that a good client author 
such as Dave will enable most/all of the 15 and use CONDSTORE, so the 
server has to process each of 150 messages 15 times.

4. The amount of CPU/whatever required when a message is injected into a 
mailbox and a client is looking at that mailbox. For VFOLDER, the 
server needs to consider whether the message matches either zero or one 
search expression(s): Zero if the client has INBOX selected, one if the 
client has one of the VFOLDERs selected. For CONTEXTS, it needs to 
consider as many search expressions as the client has enabled, 0-15.

For three of these, IMO the two are nearly the same, and for one, 
VFOLDER wins by a significant margin. Depending on what's expensive in 
your architecture, your conclusion may be different.

> some existing clients today, vmailboxes will make the situation 

True. I disregarded this because I think LIST * is already a form of 
suicide. One might as well be hanged for a sheep as for a lamb.

> What? Clients stay in sync using existing IMAP mechanisms.

How does this work? It's not clear to me from the draft. Take the case 
where I read mail using my mobile device milonga and my office machine 
libertango. When I create, modify or delete a context on milonga, how 
does libertango learn about change context?

It could be done. The two could use ACAP, IMAP METADATA, WebDAV or some 
other mechanism external to the CONTEXTS draft. IMAP doesn't have any 
such mechanisms now. To create one, you need to specify how to label 
and format the metadata, and also deal with concurrent update issues.

> It avoids it by using the ACL of the backing mailbox.

(That's what I tried to say.)

> a client might want to STATUS.

True.

> VMAILBOX doesn't help with improving  SORT or THREAD update issues. 
> CONTEXT can be extended to-do so which  will be a great benefit to 
> all clients.
>
>> work with plain IMAP servers, I prefer CONTEXTS. If I don't, or  I 
>> want the extension to be valuable to plain IMAP clients, I prefer  
> feature to servers is overly optimisic.

I agree... but we can make poor implementations worse by adding a 
badly-considered feature.

> As I pointed out VMAILBOX can  make some "plain IMAP" clients perform 
> worst. We do need to trust  developers will produce the best servers 
> and clients they can. A  great protocol doesn't prevent bad 
> implementations.

So far, IMAP clients tend not to use SEARCH at all, but instead FETCH 
1:* RFC822 and do the searching on the client. I've seen a mobile 
client written by people I personally know and respect do that. It 
takes some faith to assume that the average mobile client will begin to 
use  SEARCH+CONDSTORE+CONTEXTS optimally.

Instead of trusting implementers generally, I'd prefer to design the 
extension so that if a server implementation sucks, the pain is mostly 
on the server admin's side, and that if a client implementation sucks, 
the pain is mostly on the user's side.

>  I think most active desktop clients will start to use the tools set  

I'll accept that if you can get a Microsoft person to step forward and 
say "Outlook and Outlook Express will be updated to use LEMONADE".

Arnt

(PS. I keep saying VFOLDER. Not because I think the name is good - I 
don't. Only because it's the name in the draft at hand. I would be 
happy if Stephane would update the draft, even if it's just to rename.)

---

From: Dave Cridland <[email-masked]>
Date: Thu, 15 Jun 2006 15:19:42 +0100

On Thu Jun 15 08:52:42 2006, Ray Cromwell wrote:
> CONTEXT could do with more extensions, I think isn't a valid 
> comparison at this point.

Well, it depends on how easily rectifiable they are as ommisions. 
FWIW, CONTEXTS already has the placeholder for handling SORT - to 
paraphrase Jefferson rather ominously, it's just a matter of typing, 
so if you were referring to that, Curtis slipped up, because CONTEXTS 
has been designed specifically with SORT in mind. Whether that's 
split off into a later document or not isn't important, I would 
insist of keeping the syntax support (which is in ADDTO).

>  CONTEXT doesn't make it easy for two or more devices or users to 
> share a 'virtual' view into a mailbox.

No, but other existing mechanisms can be used to share preferences, 
such as ACAP (which Polymer uses for precisely this purpose) or 
ANNOTATEMORE/METADATA.

>  Neither spec handles multiple mailbox searches which is clearly a 
> highly desirable user feature,

Multi-mailbox searching can actually be done now using multiple 
connections. Multi-mailbox SORT cannot.

Multi-mailbox searching could be made better with a very small 
extension dealing with unsolicited STATUS responses, I think, and the 
use of the client-side-reconnect, which I'm working on today.

>  and CONTEXT doesn't handle, atleast in an obvious way, a crucial 
> part of the Lemonade OMA MEM realization for notifications)
> 
> 
No, but I'm not convinced it needs to. Your approach is to create a 
mailbox from some live search results, then monitor it for 
notifications, which means that you have a situation where VMAILBOX 
is being used for (at least) two different and orthogonal problems.

Nothing prevents you from monitoring the live search results 
directly, whether or not they're actually used as a view or not. (If 
they are, of course, by all means use the same cached set of results 
for both purposes).

Dave.

---

From: Dave Cridland <[email-masked]>
Date: Thu, 15 Jun 2006 15:20:46 +0100

On Thu Jun 15 12:26:35 2006, Arnt Gulbrandsen wrote:
>>>  both cases, the server keeps track of modsequences and updates 
>>> UID  sets in RAM. For CONTEXTS, it's one UID set, for VFOLDER, 
>>> it's two  UID sets and a mailbox name.
>>> 
>>> server. In CONTEXTS, parts of it is stored on the client, and 
>>> parts  may not be stored at all, but instead recomputed.
>>> 
>>> persistently, performance at session start time is essentially 
>>> the  same for the two. If the client doesn't use CONDSTORE, or if 
>>> it  issues SEARCHes for all contexts when it SELECTs the backing  
>>  index the cost will be same, except with CONTEXT there will be 1 
>> less  for each "view".
> 
Yes, since there's no UID allocation involved - maintaining a 
CONTEXTS index is marginally cheaper for a server.

> Take the case where a user read INBOX by mostly partitioning it 
> into fifteen groups: Ten specific to a mailing list (SEARCH HEADER 
> "List-ID" "lemonade.ietf.org" etc), one which catches other mailing 
> lists, one which catches messages with the $Junk flag set, one 
> which catches messages to/from the user's  personal address, one 
> which catches work mail (based on From/To domain), and one for the 
> rest. Perfectly plausible. Just so I have some numbers, I'll say 
> that the inbox contains 35,000 messages, of which 150 have been 
> delivered since last time I read mail. (If you think 35,000 is a 
> lot, use gmail for a while, it'll change your mind.)
> 
> 
I think 35,000 is "moderate", and I don't use gmail. :-)

> 1. The amount of persistent storage required on the server. For 
> VFOLDER, that's 15 sets of UID pairs, 15 mailbox names, 15 ACLs, 
> etc. For CONTEXTS, nothing. Agreed?
> 
> 
No.

CONTEXTS does not mandate any persistent storage, but equally does 
not mandate against it. Typically, I'd expect the server to have 
built indices, so for sake of argument you're looking at - perhaps - 
15 sets of UIDs. The wire-level efficiency is the same whatever is 
stored on the server.

This contrasts with VFOLDER, since unless you effectively make it 
into a real mailbox with shared content, and persist UIDs, the wire 
efficiency of the protocol will suffer greatly.

> 2. Ditto on the client(s). For VFOLDER, nothing needs be stored. 
> For CONTEXTS, the client must to store the search expression. If 
> the client also stores the matching UID set and its highest modseq, 
> that benefits the server greatly (but makes little difference to 
> the client).
> 
> 
For VFOLDER on a client, a substantial amount of data must be stored 
for effective use, but nothing needs to be.

For CONTEXTS on the client, the search criteria need to be available.

The client can persist result sets, of course, and this not only 
makes a difference to the server, but to the client, too, because it 
affects wire efficiency.

> 3. The amount of CPU/whatever required to start a session (SELECT 
> or equivalent). For VFOLDER, the server must consider 150 messages 
> every time the client SELECTs a VFOLDER. For CONTEXTS, the client 
> affects that. In the lowest-server-load case, the client enables 
> just one CONTEXT at a time and uses CONDSTORE optimally, so the 
> server needs to consider 150 messages each time the client switches 
> context. In the worst case, the client enables all 15 contexts 
> after issuing SELECT INBOX and does not use CONDSTORE, and the 
> server has to consider each of the 35,000 messages 15 times. I 
> suspect that a good client author such as Dave will enable most/all 
> of the 15 and use CONDSTORE, so the server has to process each of 
> 150 messages 15 times.
> 
> 
To begin with, you're still considering all 35,000 messages - it's 
simply that you're using a MODSEQ index to first-cut the search range.

I would probably not resynch all contexts, there's little point - 
Polymer does use the same view definitions for colourizing the 
display, but it can handle most criteria used for that locally, so 
there's little advantage.

It'd most likely resynch - or refetch - the context results when that 
view was selected.

But... There's nothing at all preventing the server from persistently 
caching the results from contexts, which turns the speed of context 
reaquisition to much the same as VMAILBOX resynch, but with less 
server-side state involved.

The fun thing about CONTEXTS is that the server does not have to, and 
it makes no difference - at all - to the client.

> 4. The amount of CPU/whatever required when a message is injected 
> into a mailbox and a client is looking at that mailbox. For 
> VFOLDER, the server needs to consider whether the message matches 
> either zero or one search expression(s): Zero if the client has 
> INBOX selected, one if the client has one of the VFOLDERs selected. 
> For CONTEXTS, it needs to consider as many search expressions as 
> the client has enabled, 0-15.
> 
> 
Ah, wait... A client can equally well SELECT all the real virtual 
mailboxes in the system, and doing this will mean multiple 
connections, etc.

Acquisition of every context is less work than this. What you're 
basically saying is that it's practical for a client to acquire every 
context defined for the currently selected mailbox at low cost. This 
to my mind is an advantage.

>> The performance issue is listing of all the mailboxes and 
>> vmailboxes,  more so with clients than servers. This is one 
>> biggest problems with  some existing clients today, vmailboxes 
>> will make the situation worst.
> 
> True. I disregarded this because I think LIST * is already a form 
> of suicide. One might as well be hanged for a sheep as for a lamb.
> 
> 
I've already had complaints from users noting that "LISTing is slow", 
due to large numbers of mailbox on the same heirarchy level.

> How does this work? It's not clear to me from the draft. Take the 
> case where I read mail using my mobile device milonga and my office 
> machine libertango. When I create, modify or delete a context on 
> milonga, how does libertango learn about change context?
> 
> 
It's not covered by the draft at all.

> It could be done. The two could use ACAP, IMAP METADATA, WebDAV or 
> some other mechanism external to the CONTEXTS draft. IMAP doesn't 
> have any such mechanisms now. To create one, you need to specify 
> how to label and format the metadata, and also deal with concurrent 
> update issues.
> 
> 
Polymer uses ACAP right now. *shrugs* METADATA would work too. Either 
way, it's orthogonal to the protocol.

Dave.

---

From: Dave Cridland <[email-masked]>
Date: Thu, 15 Jun 2006 15:20:26 +0100

On Thu Jun 15 01:58:47 2006, Randall Gellens wrote:
> In Section 3.4.  Notifications of changes, there should perhaps be 
> a mention of the interaction with reconnect.  Specifically, if the 
It would be nice if it did, wouldn't it?

In practise, I suspect the server implementors would scream about a 
vast increase in server-side state. Server implementors have already 
expressed concern about maintaining a note of when a message was 
expunged, expecting them to recall when a message was removed from a 
context would be asking a bit much.

However, I believe it is possible to resynchronize quite efficiently 
where CONDSTORE is available.

I need to double check this, and it may only work in practise, rather 
than in theory.

I'll most likely update the draft with an additional appendix 
describing how it can be done.

> client would have to first SELECT the target mailbox.

Yes, quite, it's very badly worded.

You have to select what would be the backing mailbox, before you 
replace the select of the virtual mailbox with the SEARCH RETURN 
(ALL).

Obviously you can't omit the select of the backing mailbox, nor can 
you really select the virtual mailbox, because, in this design, the 
virtual mailbox isn't a "real virtual mailbox", but a "virtual 
virtual mailbox".

(To move from the sublime to the ridiculous, server implementors 
under this design have the choice of storing physical data about the 
virtual virtual mailbox or not, which probably further subdivides 
implementation options into virtual virtual virtual mailboxes and 
real virtual virtual mailboxes).

Dave.

---

From: Mark Crispin <[email-masked]>
Date: Thu, 15 Jun 2006 08:01:36 -0700 (PDT)

On Thu, 15 Jun 2006, Curtis King wrote:
> Thinking you can solve poor client implementations by adding a feature to 
> servers is overly optimisic.

Indeed.  That misguided thinking has caused far more problems than it has 
solved.  It also leads to violations of RFC 3501, with the justification 
that the violation makes some broken client work better (at the cost of 
breaking some RFC 3501 compliant client).

> One problem when developing clients is the most you can expect 
> a server to support is RFC3501 until now. LEMONADE sets a new standard which 
> clients can expect from most major servers. I think most active desktop 
> clients will start to use the tools set the by LEMONADE profile.

I, on the other hand, hope that this does not happen.

-- Mark --

http://panda.com/mrc
Democracy is two wolves and a sheep deciding what to eat for lunch.
Liberty is a well-armed sheep contesting the vote.

---

From: Cyrus Daboo <[email-masked]>
Date: Thu, 15 Jun 2006 11:05:00 -0400

Hi Dave,

--On June 15, 2006 3:19:42 PM +0100 Dave Cridland <[email-masked]> wrote:

> Well, it depends on how easily rectifiable they are as ommisions. FWIW,
> CONTEXTS already has the placeholder for handling SORT - to paraphrase
> Jefferson rather ominously, it's just a matter of typing, so if you were
> referring to that, Curtis slipped up, because CONTEXTS has been designed
> specifically with SORT in mind. Whether that's split off into a later
> document or not isn't important, I would insist of keeping the syntax
> support (which is in ADDTO).

As was pointed out many times during the VIEW/WINDOW debate, THREAD is just 
as important (if not more so) than SORT, so CONTEXT really ought to provide 
a way to update the THREAD tree. Unfortunately that is not trivial as there 
can be non trivial changes to the tree when messages are inserted or 
deleted. The simple case of an insert or delete in a 'branch' would be easy 
to handle, but merges/fragmentation of branches is harder - in the later 
case the simplest thing to do is probably just resend the entire THREAD 
response rather than attempt a diff.

In any case I think you need to address THREAD is some fashion, as I 
believe those clients that are likely to use CONTEXT ("real" IMAP clients 
that like to take advantage of server-side processing) will also likely be 
using THREAD.

-- 
Cyrus Daboo

---

From: Dave Cridland <[email-masked]>
Date: Thu, 15 Jun 2006 16:18:37 +0100

On Thu Jun 15 16:05:00 2006, Cyrus Daboo wrote:
>> FWIW,
> As was pointed out many times during the VIEW/WINDOW debate, THREAD 
> is just as important (if not more so) than SORT, so CONTEXT really 
> ought to provide a way to update the THREAD tree. Unfortunately 
> that is not trivial as there can be non trivial changes to the tree 
> when messages are inserted or deleted. The simple case of an insert 
> or delete in a 'branch' would be easy to handle, but 
> merges/fragmentation of branches is harder - in the later case the 
> simplest thing to do is probably just resend the entire THREAD 
> In any case I think you need to address THREAD is some fashion, as 
> I believe those clients that are likely to use CONTEXT ("real" IMAP 
> clients that like to take advantage of server-side processing) will 
> also likely be using THREAD.

I've "handled" THREAD by quite deliberately leaving expansion space 
in the CAPABILITY string, but that's as far as I'm really prepared to 
go in this iteration. SORT is there at all really because it's quite 
low-hanging fruit. Whether it's low enough, I don't know, but 
syntactically, it was terribly easy to put in.

For THREAD, you not only have to deal with updates, you have to deal 
with partial trees, as well. I'd personally want to sit back and see 
what use-cases other than the obvious are made from CONTEXTS, first 
off.

Dave.

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Thu, 15 Jun 2006 17:50:23 +0200

Dave Cridland writes:
> CONTEXTS does not mandate any persistent storage, but equally does not 
> mandate against it. Typically, I'd expect the server to have built 
> indices, so for sake of argument you're looking at - perhaps - 15 
> sets of UIDs. The wire-level efficiency is the same whatever is 
> stored on the server.

Yes. Sorry, I wasn't thorough enough. I only considered keying on 
search-correlator, and didn't realise until later that one can key on 
the verbatim search expression.

But if you store on the server, you store the same information, do the 
same work, and all you get from CONTEXTS is incompatibility with 
Outlook.

> This contrasts with VFOLDER, since unless you effectively make it into 
> a real mailbox with shared content, and persist UIDs, the wire 

Explain? I don't see that. The only inefficiency I know is that if you 
read both the vfolder and the backing mailbox, you lose, and in my 
experience, people hate doing that in the first place. (That's why I 
suggested INVFOLDER as a new search-key a few weeks ago.)

>> CONTEXTS, the client must to store the search expression. If the 
>> client also stores the matching UID set and its highest modseq, that 
>> benefits the server greatly (but makes little difference to the 
>> client).
>

How? The client does some work to CREATE the thing, and after that, no 
client does anything particular.

> simply that you're using a MODSEQ index to first-cut the search 

The modseq search doesn't consider all 35,000. Depending on data 
structure, it's a loop downwards, a tree lookup, etc. In my case, the 
code would look at maybe 160 nodes.

> server-side state involved.

I looked at the code so see how I'd implement that. I would use the same 
classes that I already have, with pretty much the same member 
variables. Two variables would be replaced by two other variables.

>> into a mailbox and a client is looking at that mailbox. For VFOLDER, 
>> the server needs to consider whether the message matches either zero 
>> or one search expression(s): Zero if the client has INBOX selected, 
>> one if the client has one of the VFOLDERs selected. For CONTEXTS, it 
>> needs to consider as many search expressions as the client has 
>> enabled, 0-15.
>

Yes. But none actually do ;) I've seen much broken use of IMAP, but I've 
never seen anyhing as broken as

    for all x
       send "SELECT x"

Eventually, clients select most/all values of x, with some intervening 
FETCH commands etc. That's different, because there's no huge load 
peak.

> to my mind is an advantage.

That's exactly what I meant to say. Now, whether it's an advantage... I 
can see the attraction, but I notice that despite repeated proposals, 
noone has done a MULTISELECT yet. So maybe the advantage isn't as large 
as one would think.

(Is it more desirable to select multiple search results than multiple 
mailboxes? I can't see any reason for that, but I've never tried to 
find any.)

>> have any such mechanisms now. To create one, you need to specify how 
>> to label and format the metadata, and also deal with concurrent 
>> update issues.
>

I sort of disagree strongly. Sort of. IMO it's extremely important to 
share this data among clients.

On the other hand, a user with one mobile device and one stationary 
device (most commonly Outlook/OE) is important to me. Since CONTEXTS 
can't share in this case, there may not be much point in sharing.

Arnt

---

From: Curtis King <[email-masked]>
Date: Thu, 15 Jun 2006 10:02:38 -0600

On 15-Jun-06, at 9:50 AM, Arnt Gulbrandsen wrote:

> Yes. But none actually do ;) I've seen much broken use of IMAP, but  
> I've never seen anyhing as broken as
>
>    for all x
>       send "SELECT x"

I sure have seen clients do:

for all x
    send "STATUS x  (..)"

ck

---

From: Mark Crispin <[email-masked]>
Date: Thu, 15 Jun 2006 09:12:37 -0700 (PDT)

> I sure have seen clients do:

So have I.

The reason that STATUS does not take a wildcard is to make it obvious to 
anyone who looks at the protocol transcript that the client is badly 
broken, and that the server is not to blame for the session being so slow. 
But that doesn't stop certain individuals from writing such programs.

-- Mark --

http://panda.com/mrc

---

From: Curtis King <[email-masked]>
Date: Thu, 15 Jun 2006 12:10:42 -0600

On 15-Jun-06, at 5:26 AM, Arnt Gulbrandsen wrote:

>> I would hate to create VMAILBOX on such a server ;) An index is  
>> an  index the cost will be same, except with CONTEXT there will be  
>> 1 less  for each "view".
>

Why not just create an index for each search for either VMAILBOX or  
CONTEXT and test each message when it's injected into the backing  
mailbox. VMAILBOX requires the server to have an index storing the  
fake VMAILBOX uids, CONTEXT does not. So, for servers which already  
have really good search performance VMAILBOX has higher requirements  
than CONTEXT.

> of suicide. One might as well be hanged for a sheep as for a lamb.

I haven't seen clients do LIST * for long time. Clients trying to  
discover the whole mailbox hierarchy at every startup using LIST ""  
%, LIST "" foo/% which have the performance problems. In some cases  
issuing LIST * would be better than their existing behavior ;)

>
>>> For CONTEXTS, there is a nice fallback to working with stock  
>>> IMAP  servers. For VFOLDER, there is none.
>>>
>>> share state in some way outside the extension. VFOLDER works  
>>> with  vanilla and extended IMAP clients.
>>

Why do I need to share the context? Isn't the client interested in  
the changes to message? Most likely I won't even want the same  
VMAILBOX on all my clients. I would argue it's better not to force  
clients to have the same view, but instead allow them to create the  
views which best suit their usage model.

>
>> VMAILBOX doesn't help with improving  SORT or THREAD update  
>> issues. CONTEXT can be extended to-do so which  will be a great  
>> benefit to all clients.
>>
>>> If I trust clients to use good IMAP, have decent caching and  
>>> want  to work with plain IMAP servers, I prefer CONTEXTS. If I  
>>> don't, or  I want the extension to be valuable to plain IMAP  
>>> clients, I prefer  VFOLDER. So what's most desirable within the  
>>> charter of Lemonade?
>>

Which I think VMAILBOX is. I have tried to implement it and there are  
too many holes in it.

>
>> As I pointed out VMAILBOX can  make some "plain IMAP" clients  
>> perform worst. We do need to trust  developers will produce the  
>> best servers and clients they can. A  great protocol doesn't  
>> prevent bad implementations.
>
> So far, IMAP clients tend not to use SEARCH at all, but instead  
> FETCH 1:* RFC822 and do the searching on the client. I've seen a  
> mobile client written by people I personally know and respect do  
> that. It takes some faith to assume that the average mobile client  
> will begin to use  SEARCH+CONDSTORE+CONTEXTS optimally.
>
> Instead of trusting implementers generally, I'd prefer to design  
> the extension so that if a server implementation sucks, the pain is  
> mostly on the server admin's side, and that if a client  
> implementation sucks, the pain is mostly on the user's side.

Honestly I don't think thats achievable and history has shown  
opposite happens when you try to-do this.

>> better clients with. One problem when developing clients is the  
>> most  you can expect a server to support is RFC3501 until now.  
>> LEMONADE  sets a new standard which clients can expect from most  
>> major servers.  I think most active desktop clients will start to  
>> use the tools set  the by LEMONADE profile.
>
> I'll accept that if you can get a Microsoft person to step forward  
> and say "Outlook and Outlook Express will be updated to use LEMONADE".

Outlook is one of those clients which has problems when there are a  
large number of mailboxes. It's a hugh mistake to think VMAILBOX will  
make existing clients more usable. Just consider that existing  
clients will still select INBOX no matter what virtual mailboxes are  
defined.

>
> Arnt
>
> (PS. I keep saying VFOLDER. Not because I think the name is good -  
> I don't. Only because it's the name in the draft at hand. I would  
> be happy if Stephane would update the draft, even if it's just to  
> rename.)

I don't think that will happen unless we refer to it as VMAILBOX.

ck

---

From: Ray Cromwell <[email-masked]>
Date: Thu, 15 Jun 2006 11:48:40 -0700

> have really good search performance VMAILBOX has higher requirements  

I think this entire thread is predicated on the assumption that VMAILBOX 
mandates that servers must allow search criteria on dynamic attributes 
or criteria that create the need to assign new UIDs. The original 
VFOLDER proposal did not have this feature and we have not decided 
whether or not this feature is necessary for VFOLDERs or if it will be 
included in the final spec. It was made optional in the latest VFOLDER 
draft (adopted from Arnt's VIEW proposal) precisely so that a) we can 
have a discussion about the costs of implementing and b) some people 
could choose not to implement it.

-Ray

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Thu, 15 Jun 2006 21:28:08 +0200

Curtis King writes:
> fake VMAILBOX uids, CONTEXT does not.

That seems to be a big point for you. I really don't see why. Can you explain?

To me, CONTEXT and VFOLDER seem essentially equivalent in this respect. 
VFOLDER needs to store two uints per match or else invent a new 
UIDVALIDITY and suffer recomputing, CONTEXT needs to store one uint per 
match or else suffer recomputing. What's the big deal?

> So, for servers which already  have really good search performance 
> VMAILBOX has higher requirements  than CONTEXT.

I doubt it. (My code goes to some effort to search fast, and it's all 
ineffective for a single message. The optimizations are more effective 
the bigger the mailbox is.)

The performance issue of storing 1-2 uints is, well, not a factor for me.

> issuing LIST * would be better than their existing behavior ;)

OK, OK, I should have said "LIST * or the equivalent series of LIST 
...%... commands".

> the changes to message?

Every use case I've seen so far for vfolder was of a kind that's 
naturally shared, in the sense that if you have a certain vfolder on 
your desktop, you want the same on your laptop.

Consider my message earlier today, where I mentioned 15 views. That 
contained five or six substantially different ones, all of which are 
naturally shared.

Even the case where you don't want to read the mail is naturally shared. 
Suppose you don't want to read work mail when you're at home. If you 
don't have the "work mail" view on your home computer, the work mail 
shows up in "other nonspam".

> Most likely I won't even want the same  VMAILBOX on all my clients. I 
> would argue it's better not to force  clients to have the same view, 
> but instead allow them to create the  views which best suit their 
> usage model.

(I like the term view, perhaps because it's what DB people use for this. 
Can we use it in this discussion?)

Even if users.cking.workmail exists and is available on your office 
desktop, your computer at home, your laptop and your 770, you don't 
have to read it everywhere.

> Which I think VMAILBOX is. I have tried to implement it and there are  

Tell more? I agree for the previous version of the draft. The current 
version includes lots of text from draft-gulbrandsen-imap-view, which I 
have implemented and I only know about one hole (regarding listext 
syntax). Of course, since I wrote that text I do suffer from a certain 
myopia.

>  make existing clients more usable.

I'm not arguing that. I'm arguing that, unlike CONTEXT, it will not make 
them LESS usable by opening an unneeded gap between the mail you see on 
your mobile and that which you see on your desktop.

Dave has convinced me that CONTEXT, well implemented, can have 
performance characteristics as good as VFOLDER. Not 100%, but close 
enough that I don't care. However, if it's implemented that well, the 
whole difference I see is that CONTEXT is incompatible with Outlook et 
al., and that CONTEXT offers the ability to select multiple views at a 
time.

There are SORT/THREAD aspects. I think those are a little bit too airy 
at the moment, and I feel jeery about saying that views is how one does 
that.

> I don't think that will happen unless we refer to it as VMAILBOX.

Stephane's not stubborn or unwilling (or if he is I haven't noticed). 
He'll adapt to the consensus. It just takes a little while until he 
finds time to edit and submit the draft.

Arnt

---

From: Mark Crispin <[email-masked]>
Date: Thu, 15 Jun 2006 12:32:30 -0700 (PDT)

> OK, OK, I should have said "LIST * or the equivalent series of LIST

So, are you saying that there are clients that walk the entire tree using 
% just to avoid doing a *?  Unbelievable.

The whole point of "LIST * is harmful" is that walking the entire tree is 
harmful.

I can see a client doing "LIST %", followed by "LIST %/%" if the server 
does not advertise CHILDREN and needs to know that.  But any further tree 
prowling should be deferred until the user actually looks in a name that 
has inferiors.

-- Mark --

http://panda.com/mrc

---

From: Curtis King <[email-masked]>
Date: Thu, 15 Jun 2006 13:46:25 -0600

On 15-Jun-06, at 1:52 AM, Ray Cromwell wrote:

> Curtis King wrote:
>
>> Yes, VMAILBOX while helping with one scaling problem can create  
>> an  other. The number of items returned by LIST and the number  
>> mailboxes  a client might want to STATUS. VMAILBOX doesn't help  
>> with improving  SORT or THREAD update issues. CONTEXT can be  
>> extended to-do so which  will be a great benefit to all clients.
>>
> I don't see this being a problem for the client, unless users  
> create dozens and dozens of vmailboxes and tells the client that  
> all of them are interesting enough to sync.

There will be clients which do this and it's the one on your desktop.

> Yes, in a pathological case, CONTEXT would be more efficient since  
> presumably the client would issue a single uber-search and parse  
> out the results into individual 'vmailboxes' on the client side,  
> rather than issue dozens of searches.  But, what you see as a bug  
> for a pathological case (vmailboxes show up in LIST)  I see as a  
> feature (clients can share vmailboxes easily)

Once again, what's the big deal with sharing vmailboxes?

The point I'm trying to make is we need to drop the idea that we can  
improve existing clients with VMAILBOX. Let's drop backward  
compatibility, so we can get the correct solution in a timely manner.

> comparison at this point. CONTEXT doesn't make it easy for two or  
> more devices or users to share a 'virtual' view into a mailbox.  
> highly desirable user feature, and

I would suggest we call it out of scope because the time constraints.  
Let's get the core right.

> CONTEXT doesn't handle, atleast in an obvious way, a crucial part  
> of the Lemonade OMA MEM realization for notifications)

True, but VMAILBOX has a high price to get filtered notifications.

Here are a few of the problems I have run into so far and would like  
authors to explain what should happen.

"For all intents and purposes, the virtual folder looks and behaves  
like a
real IMAP4 folder, except that it does not contain any messages  
itself. "

1. If you delete a vmailbox, should it not remove all messages just  
like deleting a real mailbox? If not, this needs to be in the draft.

2. Expunging messages and the interaction between clients. The mobile  
client uses vmailbox for trash based on a message being marked  
\deleted. Now when I access the backing mailbox using a desktop  
client which does a SELECT and a CLOSE. All \deleted messages with be  
expunged and now on my mobile client it will look like my trash was  
magically emptied. Have fun figuring out that bug report ;)

3. " The EXPUNGE Command (RFC 3501 section 6.4.3) causes the underlying
    mailbox to be expunged when a view is expunged. Servers SHOULD
    expunge only the messages visible in the view, or MAY expunge the
    entire mailbox. The former is more desirable, if possible. "

I have great mobile client which has this great option to show only  
unread messages which it does by creating a vmailbox. Now, I go back  
to my desktop and find a new mailbox which I don't remember creating  
and it has a bunch duplicate messages from my INBOX. I will just  
delete this junk mailbox using my desktop client, which does moves  
all messages to "Deleted Messages", then marks them \deleted and  
expunges. Hey where did all my new messages go? With the MAY expunge  
entire mailbox clause, there might even be more messages expunged  
before the client expects it.

Not to mention all the UID games you have to play.

I will be happy if we just drop the MAILBOX part and just have  
FILTERS for views and notifications.

CONTENT can cover the view part. Yes? If no then what is missing or  
wrong.

A new draft which applies a FILTER to IDLE. If a server always  
returns UID in the response then it should be easy for the client to  
track just the desired changes.

C: A1 IDLE (FROM "isode.com")
S: + idling
S: * 852 EXISTS
S: 1 RECENT
S: * 924 FETCH (FLAGS (\RECENT) UID 36784)
S: * 916 FETCH (FLAGS (\answered \deleted \seen NotJunk $NotJunk) UID  
36776)
C: done
S: A1 OK Idle done
C: UID FETCH 924 ...

Saving SEARCH criteria can be done using METADATA like external  
notifications. Then CONTENT and IDLE can use those named searches if  
so desired. I think most criteria will be hard coded in clients.

VMAILBOX is not needed for outband notifications as SIEVE can be used  
to filter these notifications.

ck

---

From: Curtis King <[email-masked]>
Date: Thu, 15 Jun 2006 13:53:03 -0600

On 15-Jun-06, at 12:48 PM, Ray Cromwell wrote:

> Curtis King wrote:
>
>>
>>
>> Why not just create an index for each search for either VMAILBOX  
>> or  CONTEXT and test each message when it's injected into the  
>> backing  mailbox. VMAILBOX requires the server to have an index  
>> storing the  fake VMAILBOX uids, CONTEXT does not. So, for servers  
>> which already  have really good search performance VMAILBOX has  
>> higher requirements  than CONTEXT.
>
>
> I think this entire thread is predicated on the assumption that  
> VMAILBOX mandates that servers must allow search criteria on  
> dynamic attributes or criteria that create the need to assign new  
> UIDs. The original VFOLDER proposal did not have this feature and  
> we have not decided whether or not this feature is necessary for  
> VFOLDERs or if it will be included in the final spec. It was made  
> optional in the latest VFOLDER draft (adopted from Arnt's VIEW  
> proposal) precisely so that a) we can have a discussion about the  
> costs of implementing and b) some people could choose not to  
> implement it.

Not having dynamic attributes makes VMAILBOX very weak especially for  
mobile clients, yes/no? On my mobile client I only want to see unread  
messages 90% of the time.

We are arguing some fine points, but very important ones.

ck

---

From: Dave Cridland <[email-masked]>
Date: Thu, 15 Jun 2006 21:01:59 +0100

On Thu Jun 15 19:48:40 2006, Ray Cromwell wrote:
> UIDs.

I should point out that currently, I have 10 views defined (aside 
from test ones). Of those, 3 use only static attribute searches.

The other problem is that only takes into account the predefined 
views I do - I actually do a considerable amount of ad-hoc filtering, 
which really doesn't fit the VFOLDER model.

A lot of this thread is about VFOLDER handling of dynamic criteria, 
true, but I don't think that a model that doesn't support that is 
actually viable. For what it's worth, virtual mailboxes do appeal to 
me, but I can't see them being truly useful unless they can handle 
multiple source mailboxes with dynamic criteria effectively.

For single-mailbox cases, they're arguably useful for legacy client 
support, but that is a very different use-case to anything we're 
interested in for Lemonade settings.

Unless you're counting Outlook as being a constrained platform, which 
in the sense of IMAP might arguably be the case. ;-)

As I said when I sent the proto-draft to the list, my personal 
feeling about VFOLDER et al is that it's a good thing, but I'm 
hesitant to mandate it in Lemonade. (I will, of course, if consensus 
backs it). All of Lemonade's use-cases for virtual mailboxes can be 
met with CONTEXTS, and whilst it certainly increases client 
complexity by a small amount, it's not huge.

Moreover, CONTEXTS can provide more than simply virtual mailboxing, 
it can be used to implement grouped views, summary colouring, and 
probably several things I haven't thought of.

Dave.

---

From: Dave Cridland <[email-masked]>
Date: Thu, 15 Jun 2006 21:02:52 +0100

On Thu Jun 15 16:50:23 2006, Arnt Gulbrandsen wrote:
>> basically saying is that it's practical for a client to acquire 
>> every context defined for the currently selected mailbox at low 
>> cost. This to my mind is an advantage.
> 
> That's exactly what I meant to say. Now, whether it's an 
> advantage... I can see the attraction, but I notice that despite 
> repeated proposals, noone has done a MULTISELECT yet. So maybe the 
> advantage isn't as large as one would think.
> 
> 
Or maybe multiselect is just too much of a can of worms. :-)

> (Is it more desirable to select multiple search results than 
> multiple mailboxes? I can't see any reason for that, but I've never 
> tried to find any.)
> 
> 
Yes. Group-messages-by is quite a popular Outlook and Thunderbird 
feature, and involves maintaining multiple continuously presented 
filtered views of the same mailbox.

I can see that, or something similar, being implemented using a 
couple of contexts in operation.

>>> It could be done. The two could use ACAP, IMAP METADATA, WebDAV 
>>> or some other mechanism external to the CONTEXTS draft. IMAP 
>>> doesn't have any such mechanisms now. To create one, you need to 
>>> specify how to label and format the metadata, and also deal with 
>>> concurrent update issues.
>> 
>> Polymer uses ACAP right now. *shrugs* METADATA would work too. 
>> Either way, it's orthogonal to the protocol.
> 
> I sort of disagree strongly. Sort of. IMO it's extremely important 
> to share this data among clients.
> 
> 
Yes, and I have a solution that I'm quite happy with. I'm quite happy 
to write a draft describing it, too, and I would be delighted if 
everyone stored their view criteria in ACAP. I wouldn't be so pleased 
if people moved this into METADATA, as that's much more limiting.

> device (most commonly Outlook/OE) is important to me. Since 
> CONTEXTS can't share in this case, there may not be much point in 
> sharing.

I appreciate that Outlook is a limited IMAP client with a large 
market share, but I'm afraid that much as I try, I have a client 
developer's head on today. For server implementors, you want flashy 
features to be available to even the worst of clients, but for client 
implements, we tend to want the flashier features available 
ubiquitously on servers.

For you, CONTEXTS is equal complexity, and you could probably 
implement both without any problems. I happen to think that CONTEXTS 
has more utility than VIEW, overall, but I can certainly appreciate 
that VIEW has its uses, especially for legacy client support of 
course. I would heartily recommend that you implemented both.

For some servers, however, CONTEXTS is significantly simpler to 
support. There are a range of potential implementation strategies - 
for instance, even a comparitively fast implementation does not need 
to store a full result set at any time. In general, it's much easier 
to get a bad implementation of CONTEXTS working, and it will yield 
the same wire-efficiency advantages to the client.

Dave.

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Thu, 15 Jun 2006 22:06:47 +0200

Curtis King writes:
> mobile clients, yes/no?

Yes.

> On my mobile client I only want to see unread  messages 90% of the time.

The "not spam" flag is another popular one.

Arnt

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Thu, 15 Jun 2006 22:21:16 +0200

Dave Cridland writes:
> I can see that, or something similar, being implemented using a couple 
> of contexts in operation.

Or by the client, which has to fetch the information it groups by 
anyway, in order to display it.

> I appreciate that Outlook is a limited IMAP client with a large market 
> share, but I'm afraid that much as I try, I have a client developer's 
> head on today. For server implementors, you want flashy features to 
> be available to even the worst of clients, but for client implements, 
> we tend to want the flashier features available ubiquitously on 
> servers.

And I have my user experience head on... the very idea of opening an 
unneeded gap between how peoples' clients read mail in the most common 
configuration is abhorrent to me.

> implement both without any problems.

Restating that: If I implement CONTEXTS and keep at it until it doesn't 
suck, I can do VFOLDER at no extra cost and get 100% reader 
compatibility.

Arnt

---

From: Curtis King <[email-masked]>
Date: Thu, 15 Jun 2006 14:19:35 -0600

> That seems to be a big point for you. I really don't see why. Can  

Because UIDs need to-be permanent, even more so with mobile clients  
to avoid download the same message more than once. This means UIDs  
need to be handled with special care. Search indexes, message caches,  
etc all can be lost from time to time with no impact to the client.  
To me this is a big point.

> the bigger the mailbox is.)

That's my point. If you already indexing new messages supporting  
CONTEXT is easy.

> your desktop, you want the same on your laptop.

This works best when you are using the same client. The longer the  
discussion goes on, the more I think we need ACAP light or IMSP. (I'm  
only half joking here)

> shows up in "other nonspam".

Wouldn't the messages still be in my INBOX? Which clients insist on  
selecting no matter what ;) For clients to take advantage of views  
they need to-be updated to least let me set which mailbox to use as  
the INBOX. Which is one of the reason we should forget about trying  
to support non-lemonade clients.

> Can we use it in this discussion?)

yes, there are two parts to all of this views and notifications.

> syntax). Of course, since I wrote that text I do suffer from a certain

see the message I sent to Ray on how clients interact with messages  
being expunged.

ck

---

From: Dave Cridland <[email-masked]>
Date: Thu, 15 Jun 2006 22:29:45 +0100

On Thu Jun 15 21:21:16 2006, Arnt Gulbrandsen wrote:
>>> multiple mailboxes? I can't see any reason for that, but I've 
>>> never tried to find any.)
>> 
No it doesn't. :-)

It only needs to fetch the entire summary data if it needs to display 
it all, and the grouping information may potentially not be within 
that data anyway.

>>> On the other hand, a user with one mobile device and one 
>>> stationary device (most commonly Outlook/OE) is important to me. 
>>> Since CONTEXTS can't share in this case, there may not be much 
>>> point in sharing.
>> 
>> developer's head on today. For server implementors, you want 
>> flashy features to be available to even the worst of clients, but 
>> for client implements, we tend to want the flashier features 
>> available ubiquitously on servers.
> 
> And I have my user experience head on... the very idea of opening 
> an unneeded gap between how peoples' clients read mail in the most 
> common configuration is abhorrent to me.
> 
> 
I will happily encourage you into doing VFOLDER, I think it's an 
excellent tool for providing legacy support, if carefully managed.

> Restating that: If I implement CONTEXTS and keep at it until it 
> doesn't suck, I can do VFOLDER at no extra cost and get 100% reader 
> compatibility.

Probably, in your implementation, but not in others. In others, the 
cost of VFOLDER is higher.

Consider:

CONTEXT itself requires just parsing and ignoring.

NOTIFY does not require the full set of results in memory for 
efficiency, all you need is the ability to detect, on a change in the 
mailbox, whether the criteria match before the change, and whether 
they match after the change. (One method of handling detection of 
whether criteria match before the change is to store the full set of 
results, of course, and this might even be the only practical method, 
depending on your implementation).

PARTIAL does benefit from storage of results, generally from the 
beginning of the mailbox up to and including the highest matching 
message returned by a PARTIAL. In other words, you just iterate 
through the mailbox as required to satisfy PARTIAL, and no further. 
This is still much more efficient than performing the entire search 
each time.

For SORT, things are a bit different, since both NOTIFY and PARTIAL 
require a full set of results, but that's a different state of 
affairs.

Dave.

---

From: Randall Gellens <[email-masked]>
Date: Thu, 15 Jun 2006 17:17:40 -0700

At 11:21 PM -0700 6/14/06, Ray Cromwell wrote:

>  Dave Cridland wrote:
>
>>  This is primarily aimed at item (2). I don't perceive getting 
>> state changes for messages outside the criteria to be a problem, I 
>> simply don't think mailboxes with sufficient activity for this to 
>> be a problem exist such that they're likely to be looked at on a 
>> mobile device.
>>
>  The original motivation for VFOLDER was an attempt to solve 
> multiple use case issues, such as #1 and #2, and handle legacy 
> clients, among other things. It is true that separate mechanisms 
> can be designed for this (notifications), but I am concerned about 
> duplicating effort and complicating the job that the mobile client 
> has to do to provide the illusion that they are not separate (to 
> the user) (i.e. messages grouped into a "folder" on the client for 
> reasons of client-side cache and network traffic, and messages 
> grouped into a "folder" on the client for reasons for notifications)
>
>   I had thought that there was consensus that out-of-band 
> notifications were important for several reasons, such as 
> satisfying OMA requirements and enhancing battery life on mobile 
> clients. The work done on notifications so far has assumed the 
> existence of something like VFOLDER in its architecture, because 
> the assumption is that users do want to control the set of messages 
> and events which generate out-of-band notifications. I disagree 
> that mailboxes with sufficient activity for this to be a problem 
> don't exist, in fact, a mailbox that gets only a few dozen messages 
> and state changes per day would still be a candidate, depending on 
> the cost of out-of-band notifications.

Hi Ray,

Reading Dave's reply to my message, and your reply to Dave, makes me 
think you misunderstood something Dave said.  I had posited two use 
cases for vmailboxes:

At 11:59 AM -0700 6/13/06, Randall Gellens wrote:
>  (1) allowing a client to get notifications only for state changes 
> to a subset of "interesting" messages, and (2) allowing a client to 
> with the server.

In Dave's reply, he suggested that, in the case of (2), that it 
wasn't a problem if clients got notifications of state changes for 
all messages in a mailbox, even those outside the subset of those 
that were "interesting":

At 11:09 PM +0100 6/13/06, Dave Cridland wrote:
>  I don't perceive getting state changes for messages outside the 
> criteria to be a problem, I simply don't think mailboxes with 
> sufficient activity for this to be a problem exist such that 
> they're likely to be looked at on a mobile device.

So Dave was talking about notifications for state changes to messages 
outside the selection criteria, that is, "uninteresting" ones.  From 
your reply, it seemed to me that you may have misread Dave's reply as 
talking about notifications for messages inside the selection 
criteria, that is, "interesting" ones.

As I read his reply, Dave was saying it was OK for clients to simply 
ignore notifications for messages outside the criteria.  Your reply 
seemed to be based on an understanding that Dave was saying clients 
don't need to be notified of messages inside the selection criteria.

Naturally, if I'm the one that misunderstood either your reply or 
Dave's, please correct me.

>  2) user tells server which criteria it wants for these mobile 
> messages (e.g. create a vfolder that includes only those messages)
>  3) user tells server which events in this folder it wants to 
> receive notifications for
>
>  So VFOLDER fulfills dual purposes in use case #2, it allows 
> reduction of client cache and network traffic, but it also provided 
> a named grouping which could have attached notification semantics 
> (#3).  We haven't yet decided how #3 works (sieve?) but part of the 
> motiviation for making vfolders behave like ordinary folders was 
> that presumably, whatever notification mechanisms the group decided 
> on would work equally well with regular folders and with vfolders, 
> so there wouldn't be a need for separate mechanisms for each.
>
>  So my concern is, how do out-of-band notifications work with the 
> context proposal? And secondly, how does inband notifications in 
> the context proposal, and IDLE, and the msgevents draft go 
> together? Are we heading along the way of having 3 separate in-band 
> mechanisms for delivering state changes?

I think Dave's proposal covers in-band notifications, although there 
is still some discussion on how this interacts with reconnect, that 
is, what happens when the client disconnects and preserves the 
session.

-------------- Randomly-selected tag: ---------------
Ignorance is nothing more than indifference to what is before us;
we have only to pay attention--and we are paying attention in a way,
but to pretty noise, the newer the better.
--Wyatt Mason, review of E.E. Cummings biography, Harper's May 2005.

---

From: Randall Gellens <[email-masked]>
Date: Thu, 15 Jun 2006 18:46:00 -0700

> index the cost will be same, except with CONTEXT there will be 1 

One fewer index or one fewer data item per index (the second UID)?

> vmailboxes, more so with clients than servers. This is one biggest 
> problems with some existing clients today, vmailboxes will make the 
> situation worst.

The performance issue is that it is hard for clients to manage large 
numbers of mailboxes?  Because they need to poll each one for state 
changes?

>> CONTEXTS makes it depend on whether the rename/delete was issued 
>> by the context-owning client or some other.
>

Since a CONTEXT, as I understand it, only "sort of" exists, if the 
underlying mailbox is deleted then it can't exist any more.  The 
difference between the client that created the context and others is 
that the context never existed for the others, right?

> feature to servers is overly optimisic. As I pointed out VMAILBOX 
> can make some "plain IMAP" clients perform worst. We do need to 
> trust developers will produce the best servers and clients they 
> can. A great protocol doesn't prevent bad implementations.

One thing we can do is to provide more advice on the right way to use 
the protocol.  There have been some attempts at this before (offhand, 
RFC 2180 and RFC 2683).  It may be helpful to have another, and to 
mention the previous ones in profile-bis.

I agree.

-------------- Randomly-selected tag: ---------------
...All repairs tend to destroy the structure [of the software], to
  increase the entropy and disorder of the system.  Less and less
  effort is spent on fixing original design flaws; more and more is
  spent on fixing flaws introduced by earlier fixes.  As time passes,
  the system becomes less and less well-ordered.  Sooner or later the
  fixing ceases to gain any ground.  Each forward step is matched by a
  backward one.  Although in principle usable forever, the system has
  worn out as a base for progress.  [...] Systems program building is
  an entropy-decreasing process, hence inherently metastable.  Program
  maintenance is an entropy-increasing process, and even its most
  skillful execution only delays the subsidence of the system into
  unfixable obsolescence.
                             --Fred Brooks in _The Mythical Man Month_

---

From: Curtis King <[email-masked]>
Date: Thu, 15 Jun 2006 19:53:47 -0600

On 15-Jun-06, at 1:46 PM, Curtis King wrote:

> returns UID in the response then it should be easy for the client  
> to track just the desired changes.

I missed, we would need IDLE only if the notification part of CONTEXT  
was an issue because it didn't meet some requirement or we didn't  
want two inband notification mechanisms.

ck

---

From: Curtis King <[email-masked]>
Date: Thu, 15 Jun 2006 19:55:36 -0600

On 15-Jun-06, at 1:32 PM, Mark Crispin wrote:

> So, are you saying that there are clients that walk the entire tree  
> using % just to avoid doing a *?  Unbelievable.

It's even worst than that they also issue a status for each new  
mailbox they find.

>
> The whole point of "LIST * is harmful" is that walking the entire  
> tree is harmful.
>
> I can see a client doing "LIST %", followed by "LIST %/%" if the  
> server does not advertise CHILDREN and needs to know that.  But any  
> further tree prowling should be deferred until the user actually  
> looks in a name that has inferiors.

You would think that's the obvious solution.

ck

---

From: Randall Gellens <[email-masked]>
Date: Thu, 15 Jun 2006 18:55:19 -0700

> share a 'virtual' view into a mailbox.

While this sounds attractive, I'd like to better understand how it 
would be used and how important it is.  I think this goes back to the 
use cases for vmailboxes.  I'd previously suggested two use cases, 
but I haven't seen others, and if they are in the draft I've missed 
them.

> highly desirable user feature,

I assume multiple mailbox CONTEXTs/VMAILBOXes are useful because they 
reduce the client load, over the air/wire traffic, and latency.  That 
is, clients could use one CONTEXT or VMAILBOX to have a view into 
"interesting" messages that happen to be in a any of a number of 
mailboxes.  Either way, the client needs to know the actual mailbox, 
I assume, in order to let the user move or delete a message.

> part of the Lemonade OMA MEM realization for notifications)

Meaning out of band notifications?  I though that for both context 
and vmailbox Sieve would be used for this.
-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly-selected tag: ---------------
What is mind?  No matter.
What is matter?  Never mind.
        --Thomas Hewitt Key, 1799-1875

---

From: Curtis King <[email-masked]>
Date: Thu, 15 Jun 2006 21:11:31 -0600

> One fewer index or one fewer data item per index (the second UID)?

It could be either depending on the implementation. The big win is  
for CONTEXT the index can be lost with no penalty to the client,  
where as with VMAILBOX the client would have to download the messages  
again because there would be new UID returned for each message.

>> problems with some existing clients today, vmailboxes will make  
>> the situation worst.
>
> The performance issue is that it is hard for clients to manage  
> large numbers of mailboxes?

Some clients have a great desire to walk the whole mailbox hierarchy  
using LIST "" foo/%, so they can build a nice tree. This takes a long  
time.

>   Because they need to poll each one for state changes?

It's more they want to poll for state changes right at the start when  
combined with the walking of hierarchy it makes for very long connect  
and reconnect times.

>
>
>>>  When a mailbox is deleted or renamed, VFOLDER specifies a  
>>> certain behaviour (which may or may not be considered good  
>>> behaviour), CONTEXTS makes it depend on whether the rename/delete  
>>> was issued by the context-owning client or some other.
>>
> difference between the client that created the context and others  
> is that the context never existed for the others, right?

correct. Having a mailbox deleted when there is an active context  
should be handled the same as when it has a mailbox selected.

> One thing we can do is to provide more advice on the right way to  
> use the protocol.  There have been some attempts at this before  
> (offhand, RFC 2180 and RFC 2683).  It may be helpful to have  
> another, and to mention the previous ones in profile-bis.

I agree.

ck

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Fri, 16 Jun 2006 08:39:40 +0200

Curtis King writes:
> "For all intents and purposes, the virtual folder looks and behaves  like a
> real IMAP4 folder, except that it does not contain any messages  itself. "
>

It should not, which is just like a real mailbox. If you use Sieve rules 
to ensure that mail from your SO ends up in both INBOX.lillian INBOX, 
and then delete INBOX.lillian, the same messages remain in INBOX. 
Similarly, if you create INBOX.lillian vfolder as a view into INBOX 
which maps out messages from Lillian, and then delete INBOX.lillian, 
INBOX remains unaffected. No difference.

> magically emptied. Have fun figuring out that bug report ;)

That's also just like a real mailbox. Another client empties the trash 
(by closing or expunging), and the mobile client sees that the trash 
went away.

> expunges. Hey where did all my new messages go?
>
> With the MAY expunge  entire mailbox clause, there might even be more 
> messages expunged  before the client expects it.

I remember feeling a bit uncertain about whether to allow the 't' right. 
I decided against allowing 'i', but couldn't make up my mind about 't'. 
This makes it simple: 't' must be withheld.

> Not to mention all the UID games you have to play.

In my case, those UID games were zero. I had implemented APPEND, so I 
had the code already.

Arnt

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Fri, 16 Jun 2006 08:43:17 +0200

> Once again, what's the big deal with sharing vmailboxes?

I want to share all of mine. Ask Dave how many of the ten he mentioned 
yesterday he wants to share.

> compatibility, so we can get the correct solution in a timely manner.

I understand that, and my judgment is different: We're not improving 
existing mailboxes if we show them the vfolders, we're worsening them 
if we don't.

Arnt

---

From: Curtis King <[email-masked]>
Date: Fri, 16 Jun 2006 01:07:04 -0600

On 16-Jun-06, at 12:39 AM, Arnt Gulbrandsen wrote:

> Curtis King writes:
>> Here are a few of the problems I have run into so far and would  
>> like  authors to explain what should happen.
>>
>> 1. If you delete a vmailbox, should it not remove all messages  
>> just  like deleting a real mailbox? If not, this needs to be in  
>> the draft.
>
> It should not, which is just like a real mailbox.

What? The last time I deleted a real mailbox it deleted all messages  
in it.

> If you use Sieve rules to ensure that mail from your SO ends up in  
> both INBOX.lillian INBOX, and then delete INBOX.lillian, the same  
> messages remain in INBOX.

That's because it was copied.

> which maps out messages from Lillian, and then delete  
> INBOX.lillian, INBOX remains unaffected. No difference.

Why isn't that true when I expunge messages in the virtual mailbox?

>
>> 2. Expunging messages and the interaction between clients. The  
>> mobile  client uses vmailbox for trash based on a message being  
>> marked  \deleted. Now when I access the backing mailbox using a  
>> desktop  client which does a SELECT and a CLOSE. All \deleted  
>> messages with be  expunged and now on my mobile client it will  
>> look like my trash was  magically emptied. Have fun figuring out  
>> that bug report ;)
>
> That's also just like a real mailbox. Another client empties the  
> trash (by closing or expunging), and the mobile client sees that  
> the trash went away.

No it's not. When I close my INBOX it does not expunge messages in my  
Trash folder.

>> I have great mobile client which has this great option to show  
>> only  unread messages which it does by creating a vmailbox. Now, I  
>> go back  to my desktop and find a new mailbox which I don't  
>> remember creating  and it has a bunch duplicate messages from my  
>> INBOX. I will just  delete this junk mailbox using my desktop  
>> client, which does moves  all messages to "Deleted Messages", then  
>> marks them \deleted and  expunges. Hey where did all my new  
>> messages go?
>>
>> With the MAY expunge  entire mailbox clause, there might even be  
>> more messages expunged  before the client expects it.
>
> I remember feeling a bit uncertain about whether to allow the 't'  
> right. I decided against allowing 'i', but couldn't make up my mind  
> about 't'. This makes it simple: 't' must be withheld.

We continue to move further away from "For all intents and purposes,  
the virtual folder looks and behaves  like a real IMAP4 folder,  
except that it does not contain any messages  itself."

The mailbox metaphor doesn't work for filters. Time to move on.

ck

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Fri, 16 Jun 2006 09:13:25 +0200

> cost of VFOLDER is higher.
...
> require a full set of results, but that's a different state of 

Reminds me of when I implemented what became VFOLDER last autumn, except 
that in its case, it was a different desirable feature that imposed the 
cost.

You may argue that if you do CONTEXTS badly, it doesn't suck quite so 
much as if you do VFOLDER badly.

Arnt

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Fri, 16 Jun 2006 11:17:16 +0200

Curtis King writes:
> What? The last time I deleted a real mailbox it deleted all messages  in it.

It did not delete the same messages in another mailbox.

> Why isn't that true when I expunge messages in the virtual mailbox?

Because my friend Abhijit and I couldn't make a useful case either way 
for that when we implemented this last year. We couldn't decide what 
was more useful - allowing expunge to work un the backing mailbox or 
disallowing it.

It becomes a bit of a moot point when 't' is disallowed.

> The mailbox metaphor doesn't work for filters. Time to move on.

Aren't you proving that all the clients which implement vfolders (e.g. 
gnus and apple mail) are broken? I haven't heard any complaints from 
users of these clients.

Arnt

---

From: Cyrus Daboo <[email-masked]>
Date: Fri, 16 Jun 2006 10:27:14 -0400

Hi Arnt,

> gnus and apple mail) are broken? I haven't heard any complaints from

Lets remember that most clients implementing virtual folders provide a 
clear visual 'cue' to the user that they are dealing with a virtual folder 
and not a real folder. Thus users will have some expectation that it may 
not behave the same way as a 'real' folder. The concern I raised in Dallas 
about returning virtual mailboxes in ordinary LIST applies here - users 
won't have a 'cue' to distinguish those from real mailboxes and thus 
confusion will occur. If virtual mailboxes have to be explicitly LIST'ed 
via LISTEXT then clients will know about them and can provide an 
appropriate 'cue' and different warning messages/error messages for user 
actions.

Another issue: how do different clients know that a virtual mailbox created 
by another client is equivalent to one it wants to create? e.g. the case of 
trash: say one mobile client wants to use 'Trash' as the name, and another 
'trash' (on a server with case-sensitive mailbox names). If the clients 
each create their own trash mailbox then that is wasteful. But the only way 
to know whether a virtual trash exists would be to get the underlying 
search expression for each virtual mailbox and compare that with the one 
the client wants to use for its trash. This is a case where 'context' is 
better as each client just uses a search of its own choosing.

-- 
Cyrus Daboo

---

From: Cyrus Daboo <[email-masked]>
Date: Fri, 16 Jun 2006 10:29:41 -0400

Hi Arnt,

--On June 16, 2006 8:39:40 AM +0200 Arnt Gulbrandsen 
<[email-masked]> wrote:

> to ensure that mail from your SO ends up in both INBOX.lillian INBOX, and
> then delete INBOX.lillian, the same messages remain in INBOX. Similarly,
> if you create INBOX.lillian vfolder as a view into INBOX which maps out
> messages from Lillian, and then delete INBOX.lillian, INBOX remains
> unaffected. No difference.

Can you make sure that the VMAILBOX spec describes the interaction with 
SIEVE fileinto? Even if all it says is that the v-mailbox is treated 
exactly the same as a 'real' mailbox as far as fileinto is concerned.

-- 
Cyrus Daboo

---

From: Curtis King <[email-masked]>
Date: Fri, 16 Jun 2006 11:18:56 -0600

> It did not delete the same messages in another mailbox.

That's my point!!

Are these virtual mailboxes or not?

>
>>> Similarly, if you create INBOX.lillian vfolder as a view into  
>>> INBOX  which maps out messages from Lillian, and then delete   
> Because my friend Abhijit and I couldn't make a useful case either  
> way for that when we implemented this last year. We couldn't decide  
> what was more useful - allowing expunge to work un the backing  
> mailbox or disallowing it.
>

If VMAILBOX doesn't allow flags especially the \deleted flag to-be  
set, then it's not a very effective view is it.

> Aren't you proving that all the clients which implement vfolders  
> (e.g. gnus and apple mail) are broken? I haven't heard any  
> complaints from users of these clients.

The difference here is I know when I'm working on a virtual mailbox  
versus a real mailbox. Cyrus covered much better than I very could.

ck

---

From: Randall Gellens <[email-masked]>
Date: Fri, 16 Jun 2006 13:16:14 -0700

> etc. For CONTEXTS, nothing. Agreed?

Plus, for VMAILBOX, the search criteria and backing mailbox name 
(unless that's what you meant by "mailbox name").

For CONTEXTs, if the server stores nothing that the entire CONTEXT 
needs to be recomputing for each use.  The draft suggested that 
servers might use the client hint to store or at least cache some of 
this.

> that benefits the server greatly (but makes little difference to 

It would make a difference to the client if it needs to send more 
data and wait longer to get the results.

> the client has enabled, 0-15.

So, for VMAILBOX, every time a message in added to the backing 
mailbox, instead of the server comparing it against the search 
criteria for each VMAILBOX, it only needs to compare against any 
currently SELECTed VMAILBOxes that use the backing mailbox. 
Likewise, whenever a VMAILBOX is SELECTed, the server must compare 
the previous highest UID of the compared messages in the backing 
mailbox, and if new messages arrived in the backing mailbox, compare 
those.

So the server needs to maintain a link from each mailbox to all 
VMAILBOXes that use that mailbox (which it needs to do anyway in 
order to handle mailbox renames or deletes).

> VFOLDER wins by a significant margin. Depending on what's expensive 
> in your architecture, your conclusion may be different.
>

This goes to the use cases for virtual mailboxes, I think.  It may or 
may not make sense to want to share them between clients.

If virtual mailboxes are used in lieu of server-based filtering, so 
that incoming mail for, say, mailing lists, personal mail versus work 
mail, etc. are all in INBOX instead of in individual mailboxes, then 
it does make sense to share them between clients.  If instead virtual 
mailboxes are used to create subsets of already-filtered mail so that 
limited-capacity mobile devices don't see low-priority messages, then 
it may not make sense to share them, since the criteria for 
"interesting" and low-priority may differ on each device.

> will begin to use  SEARCH+CONDSTORE+CONTEXTS optimally.

Or, we need to understand why current desktop clients do it that way. 
Off the top of my head, two possible factors are: clients were 
written originally for POP and are used to doing their own searching, 
and history of poor search performance on servers (I've certainly 
experienced the pain of this).  If, because of resource constraints, 
mobile clients are more likely to rely on SEARCH, then we should help 
that, and provide guidance to client and server developers, 
especially if this expected usage differs from current practice.

> implementation sucks, the pain is mostly on the user's side.

That makes sense.

> and say "Outlook and Outlook Express will be updated to use 

By that reasoning we may as well all go home, on the assumption that 
Microsoft will own the mobile as well as the desktop client and 
server environments.

-------------- Randomly-selected tag: ---------------
I have made it a rule, whenever in my power, to avoid becoming the
draughtsman of papers to be reviewed by a public body.
              --Benjamin Franklin

---

From: Curtis King <[email-masked]>
Date: Fri, 16 Jun 2006 15:05:47 -0600

On 16-Jun-06, at 8:27 AM, Cyrus Daboo wrote:

> Another issue: how do different clients know that a virtual mailbox  
> created by another client is equivalent to one it wants to create?  
> e.g. the case of trash: say one mobile client wants to use 'Trash'  
> as the name, and another 'trash' (on a server with case-sensitive  
> mailbox names). If the clients each create their own trash mailbox  
> then that is wasteful. But the only way to know whether a virtual  
> trash exists would be to get the underlying search expression for  
> each virtual mailbox and compare that with the one the client wants  
> to use for its trash. This is a case where 'context' is better as  
> each client just uses a search of its own choosing.

This is a big advantage for CONTEXT on the server side as the server  
can use the same underlying index for the same search criteria no  
mater what name the client gives it. You can do this for VMAILBOX as  
well, but you still need to track names and UIDs, etc, no matter if  
dynamic content is allowed or not.

ck

---

From: Mark Crispin <[email-masked]>
Date: Fri, 16 Jun 2006 13:12:10 -0700 (Pacific Daylight Time)

On Thu, 15 Jun 2006, Curtis King wrote:
>> So, are you saying that there are clients that walk the entire tree using % 
>> just to avoid doing a *?  Unbelievable.
> It's even worst than that they also issue a status for each new mailbox they 
> find.

This is a lesson to never underestimate the determination of individuals 
to do the wrong thing, no matter what measures are undertaken to prevent 
it.

> You would think that's the obvious solution.

Sadly, I think that the number of people who think that they can design 
software greatly outnumbers those who actually can.

-- Mark --

http://staff.washington.edu/mrc
Science does not emerge from voting, party politics, or public debate.
Si vis pacem, para bellum.

---

From: Randall Gellens <[email-masked]>
Date: Fri, 16 Jun 2006 15:08:29 -0700

At 3:20 PM +0100 6/15/06, Dave Cridland wrote:

> built indices, so for sake of argument you're looking at - perhaps 
> - 15 sets of UIDs. The wire-level efficiency is the same whatever 
> is stored on the server.
>

With VFOLDER the UIDs of messages selected by the criteria remain 
constant unless the message goes out of view and comes back (or the 
UIDVALIDITY of the backing mailbox changes).  How does the wire 
efficiency (by which I assume you mean the amount of data that must 
be sloshed either way) differ?

>> the client also stores the matching UID set and its highest 
>> modseq, that benefits the server greatly (but makes little 
>> difference to the client).
>>
>  For VFOLDER on a client, a substantial amount of data must be 
> stored for effective use, but nothing needs to be.

I assume the client would likely store the VMAILBOX cache, just as it 
would for a real mailbox, plus UIDNEXT, UIDVALIDITY, and the highest 
MODSEQ (also just as it would for a real maibox)?

> makes a difference to the server, but to the client, too, because 
> it affects wire efficiency.

Here the client stores the UID set of matching messages and the highest MODSEQ?

>> server needs to consider 150 messages each time the client 
>> switches context. In the worst case, the client enables all 15 
>> contexts after issuing SELECT INBOX and does not use CONDSTORE, 
>> and the server has to consider each of the 35,000 messages 15 
>> times. I suspect that a good client author such as Dave will 
>> enable most/all of the 15 and use CONDSTORE, so the server has to 
>> process each of 150 messages 15 times.
>>
>>

So only messages which have changed since the MODSEQ are considered?

-------------- Randomly-selected tag: ---------------
Finagle's Second Law:
    No matter what the anticipated result, there will always be
    someone eager to (a) misinterpret it, (b) fake it, or (c)
    believe it happened according to his own pet theory.

---

From: Randall Gellens <[email-masked]>
Date: Fri, 16 Jun 2006 15:10:07 -0700

At 8:01 AM -0700 6/15/06, Mark Crispin wrote:

>>  One problem when developing clients is the most you can expect a 
>> server to support is RFC3501 until now. LEMONADE sets a new 
>> standard which clients can expect from most major servers. I think 
>> most active desktop clients will start to use the tools set the by 
>> LEMONADE profile.
>

Why is that?
-------------- Randomly-selected tag: ---------------
Note that although IBM 'invented' virtual memory in 1972 it had
been available in a multi-processor multi-tasking environment
on the B5000 line since 1963 and the B6000 line since 1968.

---

From: Mark Crispin <[email-masked]>
Date: Fri, 16 Jun 2006 15:26:43 -0700 (Pacific Daylight Time)

On Fri, 16 Jun 2006, Randall Gellens wrote:
>>>  One problem when developing clients is the most you can expect a server 
>>> to support is RFC3501 until now. LEMONADE sets a new standard which 

It would have the effect of making an already extraordinarily-complex 
mandatory minimum-to-implement for IMAP even more complex.

I do not believe that it is a good thing to raise the bar to implement 
IMAP or to offer IMAP on existing mail stores.

The Lemonade extensions, while undeniably valuable to mobile clients, are 
of much less importance to desktop clients.  We should not encourage the 
(further) proliferation of desktop clients which do not work with servers 
that do not offer a particular extension.

-- Mark --

---

From: Randall Gellens <[email-masked]>
Date: Fri, 16 Jun 2006 16:25:31 -0700

> costs of implementing and b) some people could choose not to 

It can't be optional unless it isn't essential.  To know if it is 
needed, we need to know how VMAILBOXes would be used.  Obviously, we 
can't anticipate every possible use.  But if we knew more clearly 
which problems we were trying to solve and (perhaps more importantly) 
which we aren't, we could narrow things down, I think.
-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly-selected tag: ---------------
I employ no person who smokes.      --- Thomas Edison
[Handwritten note by Edison to Henry Ford, April 26, 1914

---

From: Randall Gellens <[email-masked]>
Date: Fri, 16 Jun 2006 17:17:22 -0700

> looks in a name that has inferiors.

I believe Mac Eudora only fetches the mailbox tree (or even the top 
level of the mailbox hierarchy) on request (by the user selecting 
"Refresh mailbox list").  In which case it does make sense to walk 
the entire tree.  I assume the idea is that mailbox 
creation/deletion/renaming by other clients is infrequent enough that 
when it does happen the user can tell the client to go update the 
list.

Probably, a conclusion of this discussion is that we need better 
guidance for developers.
-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly-selected tag: ---------------
Heuristics are bug ridden by definition.  If they didn't have bugs,
then they'd be algorithms.

---

From: Randall Gellens <[email-masked]>
Date: Fri, 16 Jun 2006 18:24:12 -0700

>  On my mobile client I only want to see unread messages 90% of the time.

So your mobile client shows you a list of unread messages.  You pick 
one and read it.  Does it then vanish from the mobile device?  I'd 
think that it would be confusing and frustrating if it immediately 
vanished, but people probably would like it to to vanish from the 
mobile device at some point.  People are also likely to try and 
delete the message from the local device only if it doesn't go away 
when they are through with it.  I'm not sure when it should vanish 
just from having been read, but I think it needs to be under explicit 
client control.  Is doing a FETCH without PEEK is the right trigger 
for it to vanish?
-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly-selected tag: ---------------
    Life is ... a shopping mall ... where items such as worldly success,
love of music, a strong backhand, close relationships with your family,
a few good friends and countless other things are on sale.
    The commodity with which they are purchased is not money, but time.
                                                   --William Rehnquist

---

From: Randall Gellens <[email-masked]>
Date: Fri, 16 Jun 2006 18:30:34 -0700

At 2:19 PM -0600 6/15/06, Curtis King wrote:

>  The longer the discussion goes on, the more I think we need ACAP 
> light or IMSP.

There has been discussion in the past about a rev of the ACAP spec 
that would make it easier to implement on servers.  As I recall, one 
key point was to not require all three of contexts, inheritance, and 
ACL on items.
-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly-selected tag: ---------------
Premature optimization is the root of all evil
                              --C. A. R. Hoare

---

From: Randall Gellens <[email-masked]>
Date: Fri, 16 Jun 2006 18:39:16 -0700

> like authors to explain what should happen.

I'm not an author of the draft but have a few comments.

> like deleting a real mailbox? If not, this needs to be in the draft.

I'd expect not, but I agree we need to be clear on what happens.

> messages with be expunged and now on my mobile client it will look 
> like my trash was magically emptied. Have fun figuring out that bug 
> report ;)

Isn't this the same as a second client expunging messages from any mailbox?

> unread messages which it does by creating a vmailbox. Now, I go 
> back to my desktop and find a new mailbox which I don't remember 
> creating and it has a bunch duplicate messages from my INBOX. I 
> will just delete this junk mailbox using my desktop client, which 
> does moves all messages to "Deleted Messages", then marks them 
> \deleted and expunges. Hey where did all my new messages go? With 
> the MAY expunge entire mailbox clause, there might even be more 

This is more serious, IMHO.  A second client could 'accidently' 
delete messages in this way.

-------------- Randomly-selected tag: ---------------
Did you know...
That no one ever reads these things?

---

From: Curtis King <[email-masked]>
Date: Fri, 16 Jun 2006 22:18:55 -0600

> I'm not an author of the draft but have a few comments.

Of course the more comments the better.

>> like my trash was magically emptied. Have fun figuring out that  
>> bug report ;)
>

True, but it still might be confusing to end users. Yes?

>> messages go? With the MAY expunge entire mailbox clause, there  
>> might even be more messages expunged before the client expects it.
>

yes.

ck

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Sat, 17 Jun 2006 12:50:39 +0200

> It becomes a bit of a moot point when 't' is disallowed.

But of course 't' must be allowed. I clearly wasn't thinking when I wrote that.

Arnt

---

From: Arnt Gulbrandsen <[email-masked]>
Date: Sat, 17 Jun 2006 14:37:33 +0200

> Microsoft will own the mobile as well as the desktop client and 

I'd rather not step overboard, neither to port nor to starboard.

I think that Lemonade will take hold in the mobile email world. I also 
think that people will continue to use regular clients, even as they 
begin to use mobile devices. I conclude that Lemonade had better 
maintain some compatibility: Lemonade should affect the relationship 
between mobile client and server, not that between sometimes mobile 
user and mail.

(More answers to this message and others later, perhaps Monday, but I 
think I've almost had my say. Right or wrong, I'll stop talking about 
this soon. Better not repeat myself more than twice.)

Arnt

---

From: Dave Cridland <[email-masked]>
Date: Sat, 17 Jun 2006 21:19:42 +0100

> all messages in a mailbox, even those outside the subset of those 

Yes, that's what I intended to convey.

Dave.

---

From: Dave Cridland <[email-masked]>
Date: Sat, 17 Jun 2006 21:39:52 +0100

On Fri Jun 16 23:08:29 2006, Randall Gellens wrote:
>>  CONTEXTS does not mandate any persistent storage, but equally 
>> does not mandate against it. Typically, I'd expect the server to 
>> have built indices, so for sake of argument you're looking at - 
>> perhaps - 15 sets of UIDs. The wire-level efficiency is the same 
>> whatever is stored on the server.
>> 
>> into a real mailbox with shared content, and persist UIDs, the 
>> wire efficiency of the protocol will suffer greatly.
> 
Because (as Curtis said a while back) it means that VFOLDER mandates 
persistent storage, to ensure those UIDs don't change. If the index 
isn't persisted (or is lost), then the client's cache is rendered 
worthless, and the mailbox data will end up being sloshed (great 
word!) down the wire (which is, of course, a virtual one).

> I assume the client would likely store the VMAILBOX cache, just as 
> it would for a real mailbox, plus UIDNEXT, UIDVALIDITY, and the 
> highest MODSEQ (also just as it would for a real maibox)?
> 
> 
Yes, but if it fails to do so, the data isn't lost, although it needs 
refetching.

> Here the client stores the UID set of matching messages and the 
That, as they say, is a good question.

I need to look at what the client needs to store, the complexity of 
doing so, and the bandwidth saved by doing so.

>>  To begin with, you're still considering all 35,000 messages - 
>> it's simply that you're using a MODSEQ index to first-cut the 
>> search range.
> 

In Arnt's scenario, on a re-select or re-acquisition of a context, 
there are 35,000 messages considered.

The most efficient, and obvious, method for cutting down the server's 
workload is to not consider adding or removing any messages which 
have not changed. It's so obvious that Arnt has forgotten to mention 
it, is all. Most servers will have a fast way of finding changed 
messages, after all.

Dave.

---

From: Dave Cridland <[email-masked]>
Date: Sat, 17 Jun 2006 22:12:32 +0100

On Fri Jun 16 07:43:17 2006, Arnt Gulbrandsen wrote:
> Curtis King writes:
>>> Yes, in a pathological case, CONTEXT would be more efficient 
>>> since  presumably the client would issue a single uber-search and 
>>> parse  out the results into individual 'vmailboxes' on the client 
>>> side,  rather than issue dozens of searches.  But, what you see 
>>> as a bug  for a pathological case (vmailboxes show up in LIST)  I 
>>> see as a  feature (clients can share vmailboxes easily)
>> 
> I want to share all of mine. Ask Dave how many of the ten he 
> mentioned yesterday he wants to share.
> 
> 
Well, I do share all of them, because my view definitions, like the 
rest of my configuration, lives in ACAP.

But I'm afraid that overall, I'm really not the best person to ask 
about the notion of sharing things between devices, since my entire 
client configuration is always shared between all devices - I've 
largely forgotten what "not shared" means in practise.

Dave.

---

From: Dave Cridland <[email-masked]>
Date: Sat, 17 Jun 2006 21:38:52 +0100

On Sat Jun 17 02:24:12 2006, Randall Gellens wrote:
> So your mobile client shows you a list of unread messages.  You 
> pick one and read it.  Does it then vanish from the mobile device?  
> I'd think that it would be confusing and frustrating if it 
> immediately vanished, but people probably would like it to to 
> vanish from the mobile device at some point.  People are also 
> likely to try and delete the message from the local device only if 
> it doesn't go away when they are through with it.  I'm not sure 
> when it should vanish just from having been read, but I think it 
> needs to be under explicit client control.  Is doing a FETCH 
> without PEEK is the right trigger for it to vanish?

I have a lot of control code in my view implementation to address 
precisely this issue - basically, adding \Seen is not enough to 
remove a message from a view.

It's not terribly easy to handle in a protocol specification, and it 
risks rendering a virtual mailbox useful for reading \Seen messages, 
unless you insist on always using PEEK.

Dave.

---

From: Cyrus Daboo <[email-masked]>
Date: Sun, 18 Jun 2006 12:41:02 -0400

Hi Dave,

--On June 17, 2006 9:38:52 PM +0100 Dave Cridland <[email-masked]> wrote:

> precisely this issue - basically, adding \Seen is not enough to remove a
> message from a view.
>

All of this is a re-hash of the VIEW extension debate. Remember the 
discussion on 'dynamic', 'semi-dynamic' and 'static' views? There are use 
cases for all of those. The \Seen flag maps to semi-dynamic - you want new 
unseen messages to automatically appear in view but you do not want 
messages that you read to automatically disappear out of view. The \Delete 
flag is dynamic - typically a deleted message should disappear from the 
view immediately the flag is set. This was extensively described in Section 
5 of VIEW.

-- 
Cyrus Daboo

---

From: Randall Gellens <[email-masked]>
Date: Mon, 19 Jun 2006 14:49:15 -0700

At 1:12 PM -0700 6/16/06, Mark Crispin wrote:

>>>  So, are you saying that there are clients that walk the entire 
>>> tree using % just to avoid doing a *?  Unbelievable.
>  This is a lesson to never underestimate the determination of 
> individuals to do the wrong thing, no matter what measures are 
> undertaken to prevent it.

I think this argues that profile-bis should reference the existing 
advice-to-implementers RFCs, and that a new such document is needed 
to cover what is missing.
-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly-selected tag: ---------------
The trouble with the world is not that people know too little, but
that they know so many things that ain't so.
--Mark Twain

---

From: Mark Crispin <[email-masked]>
Date: Mon, 19 Jun 2006 14:59:36 -0700 (Pacific Daylight Time)

On Mon, 19 Jun 2006, Randall Gellens wrote:
> advice-to-implementers RFCs, and that a new such document is needed to cover 
> what is missing.

Yes, but whose advice is taken?

-- Mark --

---

From: Randall Gellens <[email-masked]>
Date: Mon, 19 Jun 2006 15:27:25 -0700

At 1:16 PM -0700 6/16/06, Randall Gellens wrote:
> that incoming mail for, say, mailing lists, personal mail versus 
> work mail, etc. are all in INBOX instead of in individual 
> mailboxes, then it does make sense to share them between clients. 
> If instead virtual mailboxes are used to create subsets of 
> already-filtered mail so that limited-capacity mobile devices don't 
> see low-priority messages, then it may not make sense to share 
> them, since the criteria for "interesting" and low-priority may 
> differ on each device.

In thinking about this issue some more (how virtual mailboxes are 
used, and about sharing them between clients), it occurs to me that 
it doesn't really matter if the virtual mailbox itself is shared or 
not.  What matters, to the user, is that:
	- a desired view of his or her mail is available on desired devices
	- the shared views and especially the messages in them are coherent

That is, if virtual mailboxes are used in lieu of server-side 
filtering, say for mailing list traffic or personal versus work mail, 
the user may want the exact same criteria on each device, or may want 
somewhat different criteria on different devices.  But most 
importantly, if the user deletes a message, or replies to a message, 
the state of that message needs to be identical on each device.

So, I'm not sure if the actual virtual mailbox mechanism needs to be 
shared or not.  If clients are going to have a set of stock criteria 
that users select and combine (perhaps using descriptive names and 
check-boxes or something, for example, 'unread mail from this week', 
or 'mail sent explicitly to me'), then it may not make any difference 
to the client.  Each client will use the set of criteria that the 
user chose, and doesn't care if any other client uses the same set. 
I think each client needs to cache the same things regardless if the 
exact criteria is being used by another client.  Also, the user can 
always modify the criteria used on one device and may or may not want 
the same change on another device.

Does it make a difference to the server?
-- 
Randall Gellens
Opinions are personal;    facts are suspect;    I speak for myself only
-------------- Randomly-selected tag: ---------------
I personally think we developed language because of our
deep need to complain.                    --Lily Tomlin