Subject: [imapext] SEARCHRES RFC5182

From: "Adrien de Croy" <[email-masked]>
Date: Tue, 09 Mar 2021 23:21:32 +0000

Hi Everyone

I've been looking into this extension, but it has a number of problems 
and uncertainties in the spec especially:

* in relation to storing results of ESEARCH extensions, it refers to 
them as messages).
* in relation to reporting errors if the saved results are incompatible 
(e.g. stored SEQ but asked to use as UID or vice versa)
* dealing with expunging of messages.

So my first question relates to whether I'm even wasting my time.  The 
imapwiki shows very few servers support it, but are any clients known to 
support it?

Regards

Adrien

---

From: Alexey Melnikov <[email-masked]>
Date: Wed, 10 Mar 2021 10:07:25 +0000

Hi Adrien,

> * in relation to storing results of ESEARCH extensions, it refers to 

Can you elaborate/quote specific text which is confusing?

> * in relation to reporting errors if the saved results are 
> incompatible (e.g. stored SEQ but asked to use as UID or vice versa)

The document is quite clear that $ references message numbers or UIDs as 
appropriate in the corresponding context. So a saved result is never 
incompatible.

You can see this in the last paragraph of section 2.1:

    Implementation note: server implementors should note that "$" can
    reference IMAP message sequences or UID sequences, depending on the
    context where it is used.  For example, the "$" marker can be set as
    a result of a SEARCH (SAVE) command and used as a parameter to a UID
    FETCH command (which accepts a UID sequence, not a message sequence),
    or the "$" marker can be set as a result of a UID SEARCH (SAVE)
    command and used as a parameter to a FETCH command (which accepts a
    message sequence, not a UID sequence).

> * dealing with expunging of messages.

I think this is also clear, from Section 2.1:

    When a message listed in the search result variable is EXPUNGEd, it
    is automatically removed from the list.  Implementors are reminded
    that if the server stores the list as a list of message numbers, it
    MUST automatically adjust them when notifying the client about
    expunged messages, as described in Section 7.4.1 of [IMAP4].

If it helps, the best way of implementing this (IMHO) is to store a list 
of UIDs in some form and translate them to sequence numbers in the 
server, when needed. When messages are expunged, this list is purged of 
expunged messages.

> imapwiki shows very few servers support it, but are any clients known 
> to support it?
I don't know the answer to this.

Best Regards,

Alexey

---

From: "Adrien de Croy" <[email-masked]>
Date: Wed, 10 Mar 2021 20:14:22 +0000

Hi Alexey

------ Original Message ------
From: "Alexey Melnikov" <[email-masked]>
To: "Adrien de Croy" <[email-masked]>
Cc: "[email-masked]" <[email-masked]>
Sent: 10/03/2021 11:07:25 pm
Subject: Re: [imapext] SEARCHRES RFC5182

>Can you elaborate/quote specific text which is confusing?

When the SAVE result option is combined with the MIN or MAX [ESEARCH]
    result option, and none of the other ESEARCH result options are
    present, the corresponding MIN/MAX is returned (if the search result
    is not empty), but the "$" marker would contain a single message as
    returned in the MIN/MAX return item.

It's not clear if $ is storing the result of the MIN, or something else. 
  But I guess those are the same for MIN and MAX - the IDs.

$ only stores message IDs, not messages.  Maybe replace "contain a 
single message" with "refers to a single message" or "contains the 
message ID"

I guess I should have read it a few more times.  Maybe it's not so 
unclear.

>The document is quite clear that $ references message numbers or UIDs 
>as appropriate in the corresponding context. So a saved result is never 
>incompatible.
>

So we save both UIDs and SEQ and adjust these on expunge, and just use 
whichever one is appropriate in the subsequent commands.

Thanks!

Adrien

>If it helps, the best way of implementing this (IMHO) is to store a 
>list of UIDs in some form and translate them to sequence numbers in the

---

From: Dave Cridland <[email-masked]>
Date: Thu, 11 Mar 2021 11:48:43 +0000

On Wed, 10 Mar 2021 at 20:14, Adrien de Croy <[email-masked]> wrote:

> I've been looking into this extension, but it has a number of problems and
> uncertainties in the spec especially:
>
> * in relation to storing results of ESEARCH extensions, it refers to them
> as messages).
>
> $ only stores message IDs, not messages.  Maybe replace "contain a single
> message" with "refers to a single message" or "contains the message ID"
>
>
Well, it depends on how you see it conceptually.

You could have the server store a list of UIDs always, and translate as
required.

You could also have the server store a list of tuples of (UID,seq), and
filter as needed, updating the sequence numbers if an EXPUNGE occurs.

Or you could have the server store $ as a
std::vector<std::shared_ptr<Message>> or something and format as UID or seq
appropriately.

In all three cases (and more) you're conceptually storing references to
messages, though, and whether they're ids or not is somewhat irrelevant -
when executing the search itself, you're working on messages rather than
identifiers.

The one thing that "$" definitely doesn't contain is a list of integers,
though.

> I guess I should have read it a few more times.  Maybe it's not so unclear.
>
>

I suspect I'd store yet another internal identifier, myself, but loosely
store whatever makes you get the right external behaviour. If UIDs work for
you, I'd store only UIDs and treat a "$" as if it were "UID $" as
needed, for example. But remember it's never a simple string substitution -
otherwise "$" would fail if the saved set were to contain no messages.

As for whether people use this in clients, I don't know - it's been fairly
esoteric until now, but IMAP4rev2 might change this. It's very useful for
"Mark all as" from a search output for online clients, but there aren't
many of those around anymore.

Dave.

---

From: Alexey Melnikov <[email-masked]>
Date: Thu, 11 Mar 2021 16:13:27 +0000

Hi Adrian,

On 10/03/2021 20:14, Adrien de Croy wrote:
> From: "Alexey Melnikov" <[email-masked] 
> <mailto:[email-masked]>>
> To: "Adrien de Croy" <[email-masked] <mailto:[email-masked]>>
> Cc: "[email-masked] <mailto:[email-masked]>" <[email-masked] 
> <mailto:[email-masked]>>
>>> I've been looking into this extension, but it has a number of 
>>> problems and uncertainties in the spec especially:
>>>
It is.
> single message" with "refers to a single message" or "contains the 
Right. Dave has explained the intent well, so yes, I mean a message 
reference.
Best Regards,

Alexey

---

From: Michael Slusarz <[email-masked]>
Date: Tue, 16 Mar 2021 12:31:15 -0600

On 3/10/2021 1:14 PM, Adrien de Croy wrote:
>>> So my first question relates to whether I'm even wasting my time.  
>>> The imapwiki shows very few servers support it, but are any clients 
>>> known to support it?

Well, I did my small part towards improving awareness by including a 
SEARCHRES example in the PREVIEW spec :)

https://www.rfc-editor.org/rfc/rfc8970.html#section-5-5 
<https://www.rfc-editor.org/rfc/rfc8970.html#section-5-5>

michael