Hi All
Correct me if I’m wrong about the below.
Evolution when configured with imap doesn’t do searches on the imap server, it only searches locally.
So if you have access to a large set of shared folders via imap, search won’t work unless everything is downloaded locally.
Is this the correct workaround
Hi, please always provide your exact application version.
Hi,
no, the libcamel IMAP provider does search on the server, supposing the
server supports necessary IMAP extension. It’s for the body-contains
search and the like, which may require complete message in the local
store. Things which can be search locally, like those one can see in
the message list, are checked locally.
Please note the search happens per folder. You cannot search “in any
folder on the server, own or shared or public” by a single command. The
search folders run the search in each configured folder for you, one by
one.
Bye,
Milan
Modern IMAP servers can actually do server side body search and are super fast at it. Tested with dovecot FTS and mutt/claws-mail as client. So yes Evolution support for this would be very welcome.
Name : evolution
Version : 3.40.4
Release : 10.el9_5.1
Architecture : x86_64
I’m not seeing this happen when using the evolution version packaged with rocky9. The server is Cyrus IMAP on a rocky8 base.
Name : cyrus-imapd
Version : 3.0.7
Release : 27.el8_10
Architecture : x86_64
When doing simple subject/to/from searches results only appear to be seen when a local cache of the folder is downloaded.
Are there any debugging commands which might allow me to explore this further from a evolution perspective?
I will do some further testing with mutt to confirm that server side search is working as expected.
Yes I know the versions of evolution are old however we’re required to use packaged versions. Any words of wisdom are appreciated. 
Hi Milan,
In relation to your reply, are you saying that a server side search will only work on a single folder or that a search will result in a serialised search over all subscribed folders?
When doing simple subject/to/from searches results only appear to be
seen when a local cache of the folder is downloaded.
Hi,
the local cache is always needed, the part for the message list (not
whole messages). The Subject/To/From search can be done from this
summary information, though I see your version can issue server-side
search for these headers too (in addition to “body-contains” search).
Some servers do not provide precise results, thus it can be the server
returns a set of the message IDs and the IMAPx provider retries with
this search result.
Are there any debugging commands which might allow me to explore this
further from a evolution perspective?
Please see:
Lets backup a little bit.
When searching using evolution just searching on source and destination email addresses across the entire account the client was returning significantly incomplete results.
What could cause issues of this nature?
When searching using evolution just searching on source and
destination email addresses across the entire account
Just to be sure, the search panel above the message list reads like:
Show: [ All Messages ] Search: [ /Subject or Addresses contain/ ]
in [ Current Account ]
and when you enter a text into the “Search” field (the ‘/’ above denote
it’s a placeholder text, grayish), the search starts and runs and runs
and runs, for how long depends on the account data, for example how
many folders it has and how many messages in each of those folders are.
The status bar below the message list (if you’ve it enabled) shows that
the search is ongoing (and it can be cancelled there).
the client was returning significantly incomplete results.
That means precisely what, please? Like the Inbox contains a
user@no.where in the From and you wrote into the Search “user@”, and
this message is not part of the result?
What could cause issues of this nature?
Yours 3.40.4 is kinda old (3.54.x series is the current, to be replaced
by the 3.56.x series in several weeks), maybe that’s the reason, maybe
not. Hard to tell.
In any case, the “Subject or Addresses contain” filter uses the local
folder summary data, it does not ask the IMAP server, if not needed.
You can verify whether the server had been consulted with the IMAP
debugging on, as described in the link I gave to above.
Yes the setting are as you described in the search panel.
The search results were a subset of other older versions of evolution as packaged from Rocky8.
From what you’ve said, if this search is done locally there may be some local corruption or similar or the functionality is broken.
What files should be removed to force regeneration of the local cache and associated indexes forcing the client to resync afresh.
Yes we understand that the client is old however we can only the upstream build can be used.
Is there any way to explicitly make the client use the server as the default for it’s search?
For example if both the home directory and the mail server are only accessible via the network, both are equivalent.
Ideally everyone seeing identical search results would be optimal.
What files should be removed to force regeneration of the local cache
and associated indexes forcing the client to resync afresh.
Hi,
the IMAP stores its data under ~/.cache/evolution/mail/<account-uid>.
Yes we understand that the client is old however we can only the
upstream build can be used.
Well, a standard way is to try to reproduce with the latest available
upstream version, for example under Flatpak from flathub.org, to verify
whether the result is the same with the latest version or not.
Depending on the result you can report to your distro maintainers to
find and backport relevant changes (it’s their duty, you chose their
distro for their support of their users), or use the version you tested
with.
Is there any way to explicitly make the client use the server as the
default for it’s search?
You did not try the CAMEL_DEBUG thing, did you? The log may show some
UID SEARCH commands.
Do you search for plain ASCII letters or any UTF-8 are involved, both
in the search text and in the expected results, which are not shown in
the message list on the Evolution side?
Bye,
Milan
Thanks Milan,
I’ve previously deleted and recreated this directory in an attempt to solve the problem to no avail.
I had run evolution with the CAMEL debug however I wasn’t sure how this relates to the making the client use the server at the default for searching to, from and subject headers.
I thought that as the issue related to the to, from and subject headers there was a statement along the lines of searches involving these components was done locally however I reread it your comment and it wasn’t what was said.
However the question remains is there a way to make the search go directly to the imap server for subject, to & from.
Unfortunately I don’t think that it’s possible to run flatpacks in this environment as this would require disabling fapolicyd
I thought that as the issue related to the to, from and subject
headers there was a statement along the lines of searches involving
these components was done locally however I reread it your comment
and it wasn’t what was said.
Hi,
I agree, it’s confusing. Let me rephrase:
The CAMEL_DEBUG log will show you whether the search is done server-
side or locally (no SEARCH in the log → local search was done).
However the question remains is there a way to make the search go
directly to the imap server for subject, to & from.
No, the code does it on its own. The local search calls an SQLite to
select appropriate messages. If you search ASCII only, then it should
always work. If UTF-8 is involved, then it can be tricky, though it
should work too. You did not provide this kind of detail, thus it’s
hard to figure out what failed and why for some of the messages.
Bye,
Milan
Thanks for the clarification.
I will do some more digging
So from looking at the debug.
Search using the To, From and subject parameters is all done on the clients.
The only check that the client does is to check that it has the latest messages from all folders.
So search errors would imply that the client is either not searching correctly or that the server is not sending the latest messages id’s to the client?
Is the above statement correct?
Yes, that would be my conclusion too.
I do not think the search on the client is broken, even I can be wrong.
I need more information about what precisely you do. I do not see you
ever answered the ASCII/UTF-8 thing, but maybe I overlooked the answer
in this long thread. Apart of that, it would be good to know the exact
search term you use, the missing message’s related-to-the-search
headers, to see where precisely the searched text is.
Consider different options with an advanced search, accessible for
example through menu Search->Advanced Search. You can fine tune how the
strings are compared (beings with, contains, is-the-same, and so on).
Apologies, the searches were all simple ASCII searches using to/from/subject line string.
My main concern is that the search may be inconsistent, and that the instance which has led to this was an extreme case.
I’ve since removed the local cache and since it’s recreation search has been better however I need to do more testing to provide greater assurance.
It would be nice to have the ability to configure search to only use the server as it’s pretty much a black box.