Hello Evolution team -
I’m running Evolution 3.44.4 on kernel 5.15.0-60, on Linux Mint 21.1
I have a household user who had a LOT of mail in her Inbox. Thousands of messages. In an attempt to manage this issue I created a new folder and moved a great deal of mail from the Inbox to this newly-created folder for managed review and deletion.
Occasionally, now, those messages now appear back in the Inbox. The mail in the new folder is where it belongs, but it also appears back in the Inbox. For example, today I backed up her system with GRSYNC and then also ran and Evolution Backup, both to a stand-alone HD, none of which should have had anything to do with it, but just saying…, and when I restarted EVO there they were. Yikes.
Hi Milan -
This account is POP3.
When they appear they stay in both places; new folder and Inbox.
As an aside this user also accesses her mail from a tablet, and uses IMAP there. This confuses things for me a bit, but she has no folders on the tablet (inbox only) and understands that if she deletes from the tablet, it’s gone from the server.
Even though this is confusing, I don’t believe this is a mail protocol issue. To me it seems like an EVO issue. For example, mail on the server (runbox.com) looks fine.
rsv869
Hi,
I’d suggest to use IMAP also on the desktop machine, but if the POP3 is
preferred for whatever reason, then no problem with it.
The POP3 account can be configured to leave messages on the server or
not. The synchronization is then covered by missing UIDs on the local
machine. The move of the message from the On This Computer/Inbox (where
the POP3 messages are filed) should not cause re-download of the
messages, because the list of the known UIDs is stored in a different
place.
I would try to run Evolution from a terminal, maybe with:
$ CAMEL_DEBUG=pop3 evolution
to see what the POP3 accounts do (it logs for all configured POP3
accounts). If you can reproduce this at least semi-consistently, then
I’d try to move one or more messages away and then click Send/Receive
button, to refresh the state. If it’ll not re-download the message,
then close Evolution and run it again, and do the Send/Receive. Maybe
the server needs to receive a new message in the Inbox to trigger this?
A log from these steps may shed a bit of light on this.
Also, the POP3 messages contain header X-Evolution-POP3-UID. Could you
check whether the moved message and the newly appeared message, aka the
duplicates, have the same UID set in this header, please? You can see
it in the message source (Ctrl+U).
Bye,
Milan
Sorry if this is a repeat of my earlier response… wasn’t sure if I should reply via email or this portal…
Hello Milan-
Thanks for your response. I’ve done some checking and here’s what I
see:
There are two kinds of duplicate messages: Duplicates in the Inbox, and
duplicates with message in separate folders (inbox and a storage). In
both cases the UIDL numbers match. I was able to clean this up a bit
by selecting all the messages in the Inbox, opposite click, and select
Delete Duplicate messages.
Simultaneously, in the course of this I stumbled on something that
might be relevant, which is a possible misunderstanding of best-
practice in message storage. All storage folders are just sub-folders
of the Inbox. ALL of it. 6.5-gig of messages in Inbox sub-folders.
One giant Inbox. Should storage folders be created beneath “On This
Computer”?
However, at the same time, for this EVO implementation, On This
Computer is Top, and Inbox is right beneath, but Drafts, Junk, OutBox,
Sent are all below the last of many sub-folders. If I move a folder to
On This Computer it moves it out of eye level, so to speak, which is
kind of weird. But Should it all be rearranged anyway? Start moving
folders to On This Computer. Would that help?
Could all the duplicates be caused by so much Inbox content?
Thanks for your response. I’ve done some checking and here’s what I
see:
There are two kinds of duplicate messages: Duplicates in the Inbox, and
duplicates with message in separate folders (inbox and a storage). In
both cases the UIDL numbers match. I was able to clean this up a bit
by selecting all the messages in the Inbox, opposite click, and select
Delete Duplicate messages.
Simultaneously, in the course of this I stumbled on something that
might be relevant, which is a possible misunderstanding of best-
practice in message storage. All storage folders are just sub-folders
of the Inbox. ALL of it. 6.5-gig of messages in Inbox sub-folders.
One giant Inbox. Should storage folders be created beneath “On This
Computer”?
However, at the same time, for this EVO implementation, On This
Computer is Top, and Inbox is right beneath, but Drafts, Junk, OutBox,
Sent are all below the last of many sub-folders. If I move a folder to
On This Computer it moves it out of eye level, so to speak, which is
kind of weird. But Should it all be rearranged anyway? Start moving
folders to On This Computer. Would that help?
Could all the duplicates be caused by so much Inbox content?
I was able to clean this up a bit by selecting all the messages in
the Inbox, opposite click, and select Delete Duplicate messages.
Hi,
in that case it looks like the messages were re-downloaded from the
POP3 server, possibly due to a POP3 error. The debug command I talked
about in the previous message should help to diagnose that.
Should storage folders be created beneath “On This
Computer”?
If you mean under On This Computer/… versus On This
Computer/Inbox/…, then it does not matter. Both work the same for the
code.
On This Computer is Top, and Inbox is right beneath, but Drafts,
Junk, OutBox, Sent are all below the last of many sub-folders.
Inbox is special, thus made to be at/near the top. Other folders are
sorted alphabetically on each level. If you do not like it, then try
menu Folder->Edit Sort Order.
But Should it all be rearranged anyway? Start moving folders to
On This Computer. Would that help?
As I said above: no, I do not think it has any impact on this. The
problem is elsewhere.
Hi Milan -
Am running EVO from the terminal with the following command: CAMEL_DEBUG=pop3 evolution
I’m happy to look over the output but am not sure what I’m looking for. Should I forward the output to this thread? If so, is it better to .zip it up? It’s quite a lot. Is that the Upload option on this portal?
As for the other question about moving mail out of the giant Inbox, I was only trying to ask about best way to clean this up. It may or may not relate to the duplicates issue. I was only trying to find the best method for moving messages out of the Inbox but still stored on EVO.
Reid
Hi,
I’m not sure what to look for in the log, maybe anything what looks
like an error/runtime warning.
Maybe you can make a copy of the following file:
~/.local/share/evolution/mail//uid-cache
which contains a list of recognized POP3 messages since the last check
of the content on the server. These UIDs are compared with the server
content and any newly recognized are downloaded as new messages to the
On This Computer/Inbox folder. It looks like the uid-cache file content
vanishes for you for some reason. I hoped the log will provide some
info regarding that, like the error messages I mentioned above.
The log can contain private information, thus be careful before sharing
it anywhere. Especially encoded data, which you cannot read, can
contain private information, thus do not share any such blobs in the
wild.
Bye,
Milan
It’s odd, but there’s actually two uid-cache files. One with nothing at all (an empty file) and one many scrolled pages. Data looks like this:
1676988683.17969.delivery04,S=56218
1677015176.8888.delivery04,S=34432
1677006269.17097.delivery04,S=50275
nothing encoded. Is that worth sharing?
BTW no errors that I see from running Evolution from the command line with parameters. that might take multiple days before it captures anything interesting.
It’s odd, but there’s actually two uid-cache files.
Hi,
there should be one for each POP3 account you’ve configured. No need to
share it.
Try to backup the one with many lines (each line corresponds to a
message stored on the server). Compare it with the one after you notice
the duplicates. I cannot tell whether it’ll show anything useful
though.
no errors that I see from running Evolution from the command line
with parameters. that might take multiple days before it captures
anything interesting.
Correct. The interesting log is from the time the problem arises. If
you can keep running evolution with the POP3 debugging enabled, then
it’ll be helpful - it may show what happened.
Bye,
Milan