Paris, France
Comment by ~rjarry on ~rjarry/aerc
Sorry, I though I had sent a reply but apparently didn't.
The main issue is that all UI updates are processed by a single thread/goroutine. There is unfortunately no way around this. When you press
<enter>to view a message, it triggers two actions:
- Sends a "message" to the maildir backend thread/goroutine asking to fetch the whole email.
- Sends another "message" to the maildir backend thread/goroutine asking to mark the email as "seen".
Both of these messages are sent at the same time without waiting for a reply. The UI thread goes back to waiting for "messages" from the backend and will process them in the order in which they arrive.
I am suspecting that the "marked as seen" operation is passing in front of the "fetch the whole email" operation. In turn, it means renaming a file (that's the way maildir works when you change a flag) and it triggers an inotify callback to be invoked. This callback is processed by the maildir backend thread/goroutine (not blocking the UI) but since you have many messages which are invalid, the backend thread/goroutine is stuck parsing all headers and spewing thousands of error logs, delaying the "fetch the whole email" operation to be processed.
Again, this is pure speculation. I don't have a maildir with that many invalid messages to reproduce the issue and be able to debug it. Also, my time for aerc is fairly limited. I am hoping that other maildir users may take on this task on my behalf ;-)
Does that make more sense to you?
Comment by ~rjarry on ~rjarry/aerc
Until someone has time to work on a proper fix, the best course of action for you is to identify which emails have incorrect headers and try to fix them somehow using
sedor the tool of your choice.Side-note, having 300k messages stored flat in the same Maildir will always have an impact on performance, even if all your messages are RFC compliant.
I am marking this bug as duplicate of #129.
Sorry for the inconvenience...
REPORTED RESOLVED DUPLICATE
Comment by ~rjarry on ~rjarry/aerc
Since you have many messages with invalid
Referencesand/orIn-Reply-Toheaders, the (pedantic) go-message header parser complains many times.I assume that most of the CPU time is spent parsing these invalid headers and spewing thousands of log messages.
Comment by ~rjarry on ~rjarry/aerc
my wild guess is that when you open a message, aerc tries to mark it as "seen" which implies modifying the maildir flags, which in turn potentially means renaming the file, which in turn causes an inotify callback which causes aerc to re-parse all messages.
Comment by ~rjarry on ~rjarry/aerc
I trust you that it is slow, I just want to understand where the slowness comes from so I can try to fix it. Could you look into the logs if maybe there are some hints?
aerc > logI am not using maildir and if I am not aware of anyone that uses it and experiences the same performance issues when opening emails.
Comment by ~rjarry on ~rjarry/aerc
That does not help much without the call graph. Did you start capturing just before opening the email and stopped after?
Comment by ~rjarry on ~rjarry/aerc
Could you compile aerc from source and record a perf profile from that?
git clone https://git.sr.ht/~rjarry/aerc cd aerc make ./aercThen from another terminal, identify aerc's PID and start recording a perf profile:
perf record --call-graph=dwarf -p $(pidof aerc)Go back to your aerc terminal, open a message, and once it is opened, interrupt
perf.^C[ perf record: Woken up 3 times to write data ] [ perf record: Captured and wrote 1.756 MB perf.data (220 samples) ]Dump the report to a file and upload it to https://paste.sr.ht (this could get quite big and I am not sure if todo.sr.ht will handle huge code blocks.
Example:
~$ perf report --stdio | hut paste create -n 'aerc-308-maildir-open' Created paste https://paste.sr.ht/~rjarry/c59139eb0aa146305493a6d50d77346af16c2589
Comment by ~rjarry on ~rjarry/aerc
Simon Martin referenced this ticket in commit 7cb8e0e.
REPORTED RESOLVED FIXED