My maildir archive folder has over 9000 messages and most of the times aerc fail to display it with the following error:
open /path/to/maildir/path/to/one/message: too many open files
I suppose that this is due to too many files are being opened in parallel before enough is closed and the number grows larger than the OS's limit.
I think the error is in
worker/maildir/message.go
methodNewReader
. That method claims to read a message into memory but in fact opens a file, returns anio.Reader
based on it and then looks like it leaks the filehandle.I tested the patch shown below using
ulimit -n 40
and then starting aerc from the same shell. Without the change you get the "..." loading indicator that gets stuck. With the change, it seems to load just fine.I'm not very familiar with Go's standard library, so if the use of ioutil/bytes looks wrong it probably is.
diff --git a/worker/maildir/message.go b/worker/maildir/message.go index dbc9ade..5752694 100644 --- a/worker/maildir/message.go +++ b/worker/maildir/message.go @@ -1,9 +1,10 @@ package maildir import ( - "bufio" + "bytes" "fmt" "io" + "io/ioutil" "github.com/emersion/go-maildir" "github.com/emersion/go-message" @@ -25,7 +26,12 @@ func (m Message) NewReader() (io.Reader, error) { if err != nil { return nil, err } - return bufio.NewReader(f), nil + defer f.Close() + contents, err := ioutil.ReadAll(f) + if err != nil { + return nil, err + } + return bytes.NewReader(contents), nil } // Flags fetches the set of flags currently applied to the message.
That seems like a red herring, and will just replace "out of files" with "out of RAM" soon enough. A better solution would be to rework this function to return an io.ReadCloser instead of an io.Reader.
I believe this code was modified for this exact reason in this patch: https://git.sr.ht/~rjarry/aerc/commit/8ecf0b73f41f9319f9d89d61f8d4619aa4e69758
I cannot find any examples of
io.ReadCloser
. I'll trying digging in that direction.
Nguyễn Gia Phong referenced this ticket in commit 904ffac.
Nguyễn Gia Phong referenced this ticket in commit 904ffac.
Nguyễn Gia Phong referenced this ticket in commit 904ffac.
Nguyễn Gia Phong referenced this ticket in commit 904ffac.