~aw/mygit#31: 
Include read-only mailing list archive

I think the best move is to package these together. I can split them out if this project gets much traction, but this is simplest for now

Status
REPORTED
Submitter
~aw
Assigned to
No-one
Submitted
6 months ago
Updated
2 months ago
Labels
No labels applied.

Johann150 3 months ago · edit

The program would have to receive email somehow. I think it is a given that we would make use of another mail server.

I think it would not be practical to check for new mail on each request to the mailing list archive. Instead there should probably be a separate program to process new mail. This program could then be run by postfix's local(8) delivery agent or alternatively in a cron job.

Toggling the behaviour of the program by e.g. a command line switch to only process mail and not run the web server seems a bit dirty to me, so I propose to separate into mulitple programs, making use of cargo's workspace feature.

Any feedback is welcome.

~aw 3 months ago

I have been working on a separate project which is basically just a reimplementation of hypermail in rust. This is a better design than my original plan imo

Alex

On Jun 29, 2021, at 9:13 AM, Johann150 outgoing@sr.ht wrote:

The program would have to receive email somehow. I think it is a given that we would make use of another mail server.

I think it would not be practical to check for new mail on each request to the mailing list archive. Instead there should probably be a separate program to process new mail. This program could then be run by postfix's local(8) delivery agent or alternatively in a cron job.

Toggling the behaviour of the program by e.g. a command line switch to only process mail and not run the web server seems a bit dirty to me, so I propose to separate into mulitple programs, making use of cargo's workspace feature.

Any feedback is welcome.

-- View on the web: https://todo.sr.ht/~aw/mygit/31#event-88113

Johann150 3 months ago · edit

On 2021-07-01, ~aw wrote:

I have been working on a separate project which is basically just a reimplementation of hypermail in rust. This is a better design than my original plan imo

I think something like hypermail is already overkill. It complicates browsing by having each message on a separate page. For something like managing bug reports it makes more sense to have all mails of a thread on one page in chronological order. Like sourcehut does with https://lists.sr.ht and https://todo.sr.ht.

~aw 3 months ago

Yep i agree. I’ll share what i have so far when i get back from vacation

Alex

On Jul 1, 2021, at 12:21 PM, Johann150 outgoing@sr.ht wrote:

On 2021-07-01, ~aw wrote:

I have been working on a separate project which is basically just a reimplementation of hypermail in rust. This is a better design than my original plan imo

I think something like hypermail is already overkill. It complicates browsing by having each message on a separate page. For something like managing bug reports it makes more sense to have all mails of a thread on one page in chronological order. Like sourcehut does with

https://lists.sr.ht and https://todo.sr.ht.

-- View on the web: https://todo.sr.ht/~aw/mygit/31#event-88250

Johann Galle 2 months ago · edit

I think the best way to implement this in such a way to reduce setup to the minimum would be to have it work either if directly installed on the mail server itself (which would be the assumed more common use case) or also using another mail account that supports IMAP.

From having one address, subaddresses (also known as plus addressing or detail part) could be used to split the mails into separate threads/issues/merge requests etc. meaningfully, much like todo.sr.ht does.

Our part would just be on processing the incoming mails and would not actually handle receiving the emails itself. For both setup alternatives, either something like postfix's local(8) delivery using a command alias for the given address; or fetchmail with the --mda option, essentially resembling local(8) type delivery. As opposed to the real local(8) the mail delivery would have to be scheduled e.g. with a cron job and would thus be asynchronously.

In this way, our software would not need to be adapted except for idealy providing instructions and a shell script to the fetchmail version (as I would expect less technically versed users to want to use this). All we would thus need to do is to read in a mbox style mail archive on stdin and process it accordingly.

What is your current idea, did you already make progress on this?

~aw 2 months ago

Hmm, I'm not super familiar with local mail delivery, and I've been working on a version of this, but my idea was basically to just clone hypermail https://github.com/hypermail-project/hypermail -- ie, use something like isync to sync mailboxes to a maildir/mbox, then convert that maildir to a bunch of HTML files.

~aw 2 months ago

I may write it in Go to use email libraries like this: https://git.sr.ht/~emersion/go-emailthreads (or i could rewrite that lib in Rust, depends how much time I have)

I think what you’re describing is slightly different— I wouldn’t expect self hosting a mail server to be common at all (I don’t host email), and so my primary case would be to pull via imap

It’s surprisingly hard to find good IMAP scripts these days, most of them are ancient. I may bundle in something with this project or write a simple imap backup program in Rust

Johann Galle 2 months ago · edit

On 2021-07-29 00:56+02:00, ~aw rote:

Hmm, I'm not super familiar with local mail delivery, and I've been working on a version of this, but my idea was basically to just clone hypermail https://github.com/hypermail-project/hypermail -- ie, use something like isync to sync mailboxes to a maildir/mbox, then convert that maildir to a bunch of HTML files.

I have said it before and I will say it again: I think threading like hypermail is too much. I think the threading issue can be solved entirely by using the addressing scheme I described earlier. E.g. have separate threads # 1 and # 2, then you can partake in them by sending an email to mygit+1@example.com or mygit+2@example.com. Of course this requires the mail server to understand plus addressing.

I think this is also what ssokolow was after in this Reddit comment:

Please consider including either an NNTP bridge or a forum-like web-posting interface in your hypermail alternative.

I find trying to work mailing lists into my mail system to be such an onerous hassle that I just refuse to interact with any project that doesn't provide one of those two alternatives. https://www.reddit.com/r/rust/comments/omw4e7/mygit_simple_selfhosted_git/h5oy0ir/

Threading like hypermail is very annoying if you want to read an entire thread. All other collaboration platforms I know show everything for a thread (i.e. "issue", "pull request", "merge request" etc.) on one page. Sourcehut, Gitea, Gitlab, Github all do it this way.

On 2021-07-29 07:28+02:00, ~aw wrote:

I think what you’re describing is slightly different— I wouldn’t expect self hosting a mail server to be common at all (I don’t host email), and so my primary case would be to pull via imap

Yeah that might be a misrepresentation on my side, I agree. Still, my proposal would be compatible with both, just that the priorities should be adjusted towards IMAP a bit.

In the end my idea is also to process a mbox file, and I have seen that maildir should probably also be supported as more tools like mbsync seem to support it.

~aw 2 months ago

Threading like hypermail is very annoying if you want to read an entire thread. All other collaboration platforms I know show everything for a thread (i.e. "issue", "pull request", "merge request" etc.) on one page. Sourcehut, Gitea, Gitlab, Github all do it this way.

Yeah, this is what I was thinking as well. When I say "hypermail style" I just mean a static script that takes an mbox and outputs HTML, rather than a dynamic serer. I hadn't thought about the plus addressing system -- I was thinking just threading based on the in-reply-to header

In the end my idea is also to process a mbox file, and I have seen that maildir should probably also be supported as more tools like mbsync seem to support it.

Makes sense to me!

~aw 2 months ago

Here's a very very rough sketch of what I have so far if you're interested: https://git.alexwennerberg.com/crabmail

Register here or Log in to comment, or comment via email.