I write code.
For sourcehut support, see the wiki.
If we want to instrument some internals which aren't exposed to plugins, we should implement it internally. Personally I wouldn't have a problem with going all in on Prometheus, it's sufficiently better than the previous state of the art that anything which comes next is likely to be compatible with it.
Attachment list: TODO
I've pushed a change which sends emails asyncronously and retries on failure with an exponential backoff, up to 5 times (and up to a couple of hours later, accounting for the backoff).
I think it's relatively reasonable to assume that the SMTP server will usually be up, and failures will be transient, so it's probably fine to just drop emails on the floor if they aren't sent. The remote server has its own queueing and redelivery system and will send a mail delivery status message if there are errors later on. The message is saved in the user's outgoing mail folder regardless, so it's not unrecoverable anyway.
If that's not sufficient, ~migadu, feel free to re-open this to discuss more options.
REPORTED RESOLVED IMPLEMENTED
~migadu what are your current requirements for infrastructure monitoring? We use Prometheus at SourceHut and it would be pretty easy to incorporate into alps.
As I'm implementing asyncronous emaild elivery, I'm also setting it up to block process termination until all queued emails are sent.
If not planned for, this can cause an outage if you wait for the process to finish shutting down before spinning up a new one, which the naieve approach with most service managers would do (e.g. openrc, systemd).
~migadu, do you have thoughts on this? Should we discuss deployment planning with this in mind?