#293 Email is empty when body is only a link 14 days ago

Comment by ~alexwennerberg on ~sircmpwn/aerc2

Discussed on IRC -- not a bug / working as intended. Perhaps there's some way to improve usability if others encounter this issue? Like listing all the added headers on the confirmation page? Probably not necessary though

#341 "Message contains invalid header" when first line of email body ends in a colon 22 days ago

Comment by ~alexwennerberg on ~sircmpwn/aerc2

Issue with go-message resolved by this commit: https://github.com/emersion/go-message/commit/8ade7dd97dd23728130e753517da09c44e935c0e

Should be resolved in Aerc by updating this dependency

#341 "Message contains invalid header" when first line of email body ends in a colon a month ago

Comment by ~alexwennerberg on ~sircmpwn/aerc2

(Sorry for the triple-post)

Here is the relevant issue in go-message:

https://github.com/emersion/go-message/issues/68

#341 "Message contains invalid header" when first line of email body ends in a colon a month ago

Comment by ~alexwennerberg on ~sircmpwn/aerc2

Note that what I say above also appears to apply to RFC 5322

#293 Email is empty when body is only a link a month ago

bug added by ~alexwennerberg on ~sircmpwn/aerc2

#293 Email is empty when body is only a link a month ago

Comment by ~alexwennerberg on ~sircmpwn/aerc2

See my message here

It seems as though, if there is content after the URL in the email, the URL is parsed as an email header. Not sure what the best way to fix this is, as a url is technically a valid email header

#341 "Message contains invalid header" when first line of email body ends in a colon a month ago

Comment by ~alexwennerberg on ~sircmpwn/aerc2

OK -- I did a bit more research into this. Most of what I said above is off-target.

This is intended behavior to some degree. From the docs:

You can add additional headers like Cc and Reply-To by simply adding them 
to the top of your email, adding a blank line between the email's headers and body.

It seems as though the Fastmail and Gmail IMAP servers accept the email, even though it is in violation of RFC 2822 They assume the invalid header is part of the message body and add a new line. Presumably on the receiving end. However, when Aerc makes a command to append the outgoing message to Sent Mail, the server's mail parser does not transform the message, which is in violation of RFC2822 as constructed. The behavior is different in Fastmail and Gmail -- Fastmail will throw an error and not add the message to the Sent folder, while Gmail will add the message as-is to Sent, but Aerc will parse the sentence ending in a colon as a header and hide it unless you show it with toggle-headers.

This looks to be an issue with the go-message library. The headers are parsed by Aerc here

If I understand this comment correctly, what happens if, say, the first line of my message is not a valid header, is that mail.CreateReader throws an error, and the text is just added as-is. This is why you can begin an email with Hello, and it will still send properly.

This leads us to the go-message function ReadHeader, which does not throw an error in the case of a space in the message header. As ~labrat mentioned, spaces are invalid in header names: RFC2822. I am not sure if I am missing something about the RFCs or if RFC2822 has been errata'd, but I assume the parser should throw the error if the text before the colon has a space in it. No changes will be required to Aerc to fix this, just updating the dependency. I'll report a bug soon on that project.

I could imagine someone not intimately familiar with IMAP (such as myself) could unintentionally send an email with a valid email header. Like Dear: Alex and team or a URL. While there obviously is not really a perfect way to disambiguate this, it did trip me up. Perhaps someone can think of a creative solution, if they consider it necessary. (Some sort of syntax highlighting? Maybe a note on the preview page that indicates you've added headers? Not sure)

This also explains my issue #293. According to the ReadHeader parser, a url like https://www.alexwennerberg.com/ can be parsed as a valid Email header, with the key and value Https: //www.alexwennerberg.com/. Odd, but consistent with the RFCs to my knowledge.

#341 "Message contains invalid header" when first line of email body ends in a colon a month ago

Comment by ~alexwennerberg on ~sircmpwn/aerc2

Here are the logs that I'm seeing:

2020/01/18 13:56:21 Sending mail
smtps
2020/01/18 13:56:21 rcpt to: [alex@alexwennerberg.com]
2020/01/18 13:56:21 (ui)=> *types.AppendMessage
2020/01/18 13:56:21 <-(ui) *types.AppendMessage(86)
DONE
6N5jLw OK Completed
q5F7Bg APPEND "Sent" (\Seen) "18-Jan-2020 13:56:21 -0600" {303+}
Content-Transfer-Encoding: quoted-printable
Content-Type: text/plain; charset=UTF-8
Date: Sat, 18 Jan 2020 13:56:05 -0600
Subject: email
From: "alex wennerberg" <alex@alexwennerberg.com>
To: <alex@alexwennerberg.com>
Message-Id: <BZZ6JTHLTOGD.2CUNYE3DS5NKE@debian-thinkpad>
a mail adsf:

asdf

q5F7Bg NO Message contains invalid header
2020/01/18 13:56:21 ->(ui) *types.Error:*types.AppendMessage
T5KG6Q IDLE
2020/01/18 13:56:21 (ui)<= *types.Error(87):*types.AppendMessage(86)
2020/01/18 13:56:21 Message contains invalid header
+ idling
* 59 EXISTS
* 1 RECENT

It seems like it is potentially an error on the server end, not aerc's end

#341 "Message contains invalid header" when first line of email body ends in a colon a month ago

Comment by ~alexwennerberg on ~sircmpwn/aerc2

On Thu Jan 16, 2020 at 7:04 PM, ~labrat wrote:

No, my own server or gmail

I'm only encountering this issue on Fastmail. Was unable to reproduce it on Gmail

#341 "Message contains invalid header" when first line of email body ends in a colon a month ago

Comment by ~alexwennerberg on ~sircmpwn/aerc2

@labrat

Are you using fastmail as your client?

Two additional details:

1. I updated Aerc to the latest commit and confirmed that I still get this bug
2. This seems to occur when copying the email to the Sent folder. That is when the error occurs and the email is not copied to the sent folder. Other emails are properly getting copied to the Sent folder in my client