#453 could not read header: message: malformed MIME header line 3 years ago

Comment by ~mato on ~sircmpwn/aerc2

~ceuk I'm using this patch against a locally vendored copy of go-message:

--- a/textproto/header.go
+++ b/textproto/header.go
@@ -485,7 +485,7 @@ func trimAroundNewlines(v []byte) string {

 const (
        maxHeaderLines = 1000
-       maxLineOctets  = 4000
+       maxLineOctets  = 8000
 )

 // ReadHeader reads a MIME header from r. The header is a sequence of possibly
@@ -493,6 +493,10 @@ const (
 func ReadHeader(r *bufio.Reader) (Header, error) {
        fs := make([]*headerField, 0, 32)

+       if buf, err := r.Peek(5); err == nil && bytes.Equal(buf, []byte("From ")) {
+               _, _ = readLineSlice(r, nil)
+       }
+
        // The first line cannot start with a leading space.
        if buf, err := r.Peek(1); err == nil && isSpace(buf[0]) {
                line, err := readLineSlice(r, nil)

#453 could not read header: message: malformed MIME header line 4 years ago

Comment by ~mato on ~sircmpwn/aerc2

~labrat I don't have any good answers, but given that e-mail is an ancient protocol some judicious application of Postel's law [1] in aerc is warranted. I managed to make things work for me by patching go-message to ignore the first header line when parsing, if and only if it starts with From<SP>; I can post that patch here if you're interested.

[1] https://en.wikipedia.org/wiki/Robustness_principle

#453 could not read header: message: malformed MIME header line 4 years ago

Comment by ~mato on ~sircmpwn/aerc2

(Edited the headers a bit for privacy)

~labrat Look, I'm just trying to be helpful. I'll see what I can do to fix my Postfix instance, but I obviously need to take care not to lose data or break a setup that has been working fine for years.

#453 could not read header: message: malformed MIME header line 4 years ago

Comment by ~mato on ~sircmpwn/aerc2

~labrat You're referring to the first line of the message headers, i.e. From xxxxxx? I agree it's not a valid header. However, something in my Postfix setup is adding it; I suspect maildrop which is being used as the LDA. Also, other mail clients (mutt, mailx, roundcube, even Windows Mail) don't complain about it, presumably due to legacy UNIX mbox-handling code?

#453 could not read header: message: malformed MIME header line 4 years ago

Comment by ~mato on ~sircmpwn/aerc2

~labrat Here you go, raw from Maildir/...:

From ocaml+verp-e302c1e62be500c5e8b6d21ad8bd39dd@discoursemail.com  Sun Sep 27 22:21:57 2020
Return-Path: <ocaml+verp-e302c1e62be500c5e8b6d21ad8bd39dd@discoursemail.com>
X-Original-To: XXXXXX@lucina.net
Delivered-To: XXXX@smtp.lucina.net
Received-SPF: Pass (sender SPF authorized) identity=mailfrom; client-ip=66.220.12.154; helo=mx-out-01b.sjc2.discourse.cloud; envelope-from=ocaml+verp-e302c1e62be500c5e8b6d21ad8bd39dd@discoursemail.com; receiver=XXXXXX@lucina.net 
Authentication-Results: smtp.lucina.net; dkim=pass
	reason="2048-bit key; unprotected key"
	header.d=discoursemail.com header.i=@discoursemail.com
	header.b=A76H01pm; dkim-adsp=pass; dkim-atps=neutral
Received: from mx-out-01b.sjc2.discourse.cloud (mx-out-01b.sjc2.discourse.cloud [66.220.12.154])
	by smtp.lucina.net (Postfix) with ESMTPS id 7E8E4122804
	for <XXXXXX@lucina.net>; Sun, 27 Sep 2020 22:21:56 +0200 (CEST)
Received: from localhost.localdomain (unknown [IPv6:2001:470:14e:2::2ef:8bb1:35b])
	by mx-out-01b.sjc2.discourse.cloud (Postfix) with ESMTP id BE2AF140F65
	for <XXXXXX@lucina.net>; Sun, 27 Sep 2020 20:21:53 +0000 (UTC)
DKIM-Signature: v=1; a=rsa-sha256; c=simple/simple; d=discoursemail.com;
	s=sjc2; t=1601238113;
	bh=SsOPhhQHxgYN9sJBiKt7oBkNnCDS9YoPf8MPnyOxHmM=;
	h=Date:From:Reply-To:To:Subject:List-Unsubscribe;
	b=A76H01pm65XtixKTwK4HQtBnowHFUG9xzbwbiWKyVrB7ADkTNYbLVlgRKkxX3SaRO
	 DPicT+5+c6EIYKFdanmq9d+LlRwgKExu1Tu96QRmFOfkOt2UHg3UTVMrfCuEhSb+BS
	 znX9jF1izJOUG94c83I9vFqqwt3XUeuFO4ydpy5ec1wxo0nrN7i1mZkjz+IXo+nJL+
	 v18PSHbHdqnCGX0iAcXR7u0pFFE7MbsKDmCE19f+Y5GOrQxzwVjkv7atWUZn8vdwUg
	 3gDYvBTwZa518O6XszgaKs50iOKl+wk6LrjyEIcb/VA5mSwCEUZ/JvCSSTWurmjw3F
	 zYL7WrLwRx7tA==
Date: Sun, 27 Sep 2020 20:21:53 +0000
From: OCaml <ocaml@discoursemail.com>
Reply-To: OCaml <ocaml@discoursemail.com>
To: XXXXXX@lucina.net
Message-ID: <070b4d50-1b7a-4af9-ad6e-34469b824755@discuss.ocaml.org>
Subject: [OCaml] Summary
Mime-Version: 1.0
Content-Type: multipart/alternative;
 boundary="--==_mimepart_5f70f46019d41_3e2b3ff44a8fe95828483eb";
 charset=UTF-8
Content-Transfer-Encoding: 7bit
List-Unsubscribe: <https://discuss.ocaml.org/email/unsubscribe/REDACTED>
X-Auto-Response-Suppress: All
Auto-Submitted: auto-generated
Feedback-ID: ocaml:digest:discoursemail

Now I'm wondering why I have a mbox-style From xxxxx envelope sender line in my Maildir/s, but AFAICT it's always been that way.

#453 could not read header: message: malformed MIME header line 4 years ago

Comment by ~mato on ~sircmpwn/aerc2

Hi. Potential new aerc user here, I just tried latest master against my mail archive of 20+ years and ran into this issue. AFAICT, in my case it's not due to the length of the header in question (see below). Here's an excerpt from the aerc log:

2020/10/01 15:29:08 ->(ui) *types.Error:*types.FetchMessageHeaders
UbePfg IDLE
2020/10/01 15:29:08 (ui)<= *types.Error(77):*types.FetchMessageHeaders(76)
2020/10/01 15:29:08 could not read header: message: malformed MIME header key: From ocaml+verp-e302c1e62be500c5e8b6d21ad8bd39dd@discoursemail.com  Sun Sep 27 22

There's nothing outlandishly long about the above header, these are just standard mails from discourse. Just to be sure, I took a look at the linked go-message issue, and patched the version aerc uses to increase maxLineOctets, it did not help.