~rjarry/aerc#96: 
awk vs gawk

awk that ships with macOS is the BSD flavor, and there are some slight inconsistencies that leads to funny behaviors. To reproduce:

brew install aerc

And open any email that needs to be piped to the colorizer Awk script. You'll see something like this:

/usr/bin/awk: syntax error at source line 22 source file /usr/local/Cellar/aerc/0.13.0/share/aerc/filters/colorize
 context is
                header_pattern = >>>  @ <<< /^[A-Z][[:alnum:]-]+:/
/usr/bin/awk: illegal statement at source line 22 source file /usr/local/Cellar/aerc/0.13.0/share/aerc/filters/colorize
/usr/bin/awk: illegal statement at source line 22 source file /usr/local/Cellar/aerc/0.13.0/share/aerc/filters/colorize

When using gawk (i.e., GNU awk), this doesn't happen. To reproduce

brew install gawk
vim /usr/local/Cellar/aerc/0.13.0/share/aerc/filters/colorize

and change the first line

#!/usr/bin/awk -f

to

#!/usr/bin/env awk -f

This is enough to make aerc pick gawk. To verify, just compare the output:

$ env awk
Usage: awk [POSIX or GNU style options] -f progfile [--] file ...
Usage: awk [POSIX or GNU style options] [--] 'program' file ...
POSIX options:          GNU long options: (standard)
        -f progfile             --file=progfile
        -F fs                   --field-separator=fs
        -v var=val              --assign=var=val
Short options:          GNU long options: (extensions)
        -b                      --characters-as-bytes
        -c                      --traditional
        -C                      --copyright
        -d[file]                --dump-variables[=file]
        -D[file]                --debug[=file]
        -e 'program-text'       --source='program-text'
        -E file                 --exec=file
        -g                      --gen-pot
        -h                      --help
        -i includefile          --include=includefile
        -I                      --trace
        -l library              --load=library
        -L[fatal|invalid|no-ext]        --lint[=fatal|invalid|no-ext]
        -M                      --bignum
        -N                      --use-lc-numeric
        -n                      --non-decimal-data
        -o[file]                --pretty-print[=file]
        -O                      --optimize
        -p[file]                --profile[=file]
        -P                      --posix
        -r                      --re-interval
        -s                      --no-optimize
        -S                      --sandbox
        -t                      --lint-old
        -V                      --version

To report bugs, use the `gawkbug' program.
For full instructions, see the node `Bugs' in `gawk.info'
which is section `Reporting Problems and Bugs' in the
printed version.  This same information may be found at
https://www.gnu.org/software/gawk/manual/html_node/Bugs.html.
PLEASE do NOT try to report bugs by posting in comp.lang.awk,
or by using a web forum such as Stack Overflow.

gawk is a pattern scanning and processing language.
By default it reads standard input and writes standard output.

Examples:
        awk '{ sum += $1 }; END { print sum }' file
        awk -F: '{ print $1 }' /etc/passwd

$ awk
usage: awk [-F fs] [-v var=value] [-f progfile | 'prog'] [file ...]
Status
RESOLVED FIXED
Submitter
~phretor
Assigned to
Submitted
1 year, 7 months ago
Updated
1 year, 7 months ago
Labels
bug

~rjarry 1 year, 7 months ago

Hi there,

thanks for the detailed report. I have submitted a patch: https://lists.sr.ht/~rjarry/aerc-devel/patches/36261

I have tested with gawk (linux) and awk (freebsd) but I don't have access to a mac. Can you check it fixes your issue?

~phretor 1 year, 7 months ago

FreeBSD's awk is 99.99999% equal to macOS awk, so I'm positive it'll work. I don't have an easy way to apply this patch without disrupting my current Homebrew-managed installation.

If you have any suggestions on how to checkout a fresh dev tree and have a self-contained build to test, I'll be more than happy to test it out.

~rjarry 1 year, 7 months ago*

Here is how you could test this without breaking your system:

git clone https://git.sr.ht/~rjarry/aerc
cd aerc
curl 'https://lists.sr.ht/~rjarry/aerc-devel/patches/36261/mbox' | git am -3
curl 'https://lists.sr.ht/~rjarry/aerc-devel/%3CCNPCJL749UT5.3SVTZCJYDVIX2%40marty%3E/raw' | \
  python3 -c 'import sys,email; print(email.message_from_file(sys.stdin).get_payload(decode=True).decode())' | \
  /usr/bin/awk -f filters/colorize

~phretor 1 year, 7 months ago*

❯ curl https://lists.sr.ht/~rjarry/aerc-devel/%3CCNPCJL749UT5.3SVTZCJYDVIX2%40marty%3E/raw |python3 -c 'import sys,email; print(email.message_from_file(sys.stdin).get_payload(decode=True).decode())' | /usr/bin/awk -f filters/colorize
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  9702  100  9702    0     0  27846      0 --:--:-- --:--:-- --:--:-- 28451
Hey there,

> diff --git a/config/config.go b/config/config.go
> index 0de1780..17757ce 100644
> --- a/config/config.go
> +++ b/config/config.go
> @@ -33,44 +33,48 @@ type GeneralConfig struct {
>  }
>
>  type UIConfig struct {
> -     AutoMarkRead        bool          `ini:"auto-mark-read"`
> -     IndexFormat         string        `ini:"index-format"`
> -     TimestampFormat     string        `ini:"timestamp-format"`
> -     ThisDayTimeFormat   string        `ini:"this-day-time-format"`
> -     ThisWeekTimeFormat  string        `ini:"this-week-time-format"`
> -     ThisYearTimeFormat  string        `ini:"this-year-time-format"`
> -     ShowHeaders         []string      `delim:","`
> -     RenderAccountTabs   string        `ini:"render-account-tabs"`
> -     PinnedTabMarker     string        `ini:"pinned-tab-marker"`
> -     SidebarWidth        int           `ini:"sidebar-width"`
> -     PreviewHeight       int           `ini:"preview-height"`
> -     EmptyMessage        string        `ini:"empty-message"`
> -     EmptyDirlist        string        `ini:"empty-dirlist"`
> -     MouseEnabled        bool          `ini:"mouse-enabled"`
> -     ThreadingEnabled    bool          `ini:"threading-enabled"`
> -     ForceClientThreads  bool          `ini:"force-client-threads"`
> -     ClientThreadsDelay  time.Duration `ini:"client-threads-delay"`
> -     FuzzyComplete       bool          `ini:"fuzzy-complete"`
> -     NewMessageBell      bool          `ini:"new-message-bell"`
> -     Spinner             string        `ini:"spinner"`
> -     SpinnerDelimiter    string        `ini:"spinner-delimiter"`
> -     IconUnencrypted     string        `ini:"icon-unencrypted"`
> -     IconEncrypted       string        `ini:"icon-encrypted"`
> -     IconSigned          string        `ini:"icon-signed"`
> -     IconSignedEncrypted string        `ini:"icon-signed-encrypted"`
> -     IconUnknown         string        `ini:"icon-unknown"`
> -     IconInvalid         string        `ini:"icon-invalid"`
> -     DirListFormat       string        `ini:"dirlist-format"`
> -     DirListDelay        time.Duration `ini:"dirlist-delay"`
> -     DirListTree         bool          `ini:"dirlist-tree"`
> -     DirListCollapse     int           `ini:"dirlist-collapse"`
> -     Sort                []string      `delim:" "`
> -     NextMessageOnDelete bool          `ini:"next-message-on-delete"`
> -     CompletionDelay     time.Duration `ini:"completion-delay"`
> -     CompletionPopovers  bool          `ini:"completion-popovers"`
> -     StyleSetDirs        []string      `ini:"stylesets-dirs" delim:":"`
> -     StyleSetName        string        `ini:"styleset-name"`
> -     style               StyleSet
> +     AutoMarkRead              bool          `ini:"auto-mark-read"`
> +     IndexFormat               string        `ini:"index-format"`
> +     TimestampFormat           string        `ini:"timestamp-format"`
> +     ThisDayTimeFormat         string        `ini:"this-day-time-format"`
> +     ThisWeekTimeFormat        string        `ini:"this-week-time-format"`
> +     ThisYearTimeFormat        string        `ini:"this-year-time-format"`
> +     MsgViewTimestampFormat    string        `ini:"message-view-timestamp-format"`
> +     MsgViewThisDayTimeFormat  string        `ini:"message-view-this-day-time-format"`
> +     MsgViewThisWeekTimeFormat string        `ini:"message-view-this-week-time-format"`
> +     MsgViewThisYearTimeFormat string        `ini:"message-view-this-year-time-format"`

The field names and ini setting names should match. So either:

    MsgView -> msg-view-...

or:

    MessageView -> message-view-...

> +     ShowHeaders               []string      `delim:","`
> +     RenderAccountTabs         string        `ini:"render-account-tabs"`
> +     PinnedTabMarker           string        `ini:"pinned-tab-marker"`
> +     SidebarWidth              int           `ini:"sidebar-width"`
> +     PreviewHeight             int           `ini:"preview-height"`
> +     EmptyMessage              string        `ini:"empty-message"`
> +     EmptyDirlist              string        `ini:"empty-dirlist"`
> +     MouseEnabled              bool          `ini:"mouse-enabled"`
> +     ThreadingEnabled          bool          `ini:"threading-enabled"`
> +     ForceClientThreads        bool          `ini:"force-client-threads"`
> +     ClientThreadsDelay        time.Duration `ini:"client-threads-delay"`
> +     FuzzyComplete             bool          `ini:"fuzzy-complete"`
> +     NewMessageBell            bool          `ini:"new-message-bell"`
> +     Spinner                   string        `ini:"spinner"`
> +     SpinnerDelimiter          string        `ini:"spinner-delimiter"`
> +     IconUnencrypted           string        `ini:"icon-unencrypted"`
> +     IconEncrypted             string        `ini:"icon-encrypted"`
> +     IconSigned                string        `ini:"icon-signed"`
> +     IconSignedEncrypted       string        `ini:"icon-signed-encrypted"`
> +     IconUnknown               string        `ini:"icon-unknown"`
> +     IconInvalid               string        `ini:"icon-invalid"`
> +     DirListFormat             string        `ini:"dirlist-format"`
> +     DirListDelay              time.Duration `ini:"dirlist-delay"`
> +     DirListTree               bool          `ini:"dirlist-tree"`
> +     DirListCollapse           int           `ini:"dirlist-collapse"`
> +     Sort                      []string      `delim:" "`
> +     NextMessageOnDelete       bool          `ini:"next-message-on-delete"`
> +     CompletionDelay           time.Duration `ini:"completion-delay"`
> +     CompletionPopovers        bool          `ini:"completion-popovers"`
> +     StyleSetDirs              []string      `ini:"stylesets-dirs" delim:":"`
> +     StyleSetName              string        `ini:"styleset-name"`
> +     style                     StyleSet

This is really bugging me but I want to split config.go into multiple
files so it would have broken git blame anyway...

I would suggest that we start grouping fields by topic separated by
empty lines and/or line comments to prevent gofmt from doing this kind
of heresy.

> +     if config.MsgViewTimestampFormat == "" {
> +             config.MsgViewTimestampFormat = config.TimestampFormat
> +     }
> +     if config.MsgViewThisDayTimeFormat == "" {
> +             config.MsgViewThisDayTimeFormat = config.ThisDayTimeFormat
> +     }
> +     if config.MsgViewThisWeekTimeFormat == "" {
> +             config.MsgViewThisWeekTimeFormat = config.ThisWeekTimeFormat
> +     }
> +     if config.MsgViewThisDayTimeFormat == "" {
> +             config.MsgViewThisDayTimeFormat = config.ThisDayTimeFormat
> +     }

Now that I think about it, this will result in a change of behaviour for
existing configs. The message view was previously using timestamp-format
for all emails, whatever their date. All these new settings should
default to timestamp-format to preserve existing behaviour.

> diff --git a/doc/aerc-config.5.scd b/doc/aerc-config.5.scd
> index 21472e6..36c1450 100644
> --- a/doc/aerc-config.5.scd
> +++ b/doc/aerc-config.5.scd
> @@ -121,6 +121,26 @@ These options are configured in the *[ui]* section of aerc.conf.
>
>       Default: ""
>
> +*message-timestamp-format*

This does not match `ini:"message-view-timestamp-format"`

> +     If set, overrides *timestamp-format* for the message view.
> +
> +     Default: ""
> +
> +*message-this-day-time-format*

This does not match `ini:"message-view-this-day-time-format"`

> +     If set, overrides *timestamp-this-day-time-format* for the message view.

Typo: s/timestamp-//

> +
> +     Default: ""
> +
> +*message-this-week-time-format*

This does not match `ini:"message-view-this-week-time-format"`

> +     If set, overrides *timestamp-this-week-time-format* for the message view.

Typo: s/timestamp-//

> +
> +     Default: ""
> +
> +*message-this-year-time-format*

This does not match `ini:"message-view-this-year-time-format"`

> +     If set, overrides *timestamp-this-year-time-format* for the message view.

Typo: s/timestamp-//

~rjarry 1 year, 7 months ago

Does it mean it worked? :)

~rjarry REPORTED FIXED 1 year, 7 months ago

Robin Jarry referenced this ticket in commit 9d71da1.

Register here or Log in to comment, or comment via email.