I'm viewing a message fullscreened ( ~ 300 columns for my display). If I have to scroll an email message, I experience 2-3 seconds of latency.
Essentially, I'm just scrolling up and down and seeing how long before the key-press takes effect. As I start decreasing the screen width, the latency is reduced. Somewhere around 120-80 columns it becomes acceptable.
I also notice 100% cpu utilization of a core while I'm seeing this latency (just crudely watching htop and aerc during this).
I tried two different pagers (less and bat), and they both exhibited similar behavior (while they work fine when not running under aerc for large column numbers).