~emersion/grim#98: 
Blurry with fractional scaling

I notice that with sway and fractional scaling enabled, the produced images (PNG) are blurry, looking like being upscaled.

Running

grim -t png test.png

I get an image of size 3840 x 2159 (on a 4K monitor and and output scale set to 1.4), but blurry.

With output scale 1 or 2 both, I get an image of size 3840 x 2160 and no noticable difference to the original screen content.

Unfortunately I wouldn't know how to show a comparison between the screenshot and the original output, as I don't know of a way to obtain a 'correct' screenshot.

I' using sway 1.10, grim 1.4.1.

Status
REPORTED
Submitter
~omgold
Assigned to
No-one
Submitted
5 months ago
Updated
2 months ago
Labels
No labels applied.

~anarcat 2 months ago*

just as a curiosity: what are you using to see the screenshot?

we have a similar issue filed in shotman and there, the maintainer figured the issue might actually be with GTK-based image viewers which do not support fractional scaling. the solution, in that case, is to use another viewer: gwenview is an alternative that doesn't seem to suffer from such issues, interestingly.

~whynothugo 2 months ago

My display is 3440x1440. Taking a screenshot with grim -t png -o DP-1 test.png produces an image of size 3441x1440. The compositor is properly reporting this size for the screen copy too, and grim produces a buffer of the right size:

[4161393.921] {Default Queue} zwlr_screencopy_frame_v1#11.buffer(875709016, 3440, 1440, 13760)
[4161393.929] {Default Queue}  -> wl_shm#4.create_pool(new id wl_shm_pool#14, fd 6, 19814400)
[4161393.932] {Default Queue}  -> wl_shm_pool#14.create_buffer(new id wl_buffer#15, 0, 3440, 1440, 13760, 875709016)
[4161393.936] {Default Queue}  -> wl_shm_pool#14.destroy()
[4161393.938] {Default Queue}  -> zwlr_screencopy_frame_v1#11.copy(wl_buffer#15)

Even rendering with mpv --loop=inf Screenshot.from.2025-01-20.at.23_16_13.873859976.png, the image is quite blurry.


Note that imv always renders blurry images if you use fractional scale, and the screenshot looks much blurrier with imv. I'm sure there's a compound effect in this scenario.

~whynothugo 2 months ago

Above should read mpv --loop=inf test.png; I can't edit the message.

~anarcat 2 months ago

~whynothugo: i think it might be useful to share what your scaling is (1x? 1.5x? 2x?) here, and whether you're using sway or something else (i assume sway, i guess?)

also: i assume you don't get this with shotman?

~whynothugo 2 months ago

Using sway with scale 1.6. The scale should be irrelevant here, since the compositor informs the exact resolution of the capture and fills a buffer of that size.

~mstoeckl 2 months ago

The scale should be irrelevant here, since the compositor informs the exact resolution of the capture and fills a buffer of that size.

grim, in order to maintain relative (logical) output sizes when making multi-monitor screenshots, performs scale adjustments based on the output logical size. These scale and output buffer size calculations can have floating point rounding errors; specifically, in xdg_output_handle_done(), when output->logical_scale is computed; and in render(), when the screenshot dimensions common_width/common_height are computed, and when the image transform com2out is made.

To see what grim is doing internally, I've been using the following patch:

diff --git a/main.c b/main.c
index ed19e36..752259d 100644
--- a/main.c
+++ b/main.c
@@ -209,6 +209,7 @@ static void xdg_output_handle_done(void *data,
        int32_t height = output->geometry.height;
        apply_output_transform(output->transform, &width, &height);
        output->logical_scale = (double)width / output->logical_geometry.width;
+       fprintf(stderr, "computing logical scale: %d/%d -> %f\n", width, output->logical_geometry.width, output->logical_scale);
 }
 
 static void xdg_output_handle_name(void *data,
@@ -702,6 +703,7 @@ int main(int argc, char *argv[]) {
 
                ++n_pending;
        }
+       fprintf(stderr, "Chosen scale: %f\n", scale);
 
        if (n_pending == 0) {
                fprintf(stderr, "supplied geometry did not intersect with any outputs\n");
diff --git a/render.c b/render.c
index e49e744..dc60091 100644
--- a/render.c
+++ b/render.c
@@ -144,6 +144,7 @@ pixman_image_t *render(struct grim_state *state, struct grim_box *geometry,
                double scale) {
        int common_width = geometry->width * scale;
        int common_height = geometry->height * scale;
+       fprintf(stderr, "Output image size; W: %d x %f -> %d, H: %d x %f -> %d\n", geometry->width, scale, common_width, geometry->height, scale, common_height);
        pixman_image_t *common_image = pixman_image_create_bits(PIXMAN_a8r8g8b8,
                common_width, common_height, NULL, 0);
        if (!common_image) {
@@ -175,6 +176,7 @@ pixman_image_t *render(struct grim_state *state, struct grim_box *geometry,
                int32_t raw_output_height = output->geometry.height;
                apply_output_transform(output->transform,
                        &raw_output_width, &raw_output_height);
+               fprintf(stderr, "Rendering image with transformed size; %d x %d\n", raw_output_width, raw_output_height);
 
                int output_flipped_x = get_output_flipped(output->transform);
                int output_flipped_y = output->screencopy_frame_flags &

which on a 2560x1600 display at scale 1.2 produces output like:

computing logical scale: 2560/2133 -> 1.200188
Chosen scale: 1.200188
Output image size; W: 2133 x 1.200188 -> 2560, H: 1333 x 1.200188 -> 1599
Rendering image with transformed size; 2560 x 1600

In this case, grim incorrectly produces a 2560x1599 image after the compositor gives it a 2560x1600 size buffer.

~whynothugo 2 months ago

grim, in order to maintain relative (logical) output sizes when making multi-monitor screenshots, performs scale adjustments based on the output logical size.

In the given example, the screenshot is a single-output screenshot: grim -t png -o DP-1 test.png.

I guess the root of the issue is doing all this calculation when it's not necessary.

~mstoeckl 2 months ago

I think there are two main directions to fix this issue:

  • Special case the single-output screenshot case to, by default, just store the provided buffer at the given size. (But then the rounding may still be slightly off if you have multiple monitors of the same size and fractional scale.)
  • Fix the scale calculations in the general case.
    • The easy approach is to add a round() call whenever floating point values are converted to integers. This will generally work when the ideal calculation result is an integer and all values involved are small, but is still slightly wrong in general: when the ideal result ends in .5, floating point error can make the rounding go either way.
    • You can get perfectly accurate results with floating point, if you a) use a wide enough floating point type to accurately represent all possible intermediate values (or limit the range of inputs to ensure this) b) use a stable rounding method (like round(x+epsilon), where epsilon depends on the details of the entire floating point calculation: too small and floating point error can influence the result; too large and you might round accurate values in the wrong direction. If code is rounding like this and doesn't explain the epsilon value, it is probably slightly wrong some of the time.) c) avoid numerically unstable operations (which grim already does). Something similar to this approach can be done with fixed point.
    • Express coordinates in rational arithmetic, with sufficiently high precision, and overflow checks. Doing this requires either using an existing library for it, or effectively writing your own mini-library; if the calculations are simple enough (which may be true for grim) the numerator and denominator will fit in a uint64_t and one will not need to reduce fractions.) To do this, the entire calculation of c2o_fixedpt in render would probably need to be rewritten, which (for lack of operator overloading) may be tedious and hard to review. Unlike the floating point approach, which may be broken by a benign-looking refactoring, using rational arithmetic should be harder to get wrong by accident, after the framework is set up.

~whynothugo 2 months ago

Special case the single-output screenshot case to, by default, just store the provided buffer at the given size. (But then the rounding may still be slightly off if you have multiple monitors of the same size and fractional scale.)

You can't have any rounding errors. If your screen resolution is 3440x1440, the compositor will inform that size, you create a buffer with that size, and the compositor fills is with the exact same pixel data that is being displayed. You don't need to do any math, so it's impossible to have any rounding error.

~mstoeckl 2 months ago*

I was too imprecise with my earlier statement. Yes, making a special case for when a screenshot of a single output is taken will work correctly (even if there are multiple outputs); but handling the special case will not fix grim's rounding problems with screenshots that span multiple outputs.