~eliasnaur/gio#192: 
Issues with the new NRGBA values

I'm getting too bright colors with the new NRGBA API on windows. For example, white with only Alpha = 1 (i.e color.NRGBA{R: 255, G: 255, B: 255, A: 1}) painted on full black, gives me R = 13, G = 13, B = 13. I check this by taking a screen capture and using painting software color picker. That's way too bright.

Is this some kind of gamma correction going bonkers? How to change that?

Status
RESOLVED BY_DESIGN
Submitter
~vsariola
Assigned to
No-one
Submitted
4 months ago
Updated
4 months ago
Labels
No labels applied.

~eliasnaur 4 months ago

Can you provide a small, runnable example?

~vsariola 4 months ago*

Sure. Here goes:

package main

import (
	"image"
	"image/color"
	"os"

	"gioui.org/app"
	"gioui.org/io/system"
	"gioui.org/layout"
	"gioui.org/op"
	"gioui.org/op/clip"
	"gioui.org/op/paint"
	"gioui.org/unit"
)

var black = color.NRGBA{R: 0, G: 0, B: 0, A: 255}
var translucent = color.NRGBA{R: 255, G: 255, B: 255, A: 1}

func main() {
	go func() {
		w := app.NewWindow(
			app.Size(unit.Dp(800), unit.Dp(600)),
		)
		var ops op.Ops
		for {
			select {
			case e := <-w.Events():
				switch e := e.(type) {
				case system.DestroyEvent:
					os.Exit(0)
				case system.FrameEvent:
					gtx := layout.NewContext(&ops, e)
					paint.FillShape(gtx.Ops, black, clip.Rect(image.Rect(0, 0, gtx.Constraints.Max.X, gtx.Constraints.Max.Y)).Op())
					paint.FillShape(gtx.Ops, translucent, clip.Rect(image.Rect(0, 0, gtx.Constraints.Max.X/2, gtx.Constraints.Max.Y)).Op())
					e.Frame(gtx.Ops)
				}
			}
		}
	}()
	app.Main()
}

And the results I get: https://ibb.co/10dFb23

The left side is R: 13, B: 13, G: 13

~eliasnaur REPORTED FIXED 4 months ago

I believe your issue is fixed by gioui.org/commit/01d5e72. Let me know if it isn't.

~eliasnaur 4 months ago*

Egon Elbre convinced me my "fix" was in fact wrong. See https://gioui.org/commit/99bfa6a. I now see this issue as an artifact of linear alpha values. From your program, using the sRGB formulas from http://ssp.impulsetrain.com/gamma-premult.html, https://www.khronos.org/registry/OpenGL/extensions/EXT/EXT_sRGB.txt, and https://www.khronos.org/registry/OpenGL-Refpages/gl4/html/glBlendFunc.xhtml:

The NRGA(255, 255, 255, 1) sRGB color is converted to (1/255, 1/255, 1/255, 1/255) pre-multiplied linear color. Our blend funcs are (1, 1-srcAlpha), so blending against your NRGBA(0, 0, 0, 255) converted to (0,0, 0, 1) pre-multiplied linear color gives: (1/2551 + 0(1-1/255), ..., (1/255)1+1(1-1/255)) = (1/255,..., 1). Converting that back to sRGB is NRGBA(12.92, ..., 255), rounded to the (13, 13, 13, 255) you see as a result.

~vsariola 4 months ago

I find these gammas always confusing :) So apologies if I'm not grasping all the issues. But if there's final ^1/2.2, why isn't there initial ^2.2 when going from NRGBA to pre-multiplied linear? Would the last system for storing color proposed in http://ssp.impulsetrain.com/gamma-premult.html solve this problem?

Anyways, I'm doing material.io style dark theme overlays, and there often the recommendation is to apply e.g. a 5% white overlay, but I find it hard to apply these translucent overlays with the new NRGBA system because even alpha = 1 white is so bright, so I don't have the granularity to apply the amount of color I'd want.

~vsariola 4 months ago*

(removed double post, sourcehut is going old school on me for clicking submit twice)

~eliasnaur 4 months ago

I find these gammas always confusing :) So apologies if I'm not grasping all the issues. But if there's final ^1/2.2, why isn't there initial ^2.2 when going from NRGBA to pre-multiplied linear?

There is a ^2.2 applied, but only to the color values. In your NRGBA(255, 255, 255, 1) case, the gamma conversion is applied to 255 which is still 255.

Would the last system for storing color proposed in http://ssp.impulsetrain.com/gamma-premult.html solve this problem?

That's what we had with color.RGBA. We switched to NRGBA because of loss of precision issue (I'll let others supply details), and because it was non-intuitive and expensive to multiply alpha (you need to un-gamma, multiply, gamma all color components).

Anyways, I'm doing material.io style dark theme overlays, and there often the recommendation is to apply e.g. a 5% white overlay, but I find it hard to apply these translucent overlays with the new NRGBA system because even alpha = 1 white is so bright, so I don't have the granularity to apply the amount of color I'd want.

Would it help to use a less brighter white and a higher alpha?

I've notified the people more informed than myself in the Slack thread, https://gophers.slack.com/archives/CM87SNCGM/p1610723351011700, to comment here.

It's possible we'll need to change the color representation again, perhaps to a new sRGB-aware color type.

~vsariola 4 months ago

For a single overlay, I can of course e.g. set higher alpha, even 255, and just give the exact value I want. But with multiple overlays or overlaying on gray, I would want the color to always be towards brighter i.e. towards white, so my understanding is that R,G,B should be 255,255,255 for this to work correctly.

~eliasnaur FIXED REPORTED 4 months ago

Good points. I don't know the best solution, so I'll re-open this issue for discussion.

~egonelbre 4 months ago

So, I walked through the calculations:

// calculations using sRGB gamma approximation
// instead of the actual formula

Source  sRGB NRGBA       u8  : R:255, A:1

// Converting from sRGB u8 non-premul to linear (srgb/0xff)^2.2 * alpha
Source  linear premul    f32 : (255/255)^2.2 * 1/255, A:1/255
                               1/255, A:1/255

// Blending with 1, 1 - srcAlpha
Blended linear premul    f32 : (255/255)^2.2 * 1/255, A:1.0
                               1/255, A:1

// Converting from linear premul to sRGB post-premultiplied,  (linear premul / alpha)^1/2.2 * alpha
Blended srgb post premul f32 : (1/255 / A)^1/2.2 * A, A:1.0
                               (1/255)^1/2.2, A:1.0
                               0.08

// Converting from linear premul to sRGB pre-premultiplied,  (linear premul)^1/2.2
Blended srgb pre  premul f32 : (1/255)^1/2.2, A:1.0
                               0.08, A:1.0

// Converting it to uint8, srgb * 255
Blended srgb * premul   u8 : 255 * 0.08 = 20.4

From pure calculation point of view and color blending view, what Gio does to colors -- seem to be correct. I think the fundamental difference comes from sRGB vs linear blending.

If we switch to non-linear blending, then it would end up with issues, such as: https://blog.johnnovak.net/2016/09/21/what-every-coder-should-know-about-gamma/

However, there's still the issue how to nicely blend colors to get different effects.

With my experiments, using alpha overlays you can easily end up with muted colors (saturation drops), which ideally it shouldn't. You could still do "small alpha" overlays with additive blending, but that would have similar problems.

For themes, I think the solution is to either:

  1. predefine the exact colors you want to use (rather than overlays)
  2. derive the color from the palette (such as https://git.sr.ht/~eliasnaur/gio/tree/main/item/internal/f32color/rgba.go#L164)

In the case of dark theme you would interpolate towards the accent color or towards white, depending on the usecase. Such approach would give you nicer colors.

But, I do agree that overlays with lower lightness or different color formats could be useful in some other cases. One option would be to pass in a more precise color paint.Color32Op or add some extra "alpha" for all the drawing, e.g. paint.AlphaOp.

~eliasnaur 4 months ago

Thanks for the detailed answer, Egon. Would color.RGBA make any difference in this case? I assume yes, and if so, do you still think the color.NRGBA colorspace is the correct choice for Gio?

Do any of you know what other GUI libraries/browsers do about loss of precision problems?

~egonelbre 4 months ago

Would color.RGBA make any difference in this case?

I don't think it would make a difference when you use color.RGBA as Gio colorspace input. The first conversions would be:

Source  sRGB RGBA        u8  : R:(255*1/255)^1/2.2, A:1
                             : R:1, A:1

// Converting from sRGB u8 non-premul to linear ((srgb/0xff)/alpha)^2.2 * alpha
Source  linear premul    f32 : ((1/255) / (1/255))^2.2 * 1/255, A:1/255
                               1/255, A:1/255

Since the linear premultiplied color is the same the blending will happen in a similar fashion.

Using color.NRGBA64 would allow to specify higher precision.

Do any of you know what other GUI libraries/browsers do about loss of precision problems?

  1. Do color blending using sRGB instead of linear (browsers, flutter, most UI-s).
  2. Use higher precision (usually in photo editing tools you can choose RGB/8, RGB/16, RGB/32).
  3. Use floating values (it's not uncommon to pass vec4 for colors in shaders).
  4. Allow switching between sRGB and Linear modes (svg https://developer.mozilla.org/en-US/docs/Web/SVG/Attribute/color-interpolation)

~eliasnaur 4 months ago

I don't understand your second calculation. Why are you mentioning "non-premul"?

// Converting from sRGB u8 non-premul to linear ((srgb/0xff)/alpha)^2.2 * alpha
Source  linear premul    f32 : ((1/255) / (1/255))^2.2 * 1/255, A:1/255
                               1/255, A:1/255

I'd say the correct calculation is

// Converting from sRGB u8 premul to linear (srgb/0xff)^2.2
Source  linear premul    f32 : (1/255)^2.2, A:1/255

and 1/255 is != (1/255)^2.2.

This is what https://gioui.org/commit/99bfa6a (incorrectly) implemented, and it did change OP's blending problem to output (1, 1, 1) as one would (naively) expect.

~egonelbre 4 months ago

Yes, it should've been "sRGB u8 premul to linear". Typo from copy-paste. And, yes, looks like I made a mistake in the formulas and calculations, however I made two mistakes, so it cancelled out :D.

Here's trying to redo the calculations:

// The definitions:
linear-premul = linear-non-premul * alpha
srgb = (linear-non-premul * alpha)^1/2.2 * 255
==>
linear-non-premul = (srgb / 255)^2.2 / alpha
linear-premul =  (srgb / 255)^2.2 / alpha * alpha = (srgb / 255)^2.2

linear-non-premul = 1.0, alpha=1/255
srgb = (1.0 * 1/255)^1/2.2 * 255
linear-premul = (((1.0 * 1/255)^1/2.2 * 255) / 255)^2.2 = 1/255

~vsariola 4 months ago*

I still quite don't see where the original 13 or the 12.92 in Elias' reply comes from... Following Elias' original post, the last conversion is from pre-multiplied linear 1/255 to NRBGA, which to my understanding should be (1/255)**(1/2.2) * 255 = 20.54 ~= 21. This is (close) to Egon's 20.4. Although if Gio would actually output 21, that would be even worse for me though, if 13 is already bright for my taste :)

Also, I don't quite see how "palettes" can be used in a generalized way, because I am applying highlights on stuff like pictures, icons and text etc., so the highlight needs be some kind of blending operation. It gets very complicated for me if every icon needs to know if the icon is being highlighted.

I'm surprised that there are performance issues - I would've guessed GPU handles all the blending and they have come a long way, even on mobile. Is standardizing all painting to linear float32 color too expensive (e.g. for mobiles)?

I read the what-every-coder-should-know-about-gamma article - the examples put forward are persuasive & I already knew weird stuff happens to pictures and gradients if blending is done without taking gamma into account. But in my use case (highlighting), blending without taking gamma into account could actually be the desired behavior - I don't care about color accuracy, but that there is always a perceptible difference between two colors.

All in all, I will probably manage with the new NRGBAs, by using a selective combination of the solutions proposed here (absolute colors instead of blended colors, accepting the low granularity and choosing colors around that etc.) Just takes me longer time to refactor than changing some color values.

~egonelbre 4 months ago

I still quite don't see where the original 13 or the 12.92 in Elias' reply comes from.

The actual linear <-> sRGB conversion isn't a simple exponentiation. See the formulas in https://en.wikipedia.org/wiki/SRGB#Specification_of_the_transformation. The exponentiation is easier to show in formulas.

Also, I don't quite see how "palettes" can be used in a generalized way, because I am applying highlights on stuff like pictures, icons and text etc., so the highlight needs be some kind of blending operation. It gets very complicated for me if every icon needs to know if the icon is being highlighted.

I'm not quite understanding which part of the material design you are implementing. So hard to recommend something specific.

Is standardizing all painting to linear float32 color too expensive (e.g. for mobiles)?

There's a performance overhead, but there's also the memory usage for the buffers. For example, iPhone it's 11.85MB vs 47.6MB for a single screen-sized buffer.

~eliasnaur 4 months ago

On Sat Jan 16, 2021 at 7:21 PM CET, ~egonelbre wrote:

Yes, it should've been "sRGB u8 premul to linear". Typo from copy-paste. And, yes, looks like I made a mistake in the formulas and calculations, however I made two mistakes, so it cancelled out :D.

Here's trying to redo the calculations:

// The definitions:
linear-premul =
linear-non-premul * alpha
srgb = (linear-non-premul * alpha)^1/2.2 *
255
==>
linear-non-premul = (srgb / 255)^2.2 / alpha
linear-premul =
(srgb / 255)^2.2 / alpha * alpha = (srgb / 255)^2.2

linear-non-
premul = 1.0, alpha=1/255
srgb = (1.0 * 1/255)^1/2.2 * 255
linear-
premul = (((1.0 * 1/255)^1/2.2 * 255) / 255)^2.2 = 1/255

I think we're both right, but under different assumptions. I'm assuming ~vsariola would represent the color.NRGBA(255, 255, 255, 1) overlay in pre.multiplied color.RGBA as (1, 1, 1, 1), and you seem to assume it will be represented as (c, c, c, 1), where c=(1.0*1/255)^(1/2.2)*255 =~ 21 (or =~ 13 with the EXT_sRGB formula). color.RGBA(1, 1, 1, 1) converted to linear results in (l, l, l, 1/255) where l=(1/255)^2.2 under my assumption, and l=1/255 under your assumption.

If I'm right (am I?), the problem is that there is a loss of precision with NRGBA that doesn't exist in RGBA (because the alpha is pre-multiplied before gamma adjustment). That's a serious flaw of NRGBA and I wonder what precision issues exist in RGBA that don't exist in NRGBA. If there are none/fewer/less severe I think we should consider switching back to RGBA, and suffer the performance penalty and non-intuitiveness of alpha-multiplying a color.RGBA value.

~egonelbre 4 months ago

color.RGBA(1, 1, 1, 1) isn't a white color with transparency, it's gray with transparency (assuming it's 255 * (linear-non-premul * alpha)^1/2.2 variant).

x := color.RGBA{1,1,1,1}
fmt.Println(f32color.RGBAToNRGBA(r))
// Output: {79 79 79 1}

As a general statement, color.NRGBA is more or as precise as color.RGBA since it doesn't multiply the colors with alpha, which can only "make numbers smaller" (i.e. smaller precision). Alpha itself isn't gamma adjusted, hence it wouldn't encode it into rest of the color components.

However, you can represent colors that are "brighter than 255 and transparent" with "color.RGBA", since you write things like: color.RGBA{255, 255, 255, 1}.

~egonelbre 4 months ago

Ah, I know how to illustrate the issue better. Roughly speaking the issue is in using linear u8 for alpha component together with linear blending. First take a look at the https://blog.johnnovak.net/2016/09/21/what-every-coder-should-know-about-gamma/#light-emission-vs-perceptual-brightness.

Moving from the left-side of the gradient if you take a "1 unit" step to the right. The value on "linear gradient" is much brighter than the "sRGB". The solutions are effectively:

  1. make a smaller step (by using uint16, float32 or srgb encoding for alpha)
  2. use sRGB "gradient" in the first place

~eliasnaur 4 months ago

On Sun Jan 17, 2021 at 10:03 AM CET, ~egonelbre wrote:

color.RGBA(1, 1, 1, 1) isn't a white color with transparency, it's gray with transparency (assuming it's 255 * (linear-non-premul * alpha)^1/2.2 variant).

x := color.RGBA{1,1,1,1}

fmt.Println(f32color.RGBAToNRGBA(r))
// Output: {79 79 79 1}

That's a good point I completely overlooked. ~vsariola, besides the unintuitive blending result, does color.NRGBA{79,79,79,1} (or something darker than 255) solve your issues? And if not, does switching to color.RGBA somehow fix your issues? If not, this issue is about switching to a higher precision, for example NRGBA64. We're bound to have to deal with higher precision anyway, given the fancy newer HDR formats which use more than 8 bits per color channel all the way through to the framebuffer.

Great discussion, thanks.

~egonelbre 4 months ago

... does color.NRGBA{79,79,79,1} (or something darker than 255) solve your issues?

When blending that value with colors that are brighter than 79, it would end up making them darker and not lighter.

~vsariola 4 months ago

First of all, do not change the color system just because what I was trying to do originally, as I will be able to overcome this by using a combination of the tricks listed here: absolute A = 255 colors whenever possible; just applying the highlights to background colors manually etc.

But carefully doing the math I am coming to the conclusion that actually I would prefer additive blending directly in sRGB, without any gamma correction. So, neither NRGBA or RGBA are optimal for this, because they both have gamma correction. The old RGBA had the advantage that I could apply very subtle (e.g. R = 10) highlight to black with A set to 0 (purely additive blending). But, because of the gamma correction, this highlight would not actually be visible on medium gray at all, so it's not what I would want.

For what I was originally trying to do (highlighting), the best blending would be additive blending directly in the sRGB, with no gamma correction whatsoever. This would make the highlights to be always ~ equally perceptible. For example: applying a subtle highlight of 10 units, black R = 0 would become R = 10 and medium gray R = 128 would become R = 138; these being the pixel sRGB values on screen. This would always make the highlight perceptible (as long as R <= 245; larger values would start to saturate). Hue shifts are a non-issue, as the GUI is mostly gray anyway.

The original RGBA would allow to do this, by setting A = 0, IF the gamma correction was dropped altogether. But dropping gamma permanently is definitely going to backfire, as soon as someone tries blending multicolor bitmaps etc. And adding BlendOp or GammaOp sound like overkill just for this & would add further complexity to Gio, which might not be just worth it. So, I'm not sure what to make of all this, except I am shocked to find myself arguing for NOT doing gamma correction for the first time in my life :)

~eliasnaur REPORTED FIXED 4 months ago

I wouldn't change the color space just for your use-case, unless it had uncovered a fundamental weakness of color.NRGBA compared to color.RGBA. I'm going to close this issue (again), and let some other work or issue deal with advanced blending and colorspaces, both a long way from my expertise.

~eliasnaur FIXED REPORTED 4 months ago

~eliasnaur REPORTED BY_DESIGN 4 months ago

Register here or Log in to comment, or comment via email.