In Metal on iOS the default colorPixelFormat is bgra8Unorm. When I change format to rgba16Float all imagery brightens. Why?
An example:
Artwork
MTKView with format bgra8Unorm.
Texture-mapped quad. Texture created with SRGB=false.
MTKView with format rgba16Float.
Texture-mapped quad. Texture created with SRGB=false.
Why is everything brighter with rgba16Float. My understanding is that SRGB=false implies that no gamma correction is done when importing artwork. The assumption is the artwork has no gamma applied.
What is going on here?
If your artwork has a gamma (it does per the first image you uploaded), you have to convert it to a linear gamma if you want to use it in a linear space.
What is happening here is you are displaying gamma encoded values of the image in a linear workspace, without using color management or transform to convert those values.
BUT: Reading some of your comments, is the texture not an image but an .svg?? Did you convert your color values to linear space?
Here's the thing: RGB values are meaningless numbers unless you define how those RGB values relate to a given space.
#00FF00 in sRGB is a different color than #00FF00 in Adobe98 for instance. In your case you are going linear, but what primaries? Still using sRGB primaries? P3 Primaries? I'm not seeing a real hue shift, so I assume you are using sRGB primaries and a linear transfer curve for the second example.
THAT SAID, an RGB value of the top middle kid's green shirt is #8DB54F, normalized to 0-1, that's 0.553 0.710 0.310 .These numbers by themselves don't know if they are gamma encoded or not.
THE RELATIONSHIP BETWEEN sRGB, Y, and Light:
For the purposes of this discussion, we will assume the SIMPLE sRGB gamma of 1/2.2 and not the piecewise version. Same for L*
In sRGB, #8DB54F when displayed on an sRGB monitor with a sRGB gamma curve, the luminance (Y) is 39
This can be found by
(0.553^2.2)*0.2126 + (0.710^2.2)*0.7152 + (0.310^2.2)*0.0722
or 0.057 + 0.33 + 0.0061 = 0.39 and 0.39 * 100 = 39 (Y)
But if color management is told the values are linear, then the gamma correction is discarded, and (more or less):
0.553*0.2126 + 0.710*0.7152 + 0.310*0.0722
or 0.1175 + 0.5078 + 0.0223 = 0.65 and 0.65 * 100 = 65 (Y)
(Assuming the same coefficients are used.)
Luminance (Y) is linear, like light. But human perception is not, and neither are sRGB values.
Y is the linear luminance from CIEXYZ, while it is spectrally weighted based on the eye's response to different wavelengths, it is NOT uniform in terms of lightness. On a scale of 0-100, 18.4 is perceived as the middle.
L* is a perceptual lightness from CIELAB (L* a* b*), it is (simplified curve of):
L* = Y^0.42 On a scale of 0-100, L* 50 is the "perceived middle" value. So that green shirt at Y 39 is L* 69 when interpreted and displayed as sRGB, and the Y 65 is about L* 84 (those numbers are based on the math, here are the values per the color picker on my MacBook):
sRGB is a gamma encoded signal, done to make the best use of the limited bit depth of 8bits per channel. The effective gamma curve is similar to human perception so that more bits are used to define darker areas as human perception is more sensitive to luminance changes in dark regions. As noted above it is a simplified curve of:
sRGB_Video = Linear_Video^0.455 (And to be noted, the MONITOR adds an exponent of about 1.1)
So if 0% is black and 100% is white, then middle gray, the point most humans will say is in between 0% and 100% is:
Y 18.4%. = L* 50% = sRGB 46.7%
That is, an sRGB hex value of #777777 will display a luminance of 18.4 Y, and is equivalent to a perceived lightness of 50 L*. Middle Grey.
BUT WAIT, THERE'S MORE
So what is happening, you are telling MTKView that you are sending it image data that references linear values. But you are actually sending it sRGB values which are lighter due to the applied gamma correction. And then color management is taking what it thinks are linear values, and transforming them to the needed values for the output display.
Color management needs to know what the values mean, what colorspace they relate to. When you set SRGB=false then you are telling it that you are sending it linear values, not gamma encoded values.
BUT you are clearly sending gamma encoded values into a linear space without transforming/decoding the values to linear. Linearization won't happen unless you implicitly do so.
SOLUTION
Linearize the image data OR set the flag SRGB=true
Please let me know if you have further questions. But also, you may wish to see the Poynton Gamma FAQ or also the Color FAQ for clarification.
Also, for your grey: A linear value of 0.216 is equivalent to an sRGB (0-1) value of 0.500
Related
I'm developing an app that does image processing.
In this app I need to apply Linear to Apple Display P3 color conversion to my texture (Linear). The operation will have to be done using shaders, I have already created a shader using these equations from Apple documentation archive that convert from Linear to sRGB and vice versa:
rgb = mix(rgb.0.0774, pow(rgb*0.9479 + 0.05213, 2.4), step(0.04045, rgb))
rgb = mix(rgb12.92, pow(rgb*0.4167) * 1.055 - 0.055, step(0.00313, rgb))
My question is what would the equation be for converting linear to Apple Display P3 color space and vice versa?
The conversion between linear and encoded RGB in the Display P3 color space is the same as for sRGB.
Note that DCI-P3 and Display P3 are not the same color space, even though they use the same red, green, and blue points (the former uses a simple power function for conversion between linear and encoded RGB; for example, pow(rgb, 2.2) for conversion to linear).
Note also that "linear RGB" is an incomplete description of a texture's colors. Rather, the texture has to be in some known color space (such as sRGB or DCI-P3); this is often sRGB if the texture has no embedded color profile. In many cases, "linear RGB" simply means that the corresponding color space's transfer function was not applied to the texture's RGB colors.
Assume we have a photo taken under insufficient lighting condition. The image is darker than usual but still recognizable.
Now we want to make it brighter so it looks like taken under sufficient lighting condition.
Should we convert the image into YUV and tune the Y channel (luminance), or convert to HSL and tune the L channel (brightness)?
The wording seems similar to me, while their formula differs a lot:
Y: 0.299*R + 0.587*G + 0.114*B
L: 0.5*(max + min), while max/min is the max/min value among RGB
EDIT:
More specifically, I am going to use opencv, cv2.cvtColor(), but unsure which input argument better suits my need: BGR2YUV or BGR2HLS
Tuning just Y and L (in YCbCr and HSL) will result in loosing information, like contrast between high pixel values. I will suggest either using some affine transformation on Y or L
255*(Y(x,y) - min(Y))/(max(Y) - min(Y))
or best would be to use histogram equalisation. It will not only give bright image, but with better contrast also, so it is good for visualisation
I wanna calculate the perceived brightness of an image and classify the image into dark, neutral and bright. And I find one problem here!
And I quote Lakshmi Narayanan's comment below. I'm confused with this method. What does "the average of the hist values from 0th channel" mean here? the 0th channel refer to gray image or value channel in hsv image? Moreover, what's the theory of that method?
Well, for such a case, I think the hsv would be better. Or try this method #2vision2. Compute the laplacian of the gray scale of the image. obtain the max value using minMacLoc. call it maxval. Estimate your sharpness/brightness index as - (maxval * average V channel values) / (average of the hist values from 0th channel), as said above. This would give you certain values. low bright images are usually below 30. 30 - 50 can b taken as ok images. and above 50 as bright images.
If you have an RGB color image you can get the brightness by converting it to another color space that separates color from intensity information like HSV or LAB.
Gray images already show local "brightness" so no conversion is necessary.
If an image is perceived as bright depends on many things. Mainly your display device, reference images, contrast, human...
Using a few intensity statistics values should give you an ok classification for one particular display device.
I'm getting all pixels' RGB values into
R=[],
G=[],
B=[]
arrays from the picture. They are 8 bits [0-255] values containing arrays. And I need to use Fourier Transform to compress image with a lossy method.
Fourier Transform
N will be the pixel numbers. n is i for array. What will be the k and imaginary j?
Can I implement this equation into a programming language and get the compressed image file?
Or I need to use the transformation equation to a different value instead of RGB?
First off, yes, you should convert from RGB to a luminance space, such as YCbCr. The human eye has higher resolution in luminance (Y) than in the color channels, so you can decimate the colors much more than the luminance for the same level of loss. It is common to begin by reducing the resolution of the Cb and Cr channels by a factor of two in both directions, reducing the size of the color channels by a factor of four. (Look up Chroma Subsampling.)
Second, you should use a discrete cosine transform (DCT), which is effectively the real part of the discrete Fourier transform of the samples shifted over one-half step. What is done in JPEG is to break the image up into 8x8 blocks for each channel, and doing a DCT on every column and row of each block. Then the DC component is in the upper left corner, and the AC components increase in frequency as you go down and to the left. You can use whatever block size you like, though the overall computation time of the DCT will go up with the size, and the artifacts from the lossy step will have a broader reach.
Now you can make it lossy by quantizing the resulting coefficients, more so in the higher frequencies. The result will generally have lots of small and zero coefficients, which is then highly compressible with run-length and Huffman coding.
What is the difference between HSI and HSV color space? I want to use HSI color space but I did not find any useful material for HSI. Is HSI the same as HSV?
HSI, HSV, and HSL are all different color spaces. Hue computation is (as far as I can find) identical between the three models, and uses a 6-piece piece-wise function to determine it, or for a simpler model that is accurate to within 1.2 degrees, atan((sqrt(3)⋅(G-B))/2(R-G-B)) can be used. For the most part, these two are interchangeable, but generally HSV and HSL use the piece-wise model, where HSI usually uses the arctan model. Different equations may be used, but these usually sacrifice precision for either simplicity or faster computation.
For lightness/value/intensity, the three spaces use slightly different representations.
Intensity is computed by simply averaging the RGB values: (1/3)⋅(R+G+B).
Lightness averages the minimum and maximum values for RGB: (1/2)⋅(max(R,G,B) + min(R,G,B)).
Value is the simplest, being the value of the maximum of RGB: max(R,G,B).
When used in subsequent calculations, L/V/I is scaled to a decimal between 0 and 1.
Saturation is where the three models differ the most. For all 3, if I/V/L is 0, then saturation is 0 (this is for black, so that its representation is unambiguous), and HSL additionally sets saturation to 0 if lightness is maximum (because for HSL maximum lightness means white).
HSL and HSV account for both the minimum and maximum of RGB, taking the difference between the two: max(R,G,B) - min(R,G,B), this value is sometimes referred to as chroma (C).
HSV then takes the chroma and divides it by the value to get the saturation: C/V.
HSL divides chroma by an expression taking lightness into account: C/(1-abs(2L-1)).
HSI doesn't use chroma, instead only taking min(R,G,B) into account: min(R,G,B)/I.
Sources
Wikipedia: HSL and HSV
Wikipedia: Hue
From the mathematical formula, the Hues are the same for HSV and HSI when you are trying to make the conversion from RGB to one of them.
Saturation in HSL is dependent on max, min, and Lightness, while HSV's Saturation is only max and min dependent. (max and min are the maximum and minimum pixel value among R, G, B space).
Value is max while the Lightness is (max + min)/2
Appendix: RGB->HSV, RGB->HSL