Should a WebGL fragment shader output gl_FragColor RGB values which are linear, or to some 1⁄γ power in order to correct for display gamma? If the latter, is there a specific value to use or must a complete application be configurable?
The WebGL Specification does not currently contain “gamma”, “γ”, or a relevant use of “linear”, and the GL_ARB_framebuffer_sRGB extension is not available in WebGL. Is there some other applicable specification? If this is underspecified, what do current implementations do? A well-sourced answer would be appreciated.
(Assume we have successfully loaded or procedurally generated linear color values; that is, gamma of texture images is not at issue.)
This is a tough one, but from what I've been able to dig up (primarily from this email thread) it seems that the current behavior is to gamma correct linear color space images(such as PNGs) as they are loaded. Things like JPEG get loaded without transformation of any sort because they are already gamma corrected. (Source: https://www.khronos.org/webgl/public-mailing-list/archives/1009/msg00013.html) This would indicate that textures may possibly be passed to WebGL in a non-linear space, which would be problematic. I'm not sure if that has changed since late 2010.
Elsewhere in that thread it's made very clear that the desired behavior should be that everything input and output from WebGL should be in a linear color space. What happens beyond that is outside the scope of the WebGL spec (which is why it's silent on the issue).
Sorry if that doesn't authoritatively answer your question, I'm just digging up what I can on the matter. As for the matter of wether or not you should be doing correction in a shader, I would say that the answer appears to be "no", since the WebGL output is going to be assumed to be linear, and attempting to self correct may lead to a double transformation of the color space.
When I mentioned this question on Freenode #webgl (June 29, 2012), Florian Boesch vigorously expressed the opinion that nearly all users' systems are hopelessly misconfigured with regard to gamma, and therefore the only way to get good results is to provide a gamma option within a WebGL application, as even if WebGL specified a color space (whether linear or non-linear) for framebuffer data, it would not be correctly converted for the monitor.
Related
I had a good search before starting here, this question:
How to set RenderState in DirectX11?
is far too general; in studying the first answer, I suspect I need the Blend State, but it's not obvious how to set up an alpha comparison.
And searching stack overflow for D3DRS_ALPHAREF produced only seven other questions: https://stackoverflow.com/search?q=D3DRS_ALPHAREF none of which are even remotely close.
I'm using this for a program that does a two pass render to transition from one image to a second. I have a control texture that is the same size as the textures I'm rendering, and is single channel luminance.
The last lines of my pixel shader are:
// Copy rgb from the source texture
out.color.rgb = source.color.rgb;
// copy alpha from the control texture.
out.color.a = control.color.r;
return out;
Then in my render setup I have:
DWORD const reference = static_cast<DWORD>(frameNum);
D3DCMPFUNC const compare = pass == 0 ? D3DCMP_GREATEREQUAL : D3DCMP_LESS;
m_pd3dDevice->SetRenderState(D3DRS_ALPHAREF, reference);
m_pd3dDevice->SetRenderState(D3DRS_ALPHAFUNC, compare);
Where frameNum is the current frame number of the transition: 0 through 255.
-- Edit -- For those not intimately familiar with this particular capability of DirectX 9, the final stage uses the compare function to compare the alpha output from the pixel shader with the reference value, and then it actually draws the pixel iff the comparison returns a true value.
The net result of all this is that the luminance level of the control texture controls how early or late each pixel changes in the transition.
So, how exactly do I do this with DirectX 11?
Yes, I realize there are other ways to achieve the same result, passing frameNum to a suitably crafted pixel shader could get me to the same place.
That's not the point here, I'm not looking for an alternative implementation, I am looking to learn how to do alpha comparisons in DirectX 11, since they have proven a useful tool from time to time in DirectX 9.
If you are moving from Direct3D 9 to Direct3D 11, it is useful to take a brief stop at what changed in Direct3D 10. This is covered in detail on MSDN. One of the points in that article is:
Removal of Fixed Function
It is sometimes surprising that even in a Direct3D 9 engine that fully exploits the programmable pipeline, there remains a number of areas that depend on the fixed-function (FF) pipeline. The most common areas are usually related to screen-space aligned rendering for UI. It is for this reason that you are likely to need to build a FF emulation shader or set of shaders which provide the necessary replacement behaviors.
This documentation contains a white paper containing replacement shader sources for the most common FF behaviors (see Fixed Function EMU Sample). Some fixed-function pixel behavior including alpha test has been moved into shaders.
IOW: You do this in a programmable shader in Direct3D 10 or later.
Take a look at DirectX Tool Kit and in particular the AlphaTestEffect (implemented in this cpp and shader file).
If I want to clear an entire depth/stencil view in Direct3D 11, I can easily call ID3D11DeviceContext::ClearDepthStencilView.
Direct3D 11.1 adds support for clearing rectangular portions of render target views using ID3D11DeviceContext1::ClearView.
But I see no way to clear only a portion of a depth/stencil view in Direct3D 11, short of rendering a quad over the desired area? This seems like an odd regression from Direct3D 9, where this was trivially easy. Am I missing something, or is this really not supported?
There is no such function that can clear only a part of depth/stencil view.
This is my way to solve the problem:
make a texture. Set Alpha of the part to clear to 1,and other part to 0.
open AlphaTest, only the pixel whose alpha is 1.
open AlphaBlend,set BlendOP to Add,set SrcBlend factor to 0,set DestBlend factor to 1.
set StencilTest and DepthTest to Always, set StencilRef to the value you want to clear.
use orthogonal projection matrix.
draw a rectangle that just covers the screen( z-coordinate/(ZFar-ZNear) will convert to depth),and paste the texture on it.
There is an excellent reason at removing the partial clears in the API. First, it is always possible to emulate them by drawing quads with proper render states, and second, all GPUs have fast clear and resolve hardware. Using them in the intend logic greatly improve performance of the rendering.
With the DX11 clear API, it is possible to use the fast clear and the latter GPU optimisation. A depth buffer fast clear also prepare for an early depth test ( prior to pixel shading, because yes, the real depth test is post pixel shading ), and some bandwidth optimisation on access to the depth buffer will rendering. If you clear with a quad, you lost all that and draw cost will rise.
According to very few related topics that I could find I am gathering that the exponentiation step to obtain proper lighting computations perhaps must be done within the final fragment shader on an iOS app.
I have been profiling with the latest and greatest Xcode 5 OpenGL debugger and the exponentiation of the fragment accounts for a significant amount of computation. It was the line that took the longest time in the entire shader (the rest of the performance got sucked out by the various norm calls needed for point-lights).
glEnable(GL_FRAMEBUFFER_SRGB); unfortunately does not work as GL_FRAMEBUFFER_SRGB is not declared.
Of course the actual enum I should be using for GL ES may be different.
According to Apple:
The following extensions are supported for the SGX 543 and 554
processors only:
EXT_color_buffer_half_float
EXT_occlusion_query_boolean
EXT_pvrtc_sRGB
EXT_shadow_samplers
EXT_sRGB
EXT_texture_rg
OES_texture_half_float_linear
Well, that's nice, the newest device that does not have a 543 or 554 is the iPhone 4.
From the extension's text file it looks like I can set SRGB8_ALPHA8_EXT to the internalformat parameter of RenderbufferStorage, but nothing is said of how to get the normal final framebuffer to apply sRGB for us for free.
Now the sRGB correction seems like the missing step to get the correct colors. What I've been doing in my app to deal with the horrible "underexposed" colors is manually applying gamma correction like this in the fragment shader:
mediump float gammaf = 1.0/1.8; // this line declared outside of `main()`
// it specifies a constant 1.8 gamma
mediump vec4 gamma = vec4(gammaf, gammaf, gammaf, 1.0);
gl_FragColor = pow(color, gamma); // last line of `main()`
Now I recognize that the typical render pipeline involves one or more renders to a texture followed by a FS quad draw, which will afford me the opportunity to make use of an SRGB8_ALPHA_EXT renderbuffer, but what am I supposed to do without one? Am I SOL?
If that is the case, the pow call is sucking up so much time that it almost seems like I can squeeze some more perf out of it by building a 1D texture to sample and use as a gamma lookup table. This texture could then be used to tweak the output color intensities in custom ways (and get a much better approximation to sRGB compared to just the raw exponentiation). But it just all seems kind of wrong because supposedly sRGB is free.
Also somewhat alarming is the absence of any mention of the string srgb anywhere in the GL ES 2.0 spec. According to the makers of glm GL ES simply ignores sRGB entirely.
I know that I have used my code to render textures (I made a basic OpenGL powered image viewer that renders PVRTC textures) and they did not get "dimmed". I think what is happening there is that due to GL ES 2's lack of sRGB awareness, the textures are loaded in as being in linear space and written back out in the same way. In that situation, since no lighting gets applied (all colors got multiplied by 1.0) nothing bad happened to the results.
iOS 7.0 adds the new color format kEAGLColorFormatSRGBA8, which you can set instead of kEAGLColorFormatRGBA8 (the default value) for the kEAGLDrawablePropertyColorFormat key in the drawableProperties dictionary of a CAEAGLLayer. If you’re using GLKit to manage your main framebuffer for you, you can get GLKView to create a sRGB renderbuffer by setting its drawableColorFormat property to GLKViewDrawableColorFormatSRGBA8888.
Note that the OpenGL ES version of EXT_sRGB behaves as if GL_FRAMEBUFFER_SRGB is always enabled. If you want to render without sRGB conversion to/from the destination framebuffer, you’ll need to use a different attachment with a non-sRGB internal format.
I think you are getting confused between the EXT_sRGB and the ARB_framebuffer_sRGB extensions. The EXT_sRGB is the more recent extension, and is the one supported by iOS devices. This differs from ARB_framebuffer_sRGB in one important way, it is not necessary to call glEnable(GL_FRAMEBUFFER_SRGB) on the framebuffer to enable gamma correction, it is always enabled. All you need to do is create the framebuffer with an sRGB internal format and render linear textures to it.
This is not hugely useful on its own, as textures are rarely in a linear colour space. Fortunately the extension also includes the ability to convert sRGB textures to linear space. By uploading your textures with an internal format of sRGB8_ALPHA8_EXT, they will be converted into linear space when sampled in a shader for free. This allows you to use sRGB textures with a better perception encoded colour range, blend in higher precision linear space, then encode the result back to sRGB in the render buffer without any shader cost and accurate gamma correction.
Here are my test results. My only iOS7 device is an A7-powered iPad5, and in order to test fillrate I had to tweak my test app a bit to enable blending. That was sufficient on iOS 6.1 to prevent fragment-discarding optimizations on opaque geometry, but for iOS 7 I also needed to write gl_FragColor.w != 1.0 in the shader. Not a problem.
Using the GLKViewDrawableColorFormatSRGBA8888 does indeed appear to be free or close to free in terms of performance. I do not have a proper timedemo style benchmark setup so I am just testing "similar" scenes and the removal of the pow shaved around 2ms off the frame time (which would e.g. take 43ms to 41ms, 22 fps to 24 fps). Then, setting the sRGB framebuffer color format did not introduce a noticeable increase in the frame time as reported by the debugger, but this isn't very scientific and it certainly could have slowed it by a half a millisecond or so. I can't actually tell if it is completely free (i.e. fully utilizing a hardware path to perform final sRGB transformation) without first building more benching software, but I already have the problem solved so more rigorous testing will have to wait.
I'm using the EMGU OpenCV wrapper for c#. I've got a disparity map being created nicely. However for my specific application I only need the disparity values of very few pixels, and I need them in real time. The calculation is taking about 100 ms now, I imagine that by getting disparity for hundreds of pixel values rather than thousands things would speed up considerably. I don't know much about what's going on "under the hood" of the stereo solver code, is there a way to speed things up by only calculating the disparity for the pixels that I need?
First of all, you fail to mention what you are really trying to accomplish, and moreover, what algorithm you are using. E.g. StereoGC is a really slow (i.e. not real-time), but usually far more accurate) compared to both StereoSGBM and StereoBM. Those last two can be used real-time, providing a few conditions are met:
The size of the input images is reasonably small;
You are not using an extravagant set of parameters (for instance, a larger value for numberOfDisparities will increase computation time).
Don't expect miracles when it comes to accuracy though.
Apart from that, there is the issue of "just a few pixels". As far as I understand, the algorithms implemented in OpenCV usually rely on information from more than 1 pixel to determine the disparity value. E.g. it needs a neighborhood to detect which pixel from image A map to which pixel in image B. As a result, in general it is not possible to just discard every other pixel of the image (by the way, if you already know the locations in both images, you would not need the stereo methods at all). So unless you can discard a large border of your input images for which you know that you'll never find your pixels of interest there, I'd say the answer to this part of your question would be "no".
If you happen to know that your pixels of interest will always be within a certain rectangle of the input images, you can specify the input image ROIs (regions of interest) to this rectangle. Assuming OpenCV does not contain a bug here this should speedup the computation a little.
With a bit of googling you can to find real-time examples of finding stereo correspondences using EmguCV (or plain OpenCV) using the GPU on Youtube. Maybe this could help you.
Disclaimer: this may have been a more complete answer if your question contained more detail.
I am doing a bit of work on some of our HLSL shaders, trying to get them to work in SM2.0. I've nearly succeeded but one of our shaders accepts a parameter:
float alignment : VFACE
My understanding from MSDN is this is an automatic var calculated in case I need it, but it's not supported under SM2.0... so, how might I reproduce this? I'm not a shader programmer so any (pseudo) code would be really helpful. I understand what VFACE does, but not how I might calculate it myself in a pixel shader, or in a VS and pass it into the PS. Calculating it per-pixel sounds expensive so maybe someone can show a skeleton to calculate it in a VS and use it in a PS?
You can't. Because VFACE means orientation of the triangle (back or front) and the VS or PS stages have not access to the whole primitive (like in SM4/5 GS stage).
The only way is to render your geometry in two passes (one with back face culling, the other with front face culling) and pass a constant value to the shader matching VFACE meaning.