When layering multiple I have to combine them at least once with an "empty" image or background. But there is nothing like "empty". It's transparent black oder transparent white. Even being completely transparent has a color which is normally ignored when displaying.
I'd like to have the folloing:
Destination = Image1 + Image2
To do this I do in fact this:
Clear(Destination, 0); // Clear it with transparent black
Blend Image1 over Destination
Blend Image2 over Destination
Although Destination is transparent black the black contributes to the final result which is what I'd like to avoid. I could clear the destination with transparent white, but it would only work in this example and produces the same problems with darker images.
I understand why the result is like it is: The directx blend formular is set like that:
BlendOp = Add;
SrcBlend = SrcAlpha;
DestBlend = InvSrcAlpha;
The alpha of the destination (DestAlpha) isn't accounted at all in this formular. What can I do to implement this everyday problem of blending two images together before rendering them somewhere else?
Afaik you can't achieve this with normal blending operations, but maybe you could write a appropiate shader to do the blending for you. You render on an extra texture as a rendertarget and set this texture in the pixelshader. Now you branch your writed alpha value, if the background-alpha is zero you write your imagecolor, otherwise there is already an image written on this pixel and you can do the normal alphablending in your shader. With this branch you detect, whether the pixel is empty or written before and then prevent the black color to have any impact. Hope this helps :)
A little example to visualize what I mean:
float4 targetColor = tex2D(backgroundSampler,screen.TexCoord);
float4 diffuse = tex2D(imageSampler,image.TexCoord);
// overwrite black background
if (targetColor.a == 0.0) {
return diffuse;
}
// do alphablending
return targetColor*(1-diffuse.a)+diffuse*(diffuse.a);
Related
I am rendering video frames using Metal Core Image shaders. One of the requirements I have is to be able to pick a particular color (and user selected nearby range) from the CIImage, keep that color in the output and turn every other else black and white (color splash). But I am confused about the right approach that would work for videos shot in all kinds of color spaces (including 10 bit HDR):
First job is to extract the color value from the CIImage at any given pixel location. From my understanding, this can be extracted using the following API:
func render(_ image: CIImage,
toBitmap data: UnsafeMutableRawPointer,
rowBytes: Int,
bounds: CGRect,
format: CIFormat,
colorSpace: CGColorSpace?)
The API says passing NULL to colorSpace will cause the output to be in ciContext outputColorSpace. It's not clear how to correctly use this API to extract the exact color at given pixel locations, given the possibility of both 8 bit and 10 bit input images?
Having extracted the value, the next issue is how to pass the values to Metal Core Image shader? Shaders use normalized color ranges dependent on the workingcolorSpace of ciContext. Do I need to create a 1D texture with the color that should be passed to shader or there is a better way?
Based on your comment, here is another alternative:
You can read the pixel value as floats using the context's working color space. By using float values, you ensure that the bit depth of the input doesn't matter and that extended color values are correctly represented.
So for instance, a 100% red in BT.2020 would result in an extended sRGB value of (1.2483, -0.3880, -0.1434).
To read the value, you could use our small helper library CoreImageExtensions (or check out the implementation to see how to use render to get float values):
let pixelColor = context.readFloat32PixelValue(from: image, at: coordinate, colorSpace: context.workingColorSpace)
// you can convert that to CIVector, which can be passed to a kernel
let vectorValue = CIVector(x: pixelColor.r, y: pixelColor.g, ...)
In your Metal kernel, you can use a float4 input parameter for that color.
You can store and use the color value on later rendering calls as long as you are using the same workingColorSpace for the context.
I think you can achieve that without worrying about color spaces and even without the intermediate rendering step (which should speed up performance a lot).
You can simply crop your image to a 1x1 px square image that contains the specific color and make the image to extent virtually infinitely in all directions.
You can then pass that image into your next kernel and sample it anywhere to retrieve the color value (in the same color space as before).
let pixelCoordinate: CGPoint // the coordinate of the pixel that contains the color
// crop down to a single pixel
let colorPixel = inputImage.cropped(to: CGRect(origin: pixelCoordinate, size: CGSize(width: 1, height: 1))
// make the pixel extent infinite
let colorImage = colorPixel.clampedToExtent()
// simply pass it to your kernel
myKernel.apply(..., arguments: [colorImage, ...])
In the Metal kernel code, you can simply access it via sampler (or sample_t in a color kernel) and sample it like this:
// you can sample at any coord since the image contains the single color everywhere
float4 pickedColor = colorImage.sample(colorImage.coord());
To read the "original" color values from a CIImage, the CIContext used to render the pixels to the bitmap needs to be created with both workingColorSpace and outputColorSpace set to NSNull(). Then there will be no color space conversion and you don't have to worry about color spaces:
let context = CIContext(options: [.workingColorSpace: NSNull(), .outputColorSpace: NSNull()])
And then, when rendering the pixels to bitmap, specify the highest precision color format CIFormat.RGBAf to make sure you are not clipping any values. And use nil for colorSpace parameter. You will get 4 Float32 values per pixel that can be passed to the shader in CIVector as suggested by the first answer.
But here is another thing you can do, borrowing the cropping and clamping idea from the second Answer.
Create an infinite image that contains only the selected color using the approach suggested in that answer.
Crop that image to the frame's extent
Use CIColorAbsoluteDifference where one input is the original frame and another is this uniform color image.
The output of that filter will make all pixels that match the selected color exactly back, and none of the other pixels will be black, since this filter calculates the absolute difference between the colors and only pixels of exactly the same color will produce the (0,0,0) output.
Pass that image to the shader. If the color sampled from the image has exactly 0 in all its color components (ignore alpha) it means you need to copy the input pixel to the output intact. Otherwise set it to whatever you need it to be set (black or white or whatever).
I'm having trouble setting up blending in Metal. Even when starting with the Hello Triangle example provided by Apple, using the following code
pipelineStateDescriptor.colorAttachments[0].blendingEnabled = YES;
pipelineStateDescriptor.colorAttachments[0].sourceAlphaBlendFactor = MTLBlendFactorZero;
pipelineStateDescriptor.colorAttachments[0].destinationAlphaBlendFactor = MTLBlendFactorZero;
and the fragment function
fragment float4 fragmentShader(RasterizerData in [[stage_in]]) {
return float4(in.color.rgb, 0);
}
the triangle still draws completely opaque. What I want to achieve in the end is blending between two shapes by using different blending factors, but I thought I would start with a simple example to understand what is going on. What am I missing?
sourceAlphaBlendFactor and destinationAlphaBlendFactor are to do with constructing a blend for the alpha channel. i.e. they control the alpha that will be written into your destination buffer, which will not really be visible to you. You are probably more interested in the RGB that is written into the frame buffer.
Try setting values for sourceRGBBlendFactor and destinationRGBBlendFactor instead. For traditional alpha blending set sourceRGBBlendFactor to MTLBlendFactorSourceAlpha and set destinationRGBBlendFactor to MTLBlendFactorOneMinusSourceAlpha
I can't achieve correct color representation in my very basic Metal-project if its CAMetalLayer pixelFormat is setted to rgba8Unorm_srgb. By correct I mean that my UIColor's and UIImage's should appear the same in Metal like in UIKit. I have find out that it's exactly that if my CAMetalLayer pixelFormat is rgba8Unorm. But I really need to use rgba8Unorm_srgb.
I've attached two screenshot of this project. It has a squared CAMetalLayer. It's gray background color is its MTLRenderPassDescriptor.colorAttachments[0].clearColor = MTLClearColorMake(0.5, 0.5, 0.5, 1). It also have white and black quads and one another quad with red gradient above them, which actually uses standard alpha blending:
colorAttachments[0].rgbBlendOperation = .add
colorAttachments[0].alphaBlendOperation = .add
colorAttachments[0].sourceRGBBlendFactor = .sourceAlpha
colorAttachments[0].sourceAlphaBlendFactor = .sourceAlpha
colorAttachments[0].destinationRGBBlendFactor = .oneMinusSourceAlpha
colorAttachments[0].destinationAlphaBlendFactor = .oneMinusSourceAlpha
The only difference between the two screenshots is their CAMetalLayer pixelFormat. But as you can see even the clear color is already different. (They actually different exactly by the gamma-correction). White and black are the same, but all the other colors are different too. Alpha blending does't work correctly too on rgba8Unorm_srgb: you can see, that its half on the white destination is desaturated, and it half on the black destination is oversaturated.
From my understanding I don't need to worry about gamma-correction, don't need to modify my shaders or colors. Metal should do all the math in linear space regardless of pixelFormat. So why is that happening?
I'm just starting game development and I thought a game like Tank wars or Worms would be nice.
The hardest part I can think of so far is making the terrain destructible and I want to know how it's done before doing the easy parts.
I thought that explosion could have a mask texture which could be scaled for different weapons. Then using that mask I should make underlying terrain transparent (and optionally draw a dark border).
(source: mikakolari.fi)
How do I achieve that?
Do I have to change the alpha value pixel by pixel or can I use some kind of masking technique? Drawing a blue circle on top of the terrain isn't an option.
I have versions 3.1 and 4.0 of XNA.
This tutorial is what you are searching:
http://www.riemers.net/eng/Tutorials/XNA/Csharp/series2d.php
Capter 20: Adding explosion craters
In short:
You have 2 textures: 1 Color Texture (visible), 1 Collision Texture (invisible)
You substract the explosion image from your collision texture.
To get the dark border: expand the explosion texture and darken the color in this area.
Now you generate a new Color Texture (old color - collison = new color).
This is a difficult question to answer - because there are many ways you could do it. And there are pros and cons to each method. I'll just give an overview:
As an overall design, you need to keep track of: the original texture, the "darkness" applied, and the "transparency" applied. One thing I can say almost for sure is you want to "accumulate" the results of the explosions somewhere - what you don't want to be doing is maintaining a list of all explosions that have ever happened.
So you have surfaces for texture, darkness and transparency. You could probably merge darkness and transparency into a single surface with a single channel that stores "normal", "dark" (or a level of darkness) and "transparent".
Because you probably don't want the dark rings to get progressively darker where they intersect, when you apply an explosion to your darkness layer with the max function (Math.Max in C#).
To produce your final texture you could just write from the darkness/transparency texture to your original texture or a copy of it (you only need to update the area that each explosion touches).
Or you could use a pixel shader to combine them - the details of which are beyond the scope of this question. (Also a pixel shader won't work on XNA 4.0 on Windows Phone 7.)
You should Make a new Texure2D with the Color of desired pixels.Alpha = 0.
Color[] bits = new Color[Texture.Width * Texture.Height];
Texture.GetData(bits);
foreach(Vector2D pixel in overlapedArea)
{
int x = (int)(pixel.X);
int y = (int)(pixel.Y);
bits[x + y * texture.Width] = Color.FromNonPremultiplied(0,0,0,0));
}
Texture2D newTexture = new Texture2D(texture.GraphicsDevice, texture.Width, texture.Height);
newTexture.SetData(bits);
Now replace the new Texture2D with the Last Texture and you're good to go!
For more code about Collision, or changing texture pixels color go to this page for codes:
http://www.codeproject.com/Articles/328894/XNA-Sprite-Class-with-useful-methods
How can I alphablend only certain parts of a texture in DX 9?
For example, layers in Photoshop (or any other photo editing program that supports layers).
You can draw something in a layer (background filled with alpha), then place the layer over the original image (draw the texture on the screen) which leads to the original image + ONLY the things I drew in the layer.
Yes, I know my english is not very "shiny".
Thank you very much, in advance!
P.S. The background of my texture IS filled with alpha.
So you have setup the alpha on the texture you wish to overlay such that 0 is transparent (ie shows whats underneath) and 1 is opaque (ie shows the overlay texture)?
If so then you just need to set up a a simple blend mode:
pDevice->SetRenderState( D3DRS_SRCBLEND, D3DBLEND_SRCALPHA );
pDevice->SetRenderState( D3DRS_DESTBLEND, D3DBLEND_INVSRCALPHA );
pDevice->SetRenderState( D3DRS_ALPHABLENDENABLE, TRUE );
Make sure you draw the background first! Also note that values between 0 and 1 represent a linear interpolation between background and the overlay texture.