When I try to implement TAA, I update the pixel color like this:
float4 PS(VertexOut pin) : SV_Target
{
float2 currPos = pin.PosH.xy * gInvRenderTargetSize;
float2 histPos = currPos + gVelocityBuffer.Sample(gsamPointClamp, currPos).xy;
float4 currPixel = gCurrFrameMap.Sample(gsamPointClamp, currPos);
float4 histPixel = gHistoryFrameMap.Sample(gsamPointClamp, histPos);
if(gFirstFrame == 1)
{
return currPixel;
}
float4 taaPixel = 0.05 * currPixel + 0.95 * histPixel;
return taaPixel;
}
But after some frames it just show the history pixel instead of the new pixel, making the render result dirty. How could I solve this?
This is the render target picture
The history frame
The current frame
The render target frame, and we can see that the pixel color do not change here
Using the 95% History/5% Current split, you should completely eliminate any ghosting artifacts after 20 frames.
Instead, you eventually end up returning just histPixel with no contribution from currPixel, which indicates that some amount of histPixel is ending up in currPixel.
I can't say for certain without a video, but I think you are using the previous displayed frame for gCurrFrameMap. This still allows a better quality image, but only 20 frames (with each frame contributing less and less) of improvement.
Also, Sample the four neighboring pixels and clamp histPixel with the min and max of currPixel and those four samples. It will eliminate ghosting artifacts without having to reset to currPixel at gFirstFrame.
After a few days I finally found the solution T_T.
This problem is caused by the low precision of DXGI_FORMAT_R8G8B8A8_UNORM, so the updated value and the previous value will have the same RGBA value. Now I changed it to DXGI_FORMAT_R16G16B16A16_FLOAT and it was solved properly.
Related
I am currently studying shadow mapping, and my biggest issue right now is the transformations between spaces. This is my current working theory/steps.
Pass 1:
Get depth of pixel from camera, store in depth buffer
Get depth of pixel from light, store in another buffer
Pass 2:
Use texture coordinate to sample camera's depth buffer at current pixel
Convert that depth to a view space position by multiplying the projection coordinate with invProj matrix. (also do a perspective divide).
Take that view position and multiply by invV (camera's inverse view) to get a world space position
Multiply world space position by light's viewProjection matrix.
Perspective divide that projection-space coordinate, and manipulate into [0..1] to sample from light depth buffer.
Get current depth from light and closest (sampled) depth, if current depth > closest depth, it's in shadow.
Shader Code
Pass1:
PS_INPUT vs(VS_INPUT input) {
output.pos = mul(input.vPos, mvp);
output.cameraDepth = output.pos.zw;
..
float4 vPosInLight = mul(input.vPos, m);
vPosInLight = mul(vPosInLight, light.viewProj);
output.lightDepth = vPosInLight.zw;
}
PS_OUTPUT ps(PS_INPUT input){
float cameraDepth = input.cameraDepth.x / input.cameraDepth.y;
//Bundle cameraDepth in alpha channel of a normal map.
output.normal = float4(input.normal, cameraDepth);
//4 Lights in total -- although only 1 is active right now. Going to use r/g/b/a for each light depth.
output.lightDepths.r = input.lightDepth.x / input.lightDepth.y;
}
Pass 2 (Screen Quad):
float4 ps(PS_INPUT input) : SV_TARGET{
float4 pixelPosView = depthToViewSpace(input.texCoord);
..
float4 pixelPosWorld = mul(pixelPosView, invV);
float4 pixelPosLight = mul(pixelPosWorld, light.viewProj);
float shadow = shadowCalc(pixelPosLight);
//For testing / visualisation
return float4(shadow,shadow,shadow,1);
}
float4 depthToViewSpace(float2 xy) {
//Get pixel depth from camera by sampling current texcoord.
//Extract the alpha channel as this holds the depth value.
//Then, transform from [0..1] to [-1..1]
float z = (_normal.Sample(_sampler, xy).a) * 2 - 1;
float x = xy.x * 2 - 1;
float y = (1 - xy.y) * 2 - 1;
float4 vProjPos = float4(x, y, z, 1.0f);
float4 vPositionVS = mul(vProjPos, invP);
vPositionVS = float4(vPositionVS.xyz / vPositionVS.w,1);
return vPositionVS;
}
float shadowCalc(float4 pixelPosL) {
//Transform pixelPosLight from [-1..1] to [0..1]
float3 projCoords = (pixelPosL.xyz / pixelPosL.w) * 0.5 + 0.5;
float closestDepth = _lightDepths.Sample(_sampler, projCoords.xy).r;
float currentDepth = projCoords.z;
return currentDepth > closestDepth; //Supposed to have bias, but for now I just want shadows working haha
}
CPP Matrices
// (Position, LookAtPos, UpDir)
auto lightView = XMMatrixLookAtLH(XMLoadFloat4(&pos4), XMVectorSet(0,0,0,1), XMVectorSet(0,1,0,0));
// (FOV, AspectRatio (1000/680), NEAR, FAR)
auto lightProj = XMMatrixPerspectiveFovLH(1.57f , 1.47f, 0.01f, 10.0f);
XMStoreFloat4x4(&_cLightBuffer.light.viewProj, XMMatrixTranspose(XMMatrixMultiply(lightView, lightProj)));
Current Outputs
White signifies that a shadow should be projected there. Black indicates no shadow.
CameraPos (0, 2.5, -2)
CameraLookAt (0, 0, 0)
CameraFOV (1.57)
CameraNear (0.01)
CameraFar (10.0)
LightPos (0, 2.5, -2)
LightLookAt (0, 0, 0)
LightFOV (1.57)
LightNear (0.01)
LightFar (10.0)
If I change the CameraPosition to be (0, 2.5, 2), basically just flipped on the Z axis, this is the result.
Obviously a shadow shouldn't change its projection depending on where the observer is, so I think I'm making a mistake with the invV. But I really don't know for sure. I've debugged the light's projView matrix, and the values seem correct - going from CPU to GPU. It's also entirely possible I've misunderstood some theory along the way because this is quite a tricky technique for me.
Aha! Found my problem. It was a silly mistake, I was calculating the depth of pixels from each light, but storing them in a texture that was based on the view of the camera. The following image should explain my mistake better than I can with words.
For future reference, the solution I decided was to scrap my idea for storing light depths in texture channels. Instead, I basically make a new pass for each light, and bind a unique depth-stencil texture to render the geometry to. When I want to do light calculations, I bind each of the depth textures to a shader resource slot and go from there. Obviously this doesn't scale well with many lights, but for my student project where I'm only required to have 2 shadow casters, it suffices.
_context->DrawIndexed(indexCount, 0, 0); //Draw to regular render target
_sunlight->use(1, _context); //Use sunlight shader (basically just runs a Vertex Shader & Null Pixel shader so depth can be written to depth map)
_sunlight->bindDSVSetNullRenderTarget(_context);
_context->DrawIndexed(indexCount, 0, 0); //Draw to sunlight depth target
bindDSVSetNullRenderTarget(ctx){
ID3D11RenderTargetView* nullrv = { nullptr };
ctx->OMSetRenderTargets(1, &nullrv, _sunlightDepthStencilView);
}
//The purpose of setting a null render target before doing the draw call is
//that a draw call with only a depth target bound is much faster.
//(At least I believe so, from my reading online)
In order to only have only one input image for a "spread like" effect I would like to do some threshold operation on some drawable or to find any other way that work.
Are there any such tools in LOVE2d/Lua ?
I'm not exactly sure about the desired outcome, the "spread like" effect, but to create thresholding, you best use pixel shader something like this.
extern float threshold; //external variable from our lua script
vec4 effect(vec4 color, Image tex, vec2 texture_coords, vec2 screen_coords)
{
vec4 texturecolor = Texel(tex, texture_coords); //default shader code
//get average color of pixel
float average = (texturecolor[0] + texturecolor[1] + texturecolor[2])/3;
//set alpha of pixel to 0 if average of RGB is below threshold
if (average < threshold) {
texturecolor[3] = 0;
}
return texturecolor * color; //default shader code
}
This code calculates the average of RGB for each pixel and if the average is below threshold, it changes alpha of that pixel to 0 to make it invisible.
To use the pixel effect in your code you need to do something like this (only once, perhaps in love.load):
shader = love.graphics.newShader([==[ ... shader code above ... ]==])
and when drawing the image:
love.graphics.setShader(shader)
love.graphics.draw(img)
love.graphics.setShader()
To adjust the threshold:
shader:send("threshold", number) --0 to 1 float
Result:
References:
LÖVE Shader object
love.graphics.newShader for examples of the default shader code
I want to read 3d model from fbx file and display it within openGL es 2.0 engine on iPhone (not using Unity) and also I want to display animations for read 3d object.
How can i get animations from fbx file?
Currently I able to get list of poses names with, as I understood, coming with transformation matrix, full list of poses in layer, stacks, layers, and a lot of curves.
How this all information can be combined for displaying proper animations?
I also try to parse some info within TakeInfo, but result a little bit strange as for me, for example:
FbxTakeInfo *ltakeInfo = pScene->GetTakeInfo(lAnimStack->GetName());
FbxTime start = ltakeInfo->mLocalTimeSpan.GetStart();
FbxTime end = ltakeInfo->mLocalTimeSpan.GetStop();
self.startTime = start.GetSecondDouble();
self.endTime = end.GetSecondDouble();
here i got start = 0 and end = 0.014 for each of parsed layers, so i guees this is incorrect (fbx file that I want to display contains 1 mesh with simple 5 sec duration animations).
Update
After few hours of investigating I got next things:
For reference this is structure of test obj that i want to display:
Here u can see a lot of bones (more concrete - 19)
I able to get (as noted above) a 19 animated obj in list (like names of bone/obj) and 19 groups of curves within 151 frames each (with frame rate 30 exactly 5 sec of anim. - 30*5=150 + 1 last identity matrix).
If I try use each of curve group to my mesh one by one (I able to parse 1 mesh only) I see animation of different part of mesh applied to all mesh (for example vertical rotation or horizontal translation), so I think that this curves in each group should be applied exactly to specific bone and as result i will got animation for my mesh. The problem is that i don't know how to animate only selected bone vertex part only.
Now the problem is how to apply this all animations divided in to separate group specific to bone to whole obj (because I have only one mesh)?
How can I get 1 global curve's list for each frame from list with all curve's groups?
Update2
Thanks to #codetiger advice I follow instruction within provided link in comment and with that technique I'm able to retrieve list of bone-specific mat's with start and end time and required transforms, but this is almost same like I got before with curves - the only difference that with curves I should create mat from 9 curves (translate/ scale/ rotate for x y z) instead of using full matrix, but problem are still present - how can i combine them in 1 global matrix?
The code that I use (found few links for it):
FbxNode* modelNode = _fbxScene->GetRootNode();
FbxAMatrix geometryTransform = GetGeometryTransformation(modelNode);
for (unsigned int deformerIndex = 0; deformerIndex < numOfDeformers; ++deformerIndex) {
FbxSkin* currSkin = reinterpret_cast<FbxSkin*>(mesh->GetDeformer(deformerIndex, FbxDeformer::eSkin));
if (!currSkin) {
continue;
}
unsigned int numOfClusters = currSkin->GetClusterCount();
for (unsigned int clusterIndex = 0; clusterIndex < numOfClusters; ++clusterIndex) {
FbxCluster* currCluster = currSkin->GetCluster(clusterIndex);
std::string currJointName = currCluster->GetLink()->GetName();
FbxAMatrix transformMatrix;
FbxAMatrix transformLinkMatrix;
FbxAMatrix globalBindposeInverseMatrix;
currCluster->GetTransformMatrix(transformMatrix);
currCluster->GetTransformLinkMatrix(transformLinkMatrix);
globalBindposeInverseMatrix = transformLinkMatrix.Inverse() * transformMatrix * geometryTransform;
FbxAnimStack* currAnimStack = _fbxScene->GetSrcObject<FbxAnimStack>(0);
FbxString animStackName = currAnimStack->GetName();
char *mAnimationName = animStackName.Buffer();
FbxTakeInfo* takeInfo = _fbxScene->GetTakeInfo(animStackName);
FbxTime start = takeInfo->mLocalTimeSpan.GetStart();
FbxTime end = takeInfo->mLocalTimeSpan.GetStop();
FbxLongLong mAnimationLength = end.GetFrameCount(FbxTime::eFrames24) - start.GetFrameCount(FbxTime::eFrames24) + 1;
for (FbxLongLong i = start.GetFrameCount(FbxTime::eFrames24); i <= end.GetFrameCount(FbxTime::eFrames24); ++i) {
FbxTime currTime;
currTime.SetFrame(i, FbxTime::eFrames24);
FbxAMatrix currentTransformOffset = modelNode->EvaluateGlobalTransform(currTime) * geometryTransform;
FbxAMatrix mat = currentTransformOffset.Inverse() * currCluster->GetLink()->EvaluateGlobalTransform(currTime);
}
}
}
Here I receive 121 matrix instead of 151 - but duration of some matrix transforms take more than duration of 1 frame draw, so the q-ty here I think also correct
float duration = end.GetSecondDouble() - start.GetSecondDouble(); //return 5 as and expected
I guess that that Autodesk SDK there is a way to get 1 Global transform per each drawCall
Any suggestions? Is this possible?
Add bounty for anyone who can describe how to display animations on iPhone within openGLES 2.0 with Autodesk SDK... (sorry typo instead of Facebook)
Here what i'm able to get
Original object in blender
If i draw bone separately (same VBO with different transform and only indices for each bone accordantly)
Here is how to render animated mesh in OpenGL ES. This will give you fare information on what details you need to read from the file.
Method 1: (GPU Skinning)
This method works only on limited number of bones based on the capabilities of the hardware. Usually it depends on the amount of matrices you can send to the shader.
Bind the mesh information to the GPU using BindAttributes once and send the matrixes to the shader in uniforms.
Step 1: Read all Bone Matrix and create an array of Matrices and send this data to the shader in uniforms.
Step 2: In the Vertex Shader, calculate the gl_position like the below shader
attribute vec3 vertPosition;
attribute vec3 vertNormal;
attribute vec2 texCoord1;
attribute vec3 vertTangent;
attribute vec3 vertBitangent;
attribute vec4 optionalData1;
attribute vec4 optionalData2;
uniform mat4 mvp, jointTransforms[jointsSize];
void main()
{
mat4 finalMatrix;
int jointId = int(optionalData1.x);
if(jointId > 0)
finalMatrix = jointTransforms[jointId - 1] * optionalData2.x;
jointId = int(optionalData1.y);
if(jointId > 0)
finalMatrix = finalMatrix + (jointTransforms[jointId - 1] * optionalData2.y);
jointId = int( optionalData1.z);
if(jointId > 0)
finalMatrix = finalMatrix + (jointTransforms[jointId - 1] * optionalData2.z);
jointId = int( optionalData1.w);
if(jointId > 0)
finalMatrix = finalMatrix + (jointTransforms[jointId - 1] * optionalData2.w);
gl_Position = mvp * finalMatrix * vec4(vertPosition, 1.0);
}
In this shader, I've send the weight paint information in the attributes optionalData1 & optionalData2. The data is packed like this: (BoneId1, BoneId2, BoneId3, BoneId4) and (Weight4Bone1, Weight4Bone2, Weight4Bone3, Weight4Bone4). So in this case you are limited with a maximum of 4 bones affecting each attributes. The fragment shader part is usual.
Method 2: (CPU Skinning)
If you cannot live with the limitation in the GPU skinning, then the only way is to do the calculations in the CPU side.
Step 1: Calculate the vertex position in the current frame by multiplying the matrixes that belong to the bones that affects the vertex.
Step 2: Collect the new positions for the current frame and send the information to GPU in Vertex Attributes.
Step 3: Recalculate new Vertex positions in every frame and update the attributes buffer.
This method bring the load of the calculations to the CPU.
I searched and couldn't find a solution to this (maybe I'm using the wrong terms?), and I feel a bit silly because I believe I'm overlooking something simple. Here's what I'd like to do:
I have an object in my game that needs to travel a specific x distance in a specified number of frames. But, I'd like to have it ease into the target point instead of moving the same distance/velocity every frame.
So rather than just dividing the number of frames by the distance, and applying that velocity each frame, I'd I'd like to slowly ease itself into the target coordinate in a specified number of frames.
So for example (I'm terrible at explaining, perhaps an example could help)... say I have a spaceship, and he needs to travel 32 pixels to the right. I'd like to input a value that specifies he'll travel 32 pixels in... say, 4 frames. In a linear system, he'd travel 8 pixels each frame, but I want him to ease into it, so maybe on frame 1 (and I'm using completely random values) he'd move 16 pixels, frame 2 he'd move 10, frame 3 he'd move 4, and frame 4 he'd move 2, and he'll end up traveling the 32 pixels distance in those 4 frames, slowly easing into the target point.
The first thing that came to mind was using exponent/logarithms somehow, but I'd like some suggestions. Any help would be greatly appreciated, thanks! :D
The general solution is the following:
You have a value (distanceTravelled) which has a range from 0.0 to 1.0.
You have a function (fancyCurve) which takes in a float from 0.0 to 1.0 and remaps it from 0.0 to 1.0, except in a curve.
Every frame, you increase distanceTravelled by a linear amount. But you get the actual value by calling fancyCurve(distanceTravelled).
Here's a pseudo-code example
float linearFunction(float t) { return t; }
float speedUpFunction(float t) { return t*t; }
float slowDownFunction(float t)
{
//do your own research. Theres plenty of curves from http://easings.net/
}
float easingCurve(float t) {
//choose one.
//return linearFunction(t);
//return speedUpFunction(t);
return slowDownFunction(t);
}
int main() {
//setting up a spaceship with starting x coordinate
spaceshipX = 2;
spaceshipTargetX = 34;
animationFrames = 8;
//Below is the actual algorithm
distanceTravelled = 0;
startValue = spaceshipX;
animRange = spaceshipTargetX - startValue; // 32
distancePerFrame = 1.0f / animationFrames; // 0.125, since 8 frames
while (distanceTravelled < 1.0f) {
WaitForEndOfFrame();
distanceTravelled += distancePerFrame;
spaceshipX = startValue + (easingCurve(distanceTravelled) * animRange);
}
}
I've been asked to split questions which I asked here:
HLSL and Pix number of questions
I thought two and three would both fit in the same question as a solution of one may help resolve the other. I'm trying to debug a shader and seem to be running into issues. Firstly Pix seems to be skipping a large amount of code when I'm running analyse mode. This is analysing an experiment with F12 captures and with D3DX analysis turned off. I have to turn it off as I'm using XNA. The shader code in question is below:
float4 PixelShaderFunction(float2 OriginalUV : TEXCOORD0) : COLOR0
{
// Get the depth buffer value at this pixel.
float4 color = float4 (0, 0,0,0);
float4 finalColor = float4(0,0,0,0);
float zOverW = tex2D(mySampler, OriginalUV);
// H is the viewport position at this pixel in the range -1 to 1.
float4 H = float4(OriginalUV.x * 2 - 1, (1 - OriginalUV.y) * 2 - 1,
zOverW, 1);
// Transform by the view-projection inverse.
float4 D = mul(H, xViewProjectionInverseMatrix);
// Divide by w to get the world position.
float4 worldPos = D / D.w;
// Current viewport position
float4 currentPos = H;
// Use the world position, and transform by the previous view-
// projection matrix.
float4 previousPos = mul(worldPos, xPreviousViewProjectionMatrix);
// Convert to nonhomogeneous points [-1,1] by dividing by w.
previousPos /= previousPos.w;
// Use this frame's position and last frame's to compute the pixel
// velocity.
float2 velocity = (currentPos - previousPos)/2.f;
// Get the initial color at this pixel.
color = tex2D(sceneSampler, OriginalUV);
OriginalUV += velocity;
for(int i = 1; i < 1; ++i, OriginalUV += velocity)
{
// Sample the color buffer along the velocity vector.
float4 currentColor = tex2D(sceneSampler, OriginalUV);
// Add the current color to our color sum.
color += currentColor;
}
// Average all of the samples to get the final blur color.
finalColor = color / xNumSamples;
return finalColor;
}
With a captured frame and when debugging a pixel I can only see two lines working. These are color = tex2D(sceneSampler, OriginalUV) and finalColor = color / xNumSamples. The rest of it Pix just skips or doesn't do.
Also can I debug in real time using Pix? I'm wondering if this method would reveal more information.
Cheers,
It would appear that most of that shader code is being optimized out (not compiled because it is irrelevant).
In the end, all that matters in the return value of finalColor which is set with color and xNumSamples.
// Average all of the samples to get the final blur color.
finalColor = color / xNumSamples;
I am not sure where xNumSamples gets set, but you can see that the only line that matters to color is color = tex2D(sceneSampler, OriginalUV); (hence it not being removed).
Every line before that is irrelevant because it will be overwritten by that line.
The only bit that follows is that for loop:
for(int i = 1; i < 1; ++i, OriginalUV += velocity)
But this would never execute because i < 1 is false from the get-go (i is assigned a starting value of 1).
Hope that helps!
To answer you second question, I believe to debug shaders in real-time you need to use something like Nvidia's FX Composer and Shader Debugger. However, those run outside of your game, so results are not always useful.