Normal calculations in vertex shader? - directx

Question

I'm using the rastertek framework for terrain generation. I've got the terrain rendered from the vertex shader but I don't know how to calculate the normals in the shader. Theres a function call in one of the classes that generates the normals from the terrain but this only works if the terrain was generated on the cpu. Heres the code for the vertex shader I'm using:
////////////////////////////////////////////////////////////////////////////////
// Filename: terrain.vs
////////////////////////////////////////////////////////////////////////////////
#include "terrain.fx"
/////////////
// GLOBALS //
/////////////
cbuffer MatrixBuffer
{
matrix worldMatrix;
matrix viewMatrix;
matrix projectionMatrix;
};
//////////////
// TYPEDEFS //
//////////////
struct VertexInputType
{
float4 position : POSITION;
float3 normal : NORMAL;
};
struct PixelInputType
{
float4 position : SV_POSITION;
float3 normal : NORMAL;
};
////////////////////////////////////////////////////////////////////////////////
// Vertex Shader
////////////////////////////////////////////////////////////////////////////////
PixelInputType TerrainVertexShader(VertexInputType input)
{
PixelInputType output;
input.position.y = input.position.y + terrain(input.position.x,input.position.z);
// Change the position vector to be 4 units for proper matrix calculations.
input.position.w = 1.0f;
// Calculate the position of the vertex against the world, view, and projection matrices.
output.position = mul(input.position, worldMatrix);
output.position = mul(output.position, viewMatrix);
output.position = mul(output.position, projectionMatrix);
// Calculate the normal vector against the world matrix only.
output.normal = mul(input.normal, (float3x3)worldMatrix);
// Normalize the normal vector.
output.normal = normalize(output.normal);
return output;
}

Answer 1

Your big problem with generating normals in a shader is that you need knowledge of surrounding vertices. This is something you can overcome with a geometry shader but not with a vertex shader. A simple way to calculate the normal is to calculate the polygon normals (take the vector formed from 2 edges and cross product them for the face normal) of all polys that hit the vertex you are looking at and then add them up and normalise. As such if you haven't got access to a geometry shader the only real solution is to use the CPU. Even then this is not the best way to calculate the vertex normals. You may still find it better to use a more complex algorithm yet and that will give you even more problems! So yeah, CPU or geometry shader ... those are, basically, your options.