So I've been working on a Directx11/hlsl rendering engine with the goal of creating a realistic planet which you can view from both on the surface and also at a planetary level. The planet is a normalized cube, which is procedurally generated using noise and as you move closer to the surface of the planet, a binary-based triangle tree splits until the desired detail level is reached. I got vertex normal calculations to work correctly, and I recently started trying to implement normal mapping for my terrain textures, and I have gotten something that seems to work for the most part. However, when the sun is pointing almost perpendicular to the ground (90 degrees), it is way more lit up
However, from the opposite angle (270 degrees), I am getting something that seems
, but may as well be just as off.
The debug lines that are being rendered are the normal, tangent, and bitangents (which all appear to be correct and fit the topology of the terrain)
Here is my shader code:
Vertex shader:
PSIn mainvs(VSIn input)
{
PSIn output;
output.WorldPos = mul(float4(input.Position, 1.f), Instances[input.InstanceID].WorldMatrix); // pass pixel world position as opposed to screen space position for lighitng calculations
output.Position = mul(output.WorldPos, CameraViewProjectionMatrix);
output.TexCoord = input.TexCoord;
output.CameraPos = CameraPosition;
output.Normal = normalize(mul(input.Normal, (float3x3)Instances[input.InstanceID].WorldMatrix));
float3 Tangent = normalize(mul(input.Tangent, (float3x3)Instances[input.InstanceID].WorldMatrix));
float3 Bitangent = normalize(cross(output.Normal, Tangent));
output.TBN = transpose(float3x3(Tangent, Bitangent, output.Normal));
return output;
}
Pixel shader (Texcoord scalar is for smaller textures closer to planet surface):
float3 FetchNormalVector(float2 TexCoord)
{
float3 Color = NormalTex.Sample(Samp, TexCoord * TexcoordScalar);
Color *= 2.f;
return normalize(float3(Color.x - 1.f, Color.y - 1.f, Color.z - 1.f));
}
float3 LightVector = -SunDirection;
float3 TexNormal = FetchNormalVector(input.TexCoord);
float3 WorldNormal = normalize(mul(input.TBN, TexNormal));
float nDotL = max(0.0, dot(WorldNormal, LightVector));
float4 SampleColor = float4(1.f, 1.f, 1.f, 1.f);
SampleColor *= nDotL;
return float4(SampleColor.xyz, 1.f);
Thanks in advance, and let me know if you have any insight as to what could be the issue here.
Edit 1: I tried it with a fixed blue value instead of sampling from the normal texture, which gives me the correct and same results as if I had not applied mapping (as expected). Still don't have a lead on what would be causing this issue.
Edit 2: I just noticed the strangest thing. At 0, 0, +Z, there are these hard seams that only appear with normal mapping enabled
It's a little hard to see, but it seems almost like there are multiple tangents associated to the same vertex (since I'm not using indexing yet) because the debug lines appear to split on the seams.
Here is my code that I'm using to generate the tangents (bitangents are calculated in the vertex shader using cross(Normal, Tangent))
v3& p0 = Chunk.Vertices[0].Position;
v3& p1 = Chunk.Vertices[1].Position;
v3& p2 = Chunk.Vertices[2].Position;
v2& uv0 = Chunk.Vertices[0].UV;
v2& uv1 = Chunk.Vertices[1].UV;
v2& uv2 = Chunk.Vertices[2].UV;
v3 deltaPos1 = p1 - p0;
v3 deltaPos2 = p2 - p0;
v2 deltaUV1 = uv1 - uv0;
v2 deltaUV2 = uv2 - uv0;
f32 r = 1.f / (deltaUV1.x * deltaUV2.y - deltaUV1.y * deltaUV2.x);
v3 Tangent = (deltaPos1 * deltaUV2.y - deltaPos2 * deltaUV1.y) * r;
Chunk.Vertices[0].Tangent = Normalize(Tangent - (Chunk.Vertices[0].Normal * DotProduct(Chunk.Vertices[0].Normal, Tangent)));
Chunk.Vertices[1].Tangent = Normalize(Tangent - (Chunk.Vertices[1].Normal * DotProduct(Chunk.Vertices[1].Normal, Tangent)));
Chunk.Vertices[2].Tangent = Normalize(Tangent - (Chunk.Vertices[2].Normal * DotProduct(Chunk.Vertices[2].Normal, Tangent)));
Also for reference, this is the main article I was looking at while implementing all of this: link
Edit 3:
Here is an image of the planet from a distance with normal mapping enabled:
And one from the same angle without:
I've tried to understand how ESM is working - I have regular Shadow Mapping in Place (occluded/not occluded) in a deferred rendering pipeline and are trying to use ESM instead.
I've tried to adapt this from Cansin:
http://homepage.lnu.se/staff/tblma/Deferred Rendering in XNA 4.pdf
But as he does not use directional lights, I may have a misunderstanding. This is basically my approach on adapting it to directional lights:
Create ShadowMap:
float4 PS(VSO input) : COLOR0
{
float depth = input.Position2D.z / input.Position2D.w;
return exp(depth);
}
I am using an Orthogonal Projection Matrix (same NearFarClip as actual cam), as I do with regular Shadow Mapping (Position2D is ScreenSpace, because it's a directional light, Z is always the distance surface/light, or am I wrong?)
Get Shadow Factor - basically like regular Shadow Mapping, I transform into Light/Screenspace, getting the depth from the ShadowMap
float4 Position = 1;
Position.xy = input.ScreenPosition.xy;
Position.z = Depth; // saved depth from gbuffer
Position = mul(Position, InverseViewProjection);
Position /= Position.w;
float4 LightScreenPos = mul(Position, LightViewProjection);
LightScreenPos /= LightScreenPos.w;
float2 LUV = 0.5f * (float2(LightScreenPos.x, -LightScreenPos.y) + 1.0f);
float shadowDepth = tex2D(sampler_shadow, LUV).r;
float shadow = shadowDepth * exp(-10 * LightScreenPos.z);
Is my thinking fundamentally wrong?
I'm playing around with WebGL, I scipted a simple flat-shaded cube.
I got a shader which takes projection matrix, view model matrix and a normal matrix, nothing fancy:
(...)
void main(void) {
gl_Position = uPMatrix * uMVMatrix * vec4(aVertexPosition, 1.0);
vTextureCoord = aTextureCoord;
vec3 transformedNormal = uNMatrix * aVertexNormal;
float directionalLightWeighting = max(dot(transformedNormal, uLightingDirection), 0.0);
vLightWeighting = uAmbientColor + uDirectionalColor * directionalLightWeighting;
}
Everything is fine, the flat shading looks good, but as soon as I resize the cube (noted as mat4.scale below), the shading does not affect the scene anymore. If I scale down the computed normal matrix by the reverse factor, it works again.
The code follows the following schema (drawing pseudo routine):
projection = mat4.ortho
// set up general camera view
view = mat4.lookAt
// set up cube position / scaling / rotation on view matrix
mat4.translate(view)
mat4.scale(view) // remove for nice shading ..
mat4.rotate(view)
// normalFromMat4 returns upper-left 3x3 inverse transpose
normal = mat4.normalFromMat4 ( view )
pass projection, view, normal to shader
gl.drawElements
I am using gl-matrix as math library.
Any ideas where my mistake lies?
I am implementing a 3d obj viewer like app and when I did a scaling on my object in the app (in OpenGL ES 1.x) it became lighter(scale down) and darker (when I scale up).
Is there a way for me to prevent this "changing of lighting" to happen? i.e. the same brightness uniformly through out.
I guess I have to do something to the lighting normals?
My Render method is as below:
void RenderingEngine::Render(const vector<Visual>& visuals, ivec2 screenSize) const
{
glClearColor(0.5f, 0.5f, 0.5f, 1);
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
vector<Visual>::const_iterator visual = visuals.begin();
for (int visualIndex = 0; visual != visuals.end(); ++visual, ++visualIndex) {
// Set the viewport transform.
ivec2 size = visual->ViewportSize;
ivec2 lowerLeft = visual->LowerLeft;
glViewport(lowerLeft.x, lowerLeft.y, size.x, size.y);
// Set the light position.
glMatrixMode(GL_MODELVIEW);
glLoadIdentity();
vec4 lightPosition(0.25, 0.25, 1, 0);
glLightfv(GL_LIGHT0, GL_POSITION, lightPosition.Pointer());
// Set the model-view transform.
mat4 rotation = visual->Orientation.ToMatrix();
mat4 translation = visual->Translate;
mat4 scale;
scale = scale.Scale(visual->Scale);
rotation = rotation * scale;
//mat4 modelview = rotation * m_translation;
mat4 modelview = rotation * m_translation * translation;
glLoadMatrixf(modelview.Pointer());
// Set the projection transform.
float h = 4.0f * size.y / size.x;
mat4 projection = mat4::Frustum(-2, 2, -h / 2, h / 2, 5, 50);
glMatrixMode(GL_PROJECTION);
glLoadMatrixf(projection.Pointer());
// Set the diffuse color.
vec3 color = visual->Color * 0.75f;
vec4 diffuse(color.x, color.y, color.z, 1);
glMaterialfv(GL_FRONT_AND_BACK, GL_DIFFUSE, diffuse.Pointer());
// Draw the surface.
int stride = 2 * sizeof(vec3);
const GLvoid* normalOffset = (const GLvoid*) sizeof(vec3);
const Drawable& drawable = m_drawables[visualIndex];
glBindBuffer(GL_ARRAY_BUFFER, drawable.VertexBuffer);
glVertexPointer(3, GL_FLOAT, stride, 0);
glNormalPointer(GL_FLOAT, stride, normalOffset);
glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, drawable.IndexBuffer);
glDrawElements(GL_TRIANGLES, drawable.IndexCount, GL_UNSIGNED_SHORT, 0);
}
}
First off, apply your rotation and scale matrices BEFORE your tranlation matrix(or matrices). So... instead of this:
rotation = rotation * scale;
//mat4 modelview = rotation * m_translation;
mat4 modelview = rotation * m_translation * translation;
Do this:
rotation = rotation * scale;
//mat4 modelview = rotation * m_translation;
mat4 modelview = m_translation * translation * rotation;
Secondly, why do you have two translation matrices? You should just need one matrix to transition your object vertices from object space into world space.
Third, why are you creating a normal matrix? Unless you are modifying normals for some sheared object post rotate, I don't really see any reason to do so. You can rotate the normals for your object using the same rotation matrix you use for your vertices.
This solution works for me. So I'll be sharing this for any newbies to the subject who run into the same problem as me.
Use:
glEnable(GL_NORMALIZE);
to always normalize the surface normals to unit length.
About 2 days ago I decided to write code to explicitly calculate the Model-View-Projection ("MVP") matrix to understand how it worked. Since then I've had nothing but trouble, seemingly because of the projection matrix I'm using.
Working with an iPhone display, I create a screen centered square described by these 4 corner vertices:
const CGFloat cy = screenHeight/2.0f;
const CGFloat z = -1.0f;
const CGFloat dim = 50.0f;
vxData[0] = cx-dim;
vxData[1] = cy-dim;
vxData[2] = z;
vxData[3] = cx-dim;
vxData[4] = cy+dim;
vxData[5] = z;
vxData[6] = cx+dim;
vxData[7] = cy+dim;
vxData[8] = z;
vxData[9] = cx+dim;
vxData[10] = cy-dim;
vxData[11] = z;
Since I am using OGLES 2.0 I pass the MVP as a uniform to my vertex shader, then simply apply the transformation to the current vertex position:
uniform mat4 mvp;
attribute vec3 vpos;
void main()
{
gl_Position = mvp * vec4(vpos, 1.0);
}
For now I have simplified my MVP to just be the P matrix. There are two projection matrices listed in the code shown below. The first is the standard perspective projection matrix, and the second is an explicit-value projection matrix I found online.
CGRect screenBounds = [[UIScreen mainScreen] bounds];
const CGFloat screenWidth = screenBounds.size.width;
const CGFloat screenHeight = screenBounds.size.height;
const GLfloat n = 0.01f;
const GLfloat f = 100.0f;
const GLfloat fov = 60.0f * 2.0f * M_PI / 360.0f;
const GLfloat a = screenWidth/screenHeight;
const GLfloat d = 1.0f / tanf(fov/2.0f);
// Standard perspective projection.
GLKMatrix4 projectionMx = GLKMatrix4Make(d/a, 0.0f, 0.0f, 0.0f,
0.0f, d, 0.0f, 0.0f,
0.0f, 0.0f, (n+f)/(n-f), -1.0f,
0.0f, 0.0f, (2*n*f)/(n-f), 0.0f);
// The one I found online.
GLKMatrix4 projectionMx = GLKMatrix4Make(2.0f/screenWidth,0.0f,0.0f,0.0f,
0.0f,2.0f/-screenHeight,0.0f,0.0f,
0.0f,0.0f,1.0f,0.0f,
-1.0f,1.0f,0.0f,1.0f);
When using the explicit value matrix, the square renders exactly as desired in the centre of the screen with correct dimension. When using the perspective projection matrix, nothing is displayed on-screen. I've done printouts of the position values generated for screen centre (screenWidth/2, screenHeight/2, 0) by the perspective projection matrix and they're enormous. The explicit value matrix correctly produces zero.
I think the explicit value matrix is an orthographic projection matrix - is that right? My frustration is that I can't work out why my perspective projection matrix fails to work.
I'd be tremendously grateful if someone could help me with this problem. Many thanks.
UPDATE For Christian Rau:
#define Zn 0.0f
#define Zf 100.0f
#define PRIMITIVE_Z 1.0f
//...
CGRect screenBounds = [[UIScreen mainScreen] bounds];
const CGFloat screenWidth = screenBounds.size.width;
const CGFloat screenHeight = screenBounds.size.height;
//...
glUseProgram(program);
//...
glViewport(0.0f, 0.0f, screenBounds.size.width, screenBounds.size.height);
//...
const CGFloat cx = screenWidth/2.0f;
const CGFloat cy = screenHeight/2.0f;
const CGFloat z = PRIMITIVE_Z;
const CGFloat dim = 50.0f;
vxData[0] = cx-dim;
vxData[1] = cy-dim;
vxData[2] = z;
vxData[3] = cx-dim;
vxData[4] = cy+dim;
vxData[5] = z;
vxData[6] = cx+dim;
vxData[7] = cy+dim;
vxData[8] = z;
vxData[9] = cx+dim;
vxData[10] = cy-dim;
vxData[11] = z;
//...
const GLfloat n = Zn;
const GLfloat f = Zf;
const GLfloat fov = 60.0f * 2.0f * M_PI / 360.0f;
const GLfloat a = screenWidth/screenHeight;
const GLfloat d = 1.0f / tanf(fov/2.0f);
GLKMatrix4 projectionMx = GLKMatrix4Make(d/a, 0.0f, 0.0f, 0.0f,
0.0f, d, 0.0f, 0.0f,
0.0f, 0.0f, (n+f)/(n-f), -1.0f,
0.0f, 0.0f, (2*n*f)/(n-f), 0.0f);
//...
// ** Here is the matrix you recommended, Christian:
GLKMatrix4 ts = GLKMatrix4Make(2.0f/screenWidth, 0.0f, 0.0f, -1.0f,
0.0f, 2.0f/screenHeight, 0.0f, -1.0f,
0.0f, 0.0f, 1.0f, 0.0f,
0.0f, 0.0f, 0.0f, 1.0f);
GLKMatrix4 mvp = GLKMatrix4Multiply(projectionMx, ts);
UPDATE 2
The new MVP code:
GLKMatrix4 ts = GLKMatrix4Make(2.0f/screenWidth, 0.0f, 0.0f, -1.0f,
0.0f, 2.0f/-screenHeight, 0.0f, 1.0f,
0.0f, 0.0f, 1.0f, 0.0f,
0.0f, 0.0f, 0.0f, 1.0f);
// Using Apple perspective, view matrix generators
// (I can solve bugs in my own implementation later..!)
GLKMatrix4 _p = GLKMatrix4MakePerspective(60.0f * 2.0f * M_PI / 360.0f,
screenWidth / screenHeight,
Zn, Zf);
GLKMatrix4 _mv = GLKMatrix4MakeLookAt(0.0f, 0.0f, 1.0f,
0.0f, 0.0f, -1.0f,
0.0f, 1.0f, 0.0f);
GLKMatrix4 _mvp = GLKMatrix4Multiply(_p, _mv);
GLKMatrix4 mvp = GLKMatrix4Multiply(_mvp, ts);
Still nothing visible at the screen centre, and the transformed x,y coordinates of the screen centre are not zero.
UPDATE 3
Using the transpose of ts instead in the above code works! But the square no longer appears square; it appears to now have aspect ratio screenHeight/screenWidth i.e. it has a longer dimension parallel to the (short) screen width, and a shorter dimension parallel to the (long) screen height.
I'd very much like to know (a) why the transpose is required and whether it is a valid fix, (b) how to correctly rectify the non-square dimension, and (c) how this additional matrix transpose(ts) that we use fits into the transformation chain of Viewport * Projection * View * Model * Point .
For (c): I understand what the matrix does, i.e. the explanation by Christian Rau as to how we transform to range [-1, 1]. But is it correct to include this additional work as a separate transformation matrix, or should some part of our MVP chain be doing this work instead?
Sincere thanks go to Christian Rau for his valuable contribution thus far.
UPDATE 4
My question about "how ts fits in" is silly isn't it - the whole point is the matrix is only needed because I'm choosing to use screen coordinates for my vertices; if I were to use coordinates in world space from the start then this work wouldn't be needed!
Thanks Christian for all your help, it's been invaluable :) Problem solved.
The reason for this is, that your first projection matrix doesn't account for the scaling and translation part of the transformation, whereas the second matrix does it.
So, since your modelview matrix is identity, the first projection matrix assumes the models' coordinates to ly somewhere in [-1,1], whereas the second matrix already contains the scaling and translation part (look at the screenWidth/Height values in there) and therefore assumes the coordinates to ly in [0,screenWidth] x [0,screenHeight].
So you have to right-multiply your projection matrix by a matrix that first scales [0,screenWidth] down to [0,2] and [0,screenHeight] down to [0,2] and then translates [0,2] into [-1,1] (using w for screenWidth and h for screenHeight):
[ 2/w 0 0 -1 ]
[ 0 2/h 0 -1 ]
[ 0 0 1 0 ]
[ 0 0 0 1 ]
which will result in the matrix
[ 2*d/h 0 0 -d/a ]
[ 0 2*d/h 0 -d ]
[ 0 0 (n+f)/(n-f) 2*n*f/(n-f) ]
[ 0 0 -1 0 ]
So you see that your second matrix corresponds to a fov of 90 degrees, an aspect ratio of 1:1 and a near-far range of [-1,1]. Additionally it also inverts the y-axis, so that the origin is in the upper-left, which results in the second row being negated:
[ 0 -2*d/h 0 d ]
But as an end comment, I suggest you to not configure the projection matrix to account for all this. Instead your projection matrix should look like the first one and you should let the modelview matrix manage any translation or scaling of your world. It is not by accident, that the transformation pipeline was seperated into modelview and projection matrix and you should keep this separation also when using shaders. You can of course still multiply both matrices together on the CPU and upload a single MVP matrix to the shader.
And in general you don't really use a screen-based coordinate system when working with a 3-dimensional world. You would only want to do this if you are drawing 2d graphics (like GUI elements or HUDs) and in this case you would use a more simple orthographic projection matrix, anyway, that is nothing more than the above mentioned scale-translate matrix without all the perspective complexity.
EDIT: To your 3rd update:
(a) The transpose is required because I guess your GLKMatrix4Make function accepts its parameters in column-major format and you put the matrix in row-wise.
(b) I made a little mistake. You should change the screenWidth in the ts matrix into screenHeight (or maybe the other way around, not sure). We actually need a uniform scale, because the aspect ratio is already taken care of by the projection matrix.
(c) It is not easy to classify this matrix into the usual MVP pipeline. This is because it is not really common. Let's look at the two common cases of rendering:
3D: When you have a 3-dimensional world it is not really common to define it's coordinates in screen-based units, because there is not et a mapping from 3d-scene to 2d-screen and using a coordinate system where units equal pixels just doesn't make sense. In this setup you most likely would classify it as part of the modelview matrix for transforming the world into another unit system. But in this case you would need real 3d transformations and not just such a half-baked 2d solution.
2D: When rendering a 2d-scene (like a GUI or a HUD or just some text), you sometimes really want a screen-based coordinate system. But in this case you most likely would use an orthographic projection (without any perspective). Such an orthographic matrix is actually nothing more than this ts matrix (with some additional scale-translate for z, based on the near-far range). So in this case the matrix belongs to, or actually is, the projection matrix. Just look at how the good old glOrtho function constructs its matrix and you'll see its nothing more than ts.