I have array with SCNVector3, one for each vertex.
var terrainArray = [SCNVector3]()
I need to provide this data per vertex in my fragment shader. Something like this:
struct TerrainVertexInput
{
float3 position [[attribute(SCNVertexSemanticPosition)]];
float4 color [[attribute(SCNVertexSemanticColor)]];
};
struct TerrainVertexOutput
{
float4 position [[position]];
float3 terrain;
float4 color;
};
vertex TerrainVertexOutput terrainVertex(TerrainVertexInput in [[stage_in]],
constant SCNSceneBuffer& scn_frame [[buffer(0)]],
constant MyNodeBuffer& scn_node [[buffer(1)]],
constant float3 terrain [[buffer(2)]])
{
TerrainVertexOutput v;
v.position = scn_node.modelViewProjectionTransform * float4(in.position, 1.0);
v.terrain = terrain;
v.color = in.color;
return v;
}
As I understand I need to create Data object with array data and provide it to program with setValue(_:forKey:) but I'm not sure if vertex function will get right element for vertex.
How to do this right?
You're on the right track, but you don't want to use SCNVector3 for data you're passing into a Metal shader. SceneKit's vector types have components of type CGFloat, the size of which is platform-dependent.
Instead, your data should use one of the simd vector types. In Swift and Metal, that means float3 or float4. Note that float3 actually occupies 16 bytes of space; there's a dummy element at the end for alignment purposes. If you want to pack your data tightly, using exactly 3 floats per vertex, you can type your buffer in Metal as packed_float3 and write 3 contiguous floats into your data buffer for each vertex. There is no three-element packed float vector type in Swift.
There are many ways to copy an array of SCNVector3 into a suitably-typed data buffer. Here's one:
// Allocate enough memory to store three floats per vertex, ensuring we free it later
let terrainBuffer = UnsafeMutableBufferPointer<Float>.allocate(capacity: terrainArray.count * 3)
defer {
terrainBuffer.deallocate()
}
// Copy each element of each vector into the buffer
terrainArray.enumerated().forEach { i, v in
terrainBuffer[i * 3 + 0] = Float(v.x)
terrainBuffer[i * 3 + 1] = Float(v.y)
terrainBuffer[i * 3 + 2] = Float(v.z)
}
// Copy the buffer data into a Data object, as expected by SceneKit
let terrainData = Data(buffer: terrainBuffer)
You can then use setValue(:forKey:) on your geometry or material:
material.setValue(terrainData, forKey: "terrain")
Rather than taking a single float3 as a parameter in your vertex function, instead take a pointer to packed_float3 and index into it according to the vertex ID:
vertex TerrainVertexOutput terrainVertex(TerrainVertexInput in [[stage_in]],
constant SCNSceneBuffer& scn_frame [[buffer(0)]],
constant MyNodeBuffer& scn_node [[buffer(1)]],
constant packed_float3 *terrain [[buffer(2)]],
uint vid [[vertex_id]]) {
// ...
v.terrain = terrain[vid];
// ...
}
This assumes an exact correspondence between vertices in your geometry and terrain data points. Rather than using the vertex ID directly, you can of course do whatever sort of fancy indexing you want to look up the terrain data for a given vertex.
Related
I'm using Metal to render a scene with a z buffer and now need to integrate this z-buffer into SceneKit's rendering. However I can't figure out how to get SceneKit to use this depth better correctly and am not even 100% sure what format SceneKit expects it's z-buffers to be in
Base on this question, my understanding was that SceneKit uses a reverse logarithmic z-buffer in the range of 1 (near) to 0 (far). However I can't get this working and objects I draw with SceneKit don't properly respect the depth buffer: they are either always showing or always hidden
First, here's how the generate a z-buffer texture in a Metal render pass:
struct FragmentOut {
float4 color [[color(0)]];
float depth [[depth(any)]];
};
fragment FragmentOut metalRenderFragment(const InOut in [[ stage_in ]]) {
FragmentOut out;
out.depth = 0; // 0 is far with reverse z buffer
...
float cameraSpaceZ = ...; // Computed in shader
// There constants are taken from SceneKit's camera and inlined here
const float zNear = 0.0010000000474974513;
const float zFar = 1000.0;
float logDepth = log(z / zNear) / log(zFar / zNear);
out.depth = 1.0 - logDepth; // Reverse the depth for scenekit
return out;
}
Then to integrate the depth buffer into SceneKit, I render a full screen quad in scenekit with a SCNProgram that uses the depth texture generated in the previous step:
fragment FragmentOut sceneKitFullScreenQuadFragment(const InOut in [[ stage_in ]],
depth2d<float, access::sample> depthTexture [[texture(1)]])
{
constexpr sampler sampler(filter::linear);
const float depth = depthTexture.sample(sampler, in.uv);
return {
.color = float4(0),
.depth = depth,
};
}
So two questions:
What format does SceneKit use for its z-buffer? Is it a reversed logarithmic z-buffer?
What am I doing wrong in generating the z-buffer values for SceneKit?
SceneKit uses a reverse logarithmic Z-Buffer. This post and this post show you how to get a normalized linear mapping space [0...1]. You need the opposite formula.
Also, you can toggle the value from reverseZ to directZ this way:
let sceneView = self.view as! SCNView
sceneView.usesReverseZ = true // default
Andy Jazz's answer helped but I still found the links confusing. Here's what ultimately worked for me (although there are possibly other ways to do this):
When generating the depth map (this would be inside the the metal shader in my original example) pass in SceneKit's projection transform matrix and use this to transform the depth value:
// In a metal shader generating the depth map
// The z distance from the camera, e.g. if the object
// at the current position is 5 units away, this would be 5.
const float z = ...;
// The camera points along the -z axis, so transform the -z position
// with SceneKit's projection matrix (you can get this from SCNCamera)
const float4 depthPos = (sceneKitState.projectionTransform * float4(0, 0, -z, 1));
// Then do perspective division to get the final depth value
out.depth = depthPos.z / depthPos.w;
Then inside of the SceneKit shader, simply write out the depth, taking into account usesReverseZ:
// In a scenekit, full screen quad shader
const float depth = depthTexture.sample(sampler, in.uv);
return {
.color = float4(0),
.depth = 1.0 - depth,
};
❗️ The above assumes you are using sceneView.usesReverseZ = true (the default). If you are using usesReverseZ = false, simply do .depth = depth instead
I have an MTLTexture in RGBA8Unorm format, and a screen texture (in MTKView) in BGRA8Unorm format (reversed). In the Metal shader, when I sample from that texture using sample(), I get float4. When I write to texture in metal shader, I also write float4. It seems that when I am inside the shader code, float4 always represents the same order of components RGBA regardless of the original format the texture is in ([0] for red, [1] for green, [2] for blue, and [3] for alpha). Is my conclusion correct that the meaning of the components of the sampled/written float4 is always the same inside the shader, regardless of what the storage format of the texture is?
UPDATE: I use the following code to write to a texture with RGBA8Unnorm format:
kernel void
computeColourMap(constant Uniforms &uniforms [[buffer(0)]],
constant array<float, 120> &s [[buffer(1)]],
constant array<float, 120> &red [[buffer(2)]],
constant array<float, 120> &green [[buffer(3)]],
constant array<float, 120> &blue [[buffer(4)]],
texture2d<float, access::write> output [[texture(0)]],
uint2 id [[thread_position_in_grid]])
{
if (id.x >= output.get_width() || id.y >= output.get_height()) {
return;
}
uint i = id.x % 120;
float4 col (0, 0, 0, 1);
col.x += amps[i] * red[i];
col.y += amps[i] * green[i];
col.z += amps[i] * blue[i];
output.write(col, id);
}
I then use the following shaders for the rendering stage:
vertex VertexOut
vertexShader(const device VertexIn *vertexArray [[buffer(0)]],
unsigned int vid [[vertex_id]])
{
VertexIn vertex_in = vertexArray[vid];
VertexOut vertex_out;
vertex_out.position = vertex_in.position;
vertex_out.textureCoord = vertex_in.textureCoord;
return vertex_out;
}
fragment float4
fragmentShader(VertexOut interpolated [[stage_in]],
texture2d<float> colorTexture [[ texture(0) ]])
{
const float4 colorSample = colorTexture.sample(nearestSampler,
interpolated.textureCoord);
return colorSample;
}
where colourTexture passed into the fragment shader is the one I generated in RGBA8Unorm format, and in Swift I have:
let renderPipelineDescriptor = MTLRenderPipelineDescriptor()
renderPipelineDescriptor.vertexFunction = library.makeFunction(name: "vertexShader")!
renderPipelineDescriptor.fragmentFunction = library.makeFunction(name: "fragmentShader")!
renderPipelineDescriptor.colorAttachments[0].pixelFormat = colorPixelFormat
the colorPixelFormat of the MTKView is BGRA8Unorm (reversed relative to texture), which is not the same as my texture, but the colours on the screen come out correct.
UPDATE 2: one further pointer that within a shader the colour represented by float4 always has order of rgba is: float4 type actually has accessors called v.r, v.g, v.b, v.rgb, etc...
The vector always has 4 components, but the type of the components is not necessarily float. When you declare a texture, you specify the component type as a template argument (texture2d<float ...> in your code).
For example, from Metal Shading Language Specification v2.1, section 5.10.1:
The following member functions can be used to sample from a 1D
texture.
Tv sample(sampler s, float coord) const
Tv is a 4-component vector type based on the templated type used
to declare the texture type. If T is float, Tv is float4. If T is half,
Tv is half4. If T is int, Tv is int4. If T is uint, Tv is uint4. If T
is short, Tv is short4 and if T is ushort, Tv is ushort4.
The same Tv type is used in the declaration of write(). The functions for other texture types are documented in a similar manner.
And, yes, component .r always contains the red component (if present), etc. And [0] always corresponds to .r (or .x).
fragment half4 fragmen_shader_test(WaterColorCloudOut params[[stage_in]],
texture2d<float , access::sample>cloud1 [[texture(0)]],
texture2d<half, access::sample> cloud2 [[texture(1)]],
texture2d<half, access::sample> cloud3 [[texture(2)]]
)
{
constexpr sampler defaultSampler;
float4 color1;
if(params.index == 0){
color1= float4(cloud1.sample(defaultSampler, float2(params.textureCoordinates))) * params.color ;
}
else if(params.index == 1){
color1= float4(cloud2.sample(defaultSampler, float2(params.textureCoordinates))) * params.color ;
} else{
color1= float4(cloud3.sample(defaultSampler, float2(params.textureCoordinates))) * params.color ;
}
return half4(color1);
}
Here I am using three textures because of If-else condition the performance drops with time. I feel if i send a texture array to the shader there is no need to perform if else statement. In CPU I have three MTLTexture. How can I bind the three texture to an array and pass it shader.
In CPU side I created three Textures and Created a MTLTexture array
var textureArray:[MTLTexture] = []
Then I append the Texture to that array.
In MTLRenderCommandEncoder
let myRange: CountableRange = 0..<2
commandEncoder.setFragmentTextures(textureArray, range: myRange)
In Shader
texture2d_array<float , access::sample> texture [[ texture(0) ]]
While sampling in Shader
float4 color = texture.sample(defaultSampler, float2(params.textureCoordinates),0) * float4(1,0,0,1.0);
I am doing like this currently ButI couldn't get the correct Texture output
You don't have to change the app code. Since you are using consecutive texture indexes, you can just replace the three separate parameters with a single texture array parameter bound to index 0 and the bindings will connect the elements of the array with the textures you've provided from the app code.
In other words, if you declare a texture array with size 3 and bound to texture index 0, the elements of the array are taken from texture index 0, 1, and 2.
Update for your edited question:
You are confusing an array of textures (also called a texture array) and an array texture. The terminology is confusingly subtle.
An array texture is a single texture with multiple 2D planes in it. All of the planes have the same size and pixel format. The number of planes in the texture does not have to be known at shader compile time.
An array of textures is an array of independent textures, which may have different sizes or pixel formats. The length of the array is a compile-time constant.
You used texture2d_array<...>. That's an array texture.
For a texture array, or array of textures, you should use array<texture2d<float, access::sample>, 3> clouds [[texture(0)]].
To sample from a texture within the array, you first index into the array and then call a texture function on that element:
float4 color = clouds[params.index].sample(defaultSampler, float2(params.textureCoordinates)) * float4(1,0,0,1.0);
I'm using vs2015 and studying dx11.
I'll show you code first.
cbuffer cbperobject {
float4x4 gWorldViewProj;
};
struct VertexIn {
float3 Pos : POSITION;
float4 Color : COLOR;
};
struct VertexOut {
float4 PosH : SV_POSITION;
float4 Color : COLOR;
};
VertexOut main( VertexIn vin )
{
VertexOut vOut;
vOut.PosH = mul(float4(vin.Pos, 1.0f), gWorldViewProj);
vOut.Color = vin.Color;
return vOut;
}
This is my vertex shader code. I rahter copied it from internet.
HRESULT result;
D3D11_MAPPED_SUBRESOURCE mappedResource;
XMMATRIX* dataPtr;
UINT bufferNumber;
// Transpose the matrices to prepare them for the shader.
// Lock the constant buffer so it can be written to.
result = mD3dDContext->Map(contantBuff, 0, D3D11_MAP_WRITE_DISCARD, 0, &mappedResource);
if (FAILED(result))
{
return false;
}
// Get a pointer to the data in the constant buffer.
dataPtr = (XMMATRIX*)mappedResource.pData;
// Copy the matrices into the constant buffer.
XMMATRIX world = XMLoadFloat4x4(&mWorld); // 버텍스의 월드변환
XMMATRIX view = XMLoadFloat4x4(&mView); // 카메라
XMMATRIX proj = XMLoadFloat4x4(&mProj); // 직교투영
XMMATRIX worldViewProj = world*view*proj;
worldViewProj = XMMatrixTranspose(worldViewProj);
*dataPtr = worldViewProj;
// Unlock the constant buffer.
mD3dDContext->Unmap(contantBuff, 0);
// Set the position of the constant buffer in the vertex shader.
bufferNumber = 0;
// Finanly set the constant buffer in the vertex shader with the updated values.
mD3dDContext->VSSetConstantBuffers(bufferNumber, 1, &contantBuff);
return true;
This is my setting constant buffer in shader code.
First, what is difference between POSITION and SV_POSITION semantic? Would you recommend good HLSL tutorial book? I'm Korean and I'm living in Korea. There is no good book in here; I don't know why, all good book is out of print. What a bad country for studying programming.
Second, why should I transpose my camera matrix(worldviewproj matrix) before CPU gives data to GPU? It's Vertex * matrix = processed Vertex. Why should I transpose it?
Well POSITION(Semantic) gives directive to GPU, that concrete values will be placed as points in coordinate space and SV_POSITION is giving directive for pixel shader. Actually it gives order to GPU about pixels location on screen mainly in range -1 to 1. Look at this https://msdn.microsoft.com/en-us/library/windows/desktop/bb509647(v=vs.85).aspx
Well seems you need Linear Algebra lessons mate. Matrix transposition is the key stone in 3d graphics. With Matrix transpositions(And same time transposed Matrix is inverse Matrix and Inverse Matrix is always Orthogonal) all Matrix Transformations are happening(Translation, Rotation, Scaling). First of all you need Linear Algebra stuff and about Rendering Api be it OpenGL or DirectX(never mind they are just API's) you can grab any book or online documentation you can look at amazon.com. Happy graphics coding pal ;).
I have mapped some values into my texture on my alpha channel. Actually I use my texture as 2Darray. What I need is a way to read the alpha value of the map at position e.g. [4][5] (representing x and y)
I need the returned value available in my pixelshader. Is there any way to do this?
I use DX9.
Thx in advance!
Do you want to use the texel at [4][5] (x,y) for your entire pixelshader?
if that is your question you could just precalc that cordinate on the vertex shader and passit along to every vertex, and then sample with that uv cords. this way it wont get interpolated. (or it will, but it will only have one value to interpolate with)
other than that you probably have to specifiy abit more on what you are trying to achive.
What are you using it for? when will it occure, what sort of mesh are you using it for?
Texture2DArray is a shader model 4 thing. I don't believe you're using it on dx9.
If you are using shader model 4, then just use the function Load(4, 5).
Otherwise, for sm1,2,3, you can put the numbers you want, e.g. 4.0f and 5.0 into your vertex as normal texcoord data. Then have the pixel shader scale it by the size of the texture.
struct VertexInput {
float4 pos : POSITION;
float2 uv : TEXCOORD0; //0.0, 1.0, 2.0, 3.0, 4.0 etc
};
struct PixelInput {
float4 position : POSITION;
float2 uv : TEXCOORD0;
};
PixelInput vsTex(VertexInput vtx)
{
PixelInput output;
float4 pos = vtx.pos;
output.position = mul(pos, MatWorld);
output.position = mul(output.position, MatView);
output.position = mul(output.position, MatProj);
output.uv = vtx.uv;
return output;
}
float4 PixelShader(PixelInput input) : SV_Target
{
float coords = pix.uv / float2(TEX_WIDTH, TEX_HEIGHT);
return tex = tex2D(mySampler, coords);
}
Where TEX_WIDTH, TEX_HEIGHT are passed in via the 'defines' parameter of D3DXCompileShader. And
OR: just do 4.0f/tex_width and 5.0/tex_height in software and just pass that number (which will be between [0.0f,1.0f] through to the pixel shader)