Programmatically writing a texture and reading it in fragment shader - metal

I've written a Compute shader that outputs to a Texture. The coordinate system of the output texture is in pixels. I then have a basic vertex and fragment shader that should simply sample the value and respond with what I thought would be in normalised coordinates. However, I thought this mapping between my programmatically drawn texture and the vertices of my rendering surface would match up.
The Compute Function
Can be summarized as
texture.write(color, uint2(x, y));
where x and y are integer pixel locations.
The Vertex Data
// position.x, position.y, texCoords.x, texCoords.y
let vertexData = [Float](arrayLiteral:
-1, 1, 0, 0,
-1, -1, 0, 1,
1, -1, 1, 1,
1, -1, 1, 1,
1, 1, 1, 0,
-1, 1, 0, 0)
The Metal Shader
typedef struct {
packed_float2 position;
packed_float2 texCoords;
} VertexIn;
typedef struct {
float4 position [[ position ]];
float2 texCoords;
} FragmentVertex;
vertex FragmentVertex simple_vertex(device VertexIn *vertexArray [[ buffer(0) ]],
uint vertexIndex [[ vertex_id ]])
{
VertexIn in = vertexArray[vertexIndex];
FragmentVertex out;
out.position = float4(in.position, 0.f, 1.f);
out.texCoords = in.texCoords;
return out;
}
fragment float4 simple_fragment(FragmentVertex in [[ stage_in ]],
texture2d<uint, access::sample> inputTexture [[ texture(0) ]],
sampler linearSampler [[ sampler(0) ]])
{
const uint2 imageSizeInPixels = uint2(360, 230);
float imageSizeInPixelsWidth = imageSizeInPixels.x;
float imageSizeInPixelsHeight = imageSizeInPixels.y;
float2 coords = float2(in.position.x / 360.f, in.position.y / 230.f);
float color = inputTexture.sample(linearSampler, in.texCoords).x / 255.f;
return float4(float3(color), 1.f);
}
The Sampler
let samplerDescriptor = MTLSamplerDescriptor()
samplerDescriptor.normalizedCoordinates = true
samplerDescriptor.minFilter = .linear
samplerDescriptor.magFilter = .linear
samplerDescriptor.sAddressMode = .clampToZero
samplerDescriptor.rAddressMode = .clampToZero
self.samplerState = self.metalDevice?.makeSamplerState(descriptor: samplerDescriptor)
In this experiment the only value that seems to work is coords, based upon the normalized in.position value. in.texCoords seems to always be zero. Shouldn't the texcoords and position received by the vertex and fragment shader be values be in the range of values defined in the vertex data?

My Vertex Buffer was right, but wrong
In the process of converting Obj-C code to Swift I failed to copy the vertex completely.
The Correct Copy
let byteCount = vertexData.count * MemoryLayout<Float>.size
let vertexBuffer = self.metalDevice?.makeBuffer(bytes: vertexData, length: byteCount, options: options)
The Source of my Woes
let vertexBuffer = self.metalDevice?.makeBuffer(bytes: vertexData, length: vertexData.count, options: options)
The Complete Vertex Buffer Creation
// Vertex data for a full-screen quad. The first two numbers in each row represent
// the x, y position of the point in normalized coordinates. The second two numbers
// represent the texture coordinates for the corresponding position.
let vertexData = [Float](arrayLiteral:
-1, 1, 0, 0,
-1, -1, 0, 1,
1, -1, 1, 1,
1, -1, 1, 1,
1, 1, 1, 0,
-1, 1, 0, 0)
// Create a buffer to hold the static vertex data
let options = MTLResourceOptions().union(.storageModeShared)
let byteCount = vertexData.count * MemoryLayout<Float>.size
let vertexBuffer = self.metalDevice?.makeBuffer(bytes: vertexData, length: byteCount, options: options)
vertexBuffer?.label = "Image Quad Vertices"
self.vertexBuffer = vertexBuffer

Related

Strange errors during drawing hollow circles in the 3D space

I am trying to draw two hollow circles that are surrounding a cube which is located at the 0, 0, 0 position..
so far I've implemented the cube and the two circles here is what I get.
there are two strange things happening here.
One is that I want to draw the circles but I can see the lines radiating from the origin.
and two is that interpolated colors, even though I set just one color for the fragment shader.
here is you can see clearly those lines with interpolated color...
here is my vertex shader code and the fragment shader code
"use strict";
const loc_aPosition = 1;
const loc_aColor = 2;
const loc_UVCoord = 3;
const VSHADER_SOURCE =
`#version 300 es
layout(location=${loc_aPosition}) in vec4 aPosition;
layout(location=${loc_aColor}) in vec4 aColor;
layout(location=${loc_UVCoord}) in vec2 UVCoord;
out vec4 vColor;
out vec2 vUVCoord;
uniform mat4 uMVP;
void main()
{
gl_Position = uMVP * aPosition;
vColor = aColor;
vUVCoord = UVCoord;
}`;
const FSHADER_SOURCE =
`#version 300 es
precision mediump float;
in vec4 vColor;
out vec4 fColor;
void main()
{
fColor = vColor;
}`;
and the initilize functions for the two circles and there is the only difference is the target plane.
function init_equator(gl)
{
let vertices = []; // for the vertices
let color = [1, 0, 0]; // red color
for(var i = 0; i <= 360; i+=10)
{
let j = i * Math.PI/180;
let vert = [R * Math.cos(j), 0, R * Math.sin(j)]; // drawing a circle at the XZ plane since it has to be an equator for the cube...
vertices.push( vert[0], vert[1], vert[2] ); // push the vertices
vertices.push( color[0], color[1], color[2]); // set the color
}
const SZ = vertices.BYTES_PER_ELEMENT;
let vao = gl.createVertexArray();
gl.bindVertexArray(vao);
let vbo = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, vbo);
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(vertices), gl.STATIC_DRAW);
gl.vertexAttribPointer(loc_aPosition, 3, gl.FLOAT, false, SZ * 6, 0); // stride is 6, 3 for positions and 3 for the color
gl.enableVertexAttribArray(loc_aPosition);
gl.vertexAttribPointer(loc_aColor, 3, gl.FLOAT, false, SZ * 6, SZ * 3); // stride is 6, offset is this is because 3 color elements are located after 3 position elements..
gl.enableVertexAttribArray(loc_aColor);
gl.bindVertexArray(null);
gl.bindBuffer(gl.ARRAY_BUFFER, null);
return { vao, n : vertices.length / 3 }; // since it has three coordinates so devide by 3
}
function init_latitude(gl)
{
let vertices = []; // for the vertices
let color = [1, 0, 0]; // supposed to be the red
for(var i = 0; i <= 360; i+=10)
{
let j = i * Math.PI/180;
let vert = [0, R * Math.cos(j), R * Math.sin(j)]; // drawing a circle on the YZ plane
vertices.push( vert[0], vert[1], vert[2] );
vertices.push( color[0], color[1], color[2]);
}
const SZ = vertices.BYTES_PER_ELEMENT;
let vao = gl.createVertexArray();
gl.bindVertexArray(vao);
let vbo = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, vbo);
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(vertices), gl.STATIC_DRAW);
gl.vertexAttribPointer(loc_aPosition, 3, gl.FLOAT, false, SZ * 6, 0); // stride is 6, 3 for positions and 3 for the color
gl.enableVertexAttribArray(loc_aPosition);
gl.vertexAttribPointer(loc_aColor, 3, gl.FLOAT, false, SZ * 6, SZ * 3); // stride is 6, offset is this is because 3 color elements are located after 3 position elements..
gl.enableVertexAttribArray(loc_aColor);
gl.bindVertexArray(null);
gl.bindBuffer(gl.ARRAY_BUFFER, null);
return { vao, n : vertices.length / 3 }; // since it has three coordinates so devide by 3
}
I refer these drawing fucntions from here drawing circle
in the main function I called the draw function like this..
........
MVP.setOrtho(LEFT, RIGHT, BOTTOM, TOP, NEAR, FAR); // setting MVP matrix to orthographic mode
MVP.lookAt(FIXED_X, FIXED_Y, FIXED_Z, 0,0,0, 0,1,0); // Eye position x, y, z Look at position 0, 0, 0 Up vector 0, 1, 0
gl.uniformMatrix4fv(loc_MVP, false, MVP.elements);
gl.bindVertexArray(cube.vao);
gl.drawElements(gl.TRIANGLES, cube.n, gl.UNSIGNED_BYTE, 0)
gl.bindVertexArray(null);
gl.bindVertexArray(equator.vao);
gl.drawArrays(gl.LINE_LOOP, 0, equator.n);
gl.bindVertexArray(null);
gl.bindVertexArray(latitudeCircle.vao);
gl.drawArrays(gl.LINE_LOOP, 0, latitudeCircle.n);
gl.bindVertexArray(null);
I have no ideas why the lines are radiating from the origin and the mixed color...
could somebody help me?
this line, which appears twice in the code you posted
const SZ = vertices.BYTES_PER_ELEMENT;
is SZ will be undefined. vertices is a native JavaScript array, not a typedarray array like Float32Array. After that every calculation with SZ will be 0 or NaN
In other words these lines
gl.vertexAttribPointer(loc_aPosition, 3, gl.FLOAT, false, SZ * 6, 0);
gl.vertexAttribPointer(loc_aColor, 3, gl.FLOAT, false, SZ * 6, SZ * 3);
Will be
gl.vertexAttribPointer(loc_aPosition, 3, gl.FLOAT, false, 0, 0);
gl.vertexAttribPointer(loc_aColor, 3, gl.FLOAT, false, 0, 0);
Which means every other position is a color, and every other color is a position which explains why lines go to the center and why colors are interpolated.
Note that if you had stepped through the code in the debugger you'd have probably seen this issue so it would be good to learn how to use the debugger.
Also FYI unrelated to your issue you don't need to call gl.bindVertexArray twice in a row, once with null and once with the next thing you want to draw with.
this
gl.bindVertexArray(cube.vao);
gl.drawElements(gl.TRIANGLES, cube.n, gl.UNSIGNED_BYTE, 0)
gl.bindVertexArray(null);
gl.bindVertexArray(equator.vao);
gl.drawArrays(gl.LINE_LOOP, 0, equator.n);
gl.bindVertexArray(null);
gl.bindVertexArray(latitudeCircle.vao);
gl.drawArrays(gl.LINE_LOOP, 0, latitudeCircle.n);
gl.bindVertexArray(null);
can just be this
gl.bindVertexArray(cube.vao);
gl.drawElements(gl.TRIANGLES, cube.n, gl.UNSIGNED_BYTE, 0)
gl.bindVertexArray(equator.vao);
gl.drawArrays(gl.LINE_LOOP, 0, equator.n);
gl.bindVertexArray(latitudeCircle.vao);
gl.drawArrays(gl.LINE_LOOP, 0, latitudeCircle.n);
gl.bindVertexArray(null); // this is also not technically needed
Also also, you can use the spread operator.
This
vertices.push( vert[0], vert[1], vert[2] ); // push the vertices
vertices.push( color[0], color[1], color[2]); // set the color
can be this
vertices.push( ...vert ); // push the vertices
vertices.push( ...color ); // set the color
Also you might find these tutorials useful.

Metal vertex shader draw points of a Texture

I want to execute Metal (or OpenGLES 3.0) shader that draws Points primitive with blending. To do that, I need to pass all the pixel coordinates of the texture to Vertex shader as vertices which computes the position of the vertex to be passed to fragment shader. The fragment shader simply outputs the color for the point with blending enabled. My problem is if there is an efficient was to pass coordinates of vertices to the vertex shader, since there would be too many vertices for 1920x1080 image, and that needs to be done 30 times in a second? Like we do in a compute shader by using dispatchThreadgroups command, except that compute shader can not draw a geometry with blending enabled.
EDIT: This is what I did -
let vertexFunctionRed = library!.makeFunction(name: "vertexShaderHistogramBlenderRed")
let fragmentFunctionAccumulator = library!.makeFunction(name: "fragmentShaderHistogramAccumulator")
let renderPipelineDescriptorRed = MTLRenderPipelineDescriptor()
renderPipelineDescriptorRed.vertexFunction = vertexFunctionRed
renderPipelineDescriptorRed.fragmentFunction = fragmentFunctionAccumulator
renderPipelineDescriptorRed.colorAttachments[0].pixelFormat = .bgra8Unorm
renderPipelineDescriptorRed.colorAttachments[0].isBlendingEnabled = true
renderPipelineDescriptorRed.colorAttachments[0].rgbBlendOperation = .add
renderPipelineDescriptorRed.colorAttachments[0].sourceRGBBlendFactor = .one
renderPipelineDescriptorRed.colorAttachments[0].destinationRGBBlendFactor = .one
do {
histogramPipelineRed = try device.makeRenderPipelineState(descriptor: renderPipelineDescriptorRed)
} catch {
print("Unable to compile render pipeline state Histogram Red!")
return
}
Drawing code:
let commandBuffer = commandQueue?.makeCommandBuffer()
let renderEncoder = commandBuffer?.makeRenderCommandEncoder(descriptor: renderPassDescriptor!)
renderEncoder?.setRenderPipelineState(histogramPipelineRed!)
renderEncoder?.setVertexTexture(metalTexture, index: 0)
renderEncoder?.drawPrimitives(type: .point, vertexStart: 0, vertexCount: 1, instanceCount: metalTexture!.width*metalTexture!.height)
renderEncoder?.drawPrimitives(type: .point, vertexStart: 0, vertexCount: metalTexture!.width*metalTexture!.height, instanceCount: 1)
and Shaders:
vertex MappedVertex vertexShaderHistogramBlenderRed (texture2d<float, access::sample> inputTexture [[ texture(0) ]],
unsigned int vertexId [[vertex_id]])
{
MappedVertex out;
constexpr sampler s(s_address::clamp_to_edge, t_address::clamp_to_edge, min_filter::linear, mag_filter::linear, coord::pixel);
ushort width = inputTexture.get_width();
ushort height = inputTexture.get_height();
float X = (vertexId % width)/(1.0*width);
float Y = (vertexId/width)/(1.0*height);
int red = inputTexture.sample(s, float2(X,Y)).r;
out.position = float4(-1.0 + (red * 0.0078125), 0.0, 0.0, 1.0);
out.pointSize = 1.0;
out.colorFactor = half3(1.0, 0.0, 0.0);
return out;
}
fragment half4 fragmentShaderHistogramAccumulator ( MappedVertex in [[ stage_in ]]
)
{
half3 colorFactor = in.colorFactor;
return half4(colorFactor*(1.0/256.0), 1.0);
}
Maybe you can draw a single point instanced 1920x1080 times. Something like:
vertex float4 my_func(texture2d<float, access::read> image [[texture(0)]],
constant uint &width [[buffer(0)]],
uint instance_id [[instance_id]])
{
// decompose the instance ID to a position
uint2 pos = uint2(instance_id % width, instance_id / width);
return float4(image.read(pos).r * 255, 0, 0, 0);
}

Execution of the command buffer was aborted due to an error during execution

I've been stuck on this for a while; this isn't a particularly expensive shader (at least, based on my very limited experience with Metal), yet I still get this message for the first few frames:
Execution of the command buffer was aborted due to an error during execution. Caused GPU Hang Error (IOAF code 3)
And then I get this one all subsequent frames:
Execution of the command buffer was aborted due to an error during execution. Ignored (for causing prior/excessive GPU errors) (IOAF code 4)
This is the vertex shader, which doesn't really do anything:
vertex VertexOut viewportProgram(uint vertexID [[ vertex_id ]],
constant float2 *positions [[ buffer(0) ]],
constant float2 *texcoords [[ buffer(1) ]]) {
VertexOut out;
out.position.xy = positions[vertexID];
out.position.z = 0.0; // Only 2D; no depth
out.position.w = 1.0; // Only 2D; no perspective divide
out.texcoord = texcoords[vertexID];
return out;
}
Here's my fragment shader, which converts colors from YUV to RGB:
fragment float4 colorConvertProgram(VertexOut in [[stage_in]],
texture2d<float, access::sample> yTexture [[ texture(0) ]],
texture2d<float, access::sample> uvTexture [[ texture(1) ]],
sampler textureSampler [[ sampler(0) ]]) {
float3 colorOffset = float3(-(16.0/255.0), -0.5, -0.5);
float3x3 colorMatrix = float3x3(float3(1.164, 0.000, 1.596),
float3(1.164, -0.392, -0.813),
float3(1.164, 2.017, 0.000));
float3 yuv = float3(yTexture.sample(textureSampler, in.texcoord).r,
uvTexture.sample(textureSampler, in.texcoord).rg);
float3 rgb = (yuv + colorOffset) * colorMatrix;
return float4(rgb, 1.0);
}
And this is my Swift code that puts it all together:
let samplerDescriptor = MTLSamplerDescriptor()
samplerDescriptor.minFilter = .linear
samplerDescriptor.mipFilter = .linear
samplerDescriptor.sAddressMode = .clampToZero
samplerDescriptor.tAddressMode = .clampToZero
let sampler = device.makeSamplerState(descriptor: samplerDescriptor)
renderEncoder.setViewport(MTLViewport(originX: 0.0, originY: 0.0,
width: Double(bounds.width),
height: Double(bounds.height),
znear: -1.0, zfar: 1.0))
renderEncoder.setRenderPipelineState(renderPipelineState)
renderEncoder.setVertexBuffer(vertexPositionBuffer, offset: 0, index: 0)
renderEncoder.setVertexBuffer(vertexTexcoordBuffer, offset: 0, index: 1)
renderEncoder.setFragmentTexture(yTexture, index: 0)
renderEncoder.setFragmentTexture(uvTexture, index: 1)
renderEncoder.setFragmentSamplerState(sampler, index: 0)
renderEncoder.drawPrimitives(type: .triangleStrip, vertexStart: 0, vertexCount: 4)
renderEncoder.endEncoding()
commandBuffer.present(drawable)
commandBuffer.commit()

Calculating mean value in the metal kernel

Anyone knows a proper way to calculate mean value of the buffer with random float numbers in the metal kernel?
Dispatching work on the compute command encoder:
threadsPerGroup = MTLSizeMake(1, 1, inputTexture.arrayLength);
numThreadGroups = MTLSizeMake(1, 1, inputTexture.arrayLength / threadsPerGroup.depth);
[commandEncoder dispatchThreadgroups:numThreadGroups
threadsPerThreadgroup:threadsPerGroup];
Kernel code:
kernel void mean(texture2d_array<float, access::read> inTex [[ texture(0) ]],
device float *means [[ buffer(1) ]],
uint3 id [[ thread_position_in_grid ]]) {
if (id.x == 0 && id.y == 0) {
float mean = 0.0;
for (uint i = 0; i < inTex.get_width(); ++i) {
for (uint j = 0; j < inTex.get_height(); ++j) {
mean += inTex.read(uint2(i, j), id.z)[0];
}
}
float textureArea = inTex.get_width() * inTex.get_height();
mean /= textureArea;
out[id.z] = mean;
}
}
The buffer is represented in the texture of texture2d_array type with R32Float pixel format.
If you can use an array of uint (instead of float) as your data source, I would suggest using an "Atomic Fetch and Modify functions" (as described in the metal shading language spec) to write atomically to a buffer.
Here's an example of a kernel function which takes an input buffer (data: an array of Float) and writes the sum of the buffer into an atomic buffer (sum, a pointer to a uint):
kernel void sum(device uint *data [[ buffer(0) ]],
volatile device atomic_uint *sum [[ buffer(1) ]],
uint gid [[ thread_position_in_grid ]])
{
atomic_fetch_add_explicit(sum, data[gid], memory_order_relaxed);
}
In your swift file, you would set the buffers:
...
let data: [UInt] = [1, 2, 3, 4]
let dataBuffer = device.makeBuffer(bytes: &data, length: (data.count * MemoryLayout<UInt>.size), options: [])
commandEncoder.setBuffer(dataBuffer, offset: 0, at: 0)
var sum:UInt = 0
let sumBuffer = device!.makeBuffer(bytes: &sum, length: MemoryLayout<UInt>.size, options: [])
commandEncoder.setBuffer(sumBuffer, offset: 0, at: 1)
commandEncoder.endEncoding()
Commit, wait and then fetch the data from the GPU:
commandBuffer.commit()
commandBuffer.waitUntilCompleted()
let nsData = NSData(bytesNoCopy: sumBuffer.contents(),
length: sumBuffer.length,
freeWhenDone: false)
nsData.getBytes(&sum, length:sumBuffer.length)
let mean = Float(sum/data.count)
print(mean)
Alternatively, if your initial data source has to be an array of float, you could use the vDSP_meanv method of the Accelerate framework which is very fast for such computation.
I Hope that helped, cheers!

Indexed drawing with metal

I am trying to load a model (form .OBJ) and draw it to the screen on iOS with MetalKit. The problem is that instead of my model, I get some random polygons...
Here is the code that is tend to load the model(The code is based on a tutorial from raywenderlich.com:
let allocator = MTKMeshBufferAllocator(device: device)
let vertexDescriptor = MDLVertexDescriptor()
let vertexLayout = MDLVertexBufferLayout()
vertexLayout.stride = sizeof(Vertex)
vertexDescriptor.layouts = [vertexLayout]
vertexDescriptor.attributes = [MDLVertexAttribute(name: MDLVertexAttributePosition, format: MDLVertexFormat.Float3, offset: 0, bufferIndex: 0),
MDLVertexAttribute(name: MDLVertexAttributeColor, format: MDLVertexFormat.Float4, offset: sizeof(float3), bufferIndex: 0),
MDLVertexAttribute(name: MDLVertexAttributeTextureCoordinate, format: MDLVertexFormat.Float2, offset: sizeof(float3)+sizeof(float4), bufferIndex: 0),
MDLVertexAttribute(name: MDLVertexAttributeNormal, format: MDLVertexFormat.Float3, offset: sizeof(float3)+sizeof(float4)+sizeof(float2), bufferIndex: 0)]
var error: NSError?
let asset = MDLAsset(URL: path, vertexDescriptor: vertexDescriptor, bufferAllocator: allocator, preserveTopology: true, error: &error)
if error != nil{
print(error)
return nil
}
let model = asset.objectAtIndex(0) as! MDLMesh
let mesh = try MTKMesh(mesh: model, device: device)
And here is my drawing method:
func render(commandQueue: MTLCommandQueue, pipelineState: MTLRenderPipelineState,drawable: CAMetalDrawable,projectionMatrix: float4x4,modelViewMatrix: float4x4, clearColor: MTLClearColor){
dispatch_semaphore_wait(bufferProvider.availibleResourcesSemaphore, DISPATCH_TIME_FOREVER)
let renderPassDescriptor = MTLRenderPassDescriptor()
renderPassDescriptor.colorAttachments[0].texture = drawable.texture
renderPassDescriptor.colorAttachments[0].loadAction = .Clear
renderPassDescriptor.colorAttachments[0].clearColor = clearColor
renderPassDescriptor.colorAttachments[0].storeAction = .Store
let commandBuffer = commandQueue.commandBuffer()
commandBuffer.addCompletedHandler { (buffer) in
dispatch_semaphore_signal(self.bufferProvider.availibleResourcesSemaphore)
}
let renderEncoder = commandBuffer.renderCommandEncoderWithDescriptor(renderPassDescriptor)
renderEncoder.setCullMode(MTLCullMode.None)
renderEncoder.setRenderPipelineState(pipelineState)
renderEncoder.setVertexBuffer(vertexBuffer, offset: 0, atIndex: 0)
renderEncoder.setFragmentTexture(texture, atIndex: 0)
if let samplerState = samplerState{
renderEncoder.setFragmentSamplerState(samplerState, atIndex: 0)
}
var nodeModelMatrix = self.modelMatrix()
nodeModelMatrix.multiplyLeft(modelViewMatrix)
uniformBuffer = bufferProvider.nextUniformsBuffer(projectionMatrix, modelViewMatrix: nodeModelMatrix, light: light)
renderEncoder.setVertexBuffer(self.uniformBuffer, offset: 0, atIndex: 1)
renderEncoder.setFragmentBuffer(uniformBuffer, offset: 0, atIndex: 1)
if indexBuffer != nil{
renderEncoder.drawIndexedPrimitives(.Triangle, indexCount: self.indexCount, indexType: self.indexType, indexBuffer: self.indexBuffer!, indexBufferOffset: 0)
}else{
renderEncoder.drawPrimitives(.Triangle, vertexStart: 0, vertexCount: vertexCount, instanceCount: vertexCount/3)
}
renderEncoder.endEncoding()
commandBuffer.presentDrawable(drawable)
commandBuffer.commit()
}
Here is my vertex shader:
struct VertexIn{
packed_float3 position;
packed_float4 color;
packed_float2 texCoord;
packed_float3 normal;
};
struct VertexOut{
float4 position [[position]];
float3 fragmentPosition;
float4 color;
float2 texCoord;
float3 normal;
};
struct Light{
packed_float3 color;
float ambientIntensity;
packed_float3 direction;
float diffuseIntensity;
float shininess;
float specularIntensity;
};
struct Uniforms{
float4x4 modelMatrix;
float4x4 projectionMatrix;
Light light;
};
vertex VertexOut basic_vertex(
const device VertexIn* vertex_array [[ buffer(0) ]],
const device Uniforms& uniforms [[ buffer(1) ]],
unsigned int vid [[ vertex_id ]]) {
float4x4 mv_Matrix = uniforms.modelMatrix;
float4x4 proj_Matrix = uniforms.projectionMatrix;
VertexIn VertexIn = vertex_array[vid];
VertexOut VertexOut;
VertexOut.position = proj_Matrix * mv_Matrix * float4(VertexIn.position,1);
VertexOut.fragmentPosition = (mv_Matrix * float4(VertexIn.position,1)).xyz;
VertexOut.color = VertexIn.color;
VertexOut.texCoord = VertexIn.texCoord;
VertexOut.normal = (mv_Matrix * float4(VertexIn.normal, 0.0)).xyz;
return VertexOut;
}
And here is how it looks like:
link
Actually I have an other class that is completely written by me to load models. It works fine, the problem is that it is not using indexing so f I try to load models that are more complex than a low-poly sphere, the GPU crashes... Anyways I tried to modify it to use indexing and I got the same result.. than I added hardcoded indices for testing and I got a really weird result. When I had 3 indices it drew a triangle, when I added 3 more, it drew the same triangle and after 3 more vertices it drew 2 triangles...
Edit:
Here is my Vertex structure:
struct Vertex:Equatable{
var x,y,z: Float
var r,g,b,a: Float
var s,t: Float
var nX,nY,nZ:Float
func floatBuffer()->[Float]{
return [x,y,z,r,g,b,a,s,t,nX,nY,nZ]
}
}
I see a couple of potential issues here.
1) Your vertex descriptor does not map exactly to your Vertex struct. The position variables (x, y, z) occupy 12 bytes, so the color variables start at an offset of 12 bytes. This matches the packed_float3 position field in your shader's VertexIn struct, but in the vertex descriptor you provide to Model I/O, you use sizeof(Float3), which is 16, as the offset of the color attribute. Because you're packing the position field, you should use sizeof(Float) * 3 for this value instead, and likewise in the subsequent offsets. I suspect this is the main cause of your problems.
More generally, it's a good idea to use strideof rather than sizeof to account for alignment, though--by chance--it wouldn't make a difference here.
2) Model I/O is allowed to use a single MTLBuffer to store both vertices and indices, so you should use the offset member of each MTKMeshBuffer when setting the vertex buffer or specifying the index buffer in each draw call, rather than assuming the offsets to be 0.

Resources