I want to use array of textures in metal shader, but it crash
when running my app(iPhone6, A8), here is the error log:Failed to created pipeline state, error Error Domain=AGXMetalA8 Code=3 "Could not resolve texture/sampler references" UserInfo={NSLocalizedDescription=Could not resolve texture/sampler references}
I have try to google but did not find useful information, can anyone give me some suggestions, Thanks.
Here is my code:
fragment float4 fragment_texture (ShaderInput vert [[stage_in]],
array<texture2d<float>, 2> u_texture [[texture(0)]],
sampler _mtlsmp_u_texture [[sampler(0)]])
{
float4 srcColor = u_texture[0].sample(_mtlsmp_u_texture, vert.v_texCoord);
float4 materialColor = u_texture[1].sample(_mtlsmp_u_texture, vert.v_texCoord);
float4 mixColor = mix(srcColor, materialColor, 0.5);
return mixColor;
}
In my app code:
[renderEncoder setFragmentTexture:_textureDemo
atIndex:0];
[renderEncoder setFragmentTexture:_textureBlend
atIndex:1];
[renderEncoder setFragmentSamplerState:_sampler atIndex:0];
Update Issues
I try to use Argument Buffers to deal with array of textures, but still crashed on my iPhone6 with version iOS 11.4 and get the same error info:Failed to create pipeline state, error Could not resolve texture/sampler references
Here are some critical steps:
In my app code, argumentEncoder is a MTLArgumentEncoder type object and I encode the texture resources into the argument buffer by calling setTexture:atIndex:
[argumentEncoder setArgumentBuffer:_fragmentShaderArgumentBuffer offset:0];
[argumentEncoder setTexture:_texture[0] atIndex:0];
[argumentEncoder setTexture:_texture[1] atIndex:1];
Calling the useResource:usage: method to make texture resources accessible to the GPU:
[renderEncoder useResource:_texture[0] usage:MTLResourceUsageSample];
[renderEncoder useResource:_texture[1] usage:MTLResourceUsageSample];
And set argument buffer _fragmentShaderArgumentBuffer as an argument to the fragment function:
[renderEncoder setFragmentBuffer:_fragmentShaderArgumentBuffer
offset:0
atIndex:0];
In my fragment shader:
typedef struct FragmentShaderArguments {
array<texture2d<float>, 2> exampleTextures [[ id(0) ]];
} FragmentShaderArguments;
fragment float4
fragmentShader( RasterizerData in [[ stage_in ]],
device FragmentShaderArguments & fragmentShaderArgs [[ buffer(0) ]])
{
constexpr sampler textureSampler (mag_filter::linear,
min_filter::linear);
float4 color = float4(0, 0, 0, 1);
float4 color1 = fragmentShaderArgs.exampleTextures[0].sample(textureSampler, in.texCoord);
float4 color2 = fragmentShaderArgs.exampleTextures[1].sample(textureSampler, in.texCoord);
color = mix(color1, color2, 0.5);
return color;
}
Sincerely hope that someone can provide ideas to me, Thanks!
I upgrade to iOS 12.0.1, and it works well.
Related
I am building a minimalistic 3D engine in Metal and I want my vertex and fragment shader code to be as much reusable as possible so that my vertex shader can for instance be used without being changed no matter its input mesh vertex data layout.
An issue I have is that I can't guarantee all meshes will have the same attributes, for instance a mesh may just contain its position and normal data while another may additionally have UV coordinates attached.
Now my first issue is that if I define my vertex shader input structure like this:
struct VertexIn {
float3 position [[ attribute(0) ]];
float3 normal [[ attribute(1) ]];
float2 textureCoordinate [[ attribute(2) ]];
};
I wonder what is the consequence of doing so if there was no specified attribute 2 in my metal vertex descriptor? My tests seem to indicate there is no crash (at least of just defining such an argument in the input texture), but I wonder if this is just undefined behavior or if this is actually safe to do?
Another issue I have is that I might want to pass the uv texture info to the fragment shader (ie: return it from my vertex shader), but what happens if it is missing? It feel like except if specifically designed this way, it would be undefined behavior to access textureCoordinate to set its value to a property of some VertexOut structure I return from my vertex shader.
Additionally I notice that Apple's RealityKit framework must have found some way around this issue: it enables users to point to "shader modifier" functions that are passed the data of both vertex and fragment shaders so that they can act on it, what surprises me is that the structures the user functions are passed define a lot of properties which I am not sure are always defined for all meshes (for instance, a second UV texture). This seems pretty similar to the problem I am trying to solve.
Am I missing some obvious way to fix this issue?
I think the intended way to deal with this is function constants. This is an example of how I deal with this in my vertex shaders.
constant bool HasColor0 [[ function_constant(FunctionConstantHasColor0) ]];
constant bool HasNormal [[ function_constant(FunctionConstantHasNormal) ]];
constant bool HasTangent [[ function_constant(FunctionConstantHasTangent) ]];
constant bool HasTexCoord0 [[ function_constant(FunctionConstantHasTexCoord0) ]];
constant bool AlphaMask [[ function_constant(FunctionConstantAlphaMask) ]];
// ...
struct VertexIn
{
float3 position [[ attribute(AttributeBindingPosition) ]];
float3 normal [[ attribute(AttributeBindingNormal), function_constant(HasNormal) ]];
float4 tangent [[ attribute(AttributeBindingTangent), function_constant(HasTangent) ]];
float4 color [[ attribute(AttributeBindingColor0), function_constant(HasColor0) ]];
float2 texCoord [[ attribute(AttributeBindingTexcoord0), function_constant(HasTexCoord0) ]];
};
struct VertexOut
{
float4 positionCS [[ position ]];
float4 tangentVS = float4();
float3 positionVS = float3();
float3 normalVS = float3();
float2 texCoord = float2();
half4 color = half4();
};
static VertexOut ForwardVertexImpl(Vertex in, constant CameraUniform& camera, constant MeshUniform& meshUniform)
{
VertexOut out;
float4x4 viewModel = camera.view * meshUniform.model;
float4 positionVS = viewModel * float4(in.position.xyz, 1.0);
out.positionCS = camera.projection * positionVS;
out.positionVS = positionVS.xyz;
float4x4 normalMatrix;
if(HasNormal || HasTangent)
{
normalMatrix = transpose(meshUniform.inverseModel * camera.inverseView);
}
if(HasNormal)
{
out.normalVS = (normalMatrix * float4(in.normal, 0.0)).xyz;
}
if(HasTexCoord0)
{
out.texCoord = in.texCoord;
}
if(HasColor0)
{
out.color = half4(in.color);
}
else
{
out.color = half4(1.0);
}
if(HasTangent)
{
// Normal matrix or viewmodel matrix?
out.tangentVS.xyz = (normalMatrix * float4(in.tangent.xyz, 0.0)).xyz;
out.tangentVS.w = in.tangent.w;
}
return out;
}
vertex VertexOut ForwardVertex(
VertexIn in [[ stage_in ]],
constant CameraUniform& camera [[ buffer(BufferBindingCamera) ]],
constant MeshUniform& meshUniform [[ buffer(BufferBindingMesh) ]])
{
Vertex v
{
.color = in.color,
.tangent = in.tangent,
.position = in.position,
.normal = in.normal,
.texCoord = in.texCoord,
};
return ForwardVertexImpl(v, camera, meshUniform);
}
And in the host application I fill out the MTLFunctionConstantValues object based on the semantics geometry actually has:
func addVertexDescriptorFunctionConstants(toConstantValues values: MTLFunctionConstantValues) {
var unusedSemantics = Set<AttributeSemantic>(AttributeSemantic.allCases)
for attribute in attributes.compactMap({ $0 }) {
unusedSemantics.remove(attribute.semantic)
if let constant = attribute.semantic.functionConstant {
values.setConstantValue(true, index: constant)
}
}
for unusedSemantic in unusedSemantics {
if let constant = unusedSemantic.functionConstant {
values.setConstantValue(false, index: constant)
}
}
}
A good thing about it is that compiler should turn those function constants ifs into code without branches, so it shouldn't really be a problem during runtime AND this lets you compile your shaders offline without having to use online compilation and defines.
In Metal Shading Language Specification, Behavior of Uniform Type:
If a variable is of the uniform type, and the variable does not have the same value for all threads executing the kernel or graphics function, then the behavior is undefined.
uniform<float> bar[10]; // elements stored in bar array are uniform
My question is what is use for uniform<T> in this context? I know some uniforms such as transformation information can be passed to shader by setVertexBytes() as uniform. In what kind of situations, do you find uniform<T> useful?
struct VertexOut {
float4 position [[position]];
};
struct Uniforms {
float4x4 transform;
};
vertex VertexOut my_vertex(
const device VertexIn * vertices [[buffer(0)]],
constant Uniforms & uniforms [[buffer(1)]],
uint vid [[vertex_id]]
) {
…
}
Thanks
I'm building an app rendering 2D geometry in Metal.
Right now, the positions of the vertices are solved from within the vertex function. What I'd like is to write the solved positions back to a buffer from inside that same vertex function.
I'm under the impression that this is possible although in my first attempt to do it i.e.:
vertex VertexOut basic_vertex(device VertexIn *vertices [[ buffer(0) ]],
device VertexOut *solvedVertices [[ buffer(1) ]],
vid [[ vertex_id ]])
{
VertexIn in vertices[vid];
VertexOut out;
out.position = ... // Solve the position of the vertex
solvedVertices[vid] = out // Write to the buffer later to be read by CPU
return out;
}
I was graced with the presence of this compile time error:
Okay, so a few solutions come to my head - I could solve for the vertex positions in a first - non-rasterizing - pass through a vertex function declared like:
vertex void solve_vertex(device VertexIn *unsolved [[ buffer(0) ]],
device VertexOut *solved [[ buffer(1) ]],
vid [[ vertex_id ]])
{
solved[vid] = ...
}
And then pipe those solved vertices into a now much simpler - rasterizing - vertex function.
Another solution that could work but seems less appealing could be to solve them in a compute function.
So, what is the best way forward in a situation like this? From my little bits of research, I could track down that this same sort of procedure is done in Transform Feedback but I've had no luck (other than the link at the begging of the question) finding examples in Apple's documentation/sample code or elsewhere on the web for best practices when facing this sort of problem.
Alright, it turns out using a non-rasterizing vertex function is the way to go. There are some things to note however for others future reference:
A non-rasterizing vertex function is simply a vertex function returning void i.e.:
vertex void non_rasterizing_vertex(...) { }
When executing a non-rasterizing "render" pass, the MTLRenderPassDescriptor still needs to have a texture set - for instance in MTLRenderPassDescriptor's colorAttachments[0].texture - for reasons I don't know (I assume it's just due to the fixed nature of GPU programming).
The MTLRenderPipelineState needs to have it's rasterizationEnabled property set to false, then you can assign the non-rasterizing vertex function to it's vertexFunction property. The fragmentFunction property can remain nil as expected.
When actually executing the pass, one of the drawPrimitives: methods (the naming of which may be misleading) still needs to be invoked on the configured MTLRenderCommandEncoder. I ended up with a call to render MTLPrimitiveType.Points since that seems the most sensical.
Doing all of this sets up "rendering" logic ready to write back to vertex buffers from the vertex function - so long as they're in device address space:
vertex void non_rasterizing_vertex(device float *writeableBuffer [[ buffer(0) ]],
uint vid [[ vertex_id ]])
{
writeableBuffer[vid] = 42; // Write away!
}
This "answer" ended up more like a blog post but I hope it remains useful for future reference.
TODO
I'd still like to investigate performance tradeoffs between doing compute-y work like this in a compute pipeline versus in the rendering pipeline like above. Once I have some more time to do that, I'll update this answer.
The correct solution is to move any code writing to buffers to a compute kernel.
You will loose a great deal of performance writing to buffers in a vertex function. It is optimized for rasterizing, not for computation.
You just need to use a compute command encoder.
guard let computeBuffer = commandQueue.makeCommandBuffer() else { return }
guard let computeEncoder = computeBuffer.makeComputeCommandEncoder() else { return }
computeEncoder.setComputePipelineState(solveVertexPipelineState)
kernel void solve_vertex(device VertexIn *unsolved [[ buffer(0) ]],
device VertexOut *solved [[ buffer(1) ]],
vid [[ instance ]])
{
solved[vid] = ...
}
When I try to write to a float texture from a kernel, I get the error:
/SourceCache/AcceleratorKit/AcceleratorKit-17.7/ToolsLayers/Debug/MTLDebugComputeCommandEncoder.mm:596: failed assertion `Non-writable texture format MTLPixelFormatR32Float is being bound at index 2 to a shader argument with write access enabled.'
However, when I go check in the documentation, that format is color-renderable and function-writeable (see table at the bottom):
https://developer.apple.com/library/prerelease/ios/documentation/Metal/Reference/MetalConstants_Ref/index.html#//apple_ref/c/tdef/MTLPixelFormat
Partial code:
// texture creation
MTLTextureDescriptor *floatTextureDescriptor = [MTLTextureDescriptor texture2DDescriptorWithPixelFormat:MTLPixelFormatR32Float width:_width height:_height mipmapped:NO];
_myFloatTexture = [self.device newTextureWithDescriptor:floatTextureDescriptor];
// texture binding
[computeCommandEncoder setTexture:_myFloatTexture atIndex:2];
// texture used in the shader
void kernel myKernel(//...
texture2d<float, access::write> myFloats [[ texture(2) ]],
uint2 gid [[ thread_position_in_grid ]])
Am I doing something wrong or might this be a bug?
They are supported only from iOS 9.
I want to apply a pixel shader onto my background sprite, to create some sort of lighting.
So i draw a Render Target with the light on it and want to merge it onto the background via the Pixel shader.
This is the essential code:
GraphicsDevice.Clear(Color.Black);
spriteBatch.Begin(SpriteSortMode.Immediate, BlendState.AlphaBlend);
lighting.Parameters["lightMask"].SetValue(lightingMask);
lighting.CurrentTechnique.Passes[0].Apply();
spriteBatch.Draw(hexBack, new Vector2(0, 0), Color.White);
spriteBatch.End();
In this case, hexBack is the Rendertarget with a simple sprite drawn in it and lightingMask is the rendertarget with the light texture in it.
Both are Backbuffer width and height.
So when i try to run the program, it crashes with:
XNA Framework Reach profile requires TextureAddressMode to be Clamp when using texture sizes that are not powers of two.
So i tried to set up clamping, but i cant find a way to get it working.
The shader code:
texture lightMask;
sampler mainSampler : register(s0);
sampler lightSampler = sampler_state{Texture = lightMask;};
struct PixelShaderInput
{
float4 TextureCoords: TEXCOORD0;
};
float4 PixelShaderFunction(PixelShaderInput input) : COLOR0
{
float2 texCoord = input.TextureCoords;
float4 mainColor = tex2D(mainSampler, texCoord);
float4 lightColor = tex2D(lightSampler, texCoord);
return mainColor * lightColor;
}
technique Technique1
{
pass Pass1
{
PixelShader = compile ps_2_0 PixelShaderFunction();
}
}
Thanks for your help!
pcnx
If you are unable to use power of two textures, you have to change your Spritebath.begin call and specify a SamplerState. The minimum to specify should be
public void Begin (
SpriteSortMode sortMode,
BlendState blendState,
SamplerState samplerState,
DepthStencilState depthStencilState,
RasterizerState rasterizerState
)
The error refers to your texture addressing mode (ie: does the texture wrap around at the edges, or is it clamped at the edges). Nothing to do with a shader.
Use one of the overloads for SpriteBatch.Begin (MSDN) that takes a SamplerState, and pass in SamplerState.LinearClamp (MSDN).
The default for SpriteBatch.Begin is SamplerState.LinearClamp, so you must be setting a different state (eg: LinearWrap) onto the graphics device somewhere else in your code? Don't do that.
(Alternately: change from the Reach profile to the HiDef profile in your project settings.)