Flickering Metal output texture... Why? - metal

Question

I am attempting to place a number of overlays (textures) on top of an existing texture. For the most part, this works fine.
However, for the life of me, I can't figure out why the output of this is sporadically "flickering" in my drawRect method of my MTKView. Everything seems fine; I do further processing on theTexture (in a kernel shader) after I loop with my placing my overlays. For some reason, I feel like this encoding is ending early and not enough work is getting done on it.
To clarify, everything starts out fine but about 5 seconds in, the flickering starts and gets progressively worse. For debugging purposes (right now, anyways) that loop runs only once -- there is only one overlay element. The input texture (theTexture) is bona-fide every time before I start (created with a descriptor where storageMode is MTLStorageModeManaged and usage is MTLTextureUsageUnknown).
I've also tried stuffing the encoder instantiation/ending inside the loop; no difference.
Can someone help me see what I'm doing wrong?
id<MTLTexture> theTexture; // valid input texture as "background"
MTLRenderPassDescriptor *myRenderPassDesc = [MTLRenderPassDescriptor renderPassDescriptor];
myRenderPassDesc.colorAttachments[0].texture = theTexture;
myRenderPassDesc.colorAttachments[0].storeAction = MTLStoreActionStore;
myRenderPassDesc.colorAttachments[0].loadAction = MTLLoadActionLoad;
id<MTLRenderCommandEncoder> myEncoder = [commandBuffer renderCommandEncoderWithDescriptor:myRenderPassDesc];
MTLViewport viewPort = {0.0, 0.0, 1920.0, 1080.0, -1.0, 1.0};
vector_uint2 imgSize = vector2((u_int32_t)1920,(u_int32_t)1080);
[myEncoder setViewport:viewPort];
[myEncoder setRenderPipelineState:metalVertexPipelineState];
for (OverlayWrapper *ow in overlays) {
id<MTLTexture> overlayTexture = ow.overlayTexture;
VertexRenderSet *v = [ow getOverlayVertexInfoPtr];
NSUInteger vSize = v->metalVertexCount*sizeof(AAPLVertex);
id<MTLBuffer> mBuff = [self.device newBufferWithBytes:v->metalVertices
length:vSize
options:MTLResourceStorageModeShared];
[myEncoder setVertexBuffer:mBuff offset:0 atIndex:0];
[myEncoder setVertexBytes:&imgSize length:sizeof(imgSize) atIndex:1];
[myEncoder setFragmentTexture:overlayTexture atIndex:0];
[myEncoder drawPrimitives:MTLPrimitiveTypeTriangle vertexStart:0 vertexCount:v->metalVertexCount];
}
[myEncoder endEncoding];
// do more work (kernel shader) with "theTexture"...
UPDATE #1:
I've attached a image of a "good" frame, with the vertex area (lower right) being shown. My encoder is responsible for placing the green stand-in "image" on top of the video frame theTexture at 30fps, which it does do. Just to clarify, theTexture is created for each frame (from a CoreVideo pixel buffer). After this encoder, I only read from the theTexture in a kernel shader to adjust brightness -- all that is working just fine.
My problems must exist elsewhere, as the video frames stop flowing (though the audio keeps going) and I end up alternating between 2 or 3 previous frames once this encoder is inserted (hence, the flicker). I believe now that my video pixel buffer vendor is being inadvertently supplanted by this "overlay" vendor.
If I comment out this entire vertex renderer, my video frames flow through just fine; it's NOT a problem with my video frame vendor.
UPDATE #2:
Here is the declaration of my rendering pipeline:
MTLRenderPipelineDescriptor *p = [[MTLRenderPipelineDescriptor alloc] init];
if (!p)
return nil;
p.label = #"Vertex Mapping Pipeline";
p.vertexFunction = [metalLibrary newFunctionWithName:#"vertexShader"];
p.fragmentFunction = [metalLibrary newFunctionWithName:#"samplingShader"];
p.colorAttachments[0].pixelFormat = MTLPixelFormatBGRA8Unorm;
NSError *error;
metalVertexPipelineState = [self.device newRenderPipelineStateWithDescriptor:p
error:&error];
if (error || !metalVertexPipelineState)
return nil;
Here is the texture descriptor used for creation of theTexture:
metalTextureDescriptor = [MTLTextureDescriptor texture2DDescriptorWithPixelFormat:MTLPixelFormatBGRA8Unorm
width:width
height:height
mipmapped:NO];
metalTextureDescriptor.storageMode = MTLStorageModePrivate;
metalTextureDescriptor.usage = MTLTextureUsageUnknown;
I haven't included the AAPLVertex and the vertex/fragment functions because of this: If I just comment out the OverlayWrapper loop in my rendering code (ie. don't even set vertex buffers or draw primitives), the video frames still flicker. The video is still playing but only 2-3 frames or so are playing in a continuous loop, from the time that this encoder "ran".
I've also added this code after the [... endEncoding] and changed the texture usage to MTLStorageModeManaged -- still, no dice:
id<MTLBlitCommandEncoder> blitEncoder = [commandBuffer blitCommandEncoder];
[blitEncoder synchronizeResource:crossfadeOutput];
[blitEncoder endEncoding];
To clarify a few things: The subsequent computer shader uses theTexture for input only. These are video frames; thusly, theTexture is re-created each time. Before it goes through this render stage, it has a bona-fide "background"
UPDATE #3:
I got this working, if by unconventional means.
I used this vertex shader to render my overlay onto a transparent background of a newly-created blank texture, specifically with my loadAction being MTLLoadActionClear with a clearColor of (0,0,0,0).
I then mixed this resulting texture with my theTexture with a kernel shader. I should not have to do this, but it works!

Answer 1

I had the same problem and wanted to explore a simpler solution before attempting #zzyzy's. This solution is also somewhat unsatisfying but at least seems to work.
The key (but inadequate in and of itself) is to reduce the buffering on the Metal layer:
metalLayer_.maximumDrawableCount = 2
Second, once the buffering was reduced, I found I had to go through a render/present/commit cycle to draw a trivial, invisible item with .clear set on the render pass descriptor — pretty straightforward:
renderPassDescriptor.colorAttachments[0].loadAction = .clear
(That there were a few invisible triangles drawn is probably irrelevant; it is probably the MTLLoadActionClear attribute that differentiates the pass. I used the same clear color as #zzyzy above and I think this echos the above solution.)
Third, I found I had to run the code through that render/present/commit cycle a second time — i.e., twice in a row. Of the three, this seems the most arbitrary and I don't pretend to understand it, but the three together worked for me.