CVOpenGLESTextureCacheCreateTextureFromImage for kCVPixelFormatType_OneComponent8 - ios

Question

I'm trying to use CVOpenGLESTextureCacheCreateTextureFromImage in order to use the reference in OpenGL, with no luck:
I have a CVPixelBufferRef pixel_bufferAlpha which gets updated using CVPixelBufferCreateWithBytes, with success :
CVReturn is successfull.
Then I try to use CVOpenGLESTextureCacheCreateTextureFromImage on a 1 channel texture ( alphaMatte ) to create a CVOpenGLESTextureRef that i can use in OpenGL.
I have initialised my CVOpenGLESTextureCacheRef _videoTextureAlphaCache :
err = CVOpenGLESTextureCacheCreate(
kCFAllocatorDefault,
nil,
eaglContext,
nil,
&_videoTextureAlphaCache);
And my CVPixelBufferRef pixel_bufferAlpha is initialised using :
cvret = CVPixelBufferCreate(kCFAllocatorDefault,
width,height,
kCVPixelFormatType_OneComponent8,
(__bridge CFDictionaryRef)cvBufferProperties,
&pixel_bufferAlpha);
if(cvret != kCVReturnSuccess)
{
assert(!"Failed to create shared opengl pixel_bufferAlpha");
}
I'm using kCVPixelFormatType_OneComponent8, as my MTLTexture passed in CVPixelBufferCreateWithBytes has a MTLPixelFormat 10 - > MTLPixelFormatR8Unorm.
When trying to use CVOpenGLESTextureCacheCreateTextureFromImage, I get an error "not opengl compatible" :
CVPixelBufferLockBaseAddress(pixel_bufferAlpha, 0);
err = noErr;
err = CVOpenGLESTextureCacheCreateTextureFromImage(kCFAllocatorDefault,
_videoTextureAlphaCache,
pixel_bufferAlpha,
NULL,
GL_TEXTURE_2D,
// internal
// GL_RGBA,
GL_ALPHA,
width,
height,
// gl format
// GL_BGRA_EXT,
// GL_R8_EXT,
GL_ALPHA8_EXT,
// gl type
// GL_UNSIGNED_INT_8_8_8_8_REV,
GL_UNSIGNED_BYTE,
NULL,
&alphaTextureGLES);
if (err != kCVReturnSuccess) {
CVBufferRelease(pixel_bufferAlpha);
if(err == kCVReturnInvalidPixelFormat){
NSLog(#"Invalid pixel format");
}
if(err == kCVReturnInvalidPixelBufferAttributes){
NSLog(#"Invalid pixel buffer attributes");
}
if(err == kCVReturnInvalidSize){
NSLog(#"invalid size");
}
if(err == kCVReturnPixelBufferNotOpenGLCompatible){
NSLog(#"CVOpenGLESTextureCacheCreateTextureFromImage::not opengl compatible");
}
}else{
NSLog(#"ok CVOpenGLESTextureCacheCreateTextureFromImage SUCCESS");
}
// ================================================================================ //
// clear texture cache
CVOpenGLESTextureCacheFlush(_videoTextureAlphaCache, 0);
CVPixelBufferUnlockBaseAddress(pixel_bufferAlpha, 0);
I'm not sure what I' doing wrong here.
Also I would appreciate any pointers as I'm not super versed in iOS and textures conversions / formats...
Best,
P
Full relevant part of the code :
alphaTexture = [matteDepthTexture generateMatteFromFrame:_session.currentFrame commandBuffer:commandBuffer];
// ===============================================================
NSUInteger texBytesPerRow = alphaTexture.bufferBytesPerRow;
NSUInteger texArrayLength = alphaTexture.arrayLength;
int width = (int) alphaTexture.width;
int height = (int) alphaTexture.height;
MTLPixelFormat texPixelFormat = alphaTexture.pixelFormat;
MTLTextureType texType = alphaTexture.textureType;
int bytesPerPixel = 8;
// MTLPixelFormatR8Unorm Ordinary format with one 8-bit normalized unsigned integer component.
NSLog(#" texPixelFormat of the texture is : %d", texPixelFormat);
NSLog(#" texType of the texture is : %d", texType);
CVReturn err = noErr;
err = CVPixelBufferCreateWithBytes(kCFAllocatorDefault,
width,
height,
kCVPixelFormatType_OneComponent8,
alphaTexture,
bytesPerPixel * width,
stillImageDataReleaseCallback,
alphaTexture,
NULL,
&pixel_bufferAlpha);
if (err != kCVReturnSuccess) {
if(err == kCVReturnInvalidPixelFormat){
NSLog(#"Invalid pixel format");
}
if(err == kCVReturnInvalidPixelBufferAttributes){
NSLog(#"Invalid pixel buffer attributes");
}
if(err == kCVReturnInvalidSize){
NSLog(#"invalid size");
}
if(err == kCVReturnPixelBufferNotOpenGLCompatible){
NSLog(#"CVPixelBufferCreateWithBytes::not opengl compatible");
}
}else{
NSLog(#"ok CVPixelBufferCreateWithBytes SUCCESS");
}
OSType sourcePixelFormat = CVPixelBufferGetPixelFormatType(pixel_bufferAlpha);
if (kCVPixelFormatType_OneComponent8 == sourcePixelFormat) {
NSLog(#" got format kCVPixelFormatType_OneComponent8");
} else{
NSLog(#" Unknown CoreVideo pixel format : %a", sourcePixelFormat);
}
// ================================================================================ //
CVPixelBufferLockBaseAddress(pixel_bufferAlpha, 0);
err = noErr;
err = CVOpenGLESTextureCacheCreateTextureFromImage(kCFAllocatorDefault,
_videoTextureAlphaCache,
pixel_bufferAlpha,
NULL,
GL_TEXTURE_2D,
// internal
// GL_RGBA,
GL_RED_EXT,
width,
height,
// gl format
// GL_BGRA_EXT,
// GL_R8_EXT,
GL_R8_EXT,
// gl type
// GL_UNSIGNED_INT_8_8_8_8_REV,
GL_UNSIGNED_BYTE,
NULL,
&alphaTextureGLES);
if (err != kCVReturnSuccess) {
CVBufferRelease(pixel_bufferAlpha);
if(err == kCVReturnInvalidPixelFormat){
NSLog(#"Invalid pixel format");
}
if(err == kCVReturnInvalidPixelBufferAttributes){
NSLog(#"Invalid pixel buffer attributes");
}
if(err == kCVReturnInvalidSize){
NSLog(#"invalid size");
}
if(err == kCVReturnPixelBufferNotOpenGLCompatible){
NSLog(#"CVOpenGLESTextureCacheCreateTextureFromImage::not opengl compatible");
}
}else{
NSLog(#"ok CVOpenGLESTextureCacheCreateTextureFromImage SUCCESS");
}
// ================================================================================ //
// clear texture cache
CVOpenGLESTextureCacheFlush(_videoTextureAlphaCache, 0);
CVPixelBufferUnlockBaseAddress(pixel_bufferAlpha, 0);

Answer 1

After digging a bit, I found an extension to MTLtexture from Alloy - great work there - :
here
getBytes to put the texture in a CVPixelBufferRef then
CVOpenGLESTextureCacheCreateTextureFromImage to store it in a CVOpenGLESTextureRef.
So in Objective - C it looks like this - with the same initialisation for the pixel_bufferAlpha and _videoTextureAlphaCache - :
alphaTexture = [matteDepthTexture generateMatteFromFrame:_session.currentFrame commandBuffer:commandBuffer];
int width = (int) alphaTexture.width;
int height = (int) alphaTexture.height;
MTLPixelFormat texPixelFormat = alphaTexture.pixelFormat;
CVPixelBufferLockBaseAddress(pixel_bufferAlpha, 0);
void * CV_NULLABLE pixelBufferBaseAdress = CVPixelBufferGetBaseAddress(pixel_bufferAlpha);
size_t bytesPerRow = CVPixelBufferGetBytesPerRow(pixel_bufferAlpha);
[alphaTexture getBytes:pixelBufferBaseAdress
bytesPerRow:bytesPerRow
fromRegion:MTLRegionMake2D(0, 0, width, height)
mipmapLevel:0];
size_t w = CVPixelBufferGetWidth(pixel_bufferAlpha);
size_t h = CVPixelBufferGetHeight(pixel_bufferAlpha);
CVReturn err = noErr;
err = CVOpenGLESTextureCacheCreateTextureFromImage(kCFAllocatorDefault,
_videoTextureAlphaCache,
pixel_bufferAlpha,
nil,
GLenum(GL_TEXTURE_2D),
GLint(GL_LUMINANCE),
w,
h,
GLenum(GL_LUMINANCE),
GLenum(GL_UNSIGNED_BYTE),
0,
&alphaTextureGLES);
if (err != kCVReturnSuccess) {
CVBufferRelease(pixel_bufferAlpha);
NSLog(#"error on CVOpenGLESTextureCacheCreateTextureFromImage");
}
CVPixelBufferUnlockBaseAddress(pixel_bufferAlpha, 0);
Hope this helps someone along the way.