Related
Curretly, I use glReadPixels in an iPad app to save the contents of an OpenGL texture in a frame buffer, which is terribly slow. The texture has a size of 1024x768, and I plan on supporting Retina display at 2048x1536. The data retrieved is saved in a file.
After reading from several sources, using CVOpenGLESTextureCache seems to be the only faster alternative. However, I could not find any guide or documentation as a good starting point.
How do I rewrite my code so it uses CVOpenGLESTextureCache? What parts of the code need to be rewritten? Using third-party libraries is not a preferred option unless there is already documentation on how to do this.
Code follows below:
//Generate a framebuffer for drawing to the texture
glGenFramebuffers(1, &textureFramebuffer);
glBindFramebuffer(GL_FRAMEBUFFER, textureFramebuffer);
//Create the texture itself
glGenTextures(1, &drawingTexture);
glBindTexture(GL_TEXTURE_2D, drawingTexture);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA32F_EXT, pixelWidth, pixelHeight, 0, GL_RGBA32F_EXT, GL_UNSIGNED_BYTE, NULL);
//When drawing to or reading the texture, change the active buffer like that:
glBindFramebuffer(GL_FRAMEBUFFER, textureFramebuffer);
glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, textureId, 0);
//When the data of the texture needs to be retrieved, use glReadPixels:
GLubyte *buffer = (GLubyte *) malloc(pixelWidth * pixelHeight * 4);
glReadPixels(0, 0, pixelWidth, pixelHeight, GL_RGBA, GL_UNSIGNED_BYTE, (GLvoid *)buffer);
I'm new to OpenGL so I'm not sure how to do this.
Currently I'm doing this to create an Alpha Texture in iOS :
GLuint framebuffer, renderBufferT;
glGenFramebuffers(1, &framebuffer);
glGenTextures(1, &renderBufferT);
glBindFramebuffer(GL_FRAMEBUFFER, framebuffer);
glBindTexture(GL_TEXTURE_2D, renderBufferT);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTexImage2D(GL_TEXTURE_2D, 0, GL_ALPHA, width, height, 0, GL_ALPHA, GL_UNSIGNED_BYTE, NULL);
glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, renderBufferT, 0);
GLenum status = glCheckFramebufferStatus(GL_FRAMEBUFFER);
if(status != GL_FRAMEBUFFER_COMPLETE)
{
NSLog(#"createAlphaBufferWithSize not complete %x", status);
return NO;
}
But it returns an error : GL_FRAMEBUFFER_INCOMPLETE_ATTACHMENT
And I also wonder how to write to this texture in the fragment shader. Is it simply the same with RGBA, like this :
gl_FragColor = vec1(0.5);
My intention is to use an efficient texture, because there are so much texture reading in my code, while I only need one component of color.
Thanks for any direction where I might go with this.
I'm not an iOS guy but that error indicates your OpenGL ES driver (PowerVR) does not support rendering to the GL_ALPHA format. I have not seen any OpenGL ES drivers that will do that on any platform. You can create GL_ALPHA textures to use with OpenGL ES using the texture compression tool in the PowerVR SDK, but I think the smallest format you can render to will be 16 bit color - if that is even available.
A better way to make textures efficient is to use compressed formats because the GPU decodes them with dedicated hardware. You really need the PowerVR Texture compression tool. It is a free download:
http://www.imgtec.com/powervr/insider/sdkdownloads/sdk_licence_agreement.asp
So I still haven't found the answers for my questions above, but I found a workaround.
In essence, since each pixel comprises of 4 color components, and I only need one for alpha only, then what I do is I use one texture to store 4 different logical alpha textures that I need. It takes a little effort to maintain these logical alpha textures.
And to draw to this one texture that contains 4 logical alpha textures, I use a shader and create a sign bit that marks which color component I intend to write to.
The blend I used is
(GL_ONE, GL_ONE_MINUS_SRC_COLOR),
And the fragment shader is like this :
uniform lowp vec4 uSignBit;
void main()
{
lowp vec4 textureColor = texture2D(uTexture,vTexCoord);
gl_FragColor = uSignBit * textureColor.a;
}
So, when I intend to write an alpha value of some texture to a logical alpha texture number 2, I write :
glBlendFunc(GL_ONE, GL_ONE_MINUS_SRC_COLOR);
glUniform4f(uSignBit, 0.0, 1.0, 0.0, 0.0);
I am using Xcode 4.5.1 and testing on an iPhone5 with iOS6.
I was using the frame capture function without problem, but suddenly it stopped working.
When I press the frame capture button, it seems the frame is captured, and the phone switches to a blank screen, only to suddenly switch back to the application screen, and the application keeps running. I can still debug and pause the application, but there's no way to get the frame capture. I don't see any errors in the console either.
The reason it stopped working is this piece of code. This code is supposed to render something to a rendertexture, but the rendertexture seems blank. I wanted to use the frame capture function to find out what's wrong, but the code itself won't let me capture... :(
Any idea why?
// ------------- init function -----------------
// Create the framebuffer and bind it
glGenFramebuffers(1, &g_framebuffer);
glBindFramebuffer(GL_FRAMEBUFFER, g_framebuffer);
//Create the destination texture, and attach it to the framebuffer’s color attachment point.
glGenTextures(1, &g_texture);
glBindTexture(GL_TEXTURE_2D, g_texture);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, w, h, 0, GL_RGBA, GL_UNSIGNED_BYTE, NULL);
glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, g_texture, 0);
//Test the framebuffer for completeness
GLenum status = glCheckFramebufferStatus(GL_FRAMEBUFFER) ;
if(status != GL_FRAMEBUFFER_COMPLETE) {
NSLog(#"failed to make complete framebuffer object %x", status);
} else {
NSLog(#"SkyPlugin initialized");
}
// ----------------- on update ------------------
glGetIntegerv(GL_FRAMEBUFFER_BINDING, &oldFBO);
glGetIntegerv(GL_VIEWPORT, oldViewPort);
// set the framebuffer and clear
glBindTexture(GL_TEXTURE_2D, 0);
glBindFramebuffer(GL_FRAMEBUFFER, g_framebuffer);
glViewport(0, 0, 32, 32);
//glClearColor(0.9f, 0.1f, 0.1f, 1.0f);
glDisable(GL_DEPTH_TEST);
glClear(GL_COLOR_BUFFER_BIT);
// Set shader
glUseProgram(m_program);
// do some glEnableVertexAttribArray
// ...
// texture setting
glActiveTexture(GL_TEXTURE0);
glUniform1i(m_uniform, 0);
ResourceManager* resourceManager = ResourceManager::GetInstance();
glBindTexture(GL_TEXTURE_2D, m_texture[0]);
// ----------- Draw -----------
// Draws a full-screen quad to copy textures
static const vertexDataUV quad[] = {
{/*v:*/{-1.f,-1.f,0}, /*t:*/{0,0}},
{/*v:*/{-1.f,1,0}, /*t:*/{0,1}},
{/*v:*/{1,-1.f,0}, /*t:*/{1,0}},
{/*v:*/{1,1,0}, /*t:*/{1,1}}
};
static const GLubyte indeces[] = {0,2,1,3};
glVertexAttribPointer(m_posAttrib, 3, GL_FLOAT, 0, sizeof(vertexDataUV), &quad[0].vertex);
glVertexAttribPointer(m_texCoordAttrib, 2, GL_FLOAT, 0, sizeof(vertexDataUV), &quad[0].uv);
glDrawElements(GL_TRIANGLE_STRIP, 4, GL_UNSIGNED_BYTE, indeces);
// ------------ End
// go back to the main framebuffer!
glBindFramebuffer(GL_FRAMEBUFFER, oldFBO);
glViewport(oldViewPort[0], oldViewPort[1], oldViewPort[2], oldViewPort[3]);
glEnable(GL_DEPTH_TEST);
//glClearColor(0.1f, 0.1f, 0.1f, 1.0f);
Edit: (2012/October/28)
I found out why the above code was not working. I forgot to bind a render buffer! The code below works, but still the frame capture fails when this code is active...
On init,
// Create the renderbuffer and bind it
glGenRenderbuffers(1, &g_renderbuffer);
glBindRenderbuffer(GL_RENDERBUFFER, g_renderbuffer);
glRenderbufferStorage(GL_RENDERBUFFER, GL_RGBA8_OES, w, h);
// Create the framebuffer and bind it
glGenFramebuffers(1, &g_framebuffer);
glBindFramebuffer(GL_FRAMEBUFFER, g_framebuffer);
glFramebufferRenderbuffer(GL_FRAMEBUFFER,
GL_COLOR_ATTACHMENT0,
GL_RENDERBUFFER, g_renderbuffer);
//Create the destination texture, and attach it to the framebuffer’s color attachment point.
glGenTextures(1, &g_texture);
glBindTexture(GL_TEXTURE_2D, g_texture);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, w, h, 0, GL_RGBA, GL_UNSIGNED_BYTE, NULL);
glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, g_texture, 0);
On update,
glGetIntegerv(GL_RENDERBUFFER_BINDING, &oldRBO);
glGetIntegerv(GL_FRAMEBUFFER_BINDING, &oldFBO);
glGetIntegerv(GL_VIEWPORT, oldViewPort);
// set the framebuffer and clear
glBindTexture(GL_TEXTURE_2D, 0);
glBindFramebuffer(GL_FRAMEBUFFER, g_framebuffer);
glBindRenderbuffer(GL_RENDERBUFFER, g_renderbuffer);
glViewport(0, 0, 32, 32);
// ... draw stuff ...
End of update,
// go back to the main framebuffer!
glBindFramebuffer(GL_FRAMEBUFFER, oldFBO);
glBindRenderbuffer(GL_RENDERBUFFER, oldRBO);
It seems it was a bug of Xcode.
The latest version, Xcode 4.5.2 lets me capture the frame :)
After I capture the frame, I get an error in this part of the code:
// ... draw stuff ...
glActiveTexture(GL_TEXTURE0);
glUniform1i(MY_TEXTURE, 0);
On the glUniform1i I get this error: "The specified operation is invalid for the current OpenGL state".
No idea why I get this error (the rendering it's working), but I suspect this error may be the reason why I wasn't able to capture a frame in the previous version of Xcode...
I've seen very similar behavior, and while it was a bit erratic, it did seem to be related to memory usage. Usually when it failed, the replay application seemed to be running out of memory (I'd see a memory warning in the console when it failed).
Switching to a device with more memory fixed it (iPad 4 from iPad 3), but I also was able to occasionally work around it by reducing the amount of texture memory used -- skipping setting the top mipmap for all my textures would usually free up enough.
I try to use the "stencil buffer" to display a part of my rendering from a texture, but my render is displayed without any mask effect.
It's for a 2D iOS project, with OpenGL ES 2.0
This is the concerned part of my code :
glClear( GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT | GL_STENCIL_BUFFER_BIT );
glEnable( GL_STENCIL_TEST );
// mask rendering
glColorMask( GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE );
glStencilFunc( GL_ALWAYS, 1, 1 );
glStencilOp( GL_KEEP, GL_KEEP, GL_REPLACE );
glBindTexture(GL_TEXTURE_2D, _maskTexture);
glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);
// scene rendering
glColorMask( GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE );
glStencilFunc( GL_EQUAL, 1, 1 );
glStencilOp( GL_KEEP, GL_KEEP, GL_KEEP );
glBindTexture(GL_TEXTURE_2D, _viewTexture);
glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);
Any help would be greatly appreciated !
(As usual for a French developer, sorry for my English !)
precision : "_maskTexture" is a black & white picture.
Solution :
I finnaly solved my problem with the indications of rotoglub and tim.
Thank you both.
1/ The stencil buffer has not been created correctly.
It should be initialized like this:
glBindRenderbufferOES(GL_RENDERBUFFER_OES, depthStencilRenderbuffer);
glRenderbufferStorageOES(GL_RENDERBUFFER_OES,
GL_DEPTH24_STENCIL8_OES,
backingWidth,
backingHeight);
It's the principal reason why my rendering was not affected by my mask.
2/ To be able to make use of a texture as a mask, I replaced the black color with an alpha channel and enable blending in my rendering.
My final rendering code looks like this :
glEnableClientState(GL_VERTEX_ARRAY);
glEnableClientState(GL_TEXTURE_COORD_ARRAY);
glVertexPointer(2, GL_FLOAT, 0, vertices);
glTexCoordPointer(2, GL_FLOAT, 0, texcoords);
glClearStencil(0);
glClearColor (0.0,0.0,0.0,1);
glClear (GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT | GL_STENCIL_BUFFER_BIT);
// mask rendering
glColorMask( GL_FALSE, GL_FALSE, GL_FALSE, GL_FALSE );
glEnable(GL_STENCIL_TEST);
glEnable(GL_ALPHA_TEST);
glBlendFunc( GL_ONE, GL_ONE );
glAlphaFunc( GL_NOTEQUAL, 0.0 );
glStencilFunc(GL_ALWAYS, 1, 1);
glStencilOp(GL_KEEP, GL_KEEP, GL_REPLACE);
glBindTexture(GL_TEXTURE_2D, _mask);
glDrawArrays(GL_TRIANGLE_STRIP, 4, 4);
// scene rendering
glColorMask(GL_TRUE, GL_TRUE, GL_TRUE, GL_TRUE);
glStencilFunc(GL_EQUAL, 1, 1);
glStencilOp(GL_KEEP, GL_KEEP, GL_KEEP);
glDisable(GL_STENCIL_TEST);
glDisable(GL_ALPHA_TEST);
glBindTexture(GL_TEXTURE_2D, _texture);
glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);
Simply, the problem is that you're just drawing a texture to the scene without doing any testing of what's in the texture. The stencil buffer doesn't care about the colors in the texture, it just checks:
Did you draw a fragment ? (Update stencil buffer) : (Don't update stencil buffer);
You're drawing a fragment for every pixel of your texture, so any mask effect in the texture is useless.
If you want to mask with a texture, you need to discard any fragments that you don't want updated in the stencil buffer.
This is either done with discard keyword in shader, or glAlphaTest/glAlphaFunc in OpenGLES 1.1
Why is glTexSubImage2D() suddenly causing GL_INVALID_OPERATION?
I'm trying to upgrade my hopelessly outdated augmented reality app from iOS4.x to iOS5.x, but I'm having difficulties. I run iOS5.0. Last week I ran iOS4.3. My device is an iPhone4.
Here is a snippet from my captureOutput:didOutputSampleBuffer:fromConnection: code
uint8_t *baseAddress = /* pointer to camera buffer */
GLuint texture = /* the texture name */
glBindTexture(GL_TEXTURE_2D, texture);
glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, 480, 360, GL_BGRA, GL_UNSIGNED_BYTE, baseAddress);
/* now glGetError(); -> returns 0x0502 GL_INVALID_OPERATION on iOS5.0, works fine on iOS4.x */
Here is a snippet from my setup code
GLuint texture = /* the texture name */
glGenTextures(1, &texture);
glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_2D, texture);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER,GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MAG_FILTER,GL_LINEAR);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, 512, 512, 0, GL_RGBA, GL_UNSIGNED_BYTE, NULL);
For simplicity I have inserted hardcoded values here. In my actual code I obtain these values with CVPixelBufferGetWidth/Height/BaseAddress. The EAGLContext is initialized with kEAGLRenderingAPIOpenGLES2.
Ah.. I fixed it immediately after posting this question. Had to change GL_RGBA into GL_BRGA.
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, 512, 512, 0, GL_BGRA, GL_UNSIGNED_BYTE, NULL);
Hope it helps someone.
BTW. If you want to write AR apps then consider using CVOpenGLESTextureCache instead of using glTexSubImage2d. It's supposed to be faster.