I am trying to draw a Core Graphics image generated (at screen resolution) into OpenGL. However, the image is rendering more aliased than the CG output (antialiasing is disabled in CG). The text is the texture (the blue background is respectively drawn in Core Graphics for the first image and OpenGL for the second).
CG Output:
OpenGL Render (in simulator):
Framebuffer setup:
context = [[EAGLContext alloc] initWithAPI:kEAGLRenderingAPIOpenGLES2];
[EAGLContext setCurrentContext:context];
glGenRenderbuffers(1, &onscrRenderBuffer);
glBindRenderbuffer(GL_RENDERBUFFER, onscrRenderBuffer);
[context renderbufferStorage:GL_RENDERBUFFER fromDrawable:self.layer];
glGenFramebuffers(1, &onscrFramebuffer);
glBindFramebuffer(GL_FRAMEBUFFER, onscrFramebuffer);
glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_RENDERBUFFER, onscrRenderBuffer);
Texture Loading Code:
-(GLuint) loadTextureFromImage:(UIImage*)image {
CGImageRef textureImage = image.CGImage;
size_t width = CGImageGetWidth(textureImage);
size_t height = CGImageGetHeight(textureImage);
GLubyte* spriteData = (GLubyte*) malloc(width*height*4);
CGColorSpaceRef cs = CGImageGetColorSpace(textureImage);
CGContextRef c = CGBitmapContextCreate(spriteData, width, height, 8, width*4, cs, kCGImageAlphaPremultipliedLast | kCGBitmapByteOrder32Big);
CGColorSpaceRelease(cs);
CGContextScaleCTM(c, 1, -1);
CGContextTranslateCTM(c, 0, -CGContextGetClipBoundingBox(c).size.height);
CGContextDrawImage(c, (CGRect){CGPointZero, {width, height}}, textureImage);
CGContextRelease(c);
GLuint glTex;
glGenTextures(1, &glTex);
glBindTexture(GL_TEXTURE_2D, glTex);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, (GLsizei)width, (GLsizei)height, 0, GL_RGBA, GL_UNSIGNED_BYTE, spriteData);
glBindTexture(GL_TEXTURE_2D, 0);
free(spriteData);
return glTex;
}
Vertices:
struct vertex {
float position[3];
float color[4];
float texCoord[2];
};
typedef struct vertex vertex;
const vertex bgVertices[] = {
{{1, -1, 0}, {0, 167.0/255.0, 253.0/255.0, 1}, {1, 0}}, // BR (0)
{{1, 1, 0}, {0, 222.0/255.0, 1.0, 1}, {1, 1}}, // TR (1)
{{-1, 1, 0}, {0, 222.0/255.0, 1.0, 1}, {0, 1}}, // TL (2)
{{-1, -1, 0}, {0, 167.0/255.0, 253.0/255.0, 1}, {0, 0}} // BL (3)
};
const vertex textureVertices[] = {
{{1, -1, 0}, {0, 0, 0, 0}, {1, 0}}, // BR (0)
{{1, 1, 0}, {0, 0, 0, 0}, {1, 1}}, // TR (1)
{{-1, 1, 0}, {0, 0, 0, 0}, {0, 1}}, // TL (2)
{{-1, -1, 0}, {0, 0, 0, 0}, {0, 0}} // BL (3)
};
const GLubyte indicies[] = {
3, 2, 0, 1
};
Render Code:
glClear(GL_COLOR_BUFFER_BIT);
GLsizei width, height;
glGetRenderbufferParameteriv(GL_RENDERBUFFER, GL_RENDERBUFFER_WIDTH, &width);
glGetRenderbufferParameteriv(GL_RENDERBUFFER, GL_RENDERBUFFER_HEIGHT, &height);
glViewport(0, 0, width, height);
glBindBuffer(GL_ARRAY_BUFFER, bgVertexBuffer);
glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, indexBuffer);
glVertexAttribPointer(positionSlot, 3, GL_FLOAT, GL_FALSE, sizeof(vertex), 0);
glVertexAttribPointer(colorSlot, 4, GL_FLOAT, GL_FALSE, sizeof(vertex), (GLvoid*)(sizeof(float)*3));
glVertexAttribPointer(textureCoordSlot, 2, GL_FLOAT, GL_FALSE, sizeof(vertex), (GLvoid*)(sizeof(float)*7));
glDrawElements(GL_TRIANGLE_STRIP, sizeof(indicies)/sizeof(indicies[0]), GL_UNSIGNED_BYTE, 0);
glBindBuffer(GL_ARRAY_BUFFER, textureVertexBuffer);
glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, indexBuffer);
glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_2D, texture);
glUniform1i(textureUniform, 0);
glVertexAttribPointer(positionSlot, 3, GL_FLOAT, GL_FALSE, sizeof(vertex), 0);
glVertexAttribPointer(colorSlot, 4, GL_FLOAT, GL_FALSE, sizeof(vertex), (GLvoid*)(sizeof(float)*3));
glVertexAttribPointer(textureCoordSlot, 2, GL_FLOAT, GL_FALSE, sizeof(vertex), (GLvoid*)(sizeof(float)*7));
glDrawElements(GL_TRIANGLE_STRIP, sizeof(indicies)/sizeof(indicies[0]), GL_UNSIGNED_BYTE, 0);
glBindTexture(GL_TEXTURE_2D, 0);
I am using the blend function glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA) in case that has anything to do with it.
Any ideas where the problem is?
Your GL-rendered output looks all pixelated because it has fewer pixels. Per the Drawing and Printing Guide for iOS, the default scale factor for a CAEAGLLayer is 1.0, so when you set up your GL render buffers, you get one pixel in the buffer per point. (Remember, a point is a unit of UI layout, which on modern devices with Retina displays works out to several hardware pixels.) When you render that buffer full-screen, everything gets scaled up (by about 2.61x on an iPhone 6(s) Plus).
To render at the native screen resolution, you need to increase the contentScaleFactor of your view. (Preferably, you should do this early on, before, setting up renderbuffers, so that they get the new scale factor from the view's layer.)
Watch out, though: you want to use the UIScreen property nativeScale, not scale. The scale property reflects UI rendering, where, on iPhone 6(s) Plus, everything gets done at 3x and then scaled down slightly to the native resolution of the display. The nativeScale property reflects the number of actual device pixels per point — if you're doing GPU rendering, you want to target that so you don't sap performance by drawing more pixels than you need to. (On current devices other than the "Plus" iPhones, scale and nativeScale are the same. But using the latter is probably a good insurance policy.)
You can avoid a lot of these kinds of issues (and others) by letting GLKView do renderbuffer setup for you. Even if you're writing cross-platform GL, that part of your code is going to have to be pretty platform- and device-specific anyway, so you might as well reduce the amount of it that you have to write and maintain.
(Addressing previous edits of the question, for posterity's sake: this has nothing to do with multisampling or the quality of the GL texture data. Multisampling has to do with rasterization of polygon edges — points in the interior of a polygon get one fragment per pixel, but points on the edges get multiple fragments whose colors are blended in the resolve stage. And if you bind the texture to an FBO and glReadPixels from it, you'll find the image is pretty much the same one you put in.)
Curretly, I use glReadPixels in an iPad app to save the contents of an OpenGL texture in a frame buffer, which is terribly slow. The texture has a size of 1024x768, and I plan on supporting Retina display at 2048x1536. The data retrieved is saved in a file.
After reading from several sources, using CVOpenGLESTextureCache seems to be the only faster alternative. However, I could not find any guide or documentation as a good starting point.
How do I rewrite my code so it uses CVOpenGLESTextureCache? What parts of the code need to be rewritten? Using third-party libraries is not a preferred option unless there is already documentation on how to do this.
Code follows below:
//Generate a framebuffer for drawing to the texture
glGenFramebuffers(1, &textureFramebuffer);
glBindFramebuffer(GL_FRAMEBUFFER, textureFramebuffer);
//Create the texture itself
glGenTextures(1, &drawingTexture);
glBindTexture(GL_TEXTURE_2D, drawingTexture);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA32F_EXT, pixelWidth, pixelHeight, 0, GL_RGBA32F_EXT, GL_UNSIGNED_BYTE, NULL);
//When drawing to or reading the texture, change the active buffer like that:
glBindFramebuffer(GL_FRAMEBUFFER, textureFramebuffer);
glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, textureId, 0);
//When the data of the texture needs to be retrieved, use glReadPixels:
GLubyte *buffer = (GLubyte *) malloc(pixelWidth * pixelHeight * 4);
glReadPixels(0, 0, pixelWidth, pixelHeight, GL_RGBA, GL_UNSIGNED_BYTE, (GLvoid *)buffer);
Why is glTexSubImage2D() suddenly causing GL_INVALID_OPERATION?
I'm trying to upgrade my hopelessly outdated augmented reality app from iOS4.x to iOS5.x, but I'm having difficulties. I run iOS5.0. Last week I ran iOS4.3. My device is an iPhone4.
Here is a snippet from my captureOutput:didOutputSampleBuffer:fromConnection: code
uint8_t *baseAddress = /* pointer to camera buffer */
GLuint texture = /* the texture name */
glBindTexture(GL_TEXTURE_2D, texture);
glTexSubImage2D(GL_TEXTURE_2D, 0, 0, 0, 480, 360, GL_BGRA, GL_UNSIGNED_BYTE, baseAddress);
/* now glGetError(); -> returns 0x0502 GL_INVALID_OPERATION on iOS5.0, works fine on iOS4.x */
Here is a snippet from my setup code
GLuint texture = /* the texture name */
glGenTextures(1, &texture);
glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_2D, texture);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER,GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MAG_FILTER,GL_LINEAR);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, 512, 512, 0, GL_RGBA, GL_UNSIGNED_BYTE, NULL);
For simplicity I have inserted hardcoded values here. In my actual code I obtain these values with CVPixelBufferGetWidth/Height/BaseAddress. The EAGLContext is initialized with kEAGLRenderingAPIOpenGLES2.
Ah.. I fixed it immediately after posting this question. Had to change GL_RGBA into GL_BRGA.
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, 512, 512, 0, GL_BGRA, GL_UNSIGNED_BYTE, NULL);
Hope it helps someone.
BTW. If you want to write AR apps then consider using CVOpenGLESTextureCache instead of using glTexSubImage2d. It's supposed to be faster.
I want to read pixels from an off-screen (not backed by a CAEAGLLayer) Framebuffer. My code to create the buffer looks like:
glGenFramebuffersOES(1, &_storeFramebuffer);
glGenRenderbuffersOES(1, &_storeRenderbuffer);
glBindFramebufferOES(GL_FRAMEBUFFER_OES, _storeFramebuffer);
glBindRenderbufferOES(GL_RENDERBUFFER_OES, _storeRenderbuffer);
glFramebufferRenderbufferOES(GL_FRAMEBUFFER_OES, GL_COLOR_ATTACHMENT0_OES, GL_RENDERBUFFER_OES, _storeRenderbuffer);
glRenderbufferStorageOES(GL_RENDERBUFFER_OES, GL_RGBA8_OES, w, h);
I read raw pixels with:
glBindFramebufferOES(GL_FRAMEBUFFER_OES, _storeFramebuffer);
glBindRenderbufferOES(GL_RENDERBUFFER_OES, _storeRenderbuffer);
glReadPixels(0, 0, _videoDimensions.width, _videoDimensions.height, GL_BGRA_EXT, GL_UNSIGNED_BYTE, CVPixelBufferGetBaseAddress(outPixelBuffer));
This works well. I can render to this buffer, and copy from it to the screen. But I can't get raw pixels. glReadPixels always returns zeros, and glReadBuffer seems not to exist. I can read from the on-screen frame buffer with glReadPixels. Any ideas?
Solved. RGBA to BGRA conversion is not supported by glReadPixels on iOS.
Changing
glReadPixels(0, 0, _videoDimensions.width, _videoDimensions.height, GL_BGRA_EXT, GL_UNSIGNED_BYTE, CVPixelBufferGetBaseAddress(outPixelBuffer));
to
glReadPixels(0, 0, _videoDimensions.width, _videoDimensions.height, GL_RGBA, GL_UNSIGNED_BYTE, CVPixelBufferGetBaseAddress(outPixelBuffer));
Solves the problem. glGetError is my new friend ;)