My OpenGL ES renderBuffer doesn't display anything - ios

First of all sorry for my broken english, I'n not native and should have worked more at school :S
Those days I'm trying to build an AR system using OpengL ES on iOS.
I retrieve camera frames using AVCaptureSession and sent then both to a tracking algorithm and my OpenGL ES display class (which wait for the rotation and translation from the tracking algorithm). In the meantime I'm reading a video to display on an OpenGL plane placed at the same position as a real plane is the scene (I don't know if I am clear).
My goal is to use only one VBO and one shader to render the whole thing (image taken from camera in the background and plan with a video texture positioned using the rotation and translation given by my tracking algorithm).
I don't know why but nothing is displayed, here is the render code (called each frame) :
glClearColor(0, 0, 0, 0);
glClear(GL_COLOR_BUFFER_BIT);
[self setProjectionMatrix];
glUniformMatrix4fv(_projectionUniform, 1, 0, projection.glMatrix);
//Wait for rotation and translation
#synchronized(self.trackeriOS){
transfo = [trackeriOS getMat];
}
[self dumpMatrix:transfo];
[self fillModelViewMatrix:transfo];
glUniformMatrix4fv(_modelViewUniform, 1, 0, finalModelView);
videoTexture = [self setupTextureFromBuffer:self.buffer];
if (videoTexture == -1) {
NSLog(#"swap");
}
if (cameraVideoTexture == -1) {
NSLog(#"swap");
}
else {
glViewport(0, 0, self.frame.size.width, self.frame.size.height);
glVertexAttribPointer(_positionSlot, 3, GL_FLOAT, GL_FALSE, sizeof(Vertex), 0);
glVertexAttribPointer(_texCoordSlot, 2, GL_FLOAT, GL_FALSE, sizeof(Vertex), (GLvoid*) (sizeof(float) * 3));
glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_2D, videoTexture);
glUniform1i(_textureUniform, 0);
glDrawElements(GL_TRIANGLES, sizeof(Indices)/sizeof(Indices[0]), GL_UNSIGNED_BYTE, 0);
glUniformMatrix4fv(_projectionUniform, 1, 0, modelViewIdentity);
glUniformMatrix4fv(_modelViewUniform, 1, 0, modelViewIdentity);
glActiveTexture(GL_TEXTURE1);
glBindTexture(GL_TEXTURE_2D, cameraVideoTexture);
glUniform1i(_cameraVideotextureUniform, 1);
glDrawElements(GL_TRIANGLES, sizeof(Indices)/sizeof(Indices[0]), GL_UNSIGNED_BYTE, 0);
[_context presentRenderbuffer:GL_RENDERBUFFER];
CVBufferRelease(self.buffer);
glDeleteTextures(1, &cameraVideoTexture);
glDeleteTextures(1,&videoTexture);
}
Thank you so much for your help.
Edit 2: This is how I create my OpenGL texture from an CVImageBufferRef given by the AVFoundation delegate function while reading camera video datas:
- (void)processNewCameraFrame:(CVImageBufferRef)cameraFrame;
{
dispatch_async(backgroundImageDisplay, ^(void) {
self.cameraVideoBuffer = cameraFrame;
CVPixelBufferLockBaseAddress(cameraFrame, 0);
int bufferHeight = CVPixelBufferGetHeight(cameraFrame);
int bufferWidth = CVPixelBufferGetWidth(cameraFrame);
glGenTextures(1, &cameraVideoTexture);
glBindTexture(GL_TEXTURE_2D, cameraVideoTexture);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, bufferWidth, bufferHeight, 0, GL_BGRA, GL_UNSIGNED_BYTE, CVPixelBufferGetBaseAddress(cameraFrame));
CVPixelBufferUnlockBaseAddress(cameraFrame, 0);
CMTime outputItemTime = [videoOutput itemTimeForHostTime:CACurrentMediaTime()];
if ([[self videoOutput] hasNewPixelBufferForItemTime:outputItemTime2]) {
self.buffer = [[self videoOutput] copyPixelBufferForItemTime:outputItemTime2 itemTimeForDisplay:NULL];
//The above render function
[self render];
}
});
}
Edit : I forgot to say I set my VBO like this :
- (void)setupVBOs {
glGenBuffers(1, &vertexBuffer);
glBindBuffer(GL_ARRAY_BUFFER, vertexBuffer);
glBufferData(GL_ARRAY_BUFFER, sizeof(Vertices), Vertices, GL_STATIC_DRAW);
glGenBuffers(1, &indexBuffer);
glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, indexBuffer);
glBufferData(GL_ELEMENT_ARRAY_BUFFER, sizeof(Indices), Indices, GL_STATIC_DRAW);
NSLog(#"OpenGL View, set up VBO 1");
}
And I set up the entry point to my shader like this :
-(void)setTextureProgramShader {
_positionSlot = glGetAttribLocation(programHandle1, "Position");
glEnableVertexAttribArray(_positionSlot);
_projectionUniform = glGetUniformLocation(programHandle1, "Projection");
_modelViewUniform = glGetUniformLocation(programHandle1, "Modelview");
_texCoordSlot = glGetAttribLocation(programHandle1, "TexCoordIn");
glEnableVertexAttribArray(_texCoordSlot);
_textureUniform = glGetUniformLocation(programHandle1, "Texture");
}
Finally the vertex shader is :
attribute vec4 Position; // 1
uniform mat4 Projection;
uniform mat4 Modelview;
attribute vec2 TexCoordIn;
varying vec2 TexCoordOut;
void main(void) { // 4
gl_Position = Projection * Modelview * Position; // 6
TexCoordOut = TexCoordIn;
}

Related

Correct way to unbind an Open GL ES texture?

I have drawing code that first draws the background without a texure, then overlays the texture on top of it. I'm currently using glBindTexture(GL_TEXTURE_2D, 0) in order to un-bind the texture after performing the second lot of drawing.
However, this is generating the following warning when I analyse the GPU performance in Xcode:
When I remove the code to un-bind the texture, the warning disappears, but things aren't drawn correctly. The texture that I'm drawing doesn't have mipmapping enabled, so it's not coming from there.
Drawing Code:
glClear(GL_COLOR_BUFFER_BIT);
{ // Drawing
glViewport(0, 0, frameWidth, frameHeight);
GLuint vertexCount = animBuffer.vertexCounts[animIndex];
glBindBuffer(GL_ARRAY_BUFFER, animBuffer.vertexBuffers[animIndex]);
glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, animBuffer.indexBuffers[animIndex]);
glVertexAttribPointer(positionSlot, 3, GL_FLOAT, GL_FALSE, sizeof(vertex), 0);
glVertexAttribPointer(colorSlot, 4, GL_FLOAT, GL_FALSE, sizeof(vertex), (GLvoid*)(sizeof(float)*3));
glVertexAttribPointer(textureCoordSlot, 2, GL_FLOAT, GL_FALSE, sizeof(vertex), (GLvoid*)(sizeof(float)*7));
glDrawElements(GL_TRIANGLE_STRIP, vertexCount, GL_UNSIGNED_SHORT, 0);
// glEnable(GL_TEXTURE_2D) (tried this, gives another warning)
glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_2D, texture);
glUniform1i(textureUniform, 0);
glBindBuffer(GL_ARRAY_BUFFER, textureColorBuffer); // Transparent color buffer to allow texture to be overlayed.
glVertexAttribPointer(colorSlot, 4, GL_FLOAT, GL_FALSE, 0, (GLvoid*)(sizeof(float)*3));
glDrawElements(GL_TRIANGLE_STRIP, vertexCount, GL_UNSIGNED_SHORT, 0);
// Unbind texture.... this line is causing the error.
glBindTexture(GL_TEXTURE_2D, 0);
// glDisable(GL_TEXTURE_2D) (tried this, gives another warning)
}
[context presentRenderbuffer:GL_RENDERBUFFER];
Texture Loading Code:
CGImageRef textureImage = image.CGImage;
size_t width = CGImageGetWidth(textureImage);
size_t height = CGImageGetHeight(textureImage);
GLubyte* spriteData = malloc(width*height*4);
CGColorSpaceRef cs = CGImageGetColorSpace(textureImage);
CGContextRef c = CGBitmapContextCreate(spriteData, width, height, 8, width*4, cs, kCGImageAlphaPremultipliedLast | kCGBitmapByteOrder32Big);
CGColorSpaceRelease(cs);
CGContextScaleCTM(c, 1, -1);
CGContextTranslateCTM(c, 0, -CGContextGetClipBoundingBox(c).size.height);
CGContextDrawImage(c, (CGRect){CGPointZero, {width, height}}, textureImage);
CGContextRelease(c);
GLuint glTex;
glGenTextures(1, &glTex);
glBindTexture(GL_TEXTURE_2D, glTex);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, (GLsizei)width, (GLsizei)height, 0, GL_RGBA, GL_UNSIGNED_BYTE, spriteData);
glBindTexture(GL_TEXTURE_2D, 0);
free(spriteData);
return glTex;
How should I go about un-binding textures in order to remove this warning?
EDIT:
I have tried using glEnable(GL_TEXTURE_2D) & glDisable(GL_TEXTURE_2D) before and after the textured drawing, although this now gives me the following warning:
I think the easiest way to avoid this warning is to keep the texture binded, but just send a uniform to the fragment shader indicating whether to blend the texture or not.
So, before drawing just add:
glUniform1i(drawTexInput, GL_FALSE); // Or GL_TRUE to draw.
And in the fragment shader:
varying lowp vec4 destinationColor;
uniform bool drawTex;
varying lowp vec2 texCoordOut;
uniform sampler2D tex;
void main() {
lowp vec4 result;
if (drawTex) {
lowp vec4 tex2D = texture2D(tex, texCoordOut);
result = tex2D + vec4(1.0 - tex2D.a) * destinationColor; // Alpha blending
} else {
result = destinationColor;
}
gl_FragColor = result;
}

Open GL ES object not appearing in shadow map?

This is driving me nuts! I've not done much with OpenGL before, so any pointers appreciated.
Can anyone tell me any reason why some of my objects appear in my shadow map, and others don't?
I have 2 object I am rendering; southernHemisphere and Frame. Both of these are rendered to the scene and to the shadow map texture. The scene is working perfectly showing all objects, but only southernHemisphere is appearing in the shadow map.
My shadow map texture is bound like this:
- (void)setupShadowmapBuffers {
glGenTextures(1, &_shadowTexture);
glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_2D, _shadowTexture);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, 1024, 1024, 0, GL_RGBA, GL_UNSIGNED_BYTE, NULL);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glGenFramebuffers(1, &_shadowFBO);
glGenRenderbuffers(1, &_shadowRenderBuffer);
glBindRenderbuffer(GL_RENDERBUFFER, _shadowRenderBuffer);
glRenderbufferStorage(GL_RENDERBUFFER, GL_DEPTH_COMPONENT16, 1024, 1024);
glBindFramebuffer(GL_FRAMEBUFFER, _shadowFBO);
glBindTexture(GL_TEXTURE_2D, _shadowTexture);
glFramebufferRenderbuffer(GL_FRAMEBUFFER,
GL_DEPTH_ATTACHMENT,
GL_RENDERBUFFER,
_shadowRenderBuffer);
glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, _shadowTexture, 0);
if (glCheckFramebufferStatus(GL_FRAMEBUFFER) != GL_FRAMEBUFFER_COMPLETE)
{
printf("Error: FrameBufferObject is not complete!\n");
}
}
In my render function, I render to the shadow map texture:
// Render the shadow map
glUseProgram(_shadowProgramHandle);
glUniformMatrix4fv(_shadowProjectionUniform, 1, 0, LSprojection.glMatrix);
glUniformMatrix4fv(_shadowLightMPVUniform, 1, 0, lightsourcemat.glMatrix);
glBindFramebuffer(GL_FRAMEBUFFER, _shadowFBO);
glBindRenderbuffer(GL_RENDERBUFFER, _shadowRenderBuffer);
glClearColor(0,0,0,0);
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
glViewport(0, 0, 1024, 1024);
glUniformMatrix4fv(_shadowModelViewUniform, 1, 0, southernHemispheremat.glMatrix);
[self drawSouthernHemisphere];
glUniformMatrix4fv(_shadowModelViewUniform, 1, 0, framemat.glMatrix);
[self drawFrame];
Then I render to the scene:
// Reset buffers and finally render the scene
glUseProgram(_mainProgramHandle);
glBindFramebuffer(GL_FRAMEBUFFER, msaaFramebuffer);
glBindRenderbuffer(GL_RENDERBUFFER, msaaRenderBuffer);
glClearColor(0,0,0,0);
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
glViewport(0, 0, vFrame.size.width, vFrame.size.height);
glUniformMatrix4fv(_modelViewUniform, 1, 0, shadowmat.glMatrix);
[self drawShadow];
glUniformMatrix4fv(_modelViewUniform, 1, 0, southernHemispheremat.glMatrix);
[self drawSouthernHemisphere];
glUniformMatrix4fv(_modelViewUniform, 1, 0, framemat.glMatrix);
[self drawFrame];
My drawing functions look like this:
- (void)drawSouthernHemisphere {
// Southern Hemisphere
glEnable(GL_CULL_FACE);
glBindTexture(GL_TEXTURE_2D, _globeTexture);
glBindVertexArrayOES(_southernHemisphere);
// Draw outside (textured)
glUniform1f(_useTextureSlot, 1);
glCullFace(GL_BACK);
glDrawElements(GL_TRIANGLES, sizeof(Southern)/sizeof(Southern[0]), GL_UNSIGNED_SHORT, 0);
// Draw inside (coloured)
glCullFace(GL_FRONT);
glUniform1f(_useTextureSlot, 0);
glUniform4f(_runTimeColorSlot, 0, 0, 0, 0);
glDrawElements(GL_TRIANGLES, sizeof(Southern)/sizeof(Southern[0]), GL_UNSIGNED_SHORT, 0);
glBindVertexArrayOES(0);
}
- (void)drawFrame {
// Frame
glEnable(GL_CULL_FACE);
glBindTexture(GL_TEXTURE_2D, _frameTexture);
glBindVertexArrayOES(_frame);
// Draw outside (textured)
glUniform1f(_useTextureSlot, 1);
glCullFace(GL_BACK);
glDrawElements(GL_LINES, sizeof(Frame)/sizeof(Frame[0]), GL_UNSIGNED_SHORT, 0);
glBindVertexArrayOES(0);
}
Thanks.
EDIT:
I have isolated the issue being down to my vertex arrays - in particular the normals.
I swapped the values in the frame array with the ones in my southernHemisphere array, and bingo - the object appears in both the scene, and the shadow map!
After some adjustment of values, it appears that it will only work if the direction of the normals are the same as the direction of each vector itself.
What baffles me is that in my shadow map shader, I don't even access the normals so why do they have any effect on the rendering at all??
Here is my vertex shader for my shadow map:
attribute vec4 Position;
uniform mat4 Projection;
uniform mat4 Modelview;
uniform mat4 Light;
varying vec4 TexCoord;
void main()
{
TexCoord = Projection * Light * Modelview * Position;
gl_Position = Projection * Light * Modelview * Position;
}
and fragment:
precision mediump float;
varying vec4 TexCoord;
void main()
{
/* Generate shadow map - write fragment depth. */
float value = 10.0 - TexCoord.z;
float v = floor(value);
float f = value - v;
float vn = v * 0.1;
gl_FragColor = vec4(vn, f, 0.0, 1.0);
}
EDIT2:
Here are my set up function for VAOs
// Southern Hemisphere
glGenVertexArraysOES(1, &_southernHemisphere);
glBindVertexArrayOES(_southernHemisphere);
GLuint vb2;
glGenBuffers(1, &vb2);
glBindBuffer(GL_ARRAY_BUFFER, vb2);
glBufferData(GL_ARRAY_BUFFER, sizeof(Vertices), Vertices, GL_STATIC_DRAW);
glEnableVertexAttribArray(_positionSlot);
glVertexAttribPointer(_positionSlot, 3, GL_FLOAT, GL_FALSE, sizeof(Vertex), (void*)offsetof(Vertex, Position));
glEnableVertexAttribArray(_colorSlot);
glVertexAttribPointer(_colorSlot, 4, GL_FLOAT, GL_FALSE, sizeof(Vertex), (void*)offsetof(Vertex, Color));
glEnableVertexAttribArray(_normalSlot);
glVertexAttribPointer(_normalSlot, 3, GL_FLOAT, GL_FALSE, sizeof(Vertex), (void*)offsetof(Vertex, Normal));
glEnableVertexAttribArray(_texCoordSlot);
glVertexAttribPointer(_texCoordSlot, 2, GL_FLOAT, GL_FALSE, sizeof(Vertex), (void*)offsetof(Vertex, TexCoord));
GLuint southernBuffer;
glGenBuffers(1, &southernBuffer);
glBindBuffer(GL_ELEMENT_ARRAY_BUFFER,southernBuffer);
glBufferData(GL_ELEMENT_ARRAY_BUFFER, sizeof(Southern) * sizeof(Southern[0]), Southern, GL_STATIC_DRAW);
glBindBuffer(GL_ARRAY_BUFFER, 0);
glBindVertexArrayOES(0);
// Frame
glGenVertexArraysOES(1, &_frame);
glBindVertexArrayOES(_frame);
GLuint vb4;
glGenBuffers(1, &vb4);
glBindBuffer(GL_ARRAY_BUFFER, vb4);
glBufferData(GL_ARRAY_BUFFER, sizeof(Frame_Vertices), Frame_Vertices, GL_STATIC_DRAW);
glEnableVertexAttribArray(_positionSlot);
glVertexAttribPointer(_positionSlot, 3, GL_FLOAT, GL_FALSE, sizeof(Vertex), (void*)offsetof(Vertex, Position));
glEnableVertexAttribArray(_colorSlot);
glVertexAttribPointer(_colorSlot, 4, GL_FLOAT, GL_FALSE, sizeof(Vertex), (void*)offsetof(Vertex, Color));
glEnableVertexAttribArray(_normalSlot);
glVertexAttribPointer(_normalSlot, 3, GL_FLOAT, GL_FALSE, sizeof(Vertex), (void*)offsetof(Vertex, Normal));
glEnableVertexAttribArray(_texCoordSlot);
glVertexAttribPointer(_texCoordSlot, 2, GL_FLOAT, GL_FALSE, sizeof(Vertex), (void*)offsetof(Vertex, TexCoord));
GLuint legsBuffer;
glGenBuffers(1, &frameBuffer);
glBindBuffer(GL_ELEMENT_ARRAY_BUFFER,frameBuffer);
glBufferData(GL_ELEMENT_ARRAY_BUFFER, sizeof(Frame) * sizeof(Frame[0]), Frame, GL_STATIC_DRAW);
glBindBuffer(GL_ARRAY_BUFFER, 0);
glBindVertexArrayOES(0);
My Arrays all follow the same form of:
const Vertex Vertices[] = {
{{0.0000,-1.5000,0.0000},{0.3843137254902,0.14117647058824,0.082352941176471,1},{0.0000,-1.0000,0.0000},{0.0000,1.0000}},
{{0.0000,-1.4872,0.1958},{0.3843137254902,0.14117647058824,0.082352941176471,1},{0.0000,-0.9914,0.1305},{0.0000,0.9583}},
....
}
which consists of Position, Colour, Normal, Texture Coordinates as you can see from the set up code.
After my shaders are linked, I finish set up with these:
- (void)useMainProgram:(GLfloat[])lightsource {
const GLfloat l_ambient = 0.7;
glUseProgram(_mainProgramHandle);
_positionSlot = glGetAttribLocation(_mainProgramHandle, "Position");
_colorSlot = glGetAttribLocation(_mainProgramHandle, "Color");
_normalSlot = glGetAttribLocation(_mainProgramHandle, "Normal");
_lightingSlot = glGetUniformLocation(_mainProgramHandle, "l1");
_ambientLightingSlot = glGetUniformLocation(_mainProgramHandle, "l_ambient");
_texCoordSlot = glGetAttribLocation(_mainProgramHandle, "TexCoordIn");
_useTextureSlot = glGetUniformLocation(_mainProgramHandle, "use_texture");
_runTimeColorSlot = glGetUniformLocation(_mainProgramHandle, "RunTimeColor");
glEnableVertexAttribArray(_positionSlot);
glEnableVertexAttribArray(_colorSlot);
glEnableVertexAttribArray(_normalSlot);
glEnableVertexAttribArray(_texCoordSlot);
_projectionUniform = glGetUniformLocation(_mainProgramHandle, "Projection");
_modelViewUniform = glGetUniformLocation(_mainProgramHandle, "Modelview");
for (int i=0;i<_num_bottles; i++){
_bottlesUniform[i] = glGetUniformLocation(_mainProgramHandle, "Modelview");
}
_textureUniform = glGetUniformLocation(_mainProgramHandle, "Texture");
// Lighting doesn't change, so we set it here.
glUniform3fv(_lightingSlot, 1, lightsource);
glUniform1fv(_ambientLightingSlot, 1, &l_ambient);
}
- (void)useShadowProgram:(GLfloat[])lightsource {
glUseProgram(_shadowProgramHandle);
_shadowPositionSlot = glGetAttribLocation(_shadowProgramHandle, "Position");
_shadowLightMPVUniform = glGetUniformLocation(_shadowProgramHandle, "Light");
_shadowProjectionUniform = glGetUniformLocation(_shadowProgramHandle, "Projection");
_shadowModelViewUniform = glGetUniformLocation(_shadowProgramHandle, "Modelview");
glEnableVertexAttribArray(_shadowPositionSlot);
}

Can't retrieve multiple gl_Fragdata buffers in OpenGL in IOS (NV12 -> YV12/i420 conversion)

I'm attempting to convert camera frames which are in YUV 4.2.2 (NV12) format to YUV 4.2.0 (YV12/i420) format. The U and V channels of NV12 are interleaved, while the YV12 are not.
For performance reasons, I'm trying to do this conversion in an OpenGl ES 2.0 shader.
The shaders are fairly easy. I separate out the Y and the UV channels and then pass the UV to a shader that separates it out into a U channel in gl_FragData[0] and V channel in gl_FragData[1]
Here is the Fragment Shader:
precision mediump float;
uniform sampler2D t_texture_uv;
varying highp vec2 uvVarying;
void main()
{
/*
vec2 uvu,uvv;
int x,y;
x = uvVarying.x;
y = uvVarying.y;
uvu = vec2(x*2,y);
uvv = vec2((x*2)+1,y);
gl_FragData[0] = texture2D(t_texture_uv, uvu);
gl_FragData[1] = texture2D(t_texture_uv, uvv);
*/
// this is the same as the logic above, but is less processor intensive
gl_FragData[0] = texture2D(t_texture_uv, vec2( uvVarying.x*2 ,uvVarying.y)); // u-plane
gl_FragData[1] = texture2D(t_texture_uv, vec2((uvVarying.x*2)+1,uvVarying.y)); // v-plane
}
And the vertex shader:
attribute vec2 position;
attribute vec2 uv;
uniform mat4 proj_matrix;
varying vec2 uvVarying;
void main()
{
gl_Position = proj_matrix * vec4(vec3(position.xy, 0.0), 1.0);
uvVarying = uv;
}
My problem is that I'm lost at how to retrieve the data from the Gl_FragData buffers back in my Objective C code. I tried glReadPixels, but it doesn't seem to work.
This is my code, but I think I've gone down the wrong path.
- (void) nv12_to_yv12_converter:(unsigned char*)YBuf UVBuff:(unsigned char*)UVBuf Width:(GLsizei)UVWidth Height:(GLsizei)UVHeight {
unsigned char *Ubufout;
unsigned char *Vbufout;
EAGLContext *context;
context = [[EAGLContext alloc] initWithAPI:kEAGLRenderingAPIOpenGLES2];
if (!context || ![EAGLContext setCurrentContext:context]) {
NSLog(#"Failed to create ES context");
}
[self loadShaders];
glGenFramebuffers(1, &_framebuffer);
glGenRenderbuffers(1, &_renderbuffer);
glBindFramebuffer(GL_FRAMEBUFFER, _framebuffer);
glBindRenderbuffer(GL_RENDERBUFFER, _renderbuffer);
;
glRenderbufferStorage(GL_RENDERBUFFER, GL_DEPTH_COMPONENT16, UVWidth, UVHeight);
// [context renderbufferStorage:GL_RENDERBUFFER fromDrawable:(CAEAGLLayer*)self.view.layer];
glGetRenderbufferParameteriv(GL_RENDERBUFFER, GL_RENDERBUFFER_WIDTH, &_backingWidth);
glGetRenderbufferParameteriv(GL_RENDERBUFFER, GL_RENDERBUFFER_HEIGHT, &_backingHeight);
glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_RENDERBUFFER, _renderbuffer);
_vertices[0] = -1.0f; // x0
_vertices[1] = -1.0f; // y0
_vertices[2] = 1.0f; // ..
_vertices[3] = -1.0f;
_vertices[4] = -1.0f;
_vertices[5] = 1.0f;
_vertices[6] = 1.0f; // x3
_vertices[7] = 1.0f; // y3
int orientation = 0;
static const GLfloat texCoords[] = {
0.0f, 1.0f,
1.0f, 1.0f,
0.0f, 0.0f,
1.0f, 0.0f,
};
glViewport(0, 0, _backingWidth, _backingHeight);
glClearColor(0.0f, 0.0f, 0.0f, 1.0f);
glClear(GL_COLOR_BUFFER_BIT);
glUseProgram(_program);
if (!validateProgram(_program))
{
NSLog(#"Failed to validate program");
return;
}
glPixelStorei(GL_UNPACK_ALIGNMENT, 1);
glDeleteTextures(1, &_textures[0]);
glActiveTexture(GL_TEXTURE0);
glGenTextures(1, &_textures[0]);
glBindTexture(GL_TEXTURE_2D, _textures[0]);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
glEnable(GL_TEXTURE_2D);
glUniform1i(glGetUniformLocation(_program, "t_texture_uv"), 0);
glTexImage2D(GL_TEXTURE_2D,
0,
GL_LUMINANCE,
16,
8,
0,
GL_LUMINANCE,
GL_UNSIGNED_BYTE,
UVBuf);
_uniformMatrix = glGetUniformLocation(_program, "proj_matrix");
GLfloat modelviewProj[16];
mat4f_LoadOrtho(-1.0f, 1.0f, -1.0f, 1.0f, -1.0f, 1.0f, modelviewProj);
glUniformMatrix4fv(_uniformMatrix, 1, GL_FALSE, modelviewProj);
//draw RECT
glVertexAttribPointer(ATTRIB_VERTEX, 2, GL_FLOAT, 0, 0, _vertices);
glEnableVertexAttribArray(ATTRIB_VERTEX);
//ATTRIB_TEXTUREPOSITON
glVertexAttribPointer(ATTRIB_UV, 2, GL_FLOAT, 0, 0, texCoords);
glEnableVertexAttribArray(ATTRIB_UV);
glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);
glReadPixels(0, 0, UVWidth, UVHeight, GL_COLOR_ATTACHMENT0, GL_UNSIGNED_BYTE, &Ubufout);
glReadPixels(0, 0, UVWidth, UVHeight, GL_COLOR_ATTACHMENT0+1, GL_UNSIGNED_BYTE, &Vbufout);
NSLog(#"Complete");
}
Can anyone shed some light on how to retrieve the gl_FragData buffers?

Render to texture, then render texture to screen in iOS

I am trying to render a simple scene into a FBO that is backed by a texture as a color attachment and then draw a quad to the screen using this texture in iOS.
This will help me do some post processing of the final scene. There are a few questions on SO that address similar (but not quite the same) questions and I've tried whatever I could understand. Nothing works.
I have two shader programs. The first one _program simply takes the vertex positions and renders them with a single color. The second one quadProgram takes a texture, the texture coords and the quad Coords. (In the interest of brevity I am omitting the code for these shaders.)
Both shaders work correctly. They independently produce correct results. However, when I try to render the "rendered texture" to the quad instead of using a sample wood texture, I simply get a black screen. Here are the relevant bits of code. I setup everything like so:
- (void)setupGL
{
[EAGLContext setCurrentContext:self.context];
_program = [self createProgramWithVertexShader:#"Shader" fragShader:#"Shader"];
GLuint GLKVertexAttribPosition = glGetAttribLocation(_program, "position");
quadProgram = [self createProgramWithVertexShader:#"PostShader" fragShader:#"PostShader"];
quadTexCoord = glGetAttribLocation(quadProgram, "a_texcoord");
quadPosition = glGetAttribLocation(quadProgram, "a_position");
diffuseTexture = glGetUniformLocation(quadProgram, "s_diffuse");
glEnable(GL_DEPTH_TEST);
glGenVertexArraysOES(1, &_vertexArray);
glBindVertexArrayOES(_vertexArray);
glGenBuffers(1, &_vertexBuffer);
glBindBuffer(GL_ARRAY_BUFFER, _vertexBuffer);
glBufferData(GL_ARRAY_BUFFER, sizeof(gCubeVertexData), gCubeVertexData, GL_STATIC_DRAW);
glEnableVertexAttribArray(GLKVertexAttribPosition);
glVertexAttribPointer(GLKVertexAttribPosition, 3, GL_FLOAT, GL_FALSE, 24, BUFFER_OFFSET(0));
glBindVertexArrayOES(0);
// ============
// ---- for render to texture
width = self.view.bounds.size.width;
height = self.view.bounds.size.height;
// create fbo
glGenFramebuffers(1, &framebuffer);
glBindFramebuffer(GL_FRAMEBUFFER, framebuffer);
// create and attach backing texture
glGenTextures(1, &texture);
glBindTexture(GL_TEXTURE_2D, texture);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, width, height, 0, GL_RGBA, GL_UNSIGNED_BYTE, NULL);
glFramebufferTexture2D(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_TEXTURE_2D, texture, 0);
// create and attach depthbuffer
glGenRenderbuffers(1, &depthRenderbuffer);
glBindRenderbuffer(GL_RENDERBUFFER, depthRenderbuffer);
glRenderbufferStorage(GL_RENDERBUFFER, GL_DEPTH_COMPONENT16, width, height);
glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_DEPTH_ATTACHMENT, GL_RENDERBUFFER, depthRenderbuffer);
// Check if framebuffer was loaded correctly
GLenum status = glCheckFramebufferStatus(GL_FRAMEBUFFER) ;
if(status != GL_FRAMEBUFFER_COMPLETE) {
NSLog(#"failed to make complete framebuffer object %x", status);
}
// sample texture
NSError *theError;
NSString *filePath = [[NSBundle mainBundle] pathForResource:#"wood_floor_256" ofType:#"jpg"]; // 1
spriteTexture = [GLKTextureLoader textureWithContentsOfFile:filePath options:nil error:&theError]; // 2
}
And then my main loop:
- (void)update
{
// First save the default frame buffer.
static GLint default_frame_buffer = 0;
glGetIntegerv(GL_FRAMEBUFFER_BINDING, &default_frame_buffer);
// render to texture
glBindFramebuffer(GL_FRAMEBUFFER, framebuffer);
glClearColor(0.65f, 0.65f, 0.65f, 1.0f);
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
glBindVertexArrayOES(_vertexArray);
glUseProgram(_program);
glDrawArrays(GL_TRIANGLES, 0, 36);
// render to screen
glBindFramebuffer(GL_FRAMEBUFFER, default_frame_buffer);
glClearColor(0.65f, 0.65f, 0.65f, 1.0f);
glClear(GL_COLOR_BUFFER_BIT | GL_DEPTH_BUFFER_BIT);
glUseProgram(quadProgram);
const float quadPositions[] = { 1.0, 1.0, 0.0,
-1.0, 1.0, 0.0,
-1.0, -1.0, 0.0,
-1.0, -1.0, 0.0,
1.0, -1.0, 0.0,
1.0, 1.0, 0.0 };
const float quadTexcoords[] = { 1.0, 1.0,
0.0, 1.0,
0.0, 0.0,
0.0, 0.0,
1.0, 0.0,
1.0, 1.0 };
// stop using VBO and other bufs.
glBindBuffer(GL_ARRAY_BUFFER, 0);
glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, 0);
glBindVertexArrayOES(0);
glBindTexture(GL_TEXTURE_2D, texture);
glActiveTexture(GL_TEXTURE_2D);
// setup buffer offsets
glVertexAttribPointer(quadPosition, 3, GL_FLOAT, GL_FALSE, 3*sizeof(float), quadPositions);
glVertexAttribPointer(quadTexCoord, 2, GL_FLOAT, GL_FALSE, 2*sizeof(float), quadTexcoords);
// ensure the proper arrays are enabled
glEnableVertexAttribArray(quadPosition);
glEnableVertexAttribArray(quadTexCoord);
// draw
glDrawArrays(GL_TRIANGLES, 0, 2*3);
}
I don't even know how to tell if the backing texture contains the rendered screen.
The render to texture part should render into the FBO the following:
If I replace the two lines:
glBindTexture(GL_TEXTURE_2D, texture);
glActiveTexture(GL_TEXTURE_2D);
with
glBindTexture(spriteTexture.target, spriteTexture.name);
glEnable(spriteTexture.target);
I get the following:
Which indicates that the code to render a quad to screen with a texture is correct.
I have no idea how to get the first screen rendered to the quad.
Since the texture you are using is NPOT, it needs a non-default wrap parameter.
Add the following right below the other glTexParameteri call. It should solve the problem:
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);

ios4.2 opengl can not work

I have get the h.264 data and decoder them to YUV buffers,and i have display the yup data by opengl on ios5 and ios6,but when I try to run it on my iPad(ios4.2.1),it can not display currently,just all the screen green color.I don't know why,here is my code:
-(void)playVideoData:(void *)data
{
if (!_textureY)
{
glGenTextures(1, &_textureY);
glGenTextures(1, &_textureU);
glGenTextures(1, &_textureV);
}
glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_2D, _textureY);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RED_EXT, _videoW, _videoH, 0, GL_RED_EXT, GL_UNSIGNED_BYTE, y);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MAG_FILTER,GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER,GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
glActiveTexture(GL_TEXTURE1);
glBindTexture(GL_TEXTURE_2D, _textureU);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RED_EXT, _videoW/2, _videoH/2, 0, GL_RED_EXT, GL_UNSIGNED_BYTE, u);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MAG_FILTER,GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER,GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
glActiveTexture(GL_TEXTURE2);
glBindTexture(GL_TEXTURE_2D, _textureV);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RED_EXT, _videoW/2, _videoH/2, 0, GL_RED_EXT, GL_UNSIGNED_BYTE, v);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MAG_FILTER,GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER,GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
[self render];
}
- (void)render
{
glViewport(viewportx,viewporty, VIEWWIDTH, VIEWHEIGHT);
glClearColor(0.0, 0, 0.0, 0);
glClear(GL_COLOR_BUFFER_BIT);
glUseProgram(programId);
// Update uniform value
//glUniform1f(uniforms[UNIFORM_TRANSLATE], 0.0f);
GLuint textureUniformY = glGetUniformLocation(programId, "SamplerY");
GLuint textureUniformU = glGetUniformLocation(programId, "SamplerU");
GLuint textureUniformV = glGetUniformLocation(programId, "SamplerV");
// Update attribute values
glVertexAttribPointer(ARDRONE_ATTRIB_POSITION, 2, GL_FLOAT, 0, 0, squareVertices);
glEnableVertexAttribArray(ARDRONE_ATTRIB_POSITION);
glVertexAttribPointer(ARDRONE_ATTRIB_TEXCOORD, 2, GL_FLOAT, 0, 0, coordVertices);
glEnableVertexAttribArray(ARDRONE_ATTRIB_TEXCOORD);
glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_2D, _textureY);
glUniform1i(textureUniformY, 0);
glActiveTexture(GL_TEXTURE1);
glBindTexture(GL_TEXTURE_2D, _textureU);
glUniform1i(textureUniformU, 1);
glActiveTexture(GL_TEXTURE2);
glBindTexture(GL_TEXTURE_2D, _textureV);
glUniform1i(textureUniformV, 2);
}
-(void)drawFrame2
{
if (context != nil)
{
//make it the current context for rendering
[EAGLContext setCurrentContext:context];
//if our framebuffers have not been created yet, do that now!
if (!defaultFramebuffer)
[self createFramebuffer];
glBindFramebuffer(GL_FRAMEBUFFER, defaultFramebuffer);
[self playVideoData];
glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);
glBindRenderbuffer(GL_RENDERBUFFER, colorRenderbuffer);
[context presentRenderbuffer:GL_RENDERBUFFER];
}
else
NSLog(#"Context not set!");
}
I have put the data to the y u v buffer
here is my fsh and vsh:
vsh:
attribute vec4 position; // 1
//uniform float translate;
attribute vec2 TexCoordIn; // New
varying vec2 TexCoordOut; // New
void main(void)
{
gl_Position = position; // 6
TexCoordOut = TexCoordIn;
}
fsh:
varying lowp vec2 TexCoordOut;
uniform sampler2D SamplerY;
uniform sampler2D SamplerU;
uniform sampler2D SamplerV;
void main(void)
{
mediump vec3 yuv;
lowp vec3 rgb;
yuv.x = texture2D(SamplerY, TexCoordOut).r;
yuv.y = texture2D(SamplerU, TexCoordOut).r - 0.5;
yuv.z = texture2D(SamplerV, TexCoordOut).r - 0.5;
rgb = mat3( 1, 1, 1,
0, -0.39465, 2.03211,
1.13983, -0.58060, 0) * yuv;
gl_FragColor = vec4(rgb, 1);
}
iOs4.2 no erros ,the shaders Compile ok, but just can not display...
This is because the GL_RED_EXT extension that you are using to upload your textures was only added in iOS 5.0, and isn't present in iOS 4.2. You won't be able to use that to upload your YUV textures in this manner, so you'll need to rewrite that part of your code. Also, I believe this extension is only supported for iPad 2 and newer devices, not the original iPad and older iPhones, so you won't be able to use it on even iOS 5+ for the older ones.

Resources