iOS - pass UIImage to shader as texture - ios

I am trying to pass UIImage to GLSL shader. The fragment shader is:
varying highp vec2 textureCoordinate;
uniform sampler2D inputImageTexture;
uniform sampler2D inputImageTexture2;
void main()
{
highp vec4 color = texture2D(inputImageTexture, textureCoordinate);
highp vec4 color2 = texture2D(inputImageTexture2, textureCoordinate);
gl_FragColor = color * color2;
}
What I want to do is send images from camera and do multiply blend with texture. When I just send data from camera, everything is fine. So problem should be with sending another texture to shader. I am doing it this way:
- (void)setTexture:(UIImage*)image forUniform:(NSString*)uniform {
CGSize sizeOfImage = [image size];
CGFloat scaleOfImage = [image scale];
CGSize pixelSizeOfImage = CGSizeMake(scaleOfImage * sizeOfImage.width, scaleOfImage * sizeOfImage.height);
//create context
GLubyte * spriteData = (GLubyte *)malloc(pixelSizeOfImage.width * pixelSizeOfImage.height * 4 * sizeof(GLubyte));
CGContextRef spriteContext = CGBitmapContextCreate(spriteData, pixelSizeOfImage.width, pixelSizeOfImage.height, 8, pixelSizeOfImage.width * 4, CGImageGetColorSpace(image.CGImage), kCGImageAlphaPremultipliedLast);
//draw image into context
CGContextDrawImage(spriteContext, CGRectMake(0.0, 0.0, pixelSizeOfImage.width, pixelSizeOfImage.height), image.CGImage);
//get uniform of texture
GLuint uniformIndex = glGetUniformLocation(__programPointer, [uniform UTF8String]);
//generate texture
GLuint textureIndex;
glGenTextures(1, &textureIndex);
glBindTexture(GL_TEXTURE_2D, textureIndex);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
//create texture
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, pixelSizeOfImage.width, pixelSizeOfImage.height, 0, GL_RGBA, GL_UNSIGNED_BYTE, spriteData);
glActiveTexture(GL_TEXTURE1);
glBindTexture(GL_TEXTURE_2D, textureIndex);
//"send" to shader
glUniform1i(uniformIndex, 1);
free(spriteData);
CGContextRelease(spriteContext);
}
Uniform for texture is fine, glGetUniformLocation function do not returns -1. The texture is PNG file of resolution 2000x2000 pixels.
PROBLEM: When the texture is passed to shader, I have got "black screen". Maybe problem are parameters of the CGContext or parameters of the function glTexImage2D
UPDATE: I have just found out I need to pass texture unit number to uniform every draw cycle and everything works just fine. It is really necessary to do it?
Thank you

Related

Correct way to unbind an Open GL ES texture?

I have drawing code that first draws the background without a texure, then overlays the texture on top of it. I'm currently using glBindTexture(GL_TEXTURE_2D, 0) in order to un-bind the texture after performing the second lot of drawing.
However, this is generating the following warning when I analyse the GPU performance in Xcode:
When I remove the code to un-bind the texture, the warning disappears, but things aren't drawn correctly. The texture that I'm drawing doesn't have mipmapping enabled, so it's not coming from there.
Drawing Code:
glClear(GL_COLOR_BUFFER_BIT);
{ // Drawing
glViewport(0, 0, frameWidth, frameHeight);
GLuint vertexCount = animBuffer.vertexCounts[animIndex];
glBindBuffer(GL_ARRAY_BUFFER, animBuffer.vertexBuffers[animIndex]);
glBindBuffer(GL_ELEMENT_ARRAY_BUFFER, animBuffer.indexBuffers[animIndex]);
glVertexAttribPointer(positionSlot, 3, GL_FLOAT, GL_FALSE, sizeof(vertex), 0);
glVertexAttribPointer(colorSlot, 4, GL_FLOAT, GL_FALSE, sizeof(vertex), (GLvoid*)(sizeof(float)*3));
glVertexAttribPointer(textureCoordSlot, 2, GL_FLOAT, GL_FALSE, sizeof(vertex), (GLvoid*)(sizeof(float)*7));
glDrawElements(GL_TRIANGLE_STRIP, vertexCount, GL_UNSIGNED_SHORT, 0);
// glEnable(GL_TEXTURE_2D) (tried this, gives another warning)
glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_2D, texture);
glUniform1i(textureUniform, 0);
glBindBuffer(GL_ARRAY_BUFFER, textureColorBuffer); // Transparent color buffer to allow texture to be overlayed.
glVertexAttribPointer(colorSlot, 4, GL_FLOAT, GL_FALSE, 0, (GLvoid*)(sizeof(float)*3));
glDrawElements(GL_TRIANGLE_STRIP, vertexCount, GL_UNSIGNED_SHORT, 0);
// Unbind texture.... this line is causing the error.
glBindTexture(GL_TEXTURE_2D, 0);
// glDisable(GL_TEXTURE_2D) (tried this, gives another warning)
}
[context presentRenderbuffer:GL_RENDERBUFFER];
Texture Loading Code:
CGImageRef textureImage = image.CGImage;
size_t width = CGImageGetWidth(textureImage);
size_t height = CGImageGetHeight(textureImage);
GLubyte* spriteData = malloc(width*height*4);
CGColorSpaceRef cs = CGImageGetColorSpace(textureImage);
CGContextRef c = CGBitmapContextCreate(spriteData, width, height, 8, width*4, cs, kCGImageAlphaPremultipliedLast | kCGBitmapByteOrder32Big);
CGColorSpaceRelease(cs);
CGContextScaleCTM(c, 1, -1);
CGContextTranslateCTM(c, 0, -CGContextGetClipBoundingBox(c).size.height);
CGContextDrawImage(c, (CGRect){CGPointZero, {width, height}}, textureImage);
CGContextRelease(c);
GLuint glTex;
glGenTextures(1, &glTex);
glBindTexture(GL_TEXTURE_2D, glTex);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, (GLsizei)width, (GLsizei)height, 0, GL_RGBA, GL_UNSIGNED_BYTE, spriteData);
glBindTexture(GL_TEXTURE_2D, 0);
free(spriteData);
return glTex;
How should I go about un-binding textures in order to remove this warning?
EDIT:
I have tried using glEnable(GL_TEXTURE_2D) & glDisable(GL_TEXTURE_2D) before and after the textured drawing, although this now gives me the following warning:
I think the easiest way to avoid this warning is to keep the texture binded, but just send a uniform to the fragment shader indicating whether to blend the texture or not.
So, before drawing just add:
glUniform1i(drawTexInput, GL_FALSE); // Or GL_TRUE to draw.
And in the fragment shader:
varying lowp vec4 destinationColor;
uniform bool drawTex;
varying lowp vec2 texCoordOut;
uniform sampler2D tex;
void main() {
lowp vec4 result;
if (drawTex) {
lowp vec4 tex2D = texture2D(tex, texCoordOut);
result = tex2D + vec4(1.0 - tex2D.a) * destinationColor; // Alpha blending
} else {
result = destinationColor;
}
gl_FragColor = result;
}

How can I render an RGB bitmap into a GLKView using 2D textures?

I have an in-memory "bitmap" which is just a malloced Byte * array that contains pixel data in a simple RGB format (so the size of the byte array is 3 * the number of pixels).
My app is just a view controller with an instance of GLKView. I have implemented its delegate like so:
- (void)glkView:(GLKView *)view drawInRect:(CGRect)rect {
glClearColor(1.0f, 0.0f, 0.0f, 1.0f);
glClear(GL_COLOR_BUFFER_BIT);
}
and as expected this sets the background color of the GLKView.
What I would now like to do is add code to this implementation of glkView:drawInRect: so that my "bitmap" is rendered into this GLKView. But I can't seem to find any way of doing that simply; I'm kind of overwhelmed by all the different things OpenGL can do, all of which are much more complex than what I'm trying to do here.
glReadPixels seems to sort of be what I'm after here as it seems to provide a pointer to the buffer data.
Edit: apparently this can only be accomplished via the use of textures. I have attempted to implement this with this sample code (note that my "bitmap" here is 4 bytes per sample, matching the format parameters):
- (void)glkView:(GLKView *)view drawInRect:(CGRect)rect {
glClearColor(1.0f, 0.0f, 0.0f, 1.0f);
glClear(GL_COLOR_BUFFER_BIT);
// raw data
int width = 30;
int height = 30;
int pixelCount = width * height;
int byteSize = pixelCount * 4;
GLubyte *textureData = (GLubyte *)malloc(byteSize);
for (int i = 0; i < byteSize; i++) {
textureData[i] = 255; // white
}
glEnable(GL_TEXTURE_2D);
GLuint textureID;
glPixelStorei(GL_UNPACK_ALIGNMENT, 1);
glGenTextures(1, &textureID);
glBindTexture(GL_TEXTURE_2D, textureID);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, width, height, 0,
GL_RGBA, GL_UNSIGNED_BYTE, textureData);
free(textureData);
}
... but it is not working. The glClear() call works as expected and sets the entire background red; but if I understand the texture samples correctly, the code below that should be drawing a 30x30 white square in the corner, but all I'm getting is the solid red background.
Can anybody spot what I'm doing wrong here?
Sorry about pushing into OpenGL :)
I would create CGImage and then UIImage
void *baseAddress = & textureData;
size_t bytesPerRow = width * 4;
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
CGContextRef context = CGBitmapContextCreate(baseAddress, width, height, 8, bytesPerRow, colorSpace, kCGBitmapByteOrder32Little | kCGImageAlphaPremultipliedFirst);
CGImageRef cgImage = CGBitmapContextCreateImage(context);
CGContextRelease(context);
CGColorSpaceRelease(colorSpace);
UIImage *image = [UIImage imageWithCGImage:cgImage];
and then draw it with drawInRect:
Edit:
Here is code I've written. This is not production code: it lacks checking for openGL errors, some minor leaks and have global variables but it's good place to enhance it.
#import "ViewController.h"
#include <OpenGLES/ES2/gl.h>
#include <OpenGLES/ES2/glext.h>
#interface ViewController ()
#end
#implementation ViewController
GLuint tex;
float vertices[] = {
// Position Texcoords
-1.0f, 1.0f, 0.0f, 0.0f, // Top-left
1.0f, 1.0f, 1.0f, 0.0f, // Top-right
1.0f, -1.0f, 1.0f, 1.0f, // Bottom-right
-1.0, -1.0f, 0.0f, 1.0f // Bottom-left
};
const char * vertexShader = "attribute vec2 position;\n"
"attribute vec2 TexCoordIn;\n"
"varying vec2 TexCoordOut;\n"
"void main() {\n"
"gl_Position = vec4(position, 0.0, 1.0);\n"
"TexCoordOut = TexCoordIn;\n"
"}\n";
const char * fragmentShader = "precision mediump float;\n"
"varying lowp vec2 TexCoordOut;\n"
"uniform sampler2D Texture;\n"
"void main() {\n"
"gl_FragColor = texture2D(Texture, TexCoordOut);\n"
"}\n";
GLuint shaderProgram;
GLuint vao;
GLuint vbo;
-(void) initOpenGLObjects {
glGenVertexArraysOES(1, &vao);
glBindVertexArrayOES(vao);
glGenTextures(1, &tex);
glBindTexture(GL_TEXTURE_2D, tex);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glGenerateMipmap(GL_TEXTURE_2D);
GLuint vs = glCreateShader(GL_VERTEX_SHADER);
glShaderSource(vs, 1, &vertexShader, NULL);
glCompileShader(vs);
GLint status;
glGetShaderiv(vs, GL_COMPILE_STATUS, &status);
char buffer[512];
glGetShaderInfoLog(vs, 512, NULL, buffer);
GLuint fs = glCreateShader(GL_FRAGMENT_SHADER);
glShaderSource(fs, 1, &fragmentShader, NULL);
glCompileShader(fs);
glGetShaderiv(fs, GL_COMPILE_STATUS, &status);
glGetShaderInfoLog(fs, 512, NULL, buffer);
shaderProgram = glCreateProgram();
glAttachShader(shaderProgram, vs);
glAttachShader(shaderProgram, fs);
glLinkProgram(shaderProgram);
GLint posAttrib = glGetAttribLocation(shaderProgram, "position");
GLint texAttrib = glGetAttribLocation(shaderProgram, "TexCoordIn");
glGenBuffers(1, &vbo); // Generate 1 buffer
glBindBuffer(GL_ARRAY_BUFFER, vbo);
glBufferData(GL_ARRAY_BUFFER, sizeof(vertices), vertices, GL_STATIC_DRAW);
glVertexAttribPointer(posAttrib, 2, GL_FLOAT, GL_FALSE, 4*sizeof(float), 0);
glVertexAttribPointer(texAttrib, 2, GL_FLOAT, GL_FALSE, 4*sizeof(float), (void*)(2*sizeof(float)));
glEnableVertexAttribArray(posAttrib);
glEnableVertexAttribArray(texAttrib);
}
-(void) viewDidLoad{
[super viewDidLoad];
self.glkView.context = [[EAGLContext alloc] initWithAPI:
kEAGLRenderingAPIOpenGLES2];
[EAGLContext setCurrentContext:self.glkView.context];
CADisplayLink *displayLink = [CADisplayLink displayLinkWithTarget:self selector:#selector(update)];
[displayLink addToRunLoop:[NSRunLoop mainRunLoop] forMode:NSDefaultRunLoopMode];
self.glkView.delegate = self;
[self initOpenGLObjects];
}
- (void)update
{
[self.glkView setNeedsDisplay];
}
- (void)glkView:(GLKView *)view drawInRect:(CGRect)rect {
glClearColor(1.0f, 0.0f, 0.0f, 1.0f);
glClear(GL_COLOR_BUFFER_BIT);
int width = 300;
int height = 300;
int pixelCount = width * height;
int byteSize = pixelCount * 4;
GLubyte *textureData = (GLubyte *)malloc(byteSize);
static int time = 0;
time = (time+1)%256;
for (int i = 0; i < byteSize; i+=4) {
textureData[i] = 255;
textureData[i+1] = time;
textureData[i+2] = 255;
textureData[i+3] = 255;
}
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, width, height, 0,
GL_RGBA, GL_UNSIGNED_BYTE, textureData);
glUseProgram(shaderProgram);
glDrawArrays(GL_TRIANGLE_FAN, 0, 4);
free(textureData);
}
#end
I don't know how do you want to update data, maybe you don't have to malloc/free it every time and just do small changes and update it with glTexSubImage2D. Most of the time is spend on filling data.
This code worked on my MacBook with Xcode 7 in simulators and iPhone 6 with iOS 9.0.2.

Passing 2 Textures into fragmenshader

Im want to get to different textures in my fragment shader. But somehow it won't work. It is only getting one of them.
I'm starting my rendering with :
glBindVertexArrayOES(_heighMap.vertexArray);
//Get Uniforms
GLuint mapProj = glGetUniformLocation(_mapProgram, "modelViewProjectionMatrix");
GLuint mapView = glGetUniformLocation(_mapProgram, "modelViewMatrix");
GLuint mapNormal = glGetUniformLocation(_mapProgram, "normalMatrix");
GLuint map2D0 =glGetUniformLocation(_mapProgram, "uSampler0");
GLuint map2D1 =glGetUniformLocation(_mapProgram, "uSampler1");
// bind a texture
glEnable(GL_TEXTURE_2D);
glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_2D, _heighMap.textureID0);
glUniform1i(map2D0, 0);
glActiveTexture(GL_TEXTURE1);
glBindTexture(GL_TEXTURE_2D, _heighMap.textureID1);
glUniform1i(map2D1, 1);
//Lade Object
glUseProgram(_mapProgram);
glUniformMatrix4fv(mapProj, 1, 0, _heighMap.projectionMatix.m);
glUniformMatrix3fv(mapNormal, 1, 0, _heighMap.normalMatrix.m);
glDrawElements(GL_TRIANGLE_STRIP, _heighMap.sizeVertexIndeces, GL_UNSIGNED_INT, _heighMap.vertexIndeces);
glActiveTexture(GL_TEXTURE0);
my fragment shader nothing special:
uniform sampler2D uSampler0;
uniform sampler2D uSampler1;
varying lowp vec2 vTexCoord;
void main()
{
lowp vec4 texCol = texture2D(uSampler1, vTexCoord);
gl_FragColor = vec4(texCol.rgba);
}
just testing there if I got the textures.
and here I'm getting my Textures:
-(void)loadMapImages:(NSString *)p : (NSString *)type TexID:(uint *)textureID count:(uint)c{
NSString *path = [[NSBundle mainBundle] pathForResource:p ofType:type];
NSData *texData = [[NSData alloc] initWithContentsOfFile:path];
UIImage *image = [[UIImage alloc] initWithData:texData];
size_t width = CGImageGetWidth(image.CGImage);
size_t height = CGImageGetHeight(image.CGImage);
CGColorSpaceRef colorSpace = CGColorSpaceCreateDeviceRGB();
void *imageData = malloc( height * width * 4 );
CGContextRef context0 = CGBitmapContextCreate( imageData, width, height, 8, 4 * width, colorSpace, kCGImageAlphaPremultipliedLast | kCGBitmapByteOrder32Big );
CGColorSpaceRelease( colorSpace );
CGContextClearRect( context0, CGRectMake( 0, 0, width, height ) );
CGContextTranslateCTM( context0, 0, height - height );
CGContextDrawImage( context0, CGRectMake( 0, 0, width, height ), image.CGImage );
GLuint tex;
switch (c) {
case 0:
tex = GL_TEXTURE0;
break;
case 1:
tex = GL_TEXTURE1;
break;
default:
break;
}
glActiveTexture(tex);
glBindTexture(GL_TEXTURE_2D, *textureID);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER,GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MAG_FILTER,GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_REPEAT);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_REPEAT);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, (int)width, (int)height, 0, GL_RGBA, GL_UNSIGNED_BYTE, imageData);
glBindTexture(GL_TEXTURE_2D, tex);
CGContextRelease(context0);
free(imageData);
}
You are passing the uniforms of your samplers before you call glUseProgram
Try moving all calls to:
glGetUniformLocation(*);
As well as:
glUniform1i(map2D0, 0);
glUniform1i(map2D1, 1);
To be after you call glUseProgram(_mapProgram);
**I am assuming you are getting only the texture bound to location 0 because your samplers are both getting the default 0 values

Can't retrieve multiple gl_Fragdata buffers in OpenGL in IOS (NV12 -> YV12/i420 conversion)

I'm attempting to convert camera frames which are in YUV 4.2.2 (NV12) format to YUV 4.2.0 (YV12/i420) format. The U and V channels of NV12 are interleaved, while the YV12 are not.
For performance reasons, I'm trying to do this conversion in an OpenGl ES 2.0 shader.
The shaders are fairly easy. I separate out the Y and the UV channels and then pass the UV to a shader that separates it out into a U channel in gl_FragData[0] and V channel in gl_FragData[1]
Here is the Fragment Shader:
precision mediump float;
uniform sampler2D t_texture_uv;
varying highp vec2 uvVarying;
void main()
{
/*
vec2 uvu,uvv;
int x,y;
x = uvVarying.x;
y = uvVarying.y;
uvu = vec2(x*2,y);
uvv = vec2((x*2)+1,y);
gl_FragData[0] = texture2D(t_texture_uv, uvu);
gl_FragData[1] = texture2D(t_texture_uv, uvv);
*/
// this is the same as the logic above, but is less processor intensive
gl_FragData[0] = texture2D(t_texture_uv, vec2( uvVarying.x*2 ,uvVarying.y)); // u-plane
gl_FragData[1] = texture2D(t_texture_uv, vec2((uvVarying.x*2)+1,uvVarying.y)); // v-plane
}
And the vertex shader:
attribute vec2 position;
attribute vec2 uv;
uniform mat4 proj_matrix;
varying vec2 uvVarying;
void main()
{
gl_Position = proj_matrix * vec4(vec3(position.xy, 0.0), 1.0);
uvVarying = uv;
}
My problem is that I'm lost at how to retrieve the data from the Gl_FragData buffers back in my Objective C code. I tried glReadPixels, but it doesn't seem to work.
This is my code, but I think I've gone down the wrong path.
- (void) nv12_to_yv12_converter:(unsigned char*)YBuf UVBuff:(unsigned char*)UVBuf Width:(GLsizei)UVWidth Height:(GLsizei)UVHeight {
unsigned char *Ubufout;
unsigned char *Vbufout;
EAGLContext *context;
context = [[EAGLContext alloc] initWithAPI:kEAGLRenderingAPIOpenGLES2];
if (!context || ![EAGLContext setCurrentContext:context]) {
NSLog(#"Failed to create ES context");
}
[self loadShaders];
glGenFramebuffers(1, &_framebuffer);
glGenRenderbuffers(1, &_renderbuffer);
glBindFramebuffer(GL_FRAMEBUFFER, _framebuffer);
glBindRenderbuffer(GL_RENDERBUFFER, _renderbuffer);
;
glRenderbufferStorage(GL_RENDERBUFFER, GL_DEPTH_COMPONENT16, UVWidth, UVHeight);
// [context renderbufferStorage:GL_RENDERBUFFER fromDrawable:(CAEAGLLayer*)self.view.layer];
glGetRenderbufferParameteriv(GL_RENDERBUFFER, GL_RENDERBUFFER_WIDTH, &_backingWidth);
glGetRenderbufferParameteriv(GL_RENDERBUFFER, GL_RENDERBUFFER_HEIGHT, &_backingHeight);
glFramebufferRenderbuffer(GL_FRAMEBUFFER, GL_COLOR_ATTACHMENT0, GL_RENDERBUFFER, _renderbuffer);
_vertices[0] = -1.0f; // x0
_vertices[1] = -1.0f; // y0
_vertices[2] = 1.0f; // ..
_vertices[3] = -1.0f;
_vertices[4] = -1.0f;
_vertices[5] = 1.0f;
_vertices[6] = 1.0f; // x3
_vertices[7] = 1.0f; // y3
int orientation = 0;
static const GLfloat texCoords[] = {
0.0f, 1.0f,
1.0f, 1.0f,
0.0f, 0.0f,
1.0f, 0.0f,
};
glViewport(0, 0, _backingWidth, _backingHeight);
glClearColor(0.0f, 0.0f, 0.0f, 1.0f);
glClear(GL_COLOR_BUFFER_BIT);
glUseProgram(_program);
if (!validateProgram(_program))
{
NSLog(#"Failed to validate program");
return;
}
glPixelStorei(GL_UNPACK_ALIGNMENT, 1);
glDeleteTextures(1, &_textures[0]);
glActiveTexture(GL_TEXTURE0);
glGenTextures(1, &_textures[0]);
glBindTexture(GL_TEXTURE_2D, _textures[0]);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MAG_FILTER, GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_MIN_FILTER, GL_LINEAR);
glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameterf(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
glEnable(GL_TEXTURE_2D);
glUniform1i(glGetUniformLocation(_program, "t_texture_uv"), 0);
glTexImage2D(GL_TEXTURE_2D,
0,
GL_LUMINANCE,
16,
8,
0,
GL_LUMINANCE,
GL_UNSIGNED_BYTE,
UVBuf);
_uniformMatrix = glGetUniformLocation(_program, "proj_matrix");
GLfloat modelviewProj[16];
mat4f_LoadOrtho(-1.0f, 1.0f, -1.0f, 1.0f, -1.0f, 1.0f, modelviewProj);
glUniformMatrix4fv(_uniformMatrix, 1, GL_FALSE, modelviewProj);
//draw RECT
glVertexAttribPointer(ATTRIB_VERTEX, 2, GL_FLOAT, 0, 0, _vertices);
glEnableVertexAttribArray(ATTRIB_VERTEX);
//ATTRIB_TEXTUREPOSITON
glVertexAttribPointer(ATTRIB_UV, 2, GL_FLOAT, 0, 0, texCoords);
glEnableVertexAttribArray(ATTRIB_UV);
glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);
glReadPixels(0, 0, UVWidth, UVHeight, GL_COLOR_ATTACHMENT0, GL_UNSIGNED_BYTE, &Ubufout);
glReadPixels(0, 0, UVWidth, UVHeight, GL_COLOR_ATTACHMENT0+1, GL_UNSIGNED_BYTE, &Vbufout);
NSLog(#"Complete");
}
Can anyone shed some light on how to retrieve the gl_FragData buffers?

ios4.2 opengl can not work

I have get the h.264 data and decoder them to YUV buffers,and i have display the yup data by opengl on ios5 and ios6,but when I try to run it on my iPad(ios4.2.1),it can not display currently,just all the screen green color.I don't know why,here is my code:
-(void)playVideoData:(void *)data
{
if (!_textureY)
{
glGenTextures(1, &_textureY);
glGenTextures(1, &_textureU);
glGenTextures(1, &_textureV);
}
glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_2D, _textureY);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RED_EXT, _videoW, _videoH, 0, GL_RED_EXT, GL_UNSIGNED_BYTE, y);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MAG_FILTER,GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER,GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
glActiveTexture(GL_TEXTURE1);
glBindTexture(GL_TEXTURE_2D, _textureU);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RED_EXT, _videoW/2, _videoH/2, 0, GL_RED_EXT, GL_UNSIGNED_BYTE, u);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MAG_FILTER,GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER,GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
glActiveTexture(GL_TEXTURE2);
glBindTexture(GL_TEXTURE_2D, _textureV);
glTexImage2D(GL_TEXTURE_2D, 0, GL_RED_EXT, _videoW/2, _videoH/2, 0, GL_RED_EXT, GL_UNSIGNED_BYTE, v);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MAG_FILTER,GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D,GL_TEXTURE_MIN_FILTER,GL_LINEAR);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
[self render];
}
- (void)render
{
glViewport(viewportx,viewporty, VIEWWIDTH, VIEWHEIGHT);
glClearColor(0.0, 0, 0.0, 0);
glClear(GL_COLOR_BUFFER_BIT);
glUseProgram(programId);
// Update uniform value
//glUniform1f(uniforms[UNIFORM_TRANSLATE], 0.0f);
GLuint textureUniformY = glGetUniformLocation(programId, "SamplerY");
GLuint textureUniformU = glGetUniformLocation(programId, "SamplerU");
GLuint textureUniformV = glGetUniformLocation(programId, "SamplerV");
// Update attribute values
glVertexAttribPointer(ARDRONE_ATTRIB_POSITION, 2, GL_FLOAT, 0, 0, squareVertices);
glEnableVertexAttribArray(ARDRONE_ATTRIB_POSITION);
glVertexAttribPointer(ARDRONE_ATTRIB_TEXCOORD, 2, GL_FLOAT, 0, 0, coordVertices);
glEnableVertexAttribArray(ARDRONE_ATTRIB_TEXCOORD);
glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_2D, _textureY);
glUniform1i(textureUniformY, 0);
glActiveTexture(GL_TEXTURE1);
glBindTexture(GL_TEXTURE_2D, _textureU);
glUniform1i(textureUniformU, 1);
glActiveTexture(GL_TEXTURE2);
glBindTexture(GL_TEXTURE_2D, _textureV);
glUniform1i(textureUniformV, 2);
}
-(void)drawFrame2
{
if (context != nil)
{
//make it the current context for rendering
[EAGLContext setCurrentContext:context];
//if our framebuffers have not been created yet, do that now!
if (!defaultFramebuffer)
[self createFramebuffer];
glBindFramebuffer(GL_FRAMEBUFFER, defaultFramebuffer);
[self playVideoData];
glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);
glBindRenderbuffer(GL_RENDERBUFFER, colorRenderbuffer);
[context presentRenderbuffer:GL_RENDERBUFFER];
}
else
NSLog(#"Context not set!");
}
I have put the data to the y u v buffer
here is my fsh and vsh:
vsh:
attribute vec4 position; // 1
//uniform float translate;
attribute vec2 TexCoordIn; // New
varying vec2 TexCoordOut; // New
void main(void)
{
gl_Position = position; // 6
TexCoordOut = TexCoordIn;
}
fsh:
varying lowp vec2 TexCoordOut;
uniform sampler2D SamplerY;
uniform sampler2D SamplerU;
uniform sampler2D SamplerV;
void main(void)
{
mediump vec3 yuv;
lowp vec3 rgb;
yuv.x = texture2D(SamplerY, TexCoordOut).r;
yuv.y = texture2D(SamplerU, TexCoordOut).r - 0.5;
yuv.z = texture2D(SamplerV, TexCoordOut).r - 0.5;
rgb = mat3( 1, 1, 1,
0, -0.39465, 2.03211,
1.13983, -0.58060, 0) * yuv;
gl_FragColor = vec4(rgb, 1);
}
iOs4.2 no erros ,the shaders Compile ok, but just can not display...
This is because the GL_RED_EXT extension that you are using to upload your textures was only added in iOS 5.0, and isn't present in iOS 4.2. You won't be able to use that to upload your YUV textures in this manner, so you'll need to rewrite that part of your code. Also, I believe this extension is only supported for iPad 2 and newer devices, not the original iPad and older iPhones, so you won't be able to use it on even iOS 5+ for the older ones.

Resources