I am adding WebGL support in my game, but I have a strange problem : it runs even slower than in Canvas 2D rendering mode, and I do not understand why.
I checked on both Firefox PC, Chrome PC, and Chrome Android, they run WebGL demos on the web with hundreds of sprites smoothly though, so I definitly made an error in my code.
Firefox's profiler says the whole game uses only 7% of the ressources, the rendering part takes only 1.2%. It is just the title screen of the game and there are only five sprites to draw. It is slow though...
update : Chrome's profiler says idle is only 4%, program is a huge 93%, and render 2.6%.
When using Canvas 2D things are very different, 76% idle, 16% program, 2.3% for the drawing function.
There definitly is a problem in my WebGL rendering code.
update : Android Chrome's profiler (on JXD S5110) always says program is ~39%, drawArrays is ~ 8%, bufferData ~5%, and bindTexture is 3%. Everything else is quite negligible.
If a function of mines was wasting all the ressources I would know what to do, but here the bottlenecks seem to be "program" (the browser itself ?) and webgl methods, two things I can't edit.
Please someone have a look at my code and tell me what I did wrong.
Here are my shaders
<script id="2d-vertex-shader" type="x-shader/x-vertex">
attribute vec2 a_position;
attribute vec2 a_texCoord;
uniform vec2 u_resolution;
uniform vec2 u_translation;
uniform vec2 u_rotation;
varying vec2 v_texCoord;
void main()
{
// Rotate the position
vec2 rotatedPosition = vec2(
a_position.x * u_rotation.y + a_position.y * u_rotation.x,
a_position.y * u_rotation.y - a_position.x * u_rotation.x);
// Add in the translation.
vec2 position = rotatedPosition + u_translation;
// convert the rectangle from pixels to 0.0 to 1.0
vec2 zeroToOne = a_position / u_resolution;
// convert from 0->1 to 0->2
vec2 zeroToTwo = zeroToOne * 2.0;
// convert from 0->2 to -1->+1 (clipspace)
vec2 clipSpace = zeroToTwo - 1.0;
gl_Position = vec4(clipSpace * vec2(1, -1), 0, 1);
// pass the texCoord to the fragment shader
// The GPU will interpolate this value between points
v_texCoord = a_texCoord;
}
</script>
<script id="2d-fragment-shader" type="x-shader/x-fragment">
precision mediump float;
// our texture
uniform sampler2D u_image;
// the texCoords passed in from the vertex shader.
varying vec2 v_texCoord;
void main()
{
// Look up a color from the texture.
gl_FragColor = texture2D(u_image, vec2(v_texCoord.s, v_texCoord.t));
}
</script>
Here is the creation code of my canvas and their contexts when in WebGL mode.
(I use to use several layered canvas in order to avoid drawing the backgrounds and foregrounds at every frame while they never change, that is why canvas and contexts are in arrays.)
// Get A WebGL context
liste_canvas[c] = document.createElement("canvas") ;
document.getElementById('game_div').appendChild(liste_canvas[c]);
liste_ctx[c] = liste_canvas[c].getContext('webgl',{premultipliedAlpha:false}) || liste_canvas[c].getContext('experimental-webgl',{premultipliedAlpha:false});
var gl = liste_ctx[c] ;
gl.viewport(0, 0, game.res_w, game.res_h);
// setup a GLSL program
gl.vertexShader = createShaderFromScriptElement(gl, "2d-vertex-shader");
gl.fragmentShader = createShaderFromScriptElement(gl, "2d-fragment-shader");
gl.program = createProgram(gl, [gl.vertexShader, gl.fragmentShader]);
gl.useProgram(gl.program);
// look up where the vertex data needs to go.
positionLocation = gl.getAttribLocation(gl.program, "a_position");
texCoordLocation = gl.getAttribLocation(gl.program, "a_texCoord");
// provide texture coordinates for the rectangle.
texCoordBuffer = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, texCoordBuffer);
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array([
0.0, 0.0,
1.0, 0.0,
0.0, 1.0,
0.0, 1.0,
1.0, 0.0,
1.0, 1.0]), gl.STATIC_DRAW);
gl.enableVertexAttribArray(texCoordLocation);
gl.vertexAttribPointer(texCoordLocation, 2, gl.FLOAT, false, 0, 0);
gl.blendFunc(gl.SRC_ALPHA, gl.ONE_MINUS_SRC_ALPHA);
gl.enable( gl.BLEND ) ;
gl.posBuffer = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, gl.posBuffer);
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array([
0.0, 0.0,
1.0, 0.0,
0.0, 1.0,
0.0, 1.0,
1.0, 0.0,
1.0, 1.0]), gl.STATIC_DRAW);
gl.enableVertexAttribArray(positionLocation);
gl.vertexAttribPointer(positionLocation, 2, gl.FLOAT, false, 0, 0);
In the .onload function of my images, I add
var gl = liste_ctx[c] ;
this.texture = gl.createTexture();
gl.bindTexture(gl.TEXTURE_2D, this.texture);
gl.texImage2D(gl.TEXTURE_2D, 0, gl.RGBA, gl.RGBA, gl.UNSIGNED_BYTE, this );
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MIN_FILTER, gl.LINEAR);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_MAG_FILTER, gl.LINEAR);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_S, gl.CLAMP_TO_EDGE);
gl.texParameteri(gl.TEXTURE_2D, gl.TEXTURE_WRAP_T, gl.CLAMP_TO_EDGE);
gl.bindTexture(gl.TEXTURE_2D, null);
And here is the WebGL part of my draw_sprite() function :
var gl = liste_ctx[c] ;
gl.bindTexture(gl.TEXTURE_2D, sprites[d_sprite].texture);
var resolutionLocation = gl.getUniformLocation(gl.program, "u_resolution");
gl.uniform2f(resolutionLocation, liste_canvas[c].width, liste_canvas[c].height);
gl.bindBuffer(gl.ARRAY_BUFFER, gl.posBuffer);
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array([
topleft_x , topleft_y ,
topright_x , topright_y ,
bottomleft_x , bottomleft_y ,
bottomleft_x , bottomleft_y ,
topright_x , topright_y ,
bottomright_x , bottomright_y ]), gl.STATIC_DRAW);
gl.drawArrays(gl.TRIANGLES, 0, 6);
What did I do wrong ?
This may help: What do the "Not optimized" warnings in the Chrome Profiler mean?
Relevant links:
https://groups.google.com/forum/#!topic/v8-users/_oZ4fUSitRY
https://github.com/petkaantonov/bluebird/wiki/Optimization-killers
For "optimized too many times", that means the function parameters / behavior change too much, so Chrome keeps having to re-optimize it.
What was making it so slow was using several webgl canvas, I use only one now and it works way better. But it is still a bit slower than Canvas 2D though, and the profiler says 65% is idle while it lags as hell so I really don't understand...
edit : I think I got it. Since my computer is running WinXP, hardware acceleration for WebGL can't be enabled, so the browsers use software rendering, and that explains why 'program' is huge in Chrome's profiler. However, hardware acceleration seems to work for 2d context, that is why it is faster.
Related
I'm trying to follow the suggestion in Apple's OpenGL ES Programming Guide section on instanced drawing: Use Instanced Drawing to Minimize Draw Calls. I have started with the example project that XCode generates for a Game app with OpenGL and Swift and converted it to OpenGL ES 3.0, adding some instanced drawing to duplicate the cube.
This works fine when I use the gl_InstanceID technique and simply generate an offset from that. But when I try to use the 'instanced arrays' technique to pass data in via a buffer I am not seeing any results.
My updated vertex shader looks like this:
#version 300 es
in vec4 position;
in vec3 normal;
layout(location = 5) in vec2 instOffset;
out lowp vec4 colorVarying;
uniform mat4 modelViewProjectionMatrix;
uniform mat3 normalMatrix;
void main()
{
vec3 eyeNormal = normalize(normalMatrix * normal);
vec3 lightPosition = vec3(0.0, 0.0, 1.0);
vec4 diffuseColor = vec4(0.4, 0.4, 1.0, 1.0);
float nDotVP = max(0.0, dot(eyeNormal, normalize(lightPosition)));
colorVarying = diffuseColor * nDotVP;
// gl_Position = modelViewProjectionMatrix * position + vec4( float(gl_InstanceID)*1.5, float(gl_InstanceID)*1.5, 1.0,1.0);
gl_Position = modelViewProjectionMatrix * position + vec4(instOffset, 1.0, 1.0);
}
and in my setupGL() method I have added the following:
//glGenVertexArraysOES(1, &instArray) // EDIT: WRONG
//glBindVertexArrayOES(instArray) // EDIT: WRONG
let kMyInstanceDataAttrib = 5
glGenBuffers(1, &instBuffer)
glBindBuffer(GLenum(GL_ARRAY_BUFFER), instBuffer)
glBufferData(GLenum(GL_ARRAY_BUFFER), GLsizeiptr(sizeof(GLfloat) * instData.count), &instData, GLenum(GL_STATIC_DRAW))
glEnableVertexAttribArray(GLuint(kMyInstanceDataAttrib))
glVertexAttribPointer(GLuint(kMyInstanceDataAttrib), 2, GLenum(GL_FLOAT), GLboolean(GL_FALSE), 0/*or 8?*/, BUFFER_OFFSET(0))
glVertexAttribDivisor(GLuint(kMyInstanceDataAttrib), 1);
along with some simple instance offset data:
var instData: [GLfloat] = [
1.5, 1.5,
2.5, 2.5,
3.5, 3.5,
]
I am drawing the same way with the above as with the instance id technique:
glDrawArraysInstanced(GLenum(GL_TRIANGLES), 0, 36, 3)
But it seems to have no effect. I just get the single cube and it doesn't even seem to fail if I remove the buffer setup, so I suspect my setup is missing something.
EDIT: Fixed the code by removing two bogus lines from init.
I had an unecessary gen and bind for the attribute vertex array. The code as edited above now works.
I am using triangles(using vertices and face position) to draw the graphics.I am storing color information for each vertex and applying colors accordingly. But the problem is all the geometries in my scene are of single color(say cone=red, cylinder=blue). SO, storing color for each vertex is apparently of no use to me.
Is their any other approach by which coloring can be done in webgl apart from storing color information of each vertices in the scene. Maybe something like coloring the entire geometry(say a cone).
It's clear from your question you might not really understand WebGL yet? You might want to check out these tutorials.
WebGL uses shaders, those shaders use whatever inputs you define and output whatever you tell them to output. That means WebGL doesn't require vertex colors. Vertex colors are something you decide on when you write your shaders. If you don't want to use vertex colors, don't write a shader that references vertex colors.
That said there if you have a shader that happens to use vertex colors you can easily provide the shader with a constant color. Let's assume you have shaders like this that just use vertex colors.
vertex shader:
attribute vec4 a_position;
attribute vec4 a_color; // vertex colors
varying vec4 v_color; // so we can pass the colors to the fragment shader
uniform mat4 u_matrix;
void main() {
gl_Position = u_matrix * a_position;
v_color = a_color;
}
fragment shader:
precision mediump float;
varying vec4 v_color;
void main() {
gl_FragColor = v_color;
}
Now, all you have to do to use a constant color is turn off the attribute for a_color and set a constant value with gl.vertexAttrib4f like this
// at init time
var a_colorLocation = gl.getAttribLocation(program, "a_color";
// at draw time
gl.disableVertexAttribArray(a_colorLocation); // turn off the attribute
gl.vertexAttrib4f(a_colorLocation, r, g, b, a); // supply a constant color
Note that turning off attribute 0 will slow down WebGL on desktops because if differences between OpenGL and OpenGL ES. It's possible a_colorLocation is attribute 0. To avoid this problem bind your attribute locations BEFORE you link your program. Specifically since you'll always use a position (which is called "a_position" in the example above) just bind that to location 0 like this
..compile shaders..
..attach shaders to program..
// Must happen before you call linkProgram
gl.bindAttribLocation(program, 0, "a_position");
gl.linkProgram(program);
...check for errors, etc...
This will force the attribute for "a_position" to be attribute 0 so you'll always enable it.
Here's a sample
function main() {
var canvas = document.getElementById("c");
var gl = canvas.getContext("webgl");
if (!gl) {
alert("no WebGL");
return;
}
// NOTE:! This function binds attribute locations
// based on the indices of the second array
var program = twgl.createProgramFromScripts(
gl,
["vshader", "fshader"],
["a_position", "a_color"]); // a_position will get location 0
// a_color will get location 1
var a_positionLoc = 0;
var a_colorLoc = 1;
var u_matrixLoc = gl.getUniformLocation(program, "u_matrix");
gl.useProgram(program);
var verts = [
1, 1,
-1, 1,
-1, -1,
1, 1,
-1, -1,
1, -1,
];
var colors = [
255, 0, 0, 255,
0, 255, 0, 255,
0, 0, 255, 255,
255, 255, 0, 255,
0, 255, 255, 255,
255, 0, 255, 255,
];
var positionBuffer = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, positionBuffer);
gl.bufferData(gl.ARRAY_BUFFER, new Float32Array(verts), gl.STATIC_DRAW);
gl.enableVertexAttribArray(a_positionLoc);
gl.vertexAttribPointer(a_positionLoc, 2, gl.FLOAT, false, 0, 0);
var colorBuffer = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, colorBuffer);
gl.bufferData(gl.ARRAY_BUFFER, new Uint8Array(colors), gl.STATIC_DRAW);
gl.enableVertexAttribArray(a_colorLoc);
gl.vertexAttribPointer(a_colorLoc, 4, gl.UNSIGNED_BYTE, true, 0, 0);
// Draw in the bottom right corner
gl.uniformMatrix4fv(
u_matrixLoc,
false,
[0.5, 0, 0, 0,
0, 0.5, 0, 0,
0, 0, 1, 0,
-0.5, -0.5, 0, 1]);
gl.drawArrays(gl.TRIANGLES, 0, 6);
// Now turn off the a_color attribute and supply a solid color
gl.disableVertexAttribArray(a_colorLoc);
var r = 0.5;
var g = 1;
var b = 0.5;
var a = 1;
gl.vertexAttrib4f(a_colorLoc, r, g, b, a); // greenish
// Draw in the top left corner
gl.uniformMatrix4fv(
u_matrixLoc,
false,
[0.5, 0, 0, 0,
0, 0.5, 0, 0,
0, 0, 1, 0,
0.5, 0.5, 0, 1]);
gl.drawArrays(gl.TRIANGLES, 0, 6);
};
main();
canvas { border: 1px solid black; }
<script src="https://twgljs.org/dist/3.x/twgl.min.js"></script>
<script id="vshader" type="whatever">
attribute vec4 a_position;
attribute vec4 a_color;
varying vec4 v_color;
uniform mat4 u_matrix;
void main() {
gl_Position = u_matrix * a_position;
v_color = a_color;
}
</script>
<script id="fshader" type="whatever">
precision mediump float;
varying vec4 v_color;
void main() {
gl_FragColor = v_color;
}
</script>
<canvas id="c" width="300" height="300"></canvas>
If your geometry has color per object, that doesn't change across the geometry, then you should pass that color as the uniform variable.
So you en up with only one attribute - position of vertices, few matrix uniforms - say model, view, projection matrices, that for the vertex shader, and one vector uniform variable for the fragment shader for "shading" the object.
I was reading tutorials from here.
<script class = "WebGL">
var gl;
function initGL() {
// Get A WebGL context
var canvas = document.getElementById("canvas");
gl = getWebGLContext(canvas);
if (!gl) {
return;
}
}
var positionLocation;
var resolutionLocation;
var colorLocation;
var translationLocation;
var rotationLocation;
var translation = [50,50];
var rotation = [0, 1];
var angle = 0;
function initShaders() {
// setup GLSL program
vertexShader = createShaderFromScriptElement(gl, "2d-vertex-shader");
fragmentShader = createShaderFromScriptElement(gl, "2d-fragment-shader");
program = createProgram(gl, [vertexShader, fragmentShader]);
gl.useProgram(program);
// look up where the vertex data needs to go.
positionLocation = gl.getAttribLocation(program, "a_position");
// lookup uniforms
resolutionLocation = gl.getUniformLocation(program, "u_resolution");
colorLocation = gl.getUniformLocation(program, "u_color");
translationLocation = gl.getUniformLocation(program, "u_translation");
rotationLocation = gl.getUniformLocation(program, "u_rotation");
// set the resolution
gl.uniform2f(resolutionLocation, canvas.width, canvas.height);
}
function initBuffers() {
// Create a buffer.
var buffer = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, buffer);
gl.enableVertexAttribArray(positionLocation);
gl.vertexAttribPointer(positionLocation, 2, gl.FLOAT, false, 0, 0);
// Set Geometry.
setGeometry(gl);
}
function setColor(red, green, blue) {
gl.uniform4f(colorLocation, red, green, blue, 1);
}
// Draw the scene.
function drawScene() {
// Clear the canvas.
gl.clear(gl.COLOR_BUFFER_BIT);
// Set the translation.
gl.uniform2fv(translationLocation, translation);
// Set the rotation.
gl.uniform2fv(rotationLocation, rotation);
// Draw the geometry.
gl.drawArrays(gl.TRIANGLES, 0, 6);
}
// Fill the buffer with the values that define a letter 'F'.
function setGeometry(gl) {
/*Assume size1 is declared*/
var vertices = [
-size1/2, -size1/2,
-size1/2, size1/2,
size1/2, size1/2,
size1/2, size1/2,
size1/2, -size1/2,
-size1/2, -size1/2 ];
gl.bufferData(
gl.ARRAY_BUFFER,
new Float32Array(vertices),
gl.STATIC_DRAW);
}
function animate() {
translation[0] += 0.01;
translation[1] += 0.01;
angle += 0.01;
rotation[0] = Math.cos(angle);
rotation[1] = Math.sin(angle);
}
function tick() {
requestAnimFrame(tick);
drawScene();
animate();
}
function start() {
initGL();
initShaders();
initBuffers();
setColor(0.2, 0.5, 0.5);
tick();
}
</script>
<!-- vertex shader -->
<script id="2d-vertex-shader" type="x-shader/x-vertex">
attribute vec2 a_position;
uniform vec2 u_resolution;
uniform vec2 u_translation;
uniform vec2 u_rotation;
void main() {
vec2 rotatedPosition = vec2(
a_position.x * u_rotation.y + a_position.y * u_rotation.x,
a_position.y * u_rotation.y - a_position.x * u_rotation.x);
// Add in the translation.
vec2 position = rotatedPosition + u_translation;
// convert the position from pixels to 0.0 to 1.0
vec2 zeroToOne = position / u_resolution;
// convert from 0->1 to 0->2
vec2 zeroToTwo = zeroToOne * 2.0;
// convert from 0->2 to -1->+1 (clipspace)
vec2 clipSpace = zeroToTwo - 1.0;
gl_Position = vec4(clipSpace, 0, 1);
}
</script>
<!-- fragment shader -->
<script id="2d-fragment-shader" type="x-shader/x-fragment">
precision mediump float;
uniform vec4 u_color;
void main() {
gl_FragColor = u_color;
}
</script>
My WebGL program for 1 shape works something like this:
Get a context (gl) from the canvas element.
initialize buffers with the shape of my object
drawScene() : a call to gl.drawArrays()
If there is animation, an update function, which updates my shape's angles, positions,
and then drawScene() both in tick(), so that it gets called repeatedly.
Now when I need more than 1 shape, should I fill the single buffer at once with many objects and then use it to later call drawScene() drawing all the objects at once
[OR]
should I repeated call the initBuffer and drawScene() from requestAnimFrame().
In pseudo code
At init time
Get a context (gl) from the canvas element.
for each shader
create shader
look up attribute and uniform locations
for each shape
initialize buffers with the shape
for each texture
create textures and/or fill them with data.
At draw time
for each shape
if the last shader used is different than the shader needed for this shape call gl.useProgram
For each attribute needed by shader
call gl.enableVertexAttribArray, gl.bindBuffer and gl.vertexAttribPointer for each attribute needed by shape with the attribute locations for the current shader.
For each uniform needed by shader
call gl.uniformXXX with the desired values using the locations for the current shader
call gl.drawArrays or if the data is indexed called gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, bufferOfIndicesForCurrentShape) followed by gl.drawElements
Common Optimizations
1) Often you don't need to set every uniform. For example if you are drawing 10 shapes with the same shader and that shader takes a viewMatrix or cameraMatrix it's likely that viewMatrix uniform or cameraMatrix uniform is the same for every shape so just set it once.
2) You can often move the calls to gl.enableVertexAttribArray to initialization time.
Having multiple meshes in one buffer (and rendering them with a single gl.drawArrays() as you're suggesting) yields better performance in complex scenes but obviously at that point you're not able to change shader uniforms (such as transformations) per mesh.
If you want to have the meshes running around independently, you'll have to render each one separately. You could still keep all the meshes in one buffer to avoid some overhead from gl.bindBuffer() calls but imho that won't help that much, at least not in simple scenes.
Create your buffers separately for each object you want on the scene otherwise they won't be able to move and use shader effects independently.
But that is in case your objects are different. From what I got here I think you just want to draw the same shape more than once on different positions right?
The way you go about that is you just set that translationLocation uniform right there with a different translation matrix after drawing the shape for the first time. That way when you draw the shape again it will be located somewhere else and not in top of the other one so you can see it. You can set all those transformation matrices differently and then just call gl.drawElements again since you're going to draw the same buffers that are already in use.
I am having a look at web gl, and trying to render a cube, but I am having a problem when I try to add projection into the vertex shader. I have added an attribute, but when I use it to multiple the modelview and position, it stops displaying the cube. Im not sure why and was wondering if anyone could help? Ive tried looking at a few examples but just cant get this to work
vertex shader
attribute vec3 aVertexPosition;
uniform mat4 uMVMatrix;
uniform mat4 uPMatrix;
void main(void) {
gl_Position = uPMatrix * uMVMatrix * vec4(aVertexPosition, 1.0);
//gl_Position = uMVMatrix * vec4(aVertexPosition, 1.0);
}
fragment shader
#ifdef GL_ES
precision highp float; // Not sure why this is required, need to google it
#endif
uniform vec4 uColor;
void main() {
gl_FragColor = uColor;
}
function init() {
// Get a reference to our drawing surface
canvas = document.getElementById("webglSurface");
gl = canvas.getContext("experimental-webgl");
/** Create our simple program **/
// Get our shaders
var v = document.getElementById("vertexShader").firstChild.nodeValue;
var f = document.getElementById("fragmentShader").firstChild.nodeValue;
// Compile vertex shader
var vs = gl.createShader(gl.VERTEX_SHADER);
gl.shaderSource(vs, v);
gl.compileShader(vs);
// Compile fragment shader
var fs = gl.createShader(gl.FRAGMENT_SHADER);
gl.shaderSource(fs, f);
gl.compileShader(fs);
// Create program and attach shaders
program = gl.createProgram();
gl.attachShader(program, vs);
gl.attachShader(program, fs);
gl.linkProgram(program);
// Some debug code to check for shader compile errors and log them to console
if (!gl.getShaderParameter(vs, gl.COMPILE_STATUS))
console.log(gl.getShaderInfoLog(vs));
if (!gl.getShaderParameter(fs, gl.COMPILE_STATUS))
console.log(gl.getShaderInfoLog(fs));
if (!gl.getProgramParameter(program, gl.LINK_STATUS))
console.log(gl.getProgramInfoLog(program));
/* Create some simple VBOs*/
// Vertices for a cube
var vertices = new Float32Array([
-0.5, 0.5, 0.5, // 0
-0.5, -0.5, 0.5, // 1
0.5, 0.5, 0.5, // 2
0.5, -0.5, 0.5, // 3
-0.5, 0.5, -0.5, // 4
-0.5, -0.5, -0.5, // 5
-0.5, 0.5, -0.5, // 6
-0.5,-0.5, -0.5 // 7
]);
// Indices of the cube
var indicies = new Int16Array([
0, 1, 2, 1, 2, 3, // front
5, 4, 6, 5, 6, 7, // back
0, 1, 5, 0, 5, 4, // left
2, 3, 6, 6, 3, 7, // right
0, 4, 2, 4, 2, 6, // top
5, 3, 1, 5, 3, 7 // bottom
]);
// create vertices object on the GPU
vbo = gl.createBuffer();
gl.bindBuffer(gl.ARRAY_BUFFER, vbo);
gl.bufferData(gl.ARRAY_BUFFER, vertices, gl.STATIC_DRAW);
// Create indicies object on th GPU
ibo = gl.createBuffer();
gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, ibo);
gl.bufferData(gl.ELEMENT_ARRAY_BUFFER, indicies, gl.STATIC_DRAW);
gl.clearColor(0.0, 0.0, 0.0, 1.0);
gl.enable(gl.DEPTH_TEST);
// Render scene every 33 milliseconds
setInterval(render, 33);
}
var mvMatrix = mat4.create();
var pMatrix = mat4.create();
function render() {
// Set our viewport and clear it before we render
gl.viewport(0, 0, canvas.width, canvas.height);
gl.clear(gl.COLOR_BUFFER_BIT | gl.DEPTH_BUFFER_BIT);
gl.useProgram(program);
// Bind appropriate VBOs
gl.bindBuffer(gl.ARRAY_BUFFER, vbo);
gl.bindBuffer(gl.ELEMENT_ARRAY_BUFFER, ibo);
// Set the color for the fragment shader
program.uColor = gl.getUniformLocation(program, "uColor");
gl.uniform4fv(program.uColor, [0.3, 0.3, 0.3, 1.0]);
//
// code.google.com/p/glmatrix/wiki/Usage
program.uPMatrix = gl.getUniformLocation(program, "uPMatrix");
program.uMVMatrix = gl.getUniformLocation(program, "uMVMatrix");
mat4.perspective(45, gl.viewportWidth / gl.viewportHeight, 1.0, 10.0, pMatrix);
mat4.identity(mvMatrix);
mat4.translate(mvMatrix, [0.0, -0.25, -1.0]);
gl.uniformMatrix4fv(program.uPMatrix, false, pMatrix);
gl.uniformMatrix4fv(program.uMVMatrix, false, mvMatrix);
// Set the position for the vertex shader
program.aVertexPosition = gl.getAttribLocation(program, "aVertexPosition");
gl.enableVertexAttribArray(program.aVertexPosition);
gl.vertexAttribPointer(program.aVertexPosition, 3, gl.FLOAT, false, 3*4, 0); // position
// Render the Object
gl.drawElements(gl.TRIANGLES, 36, gl.UNSIGNED_SHORT, 0);
}
Thanks in advance for any help
Problem is here:
..., gl.viewportWidth / gl.viewportHeight, ...
Both gl.viewportWidth and gl.viewportHeight are undefined values.
I think you missed this two lines:
gl.viewportWidth = canvas.width;
gl.viewportHeight = canvas.height;
You will see a lot of people doing this:
canvas.width = canvas.clientWidth;
canvas.height = canvas.clientHeight;
gl.viewportWidth = canvas.width;
gl.viewportHeight = canvas.height;
But please note that WebGL context also have this two attributes:
gl.drawingBufferWidth
gl.drawingBufferHeight
So your cube shows up without the perspective matrix, correct?
At first glance I would think that you may be clipping away your geometry with the near plane. You provide a near an far plane to the perpective function as 1.0 and 10.0 respectively. This means that for any fragments to be visible they must fall in the z range of [1, 10]. You cube is 1 unit per side, centered on (0, 0, 0), and you are moving it "back" from the camera 1 unit. This means that the nearest face to the camera will actually be at 0.5 Z, which is outside the clipping range and therefore discarded. About half of your cube WILL be at z > 1, but you'll be looking at the inside of the cube at that point. If you have backface culling turned on you won't see anything.
Long story short - Your cube is probably too close to the camera. Try this instead:
mat4.translate(mvMatrix, [0.0, -0.25, -3.0]);
I'm trying to render a native planar image to an OpenGL ES 2.0 texture in iOS 4.3 on an iPhone 4. The texture however winds up all black. My camera is configured as such:
[videoOutput setVideoSettings:[NSDictionary dictionaryWithObject:[NSNumber numberWithInt:kCVPixelFormatType_420YpCbCr8BiPlanarVideoRange]
forKey:(id)kCVPixelBufferPixelFormatTypeKey]];
and I'm passing the pixel data to my texture like this:
glTexImage2D(GL_TEXTURE_2D, 0, GL_RGBA, bufferWidth, bufferHeight, 0, GL_RGB_422_APPLE, GL_UNSIGNED_SHORT_8_8_REV_APPLE, CVPixelBufferGetBaseAddress(cameraFrame));
My fragement shaders is:
varying highp vec2 textureCoordinate;
uniform sampler2D videoFrame;
void main() {
lowp vec4 color;
color = texture2D(videoFrame, textureCoordinate);
lowp vec3 convertedColor = vec3(-0.87075, 0.52975, -1.08175);
convertedColor += 1.164 * color.g; // Y
convertedColor += vec3(0.0, -0.391, 2.018) * color.b; // U
convertedColor += vec3(1.596, -0.813, 0.0) * color.r; // V
gl_FragColor = vec4(convertedColor, 1.0);
}
and my vertex shader is
attribute vec4 position;
attribute vec4 inputTextureCoordinate;
varying vec2 textureCoordinate;
void main()
{
gl_Position = position;
textureCoordinate = inputTextureCoordinate.xy;
}
This works just fine when I'm working with an BGRA image, and my fragment shader only does
gl_FragColor = texture2D(videoFrame, textureCoordinate);
What if anything am I missing here? Thanks!
OK. We have a working success here. The key was passing the Y and the UV as two separate textures to the fragment shader. Here is the final shader:
#ifdef GL_ES
precision mediump float;
#endif
varying vec2 textureCoordinate;
uniform sampler2D videoFrame;
uniform sampler2D videoFrameUV;
const mat3 yuv2rgb = mat3(
1, 0, 1.2802,
1, -0.214821, -0.380589,
1, 2.127982, 0
);
void main() {
vec3 yuv = vec3(
1.1643 * (texture2D(videoFrame, textureCoordinate).r - 0.0625),
texture2D(videoFrameUV, textureCoordinate).r - 0.5,
texture2D(videoFrameUV, textureCoordinate).a - 0.5
);
vec3 rgb = yuv * yuv2rgb;
gl_FragColor = vec4(rgb, 1.0);
}
You'll need to create your textures along like this:
int bufferHeight = CVPixelBufferGetHeight(cameraFrame);
int bufferWidth = CVPixelBufferGetWidth(cameraFrame);
glBindTexture(GL_TEXTURE_2D, videoFrameTexture);
glTexImage2D(GL_TEXTURE_2D, 0, GL_LUMINANCE, bufferWidth, bufferHeight, 0, GL_LUMINANCE, GL_UNSIGNED_BYTE, CVPixelBufferGetBaseAddressOfPlane(cameraFrame, 0));
glBindTexture(GL_TEXTURE_2D, videoFrameTextureUV);
glTexImage2D(GL_TEXTURE_2D, 0, GL_LUMINANCE_ALPHA, bufferWidth/2, bufferHeight/2, 0, GL_LUMINANCE_ALPHA, GL_UNSIGNED_BYTE, CVPixelBufferGetBaseAddressOfPlane(cameraFrame, 1));
and then pass them like this:
glActiveTexture(GL_TEXTURE0);
glBindTexture(GL_TEXTURE_2D, videoFrameTexture);
glActiveTexture(GL_TEXTURE1);
glBindTexture(GL_TEXTURE_2D, videoFrameTextureUV);
glActiveTexture(GL_TEXTURE0);
glUniform1i(videoFrameUniform, 0);
glUniform1i(videoFrameUniformUV, 1);
Boy am I relieved!
P.S. The values for the yuv2rgb matrix are from here http://en.wikipedia.org/wiki/YUV and I copied code from here http://www.ogre3d.org/forums/viewtopic.php?f=5&t=25877 to figure out how to get the correct YUV values to begin with.
Your code appears to attempt to convert a 32-bit colour in 444-plus-unused-byte to RGBA. That's not going to work too well. I don't know of anything that outputs "YUVA", for one.
Also, I think the returned alpha channel is 0 for BGRA camera output, not 1, so I'm not sure why it works (IIRC to convert it to a CGImage you need to use AlphaNoneSkipLast).
The 420 "bi planar" output is structued something like this:
A header telling you where the planes are (used by CVPixelBufferGetBaseAddressOfPlane() and friends)
The Y plane: height × bytes_per_row_1 × 1 bytes
The Cb,Cr plane: height/2 × bytes_per_row_2 × 2 bytes (2 bytes per 2x2 block).
bytes_per_row_1 is approximately width and bytes_per_row_2 is approximately width/2, but you'll want to use CVPixelBufferGetBytesPerRowOfPlane() for robustness (you also might want to check the results of ..GetHeightOfPlane and ...GetWidthOfPlane).
You might have luck treating it as a 1-component width*height texture and a 2-component width/2*height/2 texture. You'll probably want to check bytes-per-row and handle the case where it isn't simply width*number-of-components (although this is probably true for most of the video modes). AIUI, you'll also want to flush the GL context before calling CVPixelBufferUnlockBaseAddress().
Alternatively, you can copy it all to memory into your expected format (optimizing this loop might be a bit tricky). Copying has the advantage that you don't need to worry about things accessing memory after you've unlocked the pixel buffer.