Fragment shader causes serious lag when I run it on iPhone 4. I tried to comment part of calculations, however still there are some jitters even though I barely am doing any calculation in the Fragment Shader.
// Fragment Shader Code
uniform sampler2D texture;
varying lowp vec2 fragmentTexCoords;
uniform lowp float passAlpha;
uniform lowp vec2 inPosition;
uniform lowp float varUniform;
void main()
{
gl_FragColor = texture2D(texture, fragmentTexCoords);
lowp float disY = gl_FragCoord.y - inPosition.y;
lowp float disMax = 250.0;
lowp float coeff = 1.0 - varUniform;
gl_FragColor.rgb *= coeff;
}
//My render function is:
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_S, GL_CLAMP_TO_EDGE);
glTexParameteri(GL_TEXTURE_2D, GL_TEXTURE_WRAP_T, GL_CLAMP_TO_EDGE);
glBlendFunc(GL_SRC_ALPHA, GL_ONE_MINUS_SRC_ALPHA);
glDrawArrays(GL_TRIANGLE_STRIP, 0, 4);
glFlush();
I am still not sure what could be the problem, I am sure iPhone can handle way more complex calculations ... Any ideas ?
Thanks in advance.
I would try this, it avoids several situations where you are doing calculations at enhanced precision because of the way your variables are declared... if this improves your performance, I can further explain why it works.
// Fragment Shader Code
uniform lowp sampler2D texture;
varying lowp vec2 fragmentTexCoords;
uniform lowp float passAlpha;
uniform lowp vec2 inPosition;
uniform lowp float varUniform;
void main ()
{
lowp vec4 color = texture2D (texture, fragmentTexCoords);
lowp float disY = gl_FragCoord.y - inPosition.y;
lowp float disMax = 250.0;
lowp float coeff = 1.0 - varUniform;
color.rgb *= coeff;
gl_FragColor = color;
}
Are you certain the stutter is caused by your fragment shader? Can you verify this by removing most operations from the fragment shader? I'm asking because you're really not doing anything too expensive in your shader code, and it would be odd for that to cause any performance problems. Are you sure you're not doing anything else, like uploading textures, in your update loop? What do the xcode profiling tools say about your performance?
Related
I want to have prepared shader component (for multi sampler tex)
In my current state i use (activate and bind) only 2 texture image.
But this line :
gl_FragColor = textureColor + textureColor1 + textureColor2;
Makes trouble with my texture view as the texture I sample textureColor2 from is not bound.
In shaders its not possible to use console.log or any other standard debugging methods.I am interested to learn more about shaders but i am stuck.
Code :
...
precision mediump float;
varying vec2 vTextureCoord;
varying vec3 vLightWeighting;
uniform sampler2D uSampler;
uniform sampler2D uSampler1;
uniform sampler2D uSampler2;
uniform sampler2D uSampler3;
uniform sampler2D uSampler4;
uniform sampler2D uSampler5;
uniform sampler2D uSampler6;
uniform sampler2D uSampler7;
uniform sampler2D uSampler8;
uniform sampler2D uSampler9;
uniform sampler2D uSampler10;
uniform sampler2D uSampler11;
uniform sampler2D uSampler12;
uniform sampler2D uSampler13;
void main(void) {
vec4 textureColor = texture2D(uSampler, vec2(vTextureCoord.s, vTextureCoord.t));
vec4 textureColor1 = texture2D(uSampler1, vec2(vTextureCoord.s, vTextureCoord.t));
vec4 textureColor2 = texture2D(uSampler2, vec2(vTextureCoord.s, vTextureCoord.t));
// Need help here
gl_FragColor = textureColor + textureColor1 ;
//gl_FragColor = textureColor + textureColor1 + textureColor2;
//UPDATED QUESTION
if ( ${numTextures} == 1)
{
gl_FragColor = textureColor;
}
else if (${numTextures} == 2)
{
gl_FragColor = textureColor + textureColor1;
}
else if (${numTextures} == 3)
{
gl_FragColor = textureColor + textureColor1 + textureColor2;
}
// i use simple pragmatic if else for now .
// i pass value to the shader on load
// i still cant update shader in run time
///////////////////////////////////////
// This is segment of draw function :
for (var t=0;t<object.textures.length;t++) {
eval( " world.GL.gl.activeTexture(world.GL.gl.TEXTURE"+t+"); " )
world.GL.gl.bindTexture(world.GL.gl.TEXTURE_2D, object.textures[t]);
world.GL.gl.pixelStorei(world.GL.gl.UNPACK_FLIP_Y_WEBGL, false);
world.GL.gl.texParameteri(world.GL.gl.TEXTURE_2D, world.GL.gl.TEXTURE_MAG_FILTER, world.GL.gl.NEAREST);
world.GL.gl.texParameteri(world.GL.gl.TEXTURE_2D, world.GL.gl.TEXTURE_MIN_FILTER, world.GL.gl.NEAREST);
world.GL.gl.texParameteri(world.GL.gl.TEXTURE_2D, world.GL.gl.TEXTURE_WRAP_S, world.GL.gl.CLAMP_TO_EDGE);
world.GL.gl.texParameteri(world.GL.gl.TEXTURE_2D, world.GL.gl.TEXTURE_WRAP_T, world.GL.gl.CLAMP_TO_EDGE);
// -- Allocate storage for the texture
//world.GL.gl.texStorage2D(world.GL.gl.TEXTURE_2D, 1, world.GL.gl.RGB8, 512, 512);
//world.GL.gl.texSubImage2D(world.GL.gl.TEXTURE_2D, 0, 0, 0, world.GL.gl.RGB, world.GL.gl.UNSIGNED_BYTE, image);
//world.GL.gl.generateMipmap(world.GL.gl.TEXTURE_2D);
world.GL.gl.uniform1i(object.shaderProgram.samplerUniform, t);
}
...
Maybe in run time best way is to manipulate with object.textures array ?!
Finally :
Override shader with new flag
Compile shader
New material is updated
What are you trying to accomplish?
The normal way to use lots of textures is to use a texture atlas which is covered toward the bottom of this article
Otherwise, no there is no way to detect if a texture is loaded in the shader. You need to pass in your own flags. For example
uniform bool textureLoaded[NUM_TEXTURES];
or
uniform float textureMixAmount[NUM_TEXTURES];
I'd use a texture atlas though if I were you unless you really know you're doing something unique that actually needs 14 textures.
It's also common to generate shaders on the fly. Pretty much all game engines do this. Three.js does it as well. So rather than turn textures on and off, write some code that generates a shader for N textures. Then when you only have one texture generate a 1 texture shader, when you have 2 generate a 2 texture shader, etc. That's far more efficient for the GPU than having a 14 texture shader and trying to turn off 13 textures.
Example:
// note, I'm not recommending this shader, only showing some code
// that generates a shader
function generateShaderSrc(numTextures) {
return `
// shader for ${numTextures} textures
precision mediump float;
varying vec2 vTextureCoord;
varying vec3 vLightWeighting;
uniform sampler2D uSampler[${numTextures}];
uniform float uMixAmount[${numTextures}];
void main() {
vec4 color = vec4(0);
for (int i = 0; i < ${numTextures}; ++i) {
vec4 texColor = texture2D(uSampler[i], vTextureCoord);
color = mix(color, texColor, uMixAmount[i]);
}
gl_FragColor = color;
}
`;
}
log(generateShaderSrc(1));
log(generateShaderSrc(4));
function log(...args) {
const elem = document.createElement("pre");
elem.textContent = [...args].join(' ');
document.body.appendChild(elem);
}
That's a pretty simple example. Real shader generators often do a whole lot more string manipulation.
You should also be aware WebGL 1.0 only requires support for 8 texture units. According to webglstats about 15% of devices only support 8 texture units so you probably want to check how many texture units the user has and warn them your app is not going to work if they have less than your app needs.
I have just include OpenGL ES 3.0 in my iOS app and it is working fine.
I have a working shader below:
#version 300 es
precision mediump float;
uniform sampler2D texSampler;
uniform float fExposure;
in vec2 fTexCoord;
in vec3 fColor;
out vec4 fragmentColor;
void main()
{
fragmentColor = texture(texSampler, fTexCoord) * vec4(fColor, 1.0) * fExposure;
}
Now, I want to use a sampler3D so I have:
#version 300 es
precision mediump float;
uniform sampler3D texSampler;
uniform float fExposure;
in vec3 fTexCoord;
in vec3 fColor;
out vec4 fragmentColor;
void main()
{
fragmentColor = texture(texSampler, fTexCoord) * vec4(fColor, 1.0) * fExposure;
}
and it doesn't compile. Also, I changed the vec2 texCoord to vec3 texCoord in the vertex shader.
Actually, sampler3D is not recognized, but as far as i know it exists in OpenGL ES 3.0.
Any ideas?
Similar to float, sampler3D does not have a default precision. Add this at the start of your fragment shader, where you also specify the default float precision:
precision mediump sampler3D;
Of course you can use lowp instead if that gives you sufficient precision.
The only sampler types that have a default precision in ES 3.0/3.1 are sampler2D and samplerCube (both default to lowp). For all others, the precision has to be specified either as a default precision, or as part of the variable declaration.
When I'm rendering my content onto a FBO with a texture bound to it and then render this bound texture to a fullscreen quad using a basic shader the performance drops ridiculously.
For example:
Render to screen directly (with basic shader):
And when render to texture first, then render texture with fullscreen quad: (with same basic shader, would be something like blur or bloom normally):
Anyone got an idea how to speed this up? Since the current performance is not usable. Also I'm using GLKit for the basic OpenGL stuff.
Need to use precisions in places where it's needed.
lowp - for colors, textures coord, normals etc.
highp - for matrices and vertices/positions
Quick reference , check the range of precisions, on 3 page in "Qualifiers".
// BasicShader.vsh
precision mediump float;
attribute highp vec2 position;
attribute lowp vec2 texCoord;
attribute lowp vec4 color;
varying lowp vec2 textureCoord;
varying lowp vec4 textureColor;
uniform highp mat4 projectionMat;
uniform highp mat4 worldMat;
void main() {
highp mat4 worldProj = worldMat * projectionMat;
gl_Position = worldProj * vec4(position, 0.0, 1.0);
textureCoord = texCoord;
textureColor = color;
}
// BasicShader.fsh
precision mediump float;
varying lowp vec2 textureCoord;
varying lowp vec4 textureColor;
uniform sampler2D sampler;
void main() {
lowp vec4 Color = texture2D(sampler, textureCoord);
gl_FragColor = Color * textureColor;
}
This is very likely caused by ill-performant openGL ES API calls.
You should attach a real device and do an openGL ES frame capture. (It really needs a real device, the option for frame capture won't be available with a simulator).
The frame capture will indicate memory and other warnings along with suggestions to fix them alongside each API call. Step through these and fix each. The performance should improve considerably.
Here's a couple of references to get this done:
Debugging openGL ES frame
Xcode tools overview
I'm experimenting with some lighting techniques on iOS and I've been able to produce some effects that I'm pleased with by taking advantage of iOS' OpenGL ES extensions for depth lookup textures and a relatively simple Blinn-Phong shader:
The above shows 20 Suzanne monkeys being rendered at full-screen retina with multi-sampling and the following shader. I'm doing multi-sampling because it is only adding 1ms per frame. My current average render time is 30ms total (iPad 3), which is far too slow for 60fps.
Vertex shader:
//Position
uniform mat4 mvpMatrix;
attribute vec4 position;
uniform mat4 depthMVPMatrix;
uniform mat4 vpMatrix;
//Shadow out
varying vec3 ShadowCoord;
//Lighting
attribute vec3 normal;
varying vec3 normalOut;
uniform mat3 normalMatrix;
varying vec3 vertPos;
uniform vec4 lightColor;
uniform vec3 lightPosition;
void main() {
gl_Position = mvpMatrix * position;
//Used for handling shadows
ShadowCoord = (depthMVPMatrix * position).xyz;
ShadowCoord.z -= 0.01;
//Lighting calculations
normalOut = normalize(normalMatrix * normal);
vec4 vertPos4 = vpMatrix * position;
vertPos = vertPos4.xyz / vertPos4.w;
}
Fragment shader:
#extension GL_EXT_shadow_samplers : enable
precision lowp float;
uniform sampler2DShadow shadowTexture;
varying vec3 normalOut;
uniform vec3 lightPosition;
varying vec3 vertPos;
varying vec3 ShadowCoord;
uniform vec4 fillColor;
uniform vec3 specColor;
void main() {
vec3 normal = normalize(normalOut);
vec3 lightDir = normalize(lightPosition - vertPos);
float lambertian = max(dot(lightDir,normal), 0.0);
vec3 reflectDir = reflect(-lightDir, normal);
vec3 viewDir = normalize(-vertPos);
float specAngle = max(dot(reflectDir, viewDir), 0.0);"
float specular = pow(specAngle, 16.0);
gl_FragColor = vec4((lambertian * fillColor.xyz + specular * specColor) * shadow2DEXT(shadowTexture, ShadowCoord), fillColor.w);
}
I've read that it is possible to use textures as lookup tables to reduce computation in the fragment shader, however the linked example seems to be doing full Phong lighting, rather than Blinn-Phong (I'm not doing anything with surface tangents). Furthermore, when running the sample the lighting seemed fairly banded (the background on mine, which is a solid color + Phong shading, looks slightly banded as a result of compression - it looks far smoother on the device). Is it possible to use a lookup texture in my case, or am I going to have to move down to 30fps (which I can just about achieve), turn off multi-sampling and limit Phong shading to the monkeys, rather than the full screen? In a real world (i.e. game) scenario, am I going to need do be doing Phong shading across the entire screen anyway?
In my simple 2D game I have 2x framerate drop when using ES 2.0 implementation for drawing. Is it OK or 2.0 should be faster if used properly?
P.S. If you are interested in details. I use very simple shaders:
vertex program:
uniform vec2 u_xyscale;
uniform vec2 u_st_to_uv;
attribute vec2 a_vertex;
attribute vec2 a_texcoord;
attribute vec4 a_diffuse;
varying vec4 v_diffuse;
varying vec2 v_texcoord;
void main(void)
{
v_diffuse = a_diffuse;
// convert texture coordinates from ST space to UV.
v_texcoord = a_texcoord * u_st_to_uv;
// transform XY coordinates from screen space to clip space.
gl_Position.xy = a_vertex * u_xyscale + vec2( -1.0, 1.0 );
gl_Position.zw = vec2( 0.0, 1.0 );
}
fragment program:
precision mediump float;
uniform sampler2D t_bitmap;
varying lowp vec4 v_diffuse;
varying vec2 v_texcoord;
void main(void)
{
vec4 color = texture2D( t_bitmap, v_texcoord );
gl_FragColor = v_diffuse * color;
}
"color" is a mediump variable, due to the default precision that you specified. This forces the implementation to convert the lowp sampled result to mediump. It also requires that diffuse be converted to mediump to perform the multiplication.
You can fix this by declaring "color" as lowp.