I am writing a custom shader and I want to pass & change sampler2d property in OpenGL Shader in GPUImage at run time.
varying highp vec2 centerLocation;
varying highp float pointSpacing;
uniform sampler2D inputImageTexture;
void main()
{
mediump vec4 fragmentColor = texture2D(inputImageTexture, gl_PointCoord);
gl_FragColor = fragmentColor;
}
Related
I am trying to implement specular lighting (thats coming from the front) but the light is always changing size in an unnatural way. How do I fix this?
I hardcoded viewerPos to test. I'm using a halfway vector "shortcut" so I have to calculate less things as explained here: https://webglfundamentals.org/webgl/lessons/webgl-3d-lighting-point.html
Video with my lighting implemented: https://streamable.com/j95bz7
// Vertex shader program
const vsSource = `
attribute vec4 aVertexPosition;
attribute vec3 aVertexNormal;
attribute vec2 aTextureCoord;
uniform mat4 uNormalMatrix;
uniform mat4 uModelViewMatrix;
uniform mat4 uProjectionMatrix;
uniform highp vec3 uViewPos;
varying highp vec2 vTextureCoord;
varying highp vec4 vNormal;
varying highp mat4 vModelViewMatrix;
varying highp vec3 vPos;
void main(void) {
gl_Position = uProjectionMatrix * uModelViewMatrix * aVertexPosition;
vTextureCoord = aTextureCoord; //Textura
vModelViewMatrix = uModelViewMatrix;
vPos = (uModelViewMatrix * aVertexPosition).xyz;
vNormal = uNormalMatrix * vec4(aVertexNormal, 1.0);
}
`;
// Fragment shader program
const fsSource = `
varying highp vec2 vTextureCoord;
varying highp vec4 vNormal;
varying highp mat4 vModelViewMatrix;
varying highp vec3 vPos;
uniform sampler2D uSampler;
void main(void) {
// Apply lighting effect
highp vec4 texelColor = texture2D(uSampler, vTextureCoord);
//Luz Ambiente
highp vec3 ambientLight = 0.3 * vec3(1.0, 1.0, 1.0);
//Luz Difusa
highp vec3 directionalLightColor = vec3(1, 1, 1);
highp vec3 directionalVector = vec3(0.0, 0.0, 1.0);
highp float directional = max(dot(vNormal.xyz, normalize(directionalVector)), 0.0);
//Luz Especular
highp vec3 viewerPos = vec3(0, 0, -6); //NOTA: PASSAR PARA SHADERS, NAO DAR HARDCODE
highp vec3 surfaceToLightDirection = (-1.0 * directionalVector);
highp vec3 surfaceToViewDirection = (vPos - viewerPos);
highp vec3 halfVector = normalize(surfaceToLightDirection + surfaceToViewDirection);
highp float specular = max(dot(vNormal.xyz, halfVector), 0.0);
highp vec3 vLighting = ambientLight;// + (directionalLightColor * directional);
gl_FragColor = vec4(texelColor.rgb * vLighting + (specular * 0.5), texelColor.a);
}
`;
I wrote a Phong shader for WebGL. My Scene supports camera rotation using Euler Angles. The light should be fixed in the scene. As soon as I rotate the camera the specular effects on objects move as well. Translation also seems to have strange effects on the specular component. I
(hopefully) made sure that calculations are performed in eye space. I first want to find out if my shaders are correct.
Here is my code:
Vertex Shader
struct Light {
vec3 position;
/* ... */
};
uniform Light u_light;
uniform mat4 u_modelViewProjMat;
uniform mat4 u_modelViewMat;
uniform mat4 u_viewMat;
uniform mat3 u_normalMat;
in vec3 a_position;
in vec3 a_normal;
in vec2 a_texCoord;
out vec2 v_texCoord;
out vec3 v_normal;
out vec3 f_position;
out vec3 v_lightPos;
void main() {
v_normal = u_normalMat * a_normal;
v_texCoord = a_texCoord;
f_position = vec3(u_modelViewMat * vec4(a_position, 1.0));
v_lightPos = vec3(u_viewMat * vec4(u_light.position, 1.0));
gl_Position = u_modelViewProjMat * vec4(a_position, 1.0);
}
Fragment Shader
struct Material {
sampler2D diffuse;
vec3 specular;
float shininess;
};
struct Light {
/* ... */
vec3 ambient;
vec3 diffuse;
vec3 specular;
};
uniform Material u_material;
uniform Light u_light;
uniform vec3 u_eyePosition;
in vec2 v_texCoord;
in vec3 v_normal;
in vec3 f_position;
in vec3 v_lightPos;
out vec4 outColor;
void main() {
vec3 texture = vec3(texture(u_material.diffuse, v_texCoord));
// ambient
vec3 ambient = u_light.ambient * texture;
// diffuse
vec3 normal = normalize(v_normal);
vec3 lightDir = normalize(v_lightPos - f_position);
float diff = max(dot(normal, lightDir), 0.0);
vec3 diffuse = u_light.diffuse * diff * texture;
// specular
vec3 viewDir = normalize(u_eyePosition - f_position);
vec3 reflectDir = reflect(-lightDir, normal);
float spec = pow(max(dot(viewDir, reflectDir), 0.0), u_material.shininess);
vec3 specular = u_light.specular * (spec * u_material.specular);
outColor = vec4((ambient + diffuse + specular), 1.0);
}
f_position is a camera space position, so to get the viewDir vector for your specular component you should only perform vec3 viewDir = normalize(-f_position); or work with world space coordinates
I have a fragment shader code that requires four input textures like this:
precision lowp float;
varying highp vec2 textureCoordinate;
uniform sampler2D inputImageTexture;
uniform sampler2D inputImageTexture2;
uniform sampler2D inputImageTexture3;
uniform sampler2D inputImageTexture4;
void main()
{
vec4 texel = texture2D(inputImageTexture, textureCoordinate);
vec3 bbTexel = texture2D(inputImageTexture2, textureCoordinate).rgb;
texel.r = texture2D(inputImageTexture3, vec2(bbTexel.r, texel.r)).r;
texel.g = texture2D(inputImageTexture3, vec2(bbTexel.g, texel.g)).g;
texel.b = texture2D(inputImageTexture3, vec2(bbTexel.b, texel.b)).b;
vec4 mapped;
mapped.r = texture2D(inputImageTexture4, vec2(texel.r, .16666)).r;
mapped.g = texture2D(inputImageTexture4, vec2(texel.g, .5)).g;
mapped.b = texture2D(inputImageTexture4, vec2(texel.b, .83333)).b;
mapped.a = 1.0;
gl_FragColor = mapped;
}
and as the GPUImage only provide GPUImageThreeInputFilter at most, how do I write a filter with four input texture ? I have read the IFImageFilter which is available for more than 3 input textures (up to 6 input textures), but it can not be compiled with latest GPUImage.
GLSL Shader Error
ERROR: 0:1: '(' : syntax error: Constructor calls may not have precision
I'm seeing this error with Xcode 6 on an iOS 8 app based on GLPaint demo... (works fine in iOS7)
I also noticed they no longer use the "STRINGIFY" thing in version 1.13 of GLPaint demo.
.vsh
static const char* BaseVS = STRINGIFY
(
attribute highp vec4 inVertex;
uniform highp mat4 MVP;
uniform highp float pointSize;
uniform highp vec4 vertexColor;
uniform highp float brushRotation;
varying highp vec4 color;
void main()
{
gl_Position = MVP * inVertex;
gl_PointSize = pointSize;
color = vertexColor;
}
);
.fsh
static const char* BaseFS = STRINGIFY
(
uniform sampler2D texture;
uniform sampler2D normalMap;
uniform highp float brushRotation;
varying highp vec4 color;
varying highp vec3 normal;
varying highp vec3 lightDir;
varying highp vec3 eyeVec;
precision highp float;
void main (void)
{
highp float vRotation = (brushRotation/180.0)*3.14;;
highp float mid = 0.5;
highp vec2 rotated = vec2(cos(vRotation) * (gl_PointCoord.x - mid) + sin(vRotation) * (gl_PointCoord.y - mid) + mid,
cos(vRotation) * (gl_PointCoord.y - mid) - sin(vRotation) * (gl_PointCoord.x - mid) + mid);
highp vec4 rotatedTexture = texture2D( texture, rotated);
gl_FragColor = color * rotatedTexture;
}
);
The problem was in a method used for random generation. I removed the "high" before the vec2() construction. (Sigh)
highp float rand(highp vec2 co)
{
return fract(sin(dot(co.xy ,highp vec2(12.9898,78.233))) * 43758.5453);
}
I re-implemented Apple's GLImageProcessing with OpenGL ES 2 shaders. The effects are perfect but the performance of the Sharpness filter is not as good — it runs only at 20 FPS.
The shader code is simple:
Pass 0 for horizontal blur.
Pass 1 for vertical blur.
Pass 2 to mix the blur texture with the original texture.
Basically, the texture mix in Pass 2 is the cause of slowness since Pass 0 and Pass 1 are only done once and do not contribute to the bad performance.
How can I improve the performance?
Vertex shader:
attribute vec4 a_position;
attribute vec2 a_texCoord;
varying highp vec2 v_texCoord;
varying highp vec2 v_texCoord1;
varying highp vec2 v_texCoord2;
varying highp vec2 v_texCoord1_;
varying highp vec2 v_texCoord2_;
uniform mat4 u_modelViewProjectionMatrix;
uniform lowp int u_pass;
const highp float blurSizeH = 1.0 / 320.0;
const highp float blurSizeV = 1.0 / 480.0;
void main()
{
v_texCoord = a_texCoord;
if (u_pass == 0) {
v_texCoord1 = a_texCoord + vec2(1.3846153846 * blurSizeH, 0.0);
v_texCoord1_ = a_texCoord - vec2(1.3846153846 * blurSizeH, 0.0);
v_texCoord2 = a_texCoord + vec2(3.2307692308 * blurSizeH, 0.0);
v_texCoord2_ = a_texCoord - vec2(3.2307692308 * blurSizeH, 0.0);
} else if (u_pass == 1) {
v_texCoord1 = a_texCoord + vec2(0.0, 1.3846153846 * blurSizeV);
v_texCoord1_ = a_texCoord - vec2(0.0, 1.3846153846 * blurSizeV);
v_texCoord2 = a_texCoord + vec2(0.0, 3.2307692308 * blurSizeV);
v_texCoord2_ = a_texCoord - vec2(0.0, 3.2307692308 * blurSizeV);
}
gl_Position = u_modelViewProjectionMatrix * a_position;
}
Fragment shader:
varying highp vec2 v_texCoord;
varying highp vec2 v_texCoord1;
varying highp vec2 v_texCoord2;
varying highp vec2 v_texCoord1_;
varying highp vec2 v_texCoord2_;
uniform lowp int u_pass;
uniform sampler2D u_texture;
uniform sampler2D u_degenTexture;
uniform mediump mat4 u_filterMat;
void main()
{
if (u_pass == 0) {
gl_FragColor = texture2D(u_texture, v_texCoord) * 0.2270270270;
gl_FragColor += texture2D(u_texture, v_texCoord1) * 0.3162162162;
gl_FragColor += texture2D(u_texture, v_texCoord1_) * 0.3162162162;
gl_FragColor += texture2D(u_texture, v_texCoord2) * 0.0702702703;
gl_FragColor += texture2D(u_texture, v_texCoord2_) * 0.0702702703;
} else if (u_pass == 1) {
gl_FragColor = texture2D(u_degenTexture, v_texCoord) * 0.2270270270;
gl_FragColor += texture2D(u_degenTexture, v_texCoord1) * 0.3162162162;
gl_FragColor += texture2D(u_degenTexture, v_texCoord1_) * 0.3162162162;
gl_FragColor += texture2D(u_degenTexture, v_texCoord2) * 0.0702702703;
gl_FragColor += texture2D(u_degenTexture, v_texCoord2_) * 0.0702702703;
} else {
gl_FragColor = u_filterMat * texture2D(u_texture, v_texCoord) + (mat4(1.0) - u_filterMat) * texture2D(u_degenTexture, v_texCoord);
}
}
Before you continue with this, may I suggest you take a look at my open source GPUImage project? I have several hand-optimized sharpening effects in there, including the unsharp mask you're attempting here. I also make it reasonably easy to pull in image and video sources.
To your specific question, there are a couple of reasons why your shader is running slower than expected. The first is that you are using branching within your fragment shader. This kills performance on iOS devices, and should be avoided if at all possible. If you really need to have different conditions for different passes, split these apart into separate shader programs and swap the programs out as needed rather than using a uniform for control flow.
I'm also not sure that writing to gl_FragColor repeatedly is the fastest thing you can do here. I'd use a lowp or mediump intermediate color variable, add your Gaussian components to that, and then write the final result to gl_FragColor when done.
I do see that you've moved your sampling offset calculations to the vertex shader, and then passed those offsets into the fragment shader, which is a good thing that people usually miss. Once you implement the above tweaks (or give my framework a try to see how I handle this), you should get much better results from your filtering.
It turns out that it is really simple. The cause of bad performance is matrix-vector multiplication:
varying highp vec2 v_texCoord;
uniform sampler2D u_texture;
uniform sampler2D u_degenTexture;
uniform lowp float u_filterValue;
void main()
{
gl_FragColor = u_filterMat * texture2D(u_texture, v_texCoord) + (mat4(1.0) - u_filterMat) * texture2D(u_degenTexture, v_texCoord);
}
I initially wrote my code using matrix this way so that all my filters could share the same color mixing code. Now that I learned the lesson, I simply go back to write filter specific code and use scalar operations as much as possible:
varying highp vec2 v_texCoord;
uniform sampler2D u_texture;
uniform sampler2D u_degenTexture;
uniform lowp float u_filterValue;
void main()
{
gl_FragColor = u_filterValue * texture2D(u_texture, v_texCoord) + (1.0 - u_filterValue) * texture2D(u_degenTexture, v_texCoord);
}
Now it is awesome 60 fps!
Never thought it was such a naive issue, but it is.