Working with shaders using lookup data IOS - ios
I have lookup data provided by one software and I want to use this data with shader as written below:
7999745,8000001,8000258,8066051,8066308,8132357,8132614,8198407,8198664,8264457,8264969,8330762,8331019,8396812,8397069,8463118,8463375,8529168,8529425,8595218,8595730,8661523,8661780,8727573,8727830,8793879,8794136,8859929,8860186,8925979,8926491,8992284,8992541,9058334,9058591,9059104,9124897,9125154,9190947,9191204,9257252,9257509,9323302,9323559,9389352,9389865,9455658,9455915,9521708,9521965,9588013,9588270,9654063,9654320,9720113,9720626,9786419,9786676,9852469,9852726,9918774,9919031,9984824,9985081,10050874,10051387,10117180,10117437,10183230,10183743,10183999,10249792,10250049,10315842,10316355,10382148,10382405,10448198,10448455,10514503,10514760,10580553,10580810,10646603,10647116,10712909,10713166,10778959,10779216,10845264,10845521,10911314,10911571,10977364,10977877,11043670,11043927,11109720,11109977,11176025,11176282,11242075,11242332,11308125,11308638,11308895,11374688,11374945,11440738,11441250,11507043,11507300,11573093,11573350,11639399,11639656,11705449,11705706,11771499,11772011,11837804,11838061,11903854,11904111,11970160,11970417,12036210,12036467,12102260,12102772,12168565,12168822,12234615,12234872,12300921,12301178,12366971,12367228,12433277,12433278,12433535,12433536,12433793,12499330,12499587,12499588,12499845,12565382,12565639,12565896,12565897,12566154,12631691,12631948,12631949,12632206,12697743,12698000,12698001,12698258,12698515,12764052,12764310,12764311,12764568,12830105,12830362,12830363,12830620,12830621,12896414,12896671,12896672,12896929,12962466,12962723,12962724,12962981,12962982,13028775,13028776,13029033,13029290,13094827,13095084,13095086,13095343,13095344,13161137,13161138,13161395,13161396,13227189,13227446,13227447,13227704,13227705,13293498,13293499,13293756,13293757,13359550,13359807,13359808,13360065,13360066,13425859,13425860,13426117,13426119,13491912,13491913,13492170,13492427,13492428,13558221,13558222,13558479,13558480,13624273,13624274,13624531,13624532,13624789,13690582,13690583,13690840,13690841,13756634,13756635,13756892,13756893,13757151,13822688,13822945,13823202,13823203,13888996,13888997,13889254,13889255,13889512,13955049,13955306,13955307,13955564,14021357,14021358,14021615,14021616,14021873,14087410,14087667,14087668,14087925,14153719
Fragment Shader code:
precision highp float;
uniform sampler2D inputImageTexture;
uniform sampler2D inputImageTexture2;
varying vec2 textureCoordinate;
uniform float uAmount;
void main() {
vec4 color = texture2D(inputImageTexture, textureCoordinate);
vec2 pos = vec2((color.r + color.g + color.b)/ 3.0, 0.0);
vec4 dstColor = texture2D(inputImageTexture2, pos);
gl_FragColor = mix(
color,
dstColor,
uAmount);
}
Help me to pass this data to sampler2D inputimageTexture2.
I am thinking that these should converted to rgb(image texture) somehow, so I can pass this to sampler2D.
I take it that the lookup table is 1 channel and 2D (16x16?).
You could try uploading it with glTexImage2D as
GL_FLOAT with GL_LUMINANCE or GL_ALPHA and your shader would become
vec4 color = texture2D(inputImageTexture, textureCoordinate).xxxx // GL_LUMINANCE
or
vec4 color = texture2D(inputImageTexture, textureCoordinate).aaaa // GL_ALPHA
This question is tagged as GPUImage, which I don't know at all (so what follows could be completely wrong!), but I imagine it manages its own textures so you may have to ask it to make the LUT available to your shader. Looking through the source, GPUImageRawDataInput looks like a good place to start to get your lookup table into GPUImage, maybe with something like
GPUImageRawDataInput *rawInput =
[[GPUImageRawDataInput alloc] initWithBytes:yourTable
size:CGSizeMake(16, 16)
pixelFormat:GPUPixelFormatLuminance
type:GPUPixelTypeFloat];
I found solution these lookup data are 32-bit integer type.
Convert 32-bit integer to RGB, then pass RGB array as texture to shader.
Related
Quality loss (bluriness) in shader
I am trying to make a shader that either passes through an image unaltered or displays a tiled texture depending on some conditions. It more or less works, but I noticed that the tiled texture doesn't quite looks right, so I simplified the shader for testing so it would only show the tiled image:
precision highp float;
uniform sampler2D uSampler;
varying vec2 vTextureCoord;
varying vec4 vColor;
varying vec2 vFilterCoord;
uniform vec2 dimensions;
uniform vec4 filterArea;
uniform sampler2D selector;
uniform vec2 selectorSize;
uniform sampler2D alternate;
uniform vec2 alternateSize;
vec2 mapCoord( vec2 coord )
{
coord *= filterArea.xy;
coord += filterArea.zw;
return coord;
}
vec2 unmapCoord( vec2 coord )
{
coord -= filterArea.zw;
coord /= filterArea.xy;
return coord;
}
void main()
{
vec2 coord = vTextureCoord;
coord = mapCoord(coord);
// sample the alternate:
vec2 av = mod( coord, alternateSize ) / (alternateSize - 1.0);
vec4 alt = texture2D(alternate, av);
gl_FragColor = alt ;
}
I am not quite sure what's going on. The original image is 100x100, and the repeating area is 100x100. The pattern looks the same, but it's slightly blurred in in the shader (see screenshots below). Does this have to do with retina? (I haven't done anything special to setup retina) Mipmaps? Something else?
UPDATE: As suggested by #danieltran, I tried setting the texture to GL_NEAREST (In pixi, this is done by passing the Pixi.SCALE_MODES.NEAREST to the texture constructor). And it made no difference, so then I just tried making a sprite from the texture and displaying that, and it has the same problem, so I think this is either something related to retina, or something pixi-specific.
Original texture is taken from this image:
Here's what the output of the shader looks like:
Change the texture filter to GL_NEAREST then it will solve the issue. To be specific, the problem here is when GPU look up for the fragment, instead of taking the colour from 1 single texel, it calculate the colour using nearby texels also, that make the picture looks blurry.
WebGL shader checking status of texture sampler2D
I want to have prepared shader component (for multi sampler tex)
In my current state i use (activate and bind) only 2 texture image.
But this line :
gl_FragColor = textureColor + textureColor1 + textureColor2;
Makes trouble with my texture view as the texture I sample textureColor2 from is not bound.
In shaders its not possible to use console.log or any other standard debugging methods.I am interested to learn more about shaders but i am stuck.
Code :
...
precision mediump float;
varying vec2 vTextureCoord;
varying vec3 vLightWeighting;
uniform sampler2D uSampler;
uniform sampler2D uSampler1;
uniform sampler2D uSampler2;
uniform sampler2D uSampler3;
uniform sampler2D uSampler4;
uniform sampler2D uSampler5;
uniform sampler2D uSampler6;
uniform sampler2D uSampler7;
uniform sampler2D uSampler8;
uniform sampler2D uSampler9;
uniform sampler2D uSampler10;
uniform sampler2D uSampler11;
uniform sampler2D uSampler12;
uniform sampler2D uSampler13;
void main(void) {
vec4 textureColor = texture2D(uSampler, vec2(vTextureCoord.s, vTextureCoord.t));
vec4 textureColor1 = texture2D(uSampler1, vec2(vTextureCoord.s, vTextureCoord.t));
vec4 textureColor2 = texture2D(uSampler2, vec2(vTextureCoord.s, vTextureCoord.t));
// Need help here
gl_FragColor = textureColor + textureColor1 ;
//gl_FragColor = textureColor + textureColor1 + textureColor2;
//UPDATED QUESTION
if ( ${numTextures} == 1)
{
gl_FragColor = textureColor;
}
else if (${numTextures} == 2)
{
gl_FragColor = textureColor + textureColor1;
}
else if (${numTextures} == 3)
{
gl_FragColor = textureColor + textureColor1 + textureColor2;
}
// i use simple pragmatic if else for now .
// i pass value to the shader on load
// i still cant update shader in run time
///////////////////////////////////////
// This is segment of draw function :
for (var t=0;t<object.textures.length;t++) {
eval( " world.GL.gl.activeTexture(world.GL.gl.TEXTURE"+t+"); " )
world.GL.gl.bindTexture(world.GL.gl.TEXTURE_2D, object.textures[t]);
world.GL.gl.pixelStorei(world.GL.gl.UNPACK_FLIP_Y_WEBGL, false);
world.GL.gl.texParameteri(world.GL.gl.TEXTURE_2D, world.GL.gl.TEXTURE_MAG_FILTER, world.GL.gl.NEAREST);
world.GL.gl.texParameteri(world.GL.gl.TEXTURE_2D, world.GL.gl.TEXTURE_MIN_FILTER, world.GL.gl.NEAREST);
world.GL.gl.texParameteri(world.GL.gl.TEXTURE_2D, world.GL.gl.TEXTURE_WRAP_S, world.GL.gl.CLAMP_TO_EDGE);
world.GL.gl.texParameteri(world.GL.gl.TEXTURE_2D, world.GL.gl.TEXTURE_WRAP_T, world.GL.gl.CLAMP_TO_EDGE);
// -- Allocate storage for the texture
//world.GL.gl.texStorage2D(world.GL.gl.TEXTURE_2D, 1, world.GL.gl.RGB8, 512, 512);
//world.GL.gl.texSubImage2D(world.GL.gl.TEXTURE_2D, 0, 0, 0, world.GL.gl.RGB, world.GL.gl.UNSIGNED_BYTE, image);
//world.GL.gl.generateMipmap(world.GL.gl.TEXTURE_2D);
world.GL.gl.uniform1i(object.shaderProgram.samplerUniform, t);
}
...
Maybe in run time best way is to manipulate with object.textures array ?!
Finally :
Override shader with new flag
Compile shader
New material is updated
What are you trying to accomplish?
The normal way to use lots of textures is to use a texture atlas which is covered toward the bottom of this article
Otherwise, no there is no way to detect if a texture is loaded in the shader. You need to pass in your own flags. For example
uniform bool textureLoaded[NUM_TEXTURES];
or
uniform float textureMixAmount[NUM_TEXTURES];
I'd use a texture atlas though if I were you unless you really know you're doing something unique that actually needs 14 textures.
It's also common to generate shaders on the fly. Pretty much all game engines do this. Three.js does it as well. So rather than turn textures on and off, write some code that generates a shader for N textures. Then when you only have one texture generate a 1 texture shader, when you have 2 generate a 2 texture shader, etc. That's far more efficient for the GPU than having a 14 texture shader and trying to turn off 13 textures.
Example:
// note, I'm not recommending this shader, only showing some code
// that generates a shader
function generateShaderSrc(numTextures) {
return `
// shader for ${numTextures} textures
precision mediump float;
varying vec2 vTextureCoord;
varying vec3 vLightWeighting;
uniform sampler2D uSampler[${numTextures}];
uniform float uMixAmount[${numTextures}];
void main() {
vec4 color = vec4(0);
for (int i = 0; i < ${numTextures}; ++i) {
vec4 texColor = texture2D(uSampler[i], vTextureCoord);
color = mix(color, texColor, uMixAmount[i]);
}
gl_FragColor = color;
}
`;
}
log(generateShaderSrc(1));
log(generateShaderSrc(4));
function log(...args) {
const elem = document.createElement("pre");
elem.textContent = [...args].join(' ');
document.body.appendChild(elem);
}
That's a pretty simple example. Real shader generators often do a whole lot more string manipulation.
You should also be aware WebGL 1.0 only requires support for 8 texture units. According to webglstats about 15% of devices only support 8 texture units so you probably want to check how many texture units the user has and warn them your app is not going to work if they have less than your app needs.
OpenGL ES 2.0 draw Fullscreen Quad very slow
When I'm rendering my content onto a FBO with a texture bound to it and then render this bound texture to a fullscreen quad using a basic shader the performance drops ridiculously. For example: Render to screen directly (with basic shader): And when render to texture first, then render texture with fullscreen quad: (with same basic shader, would be something like blur or bloom normally): Anyone got an idea how to speed this up? Since the current performance is not usable. Also I'm using GLKit for the basic OpenGL stuff.
Need to use precisions in places where it's needed.
lowp - for colors, textures coord, normals etc.
highp - for matrices and vertices/positions
Quick reference , check the range of precisions, on 3 page in "Qualifiers".
// BasicShader.vsh
precision mediump float;
attribute highp vec2 position;
attribute lowp vec2 texCoord;
attribute lowp vec4 color;
varying lowp vec2 textureCoord;
varying lowp vec4 textureColor;
uniform highp mat4 projectionMat;
uniform highp mat4 worldMat;
void main() {
highp mat4 worldProj = worldMat * projectionMat;
gl_Position = worldProj * vec4(position, 0.0, 1.0);
textureCoord = texCoord;
textureColor = color;
}
// BasicShader.fsh
precision mediump float;
varying lowp vec2 textureCoord;
varying lowp vec4 textureColor;
uniform sampler2D sampler;
void main() {
lowp vec4 Color = texture2D(sampler, textureCoord);
gl_FragColor = Color * textureColor;
}
This is very likely caused by ill-performant openGL ES API calls. You should attach a real device and do an openGL ES frame capture. (It really needs a real device, the option for frame capture won't be available with a simulator). The frame capture will indicate memory and other warnings along with suggestions to fix them alongside each API call. Step through these and fix each. The performance should improve considerably. Here's a couple of references to get this done: Debugging openGL ES frame Xcode tools overview
OpenGl ES on iOS lightshading removes color
I am still getting used to OpenGL with shaders, been using OGL ES 1.0 before but it's time to update my knowledge! Now I have a problem with the simple shaders I'm looking at and I have searched for 2 days straight with no luck of a solution.
Problem is this: I render some cubes with a VBO in the form of (Vx, Vy, Vz, NormalX, NormalY, NormalZ, ColorR, ColorG, ColorB, ColorA) and this works nicely when I render it without the shader but I have to use the shader for translation and stuff (I know it can be done without but bear with me). Here is my vertex shader, default from OGL template in XCode:
attribute vec4 position;
attribute vec3 normal;
uniform vec3 translation;
varying lowp vec4 colorVarying;
uniform mat4 modelViewProjectionMatrix;
uniform mat3 normalMatrix;
void main()
{
vec3 eyeNormal = normalize(normalMatrix * normal);
vec3 lightPosition = vec3(0.0, 0.0, 10.0);
vec4 diffuseColor = vec4(0.4, 0.4, 1.0, 1.0);
float nDotVP = max(0.0, dot(eyeNormal, normalize(lightPosition)));
colorVarying = diffuseColor * nDotVP;
gl_Position = modelViewProjectionMatrix * (position + vec4(translation, 1));
}
And the fragment shader, also default:
varying lowp vec4 colorVarying;
void main()
{
gl_FragColor = colorVarying;
}
Now this ALWAYS renders whatever triangles I draw in the same color (defined by diffuseColor) without regard for the colors in the VBO. So I have tried and failed with other fragment shader like gl_FragColor = gl_FrontColor; but gl_FrontColor/gl_Color etc aren't included in OpenGL ES and are deprecated in OpenGL 3.x or something. I have also viewed code using texture samplers but since I'm not using textures but colors it gets a bit complicated for a beginner.
So my question is this, how would I have my fragmentshader find the Material Color of the current fragment being shaded?
If I should pass the colors in an array to the shaders, how would I do that and how, then, would I reference it with regard to the currently shading fragment?
(Some 'also's; tried not using a fragment shader but OGL doesn't allow only using vertex shader. Tried simply removing the gl_FragColor = colorVarying; but that leaves the colors really screwed up)
You need to add a colour attribute to your shader: attribute vec4 position; attribute vec3 normal; attribute vec4 colour; ...and use that attribute instead of diffuseColor. You must also tell OpenGL where to find that vertex attribute within your VBO using glVertexAttribPointer (I assume you are doing this for the position and normal attributes already).
How can I take advantage of lookup tables in my Blinn-Phong lighting shader?
I'm experimenting with some lighting techniques on iOS and I've been able to produce some effects that I'm pleased with by taking advantage of iOS' OpenGL ES extensions for depth lookup textures and a relatively simple Blinn-Phong shader:
The above shows 20 Suzanne monkeys being rendered at full-screen retina with multi-sampling and the following shader. I'm doing multi-sampling because it is only adding 1ms per frame. My current average render time is 30ms total (iPad 3), which is far too slow for 60fps.
Vertex shader:
//Position
uniform mat4 mvpMatrix;
attribute vec4 position;
uniform mat4 depthMVPMatrix;
uniform mat4 vpMatrix;
//Shadow out
varying vec3 ShadowCoord;
//Lighting
attribute vec3 normal;
varying vec3 normalOut;
uniform mat3 normalMatrix;
varying vec3 vertPos;
uniform vec4 lightColor;
uniform vec3 lightPosition;
void main() {
gl_Position = mvpMatrix * position;
//Used for handling shadows
ShadowCoord = (depthMVPMatrix * position).xyz;
ShadowCoord.z -= 0.01;
//Lighting calculations
normalOut = normalize(normalMatrix * normal);
vec4 vertPos4 = vpMatrix * position;
vertPos = vertPos4.xyz / vertPos4.w;
}
Fragment shader:
#extension GL_EXT_shadow_samplers : enable
precision lowp float;
uniform sampler2DShadow shadowTexture;
varying vec3 normalOut;
uniform vec3 lightPosition;
varying vec3 vertPos;
varying vec3 ShadowCoord;
uniform vec4 fillColor;
uniform vec3 specColor;
void main() {
vec3 normal = normalize(normalOut);
vec3 lightDir = normalize(lightPosition - vertPos);
float lambertian = max(dot(lightDir,normal), 0.0);
vec3 reflectDir = reflect(-lightDir, normal);
vec3 viewDir = normalize(-vertPos);
float specAngle = max(dot(reflectDir, viewDir), 0.0);"
float specular = pow(specAngle, 16.0);
gl_FragColor = vec4((lambertian * fillColor.xyz + specular * specColor) * shadow2DEXT(shadowTexture, ShadowCoord), fillColor.w);
}
I've read that it is possible to use textures as lookup tables to reduce computation in the fragment shader, however the linked example seems to be doing full Phong lighting, rather than Blinn-Phong (I'm not doing anything with surface tangents). Furthermore, when running the sample the lighting seemed fairly banded (the background on mine, which is a solid color + Phong shading, looks slightly banded as a result of compression - it looks far smoother on the device). Is it possible to use a lookup texture in my case, or am I going to have to move down to 30fps (which I can just about achieve), turn off multi-sampling and limit Phong shading to the monkeys, rather than the full screen? In a real world (i.e. game) scenario, am I going to need do be doing Phong shading across the entire screen anyway?