I'm using luaglut to do some graphics in lua. And I am struggling with this function glReadPixels, particularly with its last input argument GLvoid *pixels.
void glReadPixels (GLint x, GLint y, GLsizei width, GLsizei height, GLenum format, GLenum type, GLvoid *pixels);
pixels is a pointer type so in lua it is of type lightuserdata. I managed to get a lightuserdata type variable let's say img in lua, according to this post; however, after I get the frame I wanna grab into img by calling:
glReadPixels(0, 0, 250, 250, GL_RGB, GL_UNSIGNED_BYTE, img)
I could do nothing with img. I tried creating a same structure in lua using ffi and coverting this img to a torch.Tensor type, but it is too slow since I have to assign the values pixel by pixel.
So I am asking here if there is better ways to use this glReadPixels function to get img than this troublesome approach that I took? Both table and torch.Tensor types of img are OK. Thank you in advance!
Related
I've got a YUV420 pixelbuffer in a UInt8 Array. I need to create a texture out of it in order to render it with OpenGL. In Android there is an easy way to decode my array to an RGB array for the texture. The code is the following:
BitmapFactory.Options bO = new BitmapFactory.Options();
bO.inJustDecodeBounds = false;
bO.inPreferredConfig = Bitmap.Config.RGB_565;
try {
myBitmap= BitmapFactory.decodeByteArray( yuvbuffer,
0,
yuvbuffer.length,
bO);
} catch (Throwable e) {
// ...
}
I need to decode the yuv buffer on my ios platform (Xcode 8.3.3, Swift 3.1) in order to put it into the following method as data:
void glTexImage2D( GLenum target,
GLint level,
GLint internalFormat,
GLsizei width,
GLsizei height,
GLint border,
GLenum format,
GLenum type,
const GLvoid * data);
How can I achieve this decoding?
ALTERNATIVE:
I've described the way I am decoding the YUV-buffer on Android. Maybe there is an other way to create a texture based on yuvpixels without decoding it like this. I've already tried the following method using the FragmentShader (Link), but it is not working for me. I'm getting a black screen or a green screen, but the image is never rendered. There are also some methods using two seperate buffers for Y and for UV - but on this I don't know how to split my YUV-buffer into Y and UV.
Do you have any new examples/samples for yuv-rendering which are not outdated and working?
If you need only to display that image/video, then you don't really need to convert it to rgb texture. You can bind all 3 planes (Y/Cb/Cr) as separate textures, and perform yuv->rgb conversion in fragment shader, with just a three dot products.
I have setup the cudaArray, and have bound it to a texture:
texture<float, 2, cudaReadModeElementType> tex;
cudaChannelFormatDesc channelDesc =
cudaCreateChannelDesc(32, 0, 0, 0, cudaChannelFormatKindFloat);
cudaArray *cuArray;
checkCudaErrors(cudaMallocArray(&cuArray,
&channelDesc,
width,
height));
checkCudaErrors(cudaMemcpyToArray(cuArray,
0,
0,
hData,
size,
cudaMemcpyHostToDevice));
Now I am wondering, if the content within the cuArray and tex remains the same all the time during the calculation, can I pass tex and/or cuArray to another function so that I don't have to do the binding every time?
Something like this:
DoJobUsingTex(float* output, float* input, int size, texture tex)
{
\\ do something here
}
CUDA introduced texture objects when CUDA 5 and Kepler hardware were released. These are so called "bindless" textures which can be passed by value to kernels, so there isn't a need to rebind memory every time you want to run a kernel on different texture data.
You can read more about their use here.
I have a number of textures loaded using GLKTextureLoader. If I bind any of the loaded textures statically, each texture works as expected.
But I am trying to bind a random texture each glDrawArrays call, but the texture bound is always the same.
GLuint vbo = vboIDs[emitterNum];
GLKMatrix4 projectionMatrix = GLKMatrix4MakeScale(1.0f, aspectRatio, 1.0f);
glUseProgram(emitterShader[emitterNum].program);
glEnable(GL_TEXTURE_2D);
glActiveTexture (GL_TEXTURE0);
//Note: valid texture names are 0-31, but in my code I store texture names returned in an array and use them. Use arc4random here for simplicity
glBindTexture(GL_TEXTURE_2D, arc4random_uniform(31)); //use a random texture name
//glBindTexture(GL_TEXTURE_2D, 2); //If I use this line instead of the line above, it will draw texture 2, or any number I specify
glBindBuffer(GL_ARRAY_BUFFER, vbo);
glUniformMatrix4fv(emitterShader[emitterNum].uProjectionMatrix, 1, 0, projectionMatrix.m);
//I set a number of uniforms such as:
glUniform1f(emitterShader[emitterNum].uTime, timeCurrentFrame);
glUniform1i(emitterShader[emitterNum].uTexture, 0);
//I set a number of vertex arrays such as:
glEnableVertexAttribArray(emitterShader[emitterNum].aShade);
glVertexAttribPointer(emitterShader[emitterNum].aShade, // Set pointer
4, // four components per particle (vec4)
GL_FLOAT, // Data is floating point type
GL_FALSE, // No fixed point scaling
sizeof(Particles), // No gaps in data
(void*)(offsetof(Particles, shade))); // Start from "shade" offset within bound buffer
GLsizei rowsToUse = emitters[emitterNum]->rows;
//Draw the arrays
glDrawArrays(GL_POINTS, 0, rowsToUse );
//Then clean up
glBindTexture(GL_TEXTURE_2D, 0);
glUseProgram(0);
glDisable(GL_TEXTURE_2D);
glBindBuffer(GL_ARRAY_BUFFER, 0);
I have tried putting the texture calls in various places, like where shown and direct before the glDrawArrays command, but no matter what - I can't make it bind to different textures unless done so statically.
I'm trying to fill a 1D texture with values manually and pass that texture to a compute shader (these are 2 pixels that I want to set via code, they don't represent any image).
Due to the current small amount of Metal examples, all examples I could find deal with 2D textures that load the texture by converting a loaded UIImage to raw bytes data, but creating a dummy UIImage felt like a hack for me.
This is the "naive" way I started with -
...
var manualTextureData: [Float] = [ 1.0, 0.0, 0.0, 1.0,
0.0, 0.0, 1.0, 1.0 ];
let region: MTLRegion = MTLRegionMake1D(0, textureDescriptor.width);
myTexture.replaceRegion(region, mipmapLevel: 0, withBytes: &manualTextureData, bytesPerRow: 0);
but Metal doesn't recognize those values in the shader (it gets an empty texture, except for the first value).
I quickly realized that the Float array probably has to be converted into a bytes array (e.g UInt8), but couldn't find a way to convert from [Float] to [UInt8] either.
Another possible option I consider is using a CVPixelBuffer object, but that also felt like a workaround to the problem.
So whats the right way to tackle that?
Thanks in advance.
Please note I'm not familiar with Objective-C, hence I'm not sure whether using CVPixelBuffer / UIImage is exaggerated for something which should be straight-forward.
Please forgive the terse reply, but you may find it useful to take a look at my experiments with Swift and Metal. I've created a particle system in Swift which is passed to a Metal compute shader as a one dimensional array of Particle structs. By using posix_memalign, I'm able to eliminate the bottleneck caused by passing the array between Metal and Swift.
I've blogged extensively about this: http://flexmonkey.blogspot.co.uk/search/label/Metal
I hope this helps.
Simon
I don't see any reason for you to pass data using 1D texture. Instead I would go with just passing a buffer. Like this:
var dataBuffer:MTLBuffer? = device.newBufferWithBytes(&manualTextureData, length: sizeOf(manualTextureData), options: MTLResourceOptions.OptionCPUCacheModeDefault)
Then you hook it to your renderCommandEncoder like this:
renderCommandEncoder.setFragmentBuffer(dataBuffer, offset: 0, atIndex: 1)//Note that if you want this buffer to be passed to you vertex shader you should use setVertexBuffer
Then in your shader, you should add parameter like this const device float* bufferPassed [[ buffer(1) ]]
And then use it like this, inside your shader implementation:
float firstFloat = bufferPassed[0];
This will get the job done.
Not really answering your question, but you could just define an array in your metal shader instead of passing the values as a texture.
Something like:
constant float manualData[8] = { 1.0, 0.0, 0.0, 1.0,
0.0, 0.0, 1.0, 1.0 };
vertex float4 world_vertex(unsigned int vid[[vertex_id]], ...) {
int manualIndex = vid % 8;
float manualValue = manualData[manualIndex];
// something deep and meaningful here...
return float4(manualValue);
}
If you want a float texture bytesPerRow should be 4 for times the width, because a float has a size of 4 bytes. Metal copies the memory and dont care about the values. That is your task ;-)
Something like:
myTexture.replaceRegion(region, mipmapLevel: 0, withBytes: &manualTextureData, bytesPerRow: manualTextureData.count * sizeof(Float));
I just stumbled over a quite tricky issue.
In context of a openGL app for iOS I tried to call glReadPixels.
Therefore, a global buffer variable was created/allocated once at the beginning.
I tried to use the glReadPixel-Function on that buffer, but it did not succeed. I did not get any new picture, just crap.
so my question: Why do I need to use a free() on my allocated buffer space, when I want to use the location of that memory a lot of times before I finally free it?
See for example:
int bytes = width*height*3; //Color space is RGB
if(buffer == null)
buffer = (GLubyte *)malloc(bytes);
glReadPixels(0, 0, width, height, GL_RGB, GL_UNSIGNED_BYTE, buffer);
free(buffer);
EDIT: I replaced free(bytes); with free(buffer);