I would like to perform screenshots (or screen captures) the fastest possible.
Googling this question brings many answeers but my concern is more specific :
I am not interested in the image itself, I would like to grab in near real time the screen brightness, not the hardware one, but the image one, given that, for example, the firefox white google page gives a brighter image than a dark xterm (when both are maximzed).
To make me as clear as possible, here is one way I already managed to implement with X11 and CImg library :
Here is the header :
#include <CImg.h>
using namespace cimg_library;
#include <X11/Xlib.h>
#include <X11/Xutil.h>
#include <X11/Xos.h>
and the core part which extract an X11 image and make a loop on very pixel :
Display *display = XOpenDisplay(NULL);
Window root = DefaultRootWindow(display);
Screen* screen = DefaultScreenOfDisplay(display);
const int W = WidthOfScreen(screen);
const int H = HeightOfScreen(screen);
XImage *image = XGetImage(display, root, 0, 0, W, H, AllPlanes, ZPixmap);
unsigned long red_count(0), green_count(0), blue_count(0), count(0);
const unsigned long red_mask = image->red_mask;
const unsigned long green_mask = image->green_mask;
const unsigned long blue_mask = image->blue_mask;
CImg<unsigned char> screenshot(W, H, 1, 3, 0);
for (int x = 0; x < W; x += pixel_stride)
for (int y = 0; y < H; y += pixel_stride)
{
unsigned long pixel = XGetPixel(image, x, y);
screenshot(x, y, 0) = (pixel & red_mask) >> 16;
screenshot(x, y, 1) = (pixel & green_mask) >> 8;
screenshot(x, y, 2) = pixel & blue_mask;
red_count += (int) screenshot(x, y, 0);
green_count += (int) screenshot(x, y, 1);
blue_count += (int) screenshot(x, y, 2);
count++;
}
As I said, I do not keep the image itself, I just try to compute an average luminance value with respective values of red, green and blue pixels.
XFree(image);
const double luminance_relative = (red_luminance * double(red_count) +
green_luminance * double(green_count) +
blue_luminance * double(blue_count))
/ (double(255) * double(count));
The underlying idea is to adjust the hardware screen brightness depending on the image luminance. In short, the whiter is the screenshot, the more the brightness can be reduced and conversely.
I want to do that because I have sensitive eyes, it usually hurts my eyes when I switch from xterm to firefox.
To do so, the hardware brightness must be adjusted in a very short time, the screenshot, that is to say, the loop on pixels must be as fast as possible.
I began to implement it with X11 methods, but I wonder if there could be faster access methods ? Which comes to the question : what is the fastest way/library to get a screenshot ?
Thanks in advance for your help.
Regards
Related
Since the Corona situation characterizes my studies as self-study, as a Processing-Language newbie I don't have an easy time getting into the subject of image processing , more specifically convolution. Therefore I hope that you can help me.
My lecturer, who unfortunately is nearly never reachable, left me the following conv code. The theory behind convolution is clear to me, but I have many gaps in understanding related to the code. Could someone leave a line comment so that I can get into the code a bit more fluently?
The Code is following
color convolution (int x, int y, float[][] matrix, int matrix_size, PImage img){
float rtotal = 0.0;
float gtotal = 0.0;
float btotal = 0.0;
int offset = matrix_size / 2;
for (int i = 0; i < matrix_size; i++){
for (int j= 0; j < matrix_size; j++){
int xloc = x+i-offset;
int yloc = y+j-offset;
int loc = xloc + img.width*yloc;
rtotal += (red(img.pixels[loc]) * matrix[i][j]);
gtotal += (green(img.pixels[loc]) * matrix[i][j]);
btotal += (blue(img.pixels[loc]) * matrix[i][j]);
}
}
rtotal = constrain(rtotal, 0, 255);
gtotal = constrain(gtotal, 0, 255);
btotal = constrain(btotal, 0, 255);
return color(rtotal, gtotal, btotal);
}
I have to do a bit of guesswork since I'm not positive about all of the functions you're using and I'm not familiar with the Processing 3+ library, but here's my best shot at it.
color convolution (int x, int y, float[][] matrix, int matrix_size, PImage img){
// Note: the 'matrix' parameter here will also frequently be referred to as
// a 'window' or 'kernel' in research
// I'm not certain what your PImage class is from, but I'll assume
// you're using the Processing 3+ library and work off of that assumption
// how much of each color we see within the kernel (matrix) space
float rtotal = 0.0;
float gtotal = 0.0;
float btotal = 0.0;
// this offset is to zero-center our kernel
// the fact that we use matrix_size / 2 sort of implicitly
// alludes to the fact that our matrix_size should be an odd-number
// so that we can have a middle-pixel
int offset = matrix_size / 2;
// looping through the kernel. the fact that we use 'matrix_size'
// as our end-condition for both dimensions means that our 'matrix' kernel
// must always be a square
for (int i = 0; i < matrix_size; i++){
for (int j= 0; j < matrix_size; j++){
// calculating the index conversion from 2D to the 1D format that PImage uses
// refer to: https://processing.org/tutorials/pixels/
// for a better understanding of PImage indexing (about 1/3 of the way down the page)
// WARNING: by subtracting the offset it is possible to hit negative
// x,y values here if you pick an x or y position less than matrix_size / 2.
// the same index-out-of-bounds can occur on the high end.
// When you convolve using a kernel of N x N size (N here would be matrix_size)
// you can only convolve from [N / 2, Width - (N / 2)] for x and y
int xloc = x+i-offset;
int yloc = y+j-offset;
// this is the final 1D PImage index that corresponds to [xloc, yloc] in our 2D image
// really go back up and take a look at the link if this doesn't make sense, it's pretty good
int loc = xloc + img.width*yloc;
// I have to do some speculation again since I'm not certain what red(img.pixels[loc]) does
// I'll assume it returns the red red channel of the pixel
// this section just adds up all of the pixel colors multiplied by the value in the kernel
rtotal += (red(img.pixels[loc]) * matrix[i][j]);
gtotal += (green(img.pixels[loc]) * matrix[i][j]);
btotal += (blue(img.pixels[loc]) * matrix[i][j]);
}
}
// the fact that no further division or averaging happens after the for-loops implies
// that the kernel you feed in should have balanced values for your kernel size
// for example, a kernel that's designed to average out the color over the 3 x 3 area
// it covers (this would be like blurring the image) would be filled with 1/9
// in general: the kernel you're using should have a sum of 1 for all of the numbers inside
// this is just 'in general' you can play around with not doing that, but you'll probably notice a
// darkening effect for when the sum is less than 1, and a brightening effect if it's greater than 1
// for more info on kernels, read this: https://en.wikipedia.org/wiki/Kernel_(image_processing)
// I don't have the code for this constrain function,
// but it's almost certainly just your typical clamp (constrains the values to [0, 255])
// Note: this means that your values saturate at 0 and 255
// if you see a lot of black or white then that means your kernel
// probably isn't balanced as mentioned above
rtotal = constrain(rtotal, 0, 255);
gtotal = constrain(gtotal, 0, 255);
btotal = constrain(btotal, 0, 255);
// Finished!
return color(rtotal, gtotal, btotal);
}
I have a UIImagePickerViewController where the user takes a photo. My problem is how to know before uploading the photo to the server if the user is sending a dark photo. I mean a totally or nearly black.
I was researching and I found this:
const UInt8 *pixels = CFDataGetBytePtr(imageData);
UInt8 blackThreshold = 10; // or some value close to 0
int bytesPerPixel = 4;
for(int x = 0; x < width1; x++) {
for(int y = 0; y < height1; y++) {
int pixelStartIndex = (x + (y * width1)) * bytesPerPixel;
UInt8 alphaVal = pixels[pixelStartIndex]; // can probably ignore this value
UInt8 redVal = pixels[pixelStartIndex + 1];
UInt8 greenVal = pixels[pixelStartIndex + 2];
UInt8 blueVal = pixels[pixelStartIndex + 3];
if(redVal < blackThreshold && blueVal < blackThreshold && greenVal < blackThreshold) {
//This pixel is close to black...do something with it
}
}
}
However, I don't know how to apply the algorithm.
Yep that's a fairly simple way of doing it. You could, for example, iterate through and see what percentage of the pixels are pure black (i.e. clipped shadows) or nearly black. Or you could average the pixel colors throughout the whole image and see if it falls below a certain threshold. There are lots of approaches and these two might be a tad simplistic, but I'm not sure if this calls for anything particularly sophisticated. What threshold you want to use is up to you.
Also, while it has little practical impact, if I was going to be picky about the algorithm, I might only perform the "brightness" logic if the alphaVal was over a certain threshold, as well, as the color information is meaningless at transparent portions of image. Having said that, real photos rarely have any transparency, so this may be non-issue.
FYI, here is Apple's code for retrieving the pixel buffer. It's an oldie, but a goodie. (If I recall correctly, the only hassle is that the kCGImageAlphaPremultipliedFirst reference in CreateARGBBitmapContext must be cast with (CGBitmapInfo).)
By the way, if you're trying to determine the luminance of a particular pixel, one common algorithm is:
luminance = 0.2126 * red + 0.7152 * green + 0.0722 * blue
This code is for 8 bit data to make gray-scale IplImage.
IplImage* img_gray_resize = NULL;
img_gray_resize = cvCreateImage(cvSize(320, 256), IPL_DEPTH_8U, 1);
DWORD dwCount;
LVDS_SetDataMode(0); // o for 8 bit mode and 1 for 16 bit mode
dwCount = (LONG)320 * (LONG)256;
unsigned char* m_pImage = NULL;
m_pImage = new unsigned char[320 * 256];
for (int i=0; i<320 * 256; i++) m_pImage[i] = NULL;
LVDS_GetFrame(&dwCount, m_pImage);
int width = 320;
int height = 256;
int nn = 0;
int ii = 0;
for (int y=0; y<height; y++)
{
for (int x=0; x<width; x++)
{
ii = y * width + x;
if(nn < (height*width))
img_gray_resize->imageData[ii] = m_pImage[nn++];
}
}
delete [] m_pImage;
I need to display 16 bit gray-scale image. If I display 8 bit data, some information is missing from the image. However, LVDS_SetDataMode() can provide both types of data. I am using a library for frame grabber device. Please help me.
16 bit images should be stored in IPL_DEPTH_16U (or CV_16U) mode. This is the correct memory layout.
However, displaying them depends on your display hardware.
Most regular display APIs, e.g. OpenCV's highgui, can only display 8-bit images.
To actually display the image, you will have to convert your image to 8-bits for display.
You will need to decide how to do this. There are many ways to do this, depending on your application and complexity. Some options are:
Show MSB = right-shift the image by 8 pixels.
Show LSB = saturate anything above 255.
In fact, right-shift by any value between 0-8 bits, combined with a cv::saturate_cast to avoid value wrap-around.
HDR->LDR = Apply dynamic range compression algorithms.
as I know,only 8bit data can be displayed,you need to find the best way to convert the 16bit to 8bit to minimize the information you lose. Histogram equalization can be applyed to do this.
Finally, I have solved the problem by following way:
dwCount = (LONG)320 * (LONG)256 * 2;
LVDS_SetDataMode(1);
img_gray_resize->imageData[ii] = m_pImage[nn++] >> 6;
Just shift bits to right (2, 3, 4, 5, 6, ...), where you get good result, use that value.
I'm using Emgu.CV to perform some basic image manipulation and composition. My images are loaded as Image<Bgra,Byte>.
Question #1: When I use the Image<,>.Add() method, the images are always blended together, regardless of the alpha value. Instead I'd like them to be composited one atop the other, and use the included alpha channel to determine how the images should be blended. So if I call image1.Add(image2) any fully opaque pixels in image2 would completely cover the pixels from image1, while semi-transparent pixels would be blended based on the alpha value.
Here's what I'm trying to do in visual form. There's a city image with some "transparent holes" cut out, and a frog behind. This is what it should look like:
And this is what openCV produces.
How can I get this effect with OpenCV? And will it be as fast as calling Add()?
Question #2: is there a way to perform this composition in-place instead of creating a new image with each call to Add()? (e.g. image1.AddImageInPlace(image2) modifies the bytes of image1?)
NOTE: Looking for answers within Emgu.CV, which I'm using because of how well it handles perspective warping.
Before OpenCV 2.4 there was no support of PNGs with alpha channel.
To verify if your current version supports it, print the number of channels after loading an image that you are certain to be RGBA. If it supports, the application will output the number 4, else it will output number 3 (RGB). Using the C API you would do:
IplImage* t_img = cvLoadImage(argv[1], CV_LOAD_IMAGE_UNCHANGED);
if (!t_img)
{
printf("!!! Unable to load transparent image.\n");
return -1;
}
printf("Channels: %d\n", t_img->nChannels);
If you can't update OpenCV:
There are some posts around that try to bypass this limitation but I haven't tested them myself;
The easiest solution would be to use another API to load the image and blend it, check blImageBlending;
Another alternative, not as lightweight, is to use Qt.
If your version already supports PNGs with RGBA:
Take a look at Emulating photoshop’s blending modes in OpenCV. It implements several Photoshop blending modes and I imagine you are capable of converting that code to .Net.
EDIT:
I had to deal with this problem recently and I've demonstrated how to deal with it on this answer.
You'll have to iterate through each pixel. I'm assuming image 1 is the frog image, and image 2 is the city image, with image1 always being bigger than image2.
//to simulate image1.AddInPlace(image2)
int image2w = image2.Width;
int image2h = image2.Height;
int i,j;
var alpha;
for (i = 0; i < w; i++)
{
for (j = 0; j < h; j++)
{
//alpha=255 is opaque > image2 should be used
alpha = image2[3][j,i].Intensity;
image1[j, i]
= new Bgra(
image2[j, i].Blue * alpha + (image1[j, i].Blue * (255-alpha)),
image2[j, i].Green * alpha + (image1[j, i].Green * (255-alpha)),
image2[j, i].Red * alpha + (image1[j, i].Red * (255-alpha)));
}
}
Using Osiris's suggestion as a starting point, and having checked out alpha compositing on Wikipedia, i ended up with the following which worked really nicely for my purposes.
This was used this with Emgucv. I was hoping that the opencv gpu::AlphaComposite methods were available in Emgucv which I believe would have done the following for me, but alas the version I am using didn't appear to have them implemented.
static public Image<Bgra, Byte> Overlay( Image<Bgra, Byte> image1, Image<Bgra, Byte> image2 )
{
Image<Bgra, Byte> result = image1.Copy();
Image<Bgra, Byte> src = image2;
Image<Bgra, Byte> dst = image1;
int rows = result.Rows;
int cols = result.Cols;
for (int y = 0; y < rows; ++y)
{
for (int x = 0; x < cols; ++x)
{
// http://en.wikipedia.org/wiki/Alpha_compositing
double srcA = 1.0/255 * src.Data[y, x, 3];
double dstA = 1.0/255 * dst.Data[y, x, 3];
double outA = (srcA + (dstA - dstA * srcA));
result.Data[y, x, 0] = (Byte)(((src.Data[y, x, 0] * srcA) + (dst.Data[y, x, 0] * (1 - srcA))) / outA); // Blue
result.Data[y, x, 1] = (Byte)(((src.Data[y, x, 1] * srcA) + (dst.Data[y, x, 1] * (1 - srcA))) / outA); // Green
result.Data[y, x, 2] = (Byte)(((src.Data[y, x, 2] * srcA) + (dst.Data[y, x, 2] * (1 - srcA))) / outA); // Red
result.Data[y, x, 3] = (Byte)(outA*255);
}
}
return result;
}
A newer version, using emgucv methods. rather than a loop. Not sure it improves on performance.
double unit = 1.0 / 255.0;
Image[] dstS = dst.Split();
Image[] srcS = src.Split();
Image[] rs = result.Split();
Image<Gray, double> srcA = srcS[3] * unit;
Image<Gray, double> dstA = dstS[3] * unit;
Image<Gray, double> outA = srcA.Add(dstA.Sub(dstA.Mul(srcA)));// (srcA + (dstA - dstA * srcA));
// Red.
rs[0] = srcS[0].Mul(srcA).Add(dstS[0].Mul(1 - srcA)).Mul(outA.Pow(-1.0)); // Mul.Pow is divide.
rs[1] = srcS[1].Mul(srcA).Add(dstS[1].Mul(1 - srcA)).Mul(outA.Pow(-1.0));
rs[2] = srcS[2].Mul(srcA).Add(dstS[2].Mul(1 - srcA)).Mul(outA.Pow(-1.0));
rs[3] = outA.Mul(255);
// Merge image back together.
CvInvoke.cvMerge(rs[0], rs[1], rs[2], rs[3], result);
return result.Convert<Bgra, Byte>();
I found an interesting blog post on internet, which I think is related to what you are trying to do.
Please have a look at the Creating Overlays Method (archive.org link). You can use this idea to implement your own function to add two images in the way you mentioned above, making some particular areas in the image transparent while leaving the rest as it is.
I need to convert an 8-bit IplImage to a 32-bits IplImage. Using documentation from all over the web I've tried the following things:
// general code
img2 = cvCreateImage(cvSize(img->width, img->height), 32, 3);
int height = img->height;
int width = img->width;
int channels = img->nChannels;
int step1 = img->widthStep;
int step2 = img2->widthStep;
int depth1 = img->depth;
int depth2 = img2->depth;
uchar *data1 = (uchar *)img->imageData;
uchar *data2 = (uchar *)img2->imageData;
for(h=0;h<height;h++) for(w=0;w<width;w++) for(c=0;c<channels;c++) {
// attempt code...
}
// attempt one
// result: white image, two red spots which appear in the original image too.
// this is the closest result, what's going wrong?!
// see: http://files.dazjorz.com/cache/conversion.png
((float*)data2+h*step2+w*channels+c)[0] = data1[h*step1+w*channels+c];
// attempt two
// when I change float to unsigned long in both previous examples, I get a black screen.
// attempt three
// result: seemingly random data to the top of the screen.
data2[h*step2+w*channels*3+c] = data1[h*step1+w*channels+c];
data2[h*step2+w*channels*3+c+1] = 0x00;
data2[h*step2+w*channels*3+c+2] = 0x00;
// and then some other things. Nothing did what I wanted. I couldn't get an output
// image which looked the same as the input image.
As you see I don't really know what I'm doing. I'd love to find out, but I'd love it more if I could get this done correctly.
Thanks for any help I get!
The function you are looking for is cvConvertScale(). It automagically does any type conversion for you. You just have to specify that you want to scale by a factor of 1/255 (which maps the range [0...255] to [0...1]).
Example:
IplImage *im8 = cvLoadImage(argv[1]);
IplImage *im32 = cvCreateImage(cvSize(im8->width, im8->height), 32, 3);
cvConvertScale(im8, im32, 1/255.);
Note the dot in 1/255. - to force a double division. Without it you get a scale of 0.
Perhaps this link can help you?
Edit In response to the second edit of the OP and the comment
Have you tried
float value = 0.5
instead of
float value = 0x0000001;
I thought the range for a float color value goes from 0.0 to 1.0, where 1.0 is white.
Floating point colors go from 0.0 to 1.0, and uchars go from 0 to 255. The following code fixes it:
// h is height, w is width, c is current channel (0 to 2)
int b = ((uchar *)(img->imageData + h*img->widthStep))[w*img->nChannels + c];
((float *)(img2->imageData + h*img2->widthStep))[w*img2->nChannels + c] = ((float)b) / 255.0;
Many, many thanks to Stefan Schmidt for helping me fix this!
If you do not put the dot (.), some compilers will understand is as an int division, giving you a int result (zero in this case).
You can create an IplImage wrapper using boost::shared_ptr and template-metaprogramming. I have done that, and I get automatic garbage collection, together with automatic image conversions from one depth to another, or from one-channel to multi-channel images.
I have called the API blImageAPI and it can be found here:
http://www.barbato.us/2010/10/14/image-data-structure-based-shared_ptr-iplimage/
It is very fast, and make code very readable, (good for maintaining algorithms)
It is also can be used instead of IplImage in opencv algorithms without changing anything.
Good luck and have fun writing algorithms!!!
IplImage *img8,*img32;
img8 =cvLoadImage("a.jpg",1);
cvNamedWindow("Convert",1);
img32 = cvCreateImage(cvGetSize(img8),IPL_DEPTH_32F,3);
cvConvertScale(img8,img32,1.0/255.0,0.0);
//For Confirmation Check the pixel values (between 0 - 1)
for(int row = 0; row < img32->height; row++ ){
float* pt = (float*) (img32->imageData + row * img32->widthStep);
for ( int col = 0; col < width; col++ )
printf("\n %3.3f , %3.3f , %3.3f ",pt[3*col],pt[3*col+1],pt[3*col+2]);
}
cvShowImage("Convert",img32);
cvWaitKey(0);
cvReleaseImage(&img8);
cvReleaseImage(&img32);
cvDestroyWindow("Convert");