I have two webcams (both are Logitech C615). I want to adjust the webcams in a way that they make nearly the same picture in same environment. (the reason is that I want to render this images onto an occulus rift).
I'm using OpenCV to connect the cameras. My first innocent try was to get all the CV_CAP_PROPs from the one cam and set the values to the other cam. That doesn't work very well.
Is there may already a function I could use or can you give me another approach?
--- EDIT: histogram equalization ---
That's the result:
The result is better than before but as you can see the hue is different.
Try to convert them to YCrCb and equalize just the Y channel (and convert them back to BGR if you need to).
This should equalize the brightness of both images.
Snippet:
cv::cvtColor( frame, frame, CV_BGR2YCrCb);
cv::split( frame, channels);
cv::equalizeHist( channels[0], channels[0] );
cv::merge( channels, 3, frame );
cv::cvtColor( frame, frame, CV_YCrCb2BGR );
Related
How do I blend two images - thermal(80x60) and RGB(640x480) efficiently?
If I scale the thermal to 640x480 it doesn't scale up evenly or doesn't have enough quality to do any processing on it. Any ideas would be really helpful.
RGB image - http://postimg.org/image/66f9hnaj1/
Thermal image - http://postimg.org/image/6g1oxbm5n/
If you scale the resolution of the thermal image up by a factor of 8 and use Bilinear Interpolation you should get a smoother, less-blocky result.
When combining satellite images of different resolution, (I talk about satellite imagery because that is my speciality), you would normally use the highest resolution imagery as the Lightness or L channel to give you apparent resolution and detail in the shapes because the human eye is good at detecting contrast and then use the lower resolution imagery to fill in the Hue and Saturation, or a and b channels to give you the colour graduations you are hoping to see.
So, in concrete terms, I would consider converting the RGB to Lab or HSL colourspace and retaining the L channel. The take the thermal image and up-res it by 8 using bilinear interpolation and use the result as the a, or b or H or S and maybe fill in the remaining channel with the one from the RGB that has the most variance. Then convert the result back to RGB for a false-colour image. It is hard to tell without seeing the images or knowing what you are hoping to find in them. But in general terms, that would be my approach. HTH.
Note: Given that a of Lab colourspace controls the red/green relationship, I would probably try putting the thermal data in that channel so it tends to show more red the "hotter" the thermal channel is.
Updated Answer
Ok, now I can see your images and you have a couple more problems... firstly the images are not aligned, or registered, with each other which is not going to help - try using a tripod ;-) Secondly, your RGB image is very poorly exposed so it is not really going to contribute that much detail - especially in the shadows - to the combined image.
So, firstly, I used ImageMagick at the commandline to up-size the thermal image like this:
convert thermal.png -resize 640x480 thermal.png
Then, I used Photoshop to do a crude alignment/registration. If you want to try this, the easiest way is to put the two images into separate layers of the same document and set the Blending mode of the upper layer to Difference. Then use the Move Tool (shortcut v) to move the upper image around till the screen goes black which means that the details are on top of each other and when subtracted they come to zero, i.e. black. Then crop so the images are aligned and turn off one layer and save, then turn that layer back on and the other layer off and save again.
Now, I used ImageMagick again to separate the two images into Lab layers:
convert bigthermalaligned.png -colorspace Lab -separate thermal.png
convert rgbaligned.png -colorspace Lab -separate rgb.png
which gives me
thermal-0.png => L channel
thermal-1.png => a channel
thermal-2.png => b channel
rgb-0.png => L channel
rgb-1.png => a channel
rgb-2.png => b channel
Now I can take the L channel of the RGB image and the a and b channels of the thermal image and put them together:
convert rgba-0.png thermal-1.png thermal-2.png -normalize -set colorpsace lab -combine result.png
And you get this monstrosity! Obviously you can play around with the channels and colourpsaces and a tripod and proper exposures, but you should be able to see some of the details of the RGB image - especially the curtains on the left, the lights, the camera on the cellphone and the label on the water bottle - have come through into the final image.
Assuming that the images were not captured using a single camera, you need to note that the two cameras may have different parameters. Also, if it's two cameras, they are probably not located in the same world position (offset).
In order to resolve this, you need to get the intrinsic calibration matrix of each of the cameras, and find the offset between them.
Then, you can find a transformation between a pixel in one camera and the other. Unfortunately, if you don't have any depth information about the scene, the most you can do with the calibration matrix is get a ray direction from the camera position to the world.
The easy approach would be to ignore the offset (assuming the scene is not too close to the camera), and just transform the pixel.
p2=K2*(K1^-1 * p1)
Using this you can construct a new image that is a composite of both.
The more difficult approach would be to reconstruct the 3D structure of the scene by finding features that you can match between both images, and then triangulate the point with both rays.
I'm using OpenNI and OpenCV (but without the latest code with openni support). If I just send the depth channel to the screen - it will look dark and difficult to understand something. So I want to show a depth channel for the user in a color but cannot find how to do that without losing of accuracy. Now I do it like that:
xn::DepthMetaData xDepthMap;
depthGen.GetMetaData(xDepthMap);
XnDepthPixel* depthData = const_cast<XnDepthPixel*>(xDepthMap.Data());
cv::Mat depth(frame_height, frame_width, CV_16U, reinterpret_cast<void*>(depthData));
cv::Mat depthMat8UC1;
depth.convertTo(depthMat8UC1, CV_8UC1);
cv::Mat falseColorsMap;
cv::applyColorMap(depthMat8UC1, falseColorsMap, cv::COLORMAP_AUTUMN);
depthWriter << falseColorsMap;
But in this case I get worse (loosing details) output than, for instance, kinects software for windows shows me. So I'm looking for a function in OpenNI or OpenCV with a better transformation.
ghttps://github.com/OpenNI/OpenNI2/blob/master/Samples/Common/OniSampleUtilities.h
the link is the code for histogram equalization. In short, it makes the probability of each level equal and optimizes mapping between 10,000 levels and 255 levels. That is why Kinect yellowish map looks better than naive I=255*z/z_range.
NOTE: don’t use color for visualization since a human eye is more sensitive to luminance change than to color variation. So with 255 levels of luminance you will get better contrast than with 255*255*255 levels of color. If you still decide to go along the color mapping avenue use HSV color space where you can manipulate Hue 0..360 deg, Value 1..0 and better set saturation to max. Map depth to hue and value, convert to RGB and display. Than go back to histogram equalization ;)
Try this:
const float scaleFactor = 0.05f;
depth.convertTo(depthMat8UC1, CV_8UC1, scaleFactor);
imshow("depth gray",depthMat8UC1);
Play with the value to get a result you're happy with
I am trying to detect the white shapes in an object and can successfully do it for 1 video.
// Create and display a new matrix for triangles
triangles = src.clone();
GaussianBlur(triangles, triangles, Size(5, 5), 0, 0);
inRange(triangles, Scalar(150,150,150), Scalar(255, 255, 255), triangles);
imshow("triangles", triangles);
This gives me the result
http://s8.postimg.org/o9xg284jp/triangles.png
However, if I use a different video - then the scalar value of 150 may not be appropriate (for example if it is a light environment... everything gets detected)
http://s8.postimg.org/m09brgvlx/bad_triangles.png
For this video I would need to change the minimum scalar to be around 190-200 for it to work properly. My question - is there a good way to determine the correct scalar value to use? I know it sounds simple to some, but ive got a headache because of it!
http://colorizer.org/
If you check here you can see what your problem is. RGB = (255, 155, 155) is probably not a "white" but your inRange method is giving true output to that one.
Try to use HSL color space. Lightness > 90 is white for sure, no matter what H and S channel values are. Use BGR2HLS conversion. Then use inRange with L channel between 90-100.
Actually, for color detection problems, mostly used color spaces are HSV and HSL, not RGB!
There is probably no way to automatically determine a threshold that works for all kind of videos. But to make it less dependent on the overall lightning of the video you could make it depend on the mean or median pixel value of the image.
Or if you know how big your object appears in the image, you could choose the threshold accordingly.
Another approach could be to normalize the brightness of the video.
But which approach is best strongly dependents on your exact situation and requirements.
Basically, I want to be able to compare two histograms, but not of whole images just specific areas. I have image A and have a specific rectangular region on it that I want to compare to another image B. Is there a way to get the histogram of a definable rectangular region on an image? I have the x y position of the rectangular area, as well as it's width and height, and want to get its histogram. I'm using opencv with python.
Sorry if that isn't very clear :(
(I'm setting up a program that takes a picture of a circuit board, and checks each solder pad for consistency with an image of a perfect board. If one pad is off, the program raises a flag saying that specific pad is off by x percent, not the whole board.
Note: The following is in C++ but I think it is not hard to find the equivalent functions for python.
You can find the histogram of an image using this tutorial. So for example for the lena image we get:
In your case, since you have the rectangle coordinates, you can just extract the ROI of the image:
// C++ code
cv::Mat image = cv::imread("lena.png", 0);
cv::Rect roiRect = cv::Rect(150, 150, 250, 250);
cv::Mat imageRoi = image(roiRect);
and then find the histogram of just the ROI with the same way as above:
Is this what you wanted (in theory at least) or I misunderstood?
So I am very new to OpenCV (2.1), so please keep that in mind.
So I managed to calibrate my cheap web camera that I am using (with a wide angle attachment), using the checkerboard calibration method to produce the intrinsic and distortion coefficients.
I then have no trouble feeding these values back in and producing image maps, which I then apply to a video feed to correct the incoming images.
I run into an issue however. I know when it is warping/correcting the image, it creates several skewed sections, and then formats the image to crop out any black areas. My question then is can I view the complete warped image, including some regions that have black areas? Below is an example of the black regions with skewed sections I was trying to convey if my terminology was off:
An image better conveying the regions I am talking about can be found here! This image was discovered in this post.
Currently: The cvRemap() returns basically the yellow box in the image linked above, but I want to see the whole image as there is relevant data I am looking to get out of it.
What I've tried: Applying a scale conversion to the image map to fit the complete image (including stretched parts) into frame
CvMat *intrinsic = (CvMat*)cvLoad( "Intrinsics.xml" );
CvMat *distortion = (CvMat*)cvLoad( "Distortion.xml" );
cvInitUndistortMap( intrinsic, distortion, mapx, mapy );
cvConvertScale(mapx, mapx, 1.25, -shift_x); // Some sort of scale conversion
cvConvertScale(mapy, mapy, 1.25, -shift_y); // applied to the image map
cvRemap(distorted,undistorted,mapx,mapy);
The cvConvertScale, when I think I have aligned the x/y shift correctly (guess/checking), is somehow distorting the image map making the correction useless. There might be some math involved here I am not correctly following/understanding.
Does anyone have any other suggestions to solve this problem, or what I might be doing wrong? I've also tried trying to write my own code to fix distortion issues, but lets just say OpenCV knows already how to do it well.
From memory, you need to use InitUndistortRectifyMap(cameraMatrix,distCoeffs,R,newCameraMatrix,map1,map2), of which InitUndistortMap is a simplified version.
cvInitUndistortMap( intrinsic, distort, map1, map2 )
is equivalent to:
cvInitUndistortRectifyMap( intrinsic, distort, Identity matrix, intrinsic,
map1, map2 )
The new parameters are R and newCameraMatrix. R species an additional transformation (e.g. rotation) to perform (just set it to the identity matrix).
The parameter of interest to you is newCameraMatrix. In InitUndistortMap this is the same as the original camera matrix, but you can use it to get that scaling effect you're talking about.
You get the new camera matrix with GetOptimalNewCameraMatrix(cameraMat, distCoeffs, imageSize, alpha,...). You basically feed in intrinsic, distort, your original image size, and a parameter alpha (along with containers to hold the result matrix, see documentation). The parameter alpha will achieve what you want.
I quote from the documentation:
The function computes the optimal new camera matrix based on the free
scaling parameter. By varying this parameter the user may retrieve
only sensible pixels alpha=0, keep all the original image pixels if
there is valuable information in the corners alpha=1, or get something
in between. When alpha>0, the undistortion result will likely have
some black pixels corresponding to “virtual” pixels outside of the
captured distorted image. The original camera matrix, distortion
coefficients, the computed new camera matrix and the newImageSize
should be passed to InitUndistortRectifyMap to produce the maps for
Remap.
So for the extreme example with all the black bits showing you want alpha=1.
In summary:
call cvGetOptimalNewCameraMatrix with alpha=1 to obtain newCameraMatrix.
use cvInitUndistortRectifymap with R being identity matrix and newCameraMatrix set to the one you just calculated
feed the new maps into cvRemap.