I have used the excellent answer to the question here:
How to detect bullet holes on the target using python
I have verified that it works in both Python 2 and 3.6, but I would like to use the concept in an iOS application written in Objective C(++). This is my attempt at translating it. Ultimately, I need it to work with an image taken by the camera, so I don't want to use imread, but I've checked that this makes no difference.
UIImage *nsi = [UIImage imageNamed:#"CANDX.jpg"];
cv::Mat original;
UIImageToMat(nsi, original);
cv::Mat thresholded;
cv::inRange(original, cv::Scalar(40,40,40), cv::Scalar(160,160,160), thresholded);
cv::Mat kernel = cv::Mat::ones(10, 10, CV_64FC1);
cv::Mat opening;
cv::morphologyEx(thresholded, opening, cv::MORPH_OPEN, kernel);
vector<vector<cv::Point>> contours;
cv::findContours(opening, contours, CV_RETR_LIST, CV_CHAIN_APPROX_NONE);
The call to inRange, with the same values as the Python version, gives a completely black image. Indeed, it is impossible to pick values for lower- and upper-bounds that do not result in this outcome. I've tried converting the image to HSV and using HSV values for lower- and upper-bound. This makes a slight difference in that I can get some vaguely recognisable outcomes, but nothing like the useful result I should be getting.
If I substitute the 'thresholded' image from the answer and comment out the inRange call, the morphology and findContours calls work okay.
Am I doing something wrong in setting up the inRange call?
As you mention in the comments, the data type of original is CV_8UC4 -- i.e. it's a 4 channel image. However, in your call to cv::inRange, you provide ranges for only 3 channels.
cv::Scalar represents a 4-element vector. When you call the constructor with only 3 values, a default value of 0 is used for the 4-th element.
Hence, your call to inRange is actually equivalent to this:
cv::inRange(original, cv::Scalar(40,40,40,0), cv::Scalar(160,160,160,0), thresholded);
You're looking only for pixels that have the alpha channel set to 0 (fully transparent). Since the image came from a camera, it's highly unlikely there will be any transparent pixels -- the alpha channel is probably just all 255s.
There are 2 options to solve this:
Drop the unneeded alpha channel. One way to do this is to use cv::cvtColor, e.g.
cv::cvtColor(original, original, cv::COLOR_BGRA2BGR);
Specify desired range for all the channels, e.g.
cv::inRange(original, cv::Scalar(40,40,40,0), cv::Scalar(160,160,160,255), thresholded);
Related
Being a Matlab/Python guy and a novice in C++, I'm having major frustration moving to OpenCV in C++ for image processing purposes. I'm working with Kinect v2 so there is only one Windows example I found online which I'm modifying.
This example gives the depthMap as a cv::Mat and I've calculated surface normals on this depth image taken from a kinect v2. This surface normal image contains the i,j,k vector components (3 channels) per row,col element and I'm trying to visualize this surface normal image (3-D float matrix) as an RGB image. This is really easy to do in matlab since you just do an imshow(normMap) and it shows an RGB (ijk) image with the color specifying the orientation of the normal.
I'm trying to do a similar thing in C++. Since I'm using opencv, I decided to use the cv::Mat to store the ijk channels per pixel and I initialized the normal matrix (lets call it normMat) to a CV_32F matrix as follows:
int sizes[3] = { height, width, 3 };
cv::Mat normMat(3, sizes, CV_32F, cv::Scalar::all(0));
However, if I debug, the dims (normMat.rows and normMat.cols) are showing -1, so I don't know whether my initialization is bad or if I missed something, or whether it's normal.
I'm storing the surface normal components as:
normMat.at<float>(i, j, 0) = fVec3[0];
normMat.at<float>(i, j, 1) = fVec3[1];
normMat.at<float>(i, j, 2) = fVec3[2];
And they seem to be getting stored correctly as I've verified that in debug (using normMat.at<float>(i,j,k)).
Then I'm trying to display the image as:
normMat.convertTo(normColorMap, CV_8UC3, 255.0 / (4000), 255.0 / 500);
cv::imshow("Normals", normColorMap);
But the second line throws an exception:
OpenCV Error: Assertion failed (p[-1]
<= 2) in cv::MatSize::operator.....
I also tried displaying normMat directly which throws the same exception. Which means that there's obviously something wrong with displaying a 3 channel image as a 3D matrix, or converting it to a 2D-3 Channel Mat. I also tried initializing normMat as
cv::Mat normMat(height, width, CV_32FC3, cv::Scalar::all(0));
But I'm having issues adding data to the 3 channels of a "2D 3 channel matrix" (CV_32FC3), if I may say it this way.
All I want to do is display this normMat as an RGB image. Any inputs,suggestions are highly appreciated. Thanks!
I have a vector of Point2f which have color space CV_8UC4 and need to convert them to CV_64F, is the following code correct?
points1.convertTo(points1, CV_64F);
More details:
I am trying to use this function to calculate the essential matrix (rotation/translation) through the 5-point algorithm, instead of using the findFundamentalMath included in OpenCV, which is based on the 8-point algorithm:
https://github.com/prclibo/relative-pose-estimation/blob/master/five-point-nister/five-point.cpp#L69
As you can see it first converts the image to CV_64F. My input image is a CV_8UC4, BGRA image. When I tested the function, both BGRA and greyscale images produce valid matrices from the mathematical point of view, but if I pass a greyscale image instead of color, it takes way more to calculate. Which makes me think I'm not doing something correctly in one of the two cases.
I read around that when the change in color space is not linear (which I suppose is the case when you go from 4 channels to 1 like in this case), you should normalize the intensity value. Is that correct? Which input should I give to this function?
Another note, the function is called like this in my code:
vector<Point2f>imgpts1, imgpts2;
for (vector<DMatch>::const_iterator it = matches.begin(); it!= matches.end(); ++it)
{
imgpts1.push_back(firstViewFeatures.second[it->queryIdx].pt);
imgpts2.push_back(secondViewFeatures.second[it->trainIdx].pt);
}
Mat mask;
Mat E = findEssentialMat(imgpts1, imgpts2, [camera focal], [camera principal_point], CV_RANSAC, 0.999, 1, mask);
The fact I'm not passing a Mat, but a vector of Point2f instead, seems to create no problems, as it compiles and executes properly.
Is it the case I should store the matches in a Mat?
I am no sure do you mean by vector of Point2f in some color space, but if you want to convert vector of points into vector of points of another type you can use any standard C++/STL function like copy(), assign() or insert(). For example:
copy(floatPoints.begin(), floatPoints.end(), doublePoints.begin());
or
doublePoints.insert(doublePoints.end(), floatPoints.begin(), floatPoints.end());
No, it is not. A std::vector<cv::Pointf2f> cannot make use of the OpenCV convertTo function.
I think you really mean that you have a cv::Mat points1 of type CV_8UC4. Note that those are RxCx4 values (being R and C the number of rows and columns), and that in a CV_64F matrix you will have RxC values only. So, you need to be more clear on how you want to transform those values.
You can do points1.convertTo(points1, CV_64FC4) to get a RxCx4 matrix.
Update:
Some remarks after you updated the question:
Note that a vector<cv::Point2f> is a vector of 2D points that is not associated to any particular color space, they are just coordinates in the image axes. So, they represent the same 2D points in a grey, rgb or hsv image. Then, the execution time of findEssentialMat doesn't depend on the image color space. Getting the points may, though.
That said, I think your input for findEssentialMat is ok (the function takes care of the vectors and convert them into their internal representation). In this cases, it is very useful to draw the points in your image to debug the code.
I am trying to detect the white shapes in an object and can successfully do it for 1 video.
// Create and display a new matrix for triangles
triangles = src.clone();
GaussianBlur(triangles, triangles, Size(5, 5), 0, 0);
inRange(triangles, Scalar(150,150,150), Scalar(255, 255, 255), triangles);
imshow("triangles", triangles);
This gives me the result
http://s8.postimg.org/o9xg284jp/triangles.png
However, if I use a different video - then the scalar value of 150 may not be appropriate (for example if it is a light environment... everything gets detected)
http://s8.postimg.org/m09brgvlx/bad_triangles.png
For this video I would need to change the minimum scalar to be around 190-200 for it to work properly. My question - is there a good way to determine the correct scalar value to use? I know it sounds simple to some, but ive got a headache because of it!
http://colorizer.org/
If you check here you can see what your problem is. RGB = (255, 155, 155) is probably not a "white" but your inRange method is giving true output to that one.
Try to use HSL color space. Lightness > 90 is white for sure, no matter what H and S channel values are. Use BGR2HLS conversion. Then use inRange with L channel between 90-100.
Actually, for color detection problems, mostly used color spaces are HSV and HSL, not RGB!
There is probably no way to automatically determine a threshold that works for all kind of videos. But to make it less dependent on the overall lightning of the video you could make it depend on the mean or median pixel value of the image.
Or if you know how big your object appears in the image, you could choose the threshold accordingly.
Another approach could be to normalize the brightness of the video.
But which approach is best strongly dependents on your exact situation and requirements.
I am working on Demosaicing a Bayer pattern to RGB image without using OpenCV's direct conversion function. I have used Bilinear interpolation and got it to work, but I want to improve the quality by using The Freeman Method. This method requires Median Filter. OpenCV has medianBlur function which does that. But I am having trouble using this function. When the cv::Mat to which I apply medianBlur is of type CV_8UC1 then it works, but if it is of type CV_32S, then it does not work.
This does NOT work:
redGreenMedian.create(input.size(), CV_32S);
blueGreenMedian.create(input.size(), CV_32S);
blueMinusGreen.create(input.size(), CV_32S);
redMinusGreen.create(input.size(), CV_32S);
for(int i = 1; i <= 3; i += 2)
{
cv::medianBlur(redMinusGreen, redGreenMedian, i);
cv::medianBlur(blueMinusGreen, blueGreenMedian, i);
}
If I change all CV_32S to CV_8UC1 then it works. On debugging I found that it crashes in the second iteration only not in first one. However, I need it to run for both iterations. This does NOT work when written separately too:
cv::medianBlur(redMinusGreen, redGreenMedian, 3);
As an aside, I do not have to use CV_32S, but I need the ability to store negative numbers in the matrices.
NOTE: I have tried making all numbers in the matrices positive and then use medianBlur, but it still did not work.
All help is appreciated. Thanks in advance.
The OpenCV documentation seems to be very clear:
src – input 1-, 3-, or 4-channel image; when ksize is 3 or 5, the image depth should be CV_8U, CV_16U, or CV_32F, for larger aperture sizes, it can only be CV_8U.
As you need to cater for signed values, I think your best option is to use CV_32F.
Also, the documentation says
ksize – aperture linear size; it must be odd and greater than 1, for example: 3, 5, 7
Your loop applies sizes of 1 and 3 (if I read your code correctly), the first of which is invalid, which possibly explains why your first iteration doesn't crash (because it fails earlier).
I'm writing an Android app in OpenCV to detect blobs. One task is to threshold the image to differentiate the foreground objects from the background (see image).
It works fine as long as the image is known and I can manually pass a threshold value to threshold()--in this particular image say, 200. But assuming that the image is not known with the only knowledge that there would be a dark solid background and lighter foreground objects how can I dynamically figure out the threshold value?
I've come across the histogram where I can compute the intensity distribution of the grayscale image. But I couldn't find a method to analyze the histogram and choose the value where the objects of interest (lighter) lies. That is; I want to differ the obviously dark background spikes from the lighter foreground spikes--in this case above 200, but in another case could be say, 100 if the objects are grayish.
If all your images are like this, or can be brought to this style, i think cv2.THRESHOLD_OTSU, ie otsu's tresholding algorithm is a good shot.
Below is a sample using Python in command terminal :
>>> import cv2
>>> import numpy as np
>>> img2 = cv2.imread('D:\Abid_Rahman_K\work_space\sofeggs.jpg',0)
>>> ret,thresh = cv2.threshold(img2,0,255,cv2.THRESH_BINARY+cv2.THRESH_OTSU)
>>> ret
122.0
ret is the threshold value which is automatically calculated. We just pass '0' as threshold value for this.
I got 124 in GIMP ( which is comparable to result we got). And it also removes the noise. See result below:
If you say that the background is dark (black) and the foreground is lighter, then I recommend to use the YUV color space (or any other YXX like YCrCb, etc.), because the first component of such color spaces is luminance (or lightning).
So after the Y channel is extracted (via the extractChennel function) we need to analyse the histogram of this channel (image):
See the first (left) hump? It represents dark areas (the background in your situation) on your image. So our aim now is to find a segment (on abscissa, it's red part in the image) that contains this hump. Obviously the left point of this segment is zero. The right point is the first point where:
the (local) maximum of histogram is from the left of the point
the value of histogram is less than some small epsilon (you can set it to 10)
I drew a green vertical line to show the location of the right point of the segment in this histogram.
And that's it! This right point of the segment is the needed threshold. Here's the result (epsilon is 10 and the calculated threshold is 50):
I think that it's not a problem for you to delete the noise in the image above.
The following is a C++ implementation of Abid's answer that works with OpenCV 3.x:
// Convert the source image to a 1 channel grayscale:
Mat gray;
cvtColor(src, gray, CV_BGR2GRAY);
// Apply the threshold function with the CV_THRESH_OTSU setting as well
// You can skip having it return the value, but I include it for showing the
// results from OTSU
double thresholdValue = threshold(gray, gray, 0, 255, CV_THRESH_BINARY+CV_THRESH_OTSU);
// Present the threshold value
printf("Threshold value: %f\n", thresholdValue);
Running this against the original image, I get the following:
OpenCV calculated a threshold value of 122 for it, close to the value Abid found in his answer.
Just to verify, I altered the original image as seen here:
And produced the following, with a new threshold value of 178: