I am trying to substract background from depth images acquired with kinect. When I learned what otsu thresholding is I thought that it could with it. Converting the depth image to grayscale i can hopefully apply otsu threshold to binarize the image.
However I implemented (tried to implemented) this with OpenCV 2.3, it came in vain. The output image is binarized however, very unexpectedly. I did the thresholding continuously (i.e print the result to screen to analyze for each frame) and saw that for some frames threshold is found to be 160ish and sometimes it is found to be 0. I couldn't quite understand why this is happening. May it be due to the high number of 0's in the depth image returned by kinect, which corresponds to pixels that can not be measured. Is there a way that I could tell the algorithm to ignore pixels having the value 0? Or otsu thresholding is not good for what I am trying to do?
Here are some outputs and segment of the related code. You may notice that the second screenshot looks like it could do some good binarization, however i want to achieve one that distincly differentiates between pixels corresponding to the chair in the scene and the backgroung.
Thanks.
cv::Mat1s depthcv(depth->getHeight(), depth->getWidth());
cv::Mat1b depthcv8(depth->getHeight(), depth->getWidth());
cv::Mat1b depthcv8_th(depth->getHeight(), depth->getWidth());
depthcv.data =(uchar*) depth->getDepthMetaData().Data();
depthcv.convertTo(depthcv8,CV_8U,255/5000.f);
//apply otsu thresholding
cv::threshold(depthcv8, depthcv8_th, 128, 255, CV_THRESH_BINARY|CV_THRESH_OTSU);
std::ofstream output;
output.open("output.txt");
//output << "M = "<< endl << " " << depthcv8 << endl << endl;
cv::imshow("lab",depthcv8_th);
cv::waitKey(1);
Otsu is probably good enough for what you are trying to do, but you do need to mask out the zero values before computing the optimal threshold with the Otsu algorithm, otherwise the distribution of intensity values will be skewed lower than what you want.
OpenCV does not provide a mask argument for the cv::threshold function, so you will have to remove those values yourself. I would recommend putting all the non-zero values in a 1 by N matrix, and calling the cv::threshold function with CV_THRESH_OTSU and saving the return value (which is the estimated optimal threshold), and then running the cv::threshold function again on the original image with just the CV_THRESH_BINARY flag and the computed threshold.
Here is one possible implementation:
// move zeros to the back of a temp array
cv::Mat copyImg = origImg;
uint8* ptr = copyImg.datastart;
uint8* ptr_end = copyImg.dataend;
while (ptr < ptr_end) {
if (*ptr == 0) { // swap if zero
uint8 tmp = *ptr_end;
*ptr_end = *ptr;
*ptr = tmp;
ptr_end--; // make array smaller
} else {
ptr++;
}
}
// make a new matrix with only valid data
cv::Mat nz = cv::Mat(std::vector<uint8>(copyImg.datastart,ptr_end),true);
// compute optimal Otsu threshold
double thresh = cv::threshold(nz,nz,0,255,CV_THRESH_BINARY | CV_THRESH_OTSU);
// apply threshold
cv::threshold(origImg,origImg,thresh,255,CV_THRESH_BINARY_INV);
Related
Here is the picture in grayscale mode:
if I apply a Thresholding and set threshold to 0. According to my understanding, the thesholded image will be mostly white. but the result is opposite.
Result is:
I also tried this:
build a image and set all pixel to 255. then apply the 0 threshold thresholding, the returned image is all 255.
The question is:
in the picture is mostly zero (black) after apply thresholding.
Here are the code:
IplImage* g_image = NULL;
IplImage* g_gray = NULL;
int g_thresh = 100;
CvMemStorage* g_storage = NULL;
void on_tracker(int){
if(g_storage == NULL){
g_gray = cvCreateImage(cvGetSize(g_image), 8, 1);
g_storage = cvCreateMemStorage(0);
}else{
cvClearMemStorage(g_storage);
}
CvSeq* contours = 0;
cvCvtColor(g_image, g_gray, CV_BGR2GRAY);
cvNamedWindow("Gray");
cvShowImage("Gray", g_gray);
cvThreshold(g_gray, g_gray, g_thresh, 255, CV_THRESH_BINARY);
cvFindContours(g_gray, g_storage, &contours);
cvShowImage("Contours", g_gray);
}
int main(int argc, char** argv){
if( argc !=2 || !(g_image = cvLoadImage(argv[1]))){
return -1;
}
cvNamedWindow("Contours", CV_WINDOW_AUTOSIZE);
cvCreateTrackbar(
"Threshold",
"Contours",
&g_thresh,
255,
on_tracker
);
on_tracker(0);
cvWaitKey();
return 0;
}
Have a read of the different types of thresholding available to you in the documentation.
Starting with a 1D 'image' with a range of values (the black line) and threshold (the blue line):
...we can visualise the outcome of the different modes:
Threshold Binary
Threshold Binary Inverted
Truncate
Threshold to Zero
Threshold to Zero Inverted
Please update your question with your code so we know what mode you're using if this answer doesn't help already ;)
The basic Thresholding is to check the pixels value (say from 0 to 255) to be above the Threshold value and to assign to the pixel a value of maximum value (high intensity: black) this called Binary Thresholding.
In your case, when setting a value of 0 to the threshold, you actually filtering all your pixels since all of them (the low intensities and the higher intensities) have values above zero (0).
Maybe you would like to make a brighter picture - in this case use Inverted Binary Thresholding: in this case, you will get white picture when value is 0.
Accoring to #Miki's comments. this is caused by C API. I tried the same process with python API. the result is normal:
if I do thresholding with 0 threshold, most of pixel will be set to 255.
Currently I am trying to extract the hieroglyphics symbols from images like this one.
What I have done is used hough transform to find lines and split the image in portions to make it easier for me. But I tried a set of algorithms to extract the sunken letters from the image and I hit a dead end..
What I have tried is a mixture of morphological operations and edge detection and contour finding.
So are there any algorithms devised to do something like this or any hint will be appreciated.
You can up-sample the input image, apply some smoothing, and find the Otsu threshold, then use this threshold to find Canny edges with different window sizes.
For the larger window (5 x 5), you get a noisy image that contains almost all the edges you need, plus noise.
For the smaller window (3 x 3), you get a less noisy image, but some of the edges are missing.
If this less noisy image is not good enough, you can try morphologically reconstructing it using the noisy image as the mask. Here, I've linked some diagonal edge segments in the noisy image using a morphological hit-miss transform and then applied the reconstruction.
Using a
Mat k = (Mat_<int>(3, 3) <<
0, 0, 1,
0, -1, 0,
1, 0, 0);
kernel for linking broken edges, you get a thinner outline.
Please note that in the c++ code below, I've used a naive reconstruction.
Mat im = imread("rsSUY.png", 0);
/* up sample and smooth */
pyrUp(im, im);
GaussianBlur(im, im, Size(5, 5), 5);
/* find the Otsu threshold */
Mat bw1, bw2;
double th = threshold(im, bw1, 0, 255, THRESH_BINARY | THRESH_OTSU);
/* use the found Otsu threshold for Canny */
Canny(im, bw1, th, th/2, 5, true); /* this result would be noisy */
Canny(im, bw2, th, th/2, 3, true); /* this result would be less noisy */
/* link broken edges in more noisy image using hit-miss transform */
Mat k = (Mat_<int>(3, 3) <<
0, 0, 1,
0, -1, 0,
0, 0, 0);
Mat hitmiss;
morphologyEx(bw1, hitmiss, MORPH_HITMISS, k);
bw1 |= hitmiss;
/* apply morphological reconstruction to less noisy image using the modified noisy image */
Mat kernel = getStructuringElement(MORPH_ELLIPSE, Size(3, 3));
double prevMu = 0;
Mat recons = bw2.clone();
for (int i = 0; i < 200; i++)
{
dilate(recons, recons, kernel);
recons &= bw1;
Scalar mu = mean(recons);
if (abs(mu.val[0] - prevMu) < 0.001)
{
break;
}
prevMu = mu.val[0];
}
imshow("less noisy", bw2);
imshow("reconstructed", recons);
waitKey();
The best bet for this task is machine learning. You can:
Crop or mark a few samples for each letter
Train an SSD (Single-shot Multibox Detector) using these samples
The advantage is that you will be able to detect all letters in an image in one pass.
I have a rather simple but not so perfect solution.
1. Finding the optimal higher and lower threshold based on the median of the green channel of the image
Upper threshold image:
Lower threshold image:
2. Subtracting the two images followed by median filtering:
3. Canny edge detection:
To get a better finish you need to follow this up by some morphological operations.
I have two images that I am subtracting from one another quite simply:
Mat foo, a, b;
...//imread onto a and b or somesuch
foo = a - b;
Now, as I understand it, any pixel value that goes into the negatives (or over 255 for that matter) will be set to zero instead. If that is so, I'd like to know if there is any way to permit it to go under zero so that I may adjust the image later without information loss.
I'm working with greyscale images if that simplifies things.
This is how a simple convert => substract => convertAndScaleBack application would look like:
input:
and
int main()
{
cv::Mat input = cv::imread("../inputData/Lenna.png", CV_LOAD_IMAGE_GRAYSCALE);
cv::Mat input2 = cv::imread("../inputData/Lenna_edges.png", CV_LOAD_IMAGE_GRAYSCALE);
cv::Mat input1_16S;
cv::Mat input2_16S;
input.convertTo(input1_16S, CV_16SC1);
input2.convertTo(input2_16S, CV_16SC1);
// compute difference of 16 bit signed images
cv::Mat diffImage = input1_16S-input2_16S;
// now you have a 16S image that has some negative values
// find minimum and maximum values:
double min, max;
cv::minMaxLoc(diffImage, &min, &max);
std::cout << "min pixel value: " << min<< std::endl;
cv::Mat backConverted;
// scale the pixel values so that the smalles value is 0 and the largest one is 255
diffImage.convertTo(backConverted,CV_8UC1, 255.0/(max-min), -min);
cv::imshow("backConverted", backConverted);
cv::waitKey(0);
}
output:
here's my problem: I'm trying to create a simple program which adds Gaussian noise to an input image. The only constraints are that the input image is of type CV_64F (i.e. double) and the values are and must be kept normalized between 0 and 1.
The code I wrote is the following:
Mat my_noise;
my_ noise = Mat (input.size(), input.type());
randn(noise, 0, 5); //mean and variance
input += noise;
The above code doesn't work, the resulting image doesn't get displayed properly. I think that happens because it gets out of the 0,1 range. I modified the code like this:
Mat my_noise;
my_ noise = Mat (input.size(), input.type());
randn(noise, 0, 5); //mean and variance
input += noise;
normalize(input, input, 0.0, 1.0, CV_MINMAX, CV_64F);
but it still doesn't work. Again, the resulting image doesn't get displayed properly. Where is the problem? Remember: the input image is of type CV_64F and the values are normalized between 0 and 1 before adding noise and have to remain like also after the noise addition.
Thank you in advance.
Your problem is that Gaussian noise can have arbitrary amplitude and can't be represented in [0, 1]. Renormalizing after adding the noise is a mistake, because just one large noise value could affect the whole image.
Probably what you need to do is saturate the image when adding the noise, values that would be greater than 1.0 are clamped to 1.0, and values that would be less than 0.0 are clamped to 0.0.
Something like
cv::Mat noise(input.size(), input.type());
cv::randn(noise, 0, 5); //mean and variance
input += noise;
cv::Mat clamp_1 = cv::Mat::ones(input.size(), input.type());
cv::Mat clamp_0 = cv::Mat::zeros(input.size(), input.type());
input = cv::max(input, clamp_0);
input = cv::min(input, clamp_1);
Also a noise variance of 5 is very large, it means that there is about a 92% chance that the input + noise will be outside the range [0, 1], assuming the input is uniformly distributed on [0, 1]. So your saturated image will be mostly black and white, with the input image having little effect on the result.
I have a problem with filling white holes inside a black coin so that I can have only 0-255 binary images with filled black coins. I have used a Median filter to accomplish it but in that case connection bridge between coins grows and it goes impossible to recognize them after several times of erosion... So I need a simple floodFill like method in opencv
Here is my image with holes:
EDIT: floodfill like function must fill holes in big components without prompting X, Y coordinates as a seed...
EDIT: I tried to use the cvDrawContours function but it doesn't fill contours inside bigger ones.
Here is my code:
CvMemStorage mem = cvCreateMemStorage(0);
CvSeq contours = new CvSeq();
CvSeq ptr = new CvSeq();
int sizeofCvContour = Loader.sizeof(CvContour.class);
cvThreshold(gray, gray, 150, 255, CV_THRESH_BINARY_INV);
int numOfContours = cvFindContours(gray, mem, contours, sizeofCvContour, CV_RETR_CCOMP, CV_CHAIN_APPROX_SIMPLE);
System.out.println("The num of contours: "+numOfContours); //prints 87, ok
Random rand = new Random();
for (ptr = contours; ptr != null; ptr = ptr.h_next()) {
Color randomColor = new Color(rand.nextFloat(), rand.nextFloat(), rand.nextFloat());
CvScalar color = CV_RGB( randomColor.getRed(), randomColor.getGreen(), randomColor.getBlue());
cvDrawContours(gray, ptr, color, color, -1, CV_FILLED, 8);
}
CanvasFrame canvas6 = new CanvasFrame("drawContours");
canvas6.showImage(gray);
Result: (you can see black holes inside each coin)
There are two methods to do this:
1) Contour Filling:
First, invert the image, find contours in the image, fill it with black and invert back.
des = cv2.bitwise_not(gray)
contour,hier = cv2.findContours(des,cv2.RETR_CCOMP,cv2.CHAIN_APPROX_SIMPLE)
for cnt in contour:
cv2.drawContours(des,[cnt],0,255,-1)
gray = cv2.bitwise_not(des)
Resulting image:
2) Image Opening:
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(3,3))
res = cv2.morphologyEx(gray,cv2.MORPH_OPEN,kernel)
The resulting image is as follows:
You can see, there is not much difference in both cases.
NB: gray - grayscale image, All codes are in OpenCV-Python
Reference. OpenCV Morphological Transformations
A simple dilate and erode would close the gaps fairly well, I imagine. I think maybe this is what you're looking for.
A more robust solution would be to do an edge detect on the whole image, and then a hough transform for circles. A quick google shows there are code samples available in various languages for size invariant detection of circles using a hough transform, so hopefully that will give you something to go on.
The benefit of using the hough transform is that the algorithm will actually give you an estimate of the size and location of every circle, so you can rebuild an ideal image based on that model. It should also be very robust to overlap, especially considering the quality of the input image here (i.e. less worry about false positives, so can lower the threshold for results).
You might be looking for the Fillhole transformation, an application of morphological image reconstruction.
This transformation will fill the holes in your coins, even though at the cost of also filling all holes between groups of adjacent coins. The Hough space or opening-based solutions suggested by the other posters will probably give you better high-level recognition results.
In case someone is looking for the cpp implementation -
std::vector<std::vector<cv::Point> > contours_vector;
cv::findContours(input_image, contours_vector, CV_RETR_LIST, CV_CHAIN_APPROX_NONE);
cv::Mat contourImage(input_image.size(), CV_8UC1, cv::Scalar(0));
for ( ushort contour_index = 0; contour_index < contours_vector.size(); contour_index++) {
cv::drawContours(contourImage, contours_vector, contour_index, cv::Scalar(255), -1);
}
cv::imshow("con", contourImage);
cv::waitKey(0);
Try using cvFindContours() function. You can use it to find connected components. With the right parameters this function returns a list with the contours of each connected components.
Find the contours which represent a hole. Then use cvDrawContours() to fill up the selected contour by the foreground color thereby closing the holes.
I think if the objects are touched or crowded, there will be some problems using the contours and the math morophology opening.
Instead, the following simple solution is found and tested. It is working very well, and not only for this images, but also for any other images.
here is the steps (optimized) as seen in http://blogs.mathworks.com/steve/2008/08/05/filling-small-holes/
let I: the input image
1. filled_I = floodfill(I). // fill every hole in the image.
2. inverted_I = invert(I)`.
3. holes_I = filled_I AND inverted_I. // finds all holes
4. cc_list = connectedcomponent(holes_I) // list of all connected component in holes_I.
5. holes_I = remove(cc_list,holes_I, smallholes_threshold_size) // remove all holes from holes_I having size > smallholes_threshold_size.
6. out_I = I OR holes_I. // fill only the small holes.
In short, the algorithm is just to find all holes, remove the big ones then write the small ones only on the original image.
I've been looking around the internet to find a proper imfill function (as the one in Matlab) but working in C with OpenCV. After some reaserches, I finally came up with a solution :
IplImage* imfill(IplImage* src)
{
CvScalar white = CV_RGB( 255, 255, 255 );
IplImage* dst = cvCreateImage( cvGetSize(src), 8, 3);
CvMemStorage* storage = cvCreateMemStorage(0);
CvSeq* contour = 0;
cvFindContours(src, storage, &contour, sizeof(CvContour), CV_RETR_CCOMP, CV_CHAIN_APPROX_SIMPLE );
cvZero( dst );
for( ; contour != 0; contour = contour->h_next )
{
cvDrawContours( dst, contour, white, white, 0, CV_FILLED);
}
IplImage* bin_imgFilled = cvCreateImage(cvGetSize(src), 8, 1);
cvInRangeS(dst, white, white, bin_imgFilled);
return bin_imgFilled;
}
For this: Original Binary Image
Result is: Final Binary Image
The trick is in the parameters setting of the cvDrawContours function:
cvDrawContours( dst, contour, white, white, 0, CV_FILLED);
dst = destination image
contour = pointer to the first contour
white = color used to fill the contour
0 = Maximal level for drawn contours. If 0, only contour is drawn
CV_FILLED = Thickness of lines the contours are drawn with. If it is negative (For example, =CV_FILLED), the contour interiors are drawn.
More info in the openCV documentation.
There is probably a way to get "dst" directly as a binary image but I couldn't find how to use the cvDrawContours function with binary values.