I have a fairly blurry 432x432 image of a Sudoku puzzle that doesn't adaptively threshold well (take the mean over a block size of 5x5 pixels, then subtract 2):
As you can see, the digits are slightly distorted, there are a lot of breakages in them, and a few 5s have fused into 6s and 6s into 8s. Also, there's a ton of noise. To fix the noise, I have to make the image even blurrier using a Gaussian blur. However, even a fairly large Gaussian kernel and adaptive threshold blockSize (21x21, subtract 2) fails to remove all the breakages and fuses the digits together even more:
I've also tried dilating the image after thresholding, which has a similar effect to increasing the blockSize; and sharpening the image, which doesn't do much one way or the other. What else should I try?
A pretty good solution is to use morphological closing to make the brightness uniform and then use a regular (non-adaptive) Otsu threshold:
// Divide the image by its morphologically closed counterpart
Mat kernel = Imgproc.getStructuringElement(Imgproc.MORPH_ELLIPSE, new Size(19,19));
Mat closed = new Mat();
Imgproc.morphologyEx(image, closed, Imgproc.MORPH_CLOSE, kernel);
image.convertTo(image, CvType.CV_32F); // divide requires floating-point
Core.divide(image, closed, image, 1, CvType.CV_32F);
Core.normalize(image, image, 0, 255, Core.NORM_MINMAX);
image.convertTo(image, CvType.CV_8UC1); // convert back to unsigned int
// Threshold each block (3x3 grid) of the image separately to
// correct for minor differences in contrast across the image.
for (int i = 0; i < 3; i++) {
for (int j = 0; j < 3; j++) {
Mat block = image.rowRange(144*i, 144*(i+1)).colRange(144*j, 144*(j+1));
Imgproc.threshold(block, block, -1, 255, Imgproc.THRESH_BINARY_INV+Imgproc.THRESH_OTSU);
}
}
Result:
Take a look at Smoothing Images OpenCV tutorial. Except GaussianBlur there are also medianBlur and bilateralFilter which you can also use to reduce noise. I've got this image from your source image (top right):
Update: And the following image I got after removing small contours:
Update: also you can sharpen image (for example, using Laplacian). Look at this discussion.
Always apply gaussian for better results.
cvAdaptiveThreshold(original_image, thresh_image, 255,
CV_ADAPTIVE_THRESH_GAUSSIAN_C, CV_THRESH_BINARY, 11, 2);
Related
I want to extract the darker contours from images with opencv. I have tried using a simple threshold such as below (c++)
cv::threshold(gray, output, threshold, 255, THRESH_BINARY_INV);
I can iterate threshold lets say from 50 ~ 200
then I can get the darker contours in the middle
for images with a clear distinction such as this
here is the result of the threshold
but if the contours near the border, the threshold will fail because the pixel almost the same.
for example like this image.
What i want to ask is there any technique in opencv that can extract darker contour in the middle of image even though the contour reach the border and having almost the same pixel as the border?
(updated)
after threshold darker contour in the middle overlapped with border top.
It makes me fail to extract character such as the first two "SS".
I think you can simply add a edge preserving smoothing step to solve this :
// read input image
Mat inputImg = imread("test2.tif", IMREAD_GRAYSCALE);
Mat filteredImg;
bilateralFilter(inputImg, filteredImg, 5, 60, 20);
// compute laplacian
Mat laplaceImg;
Laplacian(filteredImg, laplaceImg, CV_16S, 1);
// threshold
Mat resImg;
threshold(laplaceImg, resImg, 10, 1, THRESH_BINARY);
// write result
imwrite("res2.tif", resImg);
This will give you the following result : result
Regards,
I think using laplacian could partialy solve your problem :
// read input image
Mat inputImg = imread("test2.tif", IMREAD_GRAYSCALE);
// compute laplacian
Mat laplaceImg;
Laplacian(inputImg, laplaceImg, CV_16S, 1);
Mat resImg;
threshold(laplaceImg, resImg, 30, 1, THRESH_BINARY);
// write result
imwrite("res2.tif", resImg);
Using this code you should obtain something like :
this result
You can then play with final threshold value and with laplacian kernel size.
You will probably have to remove small artifacts after this operation.
Regards
I am trying to find the dominant colors in dresses.
1) First step is to remove the background. I did this using the solution mentioned here. It works perfectly and makes the background black.
2) Now with the result of the first step I am trying to find dominant colors using the solution mentioned here. But I am getting black (the background) as one of the dominant colours.
How can I ignore the background pixels in step 2?
Depending on the case, you could find the bounding rectangle of the region that you're interested in. If the number of color pixels is much higher than the number of black pixels inside that bounding rectangle, black shouldn't be detected as the dominant color.
Call findContours(binaryMask) on the binary image of your mask. Make sure you found just the contour you were looking for. If not, filter them to get the best one for the application. Then call boundingRect(cnt) on the contour. Then crop the image using that rectangle and run your function. If that's insufficient, try minAreaRect(cnt), but the cropping is a bit trickier: see this answer.
If that doesn't work, I'd probably go for the "dumb" solution, by changing the color of the mask to a color that will for 99% not appear on a dress and then - knowing it exact RGB values - filter it out from the results.
Next time please remember to provide an image of your case, so the answers may be more accurate.
One easy way to do it would be to simply discard black as a dominant colour. Grab one more cluster than you really want, ignore black. If black may genuinely be the dominant colour, repeat the operation with a different background colour and discard that; compare results. This would be slow, but simple to do.
Alternatively, you could only sample from pixels in your foreground. From your foreground extraction method, you should have a binary black and white foreground/background mask. If you only sample from white areas of the mask, then only these colours should be taken into consideration.
I have a rough C++ implementation of this, but it's almost certainly not the most efficient possible. Maybe it's a start you could work from?
Mat src; //Your source image
Mat mask; //Your black & white foreground/background image
Mat samples(src.rows * src.cols, 3, CV_32F);
//Set up samples with only foreground pixels
for (int y = 0; y < src.rows; y++) {
for (int x = 0; x < src.cols; x++) {
if (mask.at<uchar>(y, x) == 255) {
for (int z = 0; z < 3; z++) {
samples.at<float>(y + x*src.rows, z) = src.at<Vec3b>(y, x)[z];
}
}
}
}
int clusterNo = 3;
int attempts = 5;
Mat labels;
Mat centers;
kmeans(samples, clusterNo, labels, TermCriteria(), attempts, KMEANS_RANDOM_CENTERS, centers);
Your dominant colours will be stored in the rows of centres, where you can do what you want with them.
Remove the background. That gives you a binary image - foreground and background pixels. Now do a morphological closing to close up little holes in foreground images and generally clean up the contours. Finally substitute pixels back in again to get a colour foreground image.
I have the following image.
this image
I would like to remove the orange boxes/rectangle around numbers and keep the original image clean without any orange grid/rectangle.
Below is my current code but it does not remove it.
Mat mask = new Mat();
Mat src = new Mat();
src = Imgcodecs.imread("enveloppe.jpg",Imgcodecs.CV_LOAD_IMAGE_COLOR);
Imgproc.cvtColor(src, hsvMat, Imgproc.COLOR_BGR2HSV);
Scalar lowerThreshold = new Scalar(0, 50, 50);
Scalar upperThreshold = new Scalar(25, 255, 255);
Mat mask = new Mat();
Core.inRange(hsvMat, lowerThreshold, upperThreshold, mask);
//src.setTo(new scalar(255,255,255),mask);
what to do next ?
How can i remove the orange boxes/rectangle from the original images ?
Update:
For information , the mask contains exactly all the boxes/rectangle that i want to remove. I don't know how to use this mask to remove boxes/rectangle from the source (src) image as if they were not present.
This is what I did to solve the problem. I solved the problem in C++ and I used OpenCV.
Part 1: Find box candidates
Firstly I wanted to isolate the signal that was specific for red channel. I splitted the image into three channels. I then subtracted the red channel from blue channel and the red from green channel. After that I subtracted both previous subtraction results from one another. The final subtraction result is shown on the image below.
using namespace cv;
using namespace std;
Mat src_rgb = imread("image.jpg");
std::vector<Mat> channels;
split(src_rgb, channels);
Mat diff_rb, diff_rg;
subtract(channels[2], channels[0], diff_rb);
subtract(channels[2], channels[1], diff_rg);
Mat diff;
subtract(diff_rb, diff_rg, diff);
My next goal was to divide the parts of obtained image into separate "groups". To do that, I smoothed the image a little bit with a Gaussian filter. Then I applied a threshold to obtain a binary image; finally I looked for external contours within that image.
GaussianBlur(diff, diff, cv::Size(11, 11), 2.0, 2.0);
threshold(diff, diff, 5, 255, THRESH_BINARY);
vector<vector<Point>> contours;
findContours(diff, contours, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_NONE);
Click to see subtraction result, Gaussian blurred image, thresholded image and detected contours.
Part 2: Inspect box candidates
After that, I had to make an estimate whether the interior of each contour contained a number or something else. I made an assumption that numbers will always be printed with black ink and that they will have sharp edges. Therefore I took a blue channel image and I applied just a little bit of Gaussian smoothing and convolved it with a Laplacian operator.
Mat blurred_ch2;
GaussianBlur(channels[2], blurred_ch2, cv::Size(7, 7), 1, 1);
Mat laplace_result;
Laplacian(blurred_ch2, laplace_result, -1, 1);
I then took the resulting image and applied the following procedure for every contour separately. I computed a standard deviation of the pixel values within the contour interior. Standard deviation was high inside the contours that surrounded numbers; and it was low inside the two contours that surrounded the dog's head and the letters on top of the stamp.
That is why I could appliy the standard deviation threshold. Standard deviation was approx. twice larger for contours containing numbers so this was an easy way to only select the contours that contained numbers. Then I drew the contour interior mask. I used erosion and subtraction to obtain the "box edge mask".
The final step was fairly easy. I computed an estimate of average pixel value nearby the box on every channel of the image. Then I changed all pixel values under the "box edge mask" to those values on every channel. After I repeated that procedure for every box contour, I merged all three channels into one.
Mat mask(src_rgb.size(), CV_8UC1);
for (int i = 0; i < contours.size(); ++i)
{
mask.setTo(0);
drawContours(mask, contours, i, cv::Scalar(200), -1);
Scalar mean, stdev;
meanStdDev(laplace_result, mean, stdev, mask);
if (stdev.val[0] < 10.0) continue;
Mat eroded;
erode(mask, eroded, cv::Mat(), cv::Point(-1, -1), 6);
subtract(mask, eroded, mask);
for (int c = 0; c < src_rgb.channels(); ++c)
{
erode(mask, eroded, cv::Mat());
subtract(mask, eroded, eroded);
Scalar mean, stdev;
meanStdDev(channels[c], mean, stdev, eroded);
channels[c].setTo(mean, mask);
}
}
Mat final_result;
merge(channels, final_result);
imshow("Final Result", final_result);
Click to see red channel of the image, the result of convolution with Laplacian operator, drawn mask of the box edges and the final result.
Please note
This code is far from being optimal, especially the last loop does quite a lot of unnecessary work. But I think that in this case readability is more important (and the author of the question did not request an optimized solution anyway).
Looking towards more general solution
After I posted the initial reply, the author of the question noted that the digits can be of any color and their edges are not necessarily sharp. That means that above procedure can fail because of various reasons. I altered the input image so that it contains different kinds of numbers (click to see the image) and you can run my algorithm on this input and analyze what goes wrong.
The way I see it, one of these approaches is needed (or perhaps a mixture of both) to obtain a more "general" solution:
concentrate only on rectangle shape and color (confirm that the box candidate is really an orange box and remove it regardless of what is inside)
concentrate on numbers only (run a proper number detection algorithm inside the interior of every box candidate; if it contains a single number, remove the box)
I will give a trivial example of the first approach. If you can assume that orange box size will always be the same, just check the box size instead of standard deviation of the signal in the last loop of the algorithm:
Rect rect = boundingRect(contours[i]);
float area = rect.area();
if (area < 1000 || area > 1200) continue;
Warning: actual area of rectangles is around 600Px^2, but I took into account the Gaussian Blurring, which caused the contour to expand. Please also note that if you use this approach you no longer need to perform blurring or laplace operations on blue channel image.
You can also add other simple constraints to that condition; ratio between width and height is the first one that comes to my mind. Geometric properties can also be a good option (right angles, straight edges, convexness ...).
Let say that we have a gray-scale image. Is there a way to calculate how the non-black pixels are distributed, i.e. whether they are grouped at one or several places or they are distributed uniformly in the whole image?
Sounds like you are looking for the histogram of an image. It is a fundamental operation in image processing.
"Histograms are collected counts of data organized into a set of predefined bins."
The document of histogram computation using OpenCV is in this link
It sounds like you're looking for is the spacial moments of a rasterized version of your image.
First you need to threshold your image to make it binary:
http://docs.opencv.org/2.4/modules/imgproc/doc/miscellaneous_transformations.html?highlight=threshold#threshold
You can then calculate the image moments:
http://docs.opencv.org/2.4/modules/imgproc/doc/structural_analysis_and_shape_descriptors.html?highlight=moments#moments
If you'd like a physical analogy of the spacial moments you can imagine that each white pixels is a unit point mass, then the second moment would be the rotational inertia of the image. If the white pixels (point masses) are tightly clustered then the second moment will be low (image will rotate easily).
I want to share another approach that I have used.
Mat img = imread(argv[1], CV_LOAD_IMAGE_COLOR);
cvtColor(img, img, CV_RGB2GRAY);
threshold(img, img, 35, 255, THRESH_BINARY);
Mat distance;
distanceTransform(img, distance, CV_DIST_L2, 3);
distance = min(distance, 1);
Scalar distribution = mean(dist);
cout << "The distribution is: " << distribution << std::endl;
The tricky part is the combination of distanceTransform and min functions. The effect of the min function will be smaller on images with good distribution and the mean value will be greater.
I have the following image:
And I'd like to obtain a thresholded image where only the tape is white, and the whole background is black.. so far I've tried this:
Mat image = Highgui.imread("C:/bezier/0.JPG");
Mat byn = new Mat();
Imgproc.cvtColor(image, byn, Imgproc.COLOR_BGR2GRAY);
Mat thresh = new Mat();
// apply filters
Imgproc.blur(byn, byn, new Size(2, 2));
Imgproc.threshold(byn, thresh, 0, 255, Imgproc.THRESH_BINARY+Imgproc.THRESH_OTSU);
Imgproc.erode(thresh, thresh, Imgproc.getStructuringElement(Imgproc.MORPH_RECT, new Size(4, 4)));
But I obtain this image, that is far away from what I want:
The tape would be always of the same color (white) and width (about 2cm), any idea? Thanks
Let's see what you know:
The tape has a lower contrast
The tape is lighter than the background
If you know the scale of the picture, you can run adaptive thresholds on two levels. Let's say that the width of the tape is 100 pixels:
Reject a pixel that has brightness outside of +/- x from the average brightness in the 50x50 (maybe smaller, but not larger) window surrounding it AND
Reject a pixel that has brightness smaller than y + the average brightness in the 100x100(maybe larger, but not smaller) window surrounding it.
You should also experiment a bit, trying both mean and median as definitions of "average" for each threshold.
From there on you should have a much better-defined image, and you can remove all but the largest contour (presumably the trail)
I think you are not taking advantage of the fact that the tape is white (and the floor is in a shade of brown).
Rather than converting to grayscale with cvtColor(src, dst, Imgproc.COLOR_BGR2GRAY) try using a custom operation that penalizes saturation... Maybe something like converting to HSV and let G = V * (1-S).