A doubt came to my mind this morning: does the findChessboardCorners OpenCV function work with a chessboard of different colours, for example blue?
If it's not the case, do you think that a quite straightforward thresholding would do the trick?
You can't pass coloured images to the findChessboardCorners because it only takes a greyscale image as #api55 pointed out in his comment.
You might be worth taking a look at the checkchessboard code provided here
// does a fast check if a chessboard is in the input image. This is a workaround to
// a problem of cvFindChessboardCorners being slow on images with no chessboard
// - src: input binary image
// - size: chessboard size
// Returns 1 if a chessboard can be in this image and findChessboardCorners should be called,
// 0 if there is no chessboard, -1 in case of error
int checkChessboardBinary(const cv::Mat & img, const cv::Size & size)
{
CV_Assert(img.channels() == 1 && img.depth() == CV_8U);
Mat white = img.clone();
Mat black = img.clone();
int result = 0;
for ( int erosion_count = 0; erosion_count <= 3; erosion_count++ )
{
if ( 1 == result )
break;
if ( 0 != erosion_count ) // first iteration keeps original images
{
erode(white, white, Mat(), Point(-1, -1), 1);
dilate(black, black, Mat(), Point(-1, -1), 1);
}
vector<pair<float, int> > quads;
fillQuads(white, black, 128, 128, quads);
if (checkQuads(quads, size))
result = 1;
}
return result;
}
With the main loop being:
CV_IMPL
int cvFindChessboardCorners( const void* arr, CvSize pattern_size,
CvPoint2D32f* out_corners, int* out_corner_count,
int flags )
is the main implementation of this method. In here they
Use cvCheckChessboard to determine if a chessboard is in the image
Convert to binary (B&W) and dilate to split the corners apart Use
icvGenerateQuads to find the squares.
So in answer to your question, as long as there is sufficient contrast in your image after you convert it to greyscale it will likely work, I would imagine a greyscaled blue and white image would be good enough, if it was a light aqua or yellow or something you might struggle without more processing
I have image with white uneven background (due to lighting). I'm trying to estimate background color and transform image into image with true white background. For this I estimated white color for each 15x15 pixels block based on its luminosity. So I've got the following map (on the right):
Now I want to interpolate color so it will be more smooth transition from 15x15 block to neighboring block, plus I want it to eliminate outliers (pink dots on left hand side). Could anyone suggest good technique/algorithm for this? (Ideally within OpenCV library, but not necessary)
Starting from this image:
You could find the text on the whiteboard as the parts of your images that have a high gradient, and apply a little dilation to deal with thick parts of the text. You'll get a mask that separates background from foreground pretty well:
Background:
Foreground:
You can then apply inpainting using the computed mask on the original image (you need OpenCV contrib module photo):
Just to show that this works independently of the text color, I tried on a different image:
Resulting in:
Code:
#include <opencv2/opencv.hpp>
#include <opencv2/photo.hpp>
using namespace cv;
void findText(const Mat3b& src, Mat1b& mask)
{
// Convert to grayscale
Mat1b gray;
cvtColor(src, gray, COLOR_BGR2GRAY);
// Compute gradient magnitude
Mat1f dx, dy, mag;
Sobel(gray, dx, CV_32F, 1, 0);
Sobel(gray, dy, CV_32F, 0, 1);
magnitude(dx, dy, mag);
// Remove low magnitude, keep only text
mask = mag > 10;
// Apply a dilation to deal with thick text
Mat1b K = getStructuringElement(MORPH_ELLIPSE, Size(3, 3));
dilate(mask, mask, K);
}
int main(int argc, const char * argv[])
{
Mat3b img = imread("path_to_image");
// Segment white
Mat1b mask;
findText(img, mask);
// Show intermediate images
Mat3b background = img.clone();
background.setTo(0, mask);
Mat3b foreground = img.clone();
foreground.setTo(0, ~mask);
// Apply inpainting
Mat3b inpainted;
inpaint(img, mask, inpainted, 21, CV_INPAINT_TELEA);
imshow("Original", img);
imshow("Foreground", foreground);
imshow("Background", background);
imshow("Inpainted", inpainted);
waitKey();
return 0;
}
We're currently trying to detect the object regions in medical instruments images using the methods available in OpenCV, C++ version. An example image is shown below:
Here are the steps we're following:
Converting the image to gray scale
Applying median filter
Find edges using sobel filter
Convert the result to binary image using a threshold of 25
Skeletonize the image to make sure we have neat edges
Finding X largest connected components
This approach works perfectly for the image 1 and here is the result:
The yellow borders are the connected components detected.
The rectangles are just to highlight the presence of a connected component.
To get understandable results, we just removed the connected components that are completely inside any another one, so the end result is something like this:
So far, everything was fine but another sample of image complicated our work shown below.
Having a small light green towel under the objects results this image:
After filtering the regions as we did earlier, we got this:
Obviously, it is not what we need..we're excepting something like this:
I'm thinking about clustering the closest connected components found(somehow!!) so we can minimize the impact of the presence of the towel, but don't know yet if it's something doable or someone has tried something like this before? Also, does anyone have any better idea to overcome this kind of problems?
Thanks in advance.
Here's what I tried.
In the images, the background is mostly greenish and the area of the background is considerably larger than that of the foreground. So, if you take a color histogram of the image, the greenish bins will have higher values. Threshold this histogram so that bins having smaller values are set to zero. This way we'll most probably retain the greenish (higher value) bins and discard other colors. Then backproject this histogram. The backprojection will highlight these greenish regions in the image.
Backprojection:
Then threshold this backprojection. This gives us the background.
Background (after some morphological filtering):
Invert the background to get foreground.
Foreground (after some morphological filtering):
Then find the contours of the foreground.
I think this gives a reasonable segmentation, and using this as mask you may be able to use a segmentation like GrabCut to refine the boundaries (I haven't tried this yet).
EDIT:
I tried the GrabCut approach and it indeed refines the boundaries. I've added the code for GrabCut segmentation.
Contours:
GrabCut segmentation using the foreground as mask:
I'm using the OpenCV C API for the histogram processing part.
// load the color image
IplImage* im = cvLoadImage("bFly6.jpg");
// get the color histogram
IplImage* im32f = cvCreateImage(cvGetSize(im), IPL_DEPTH_32F, 3);
cvConvertScale(im, im32f);
int channels[] = {0, 1, 2};
int histSize[] = {32, 32, 32};
float rgbRange[] = {0, 256};
float* ranges[] = {rgbRange, rgbRange, rgbRange};
CvHistogram* hist = cvCreateHist(3, histSize, CV_HIST_ARRAY, ranges);
IplImage* b = cvCreateImage(cvGetSize(im32f), IPL_DEPTH_32F, 1);
IplImage* g = cvCreateImage(cvGetSize(im32f), IPL_DEPTH_32F, 1);
IplImage* r = cvCreateImage(cvGetSize(im32f), IPL_DEPTH_32F, 1);
IplImage* backproject32f = cvCreateImage(cvGetSize(im), IPL_DEPTH_32F, 1);
IplImage* backproject8u = cvCreateImage(cvGetSize(im), IPL_DEPTH_8U, 1);
IplImage* bw = cvCreateImage(cvGetSize(im), IPL_DEPTH_8U, 1);
IplConvKernel* kernel = cvCreateStructuringElementEx(3, 3, 1, 1, MORPH_ELLIPSE);
cvSplit(im32f, b, g, r, NULL);
IplImage* planes[] = {b, g, r};
cvCalcHist(planes, hist);
// find min and max values of histogram bins
float minval, maxval;
cvGetMinMaxHistValue(hist, &minval, &maxval);
// threshold the histogram. this sets the bin values that are below the threshold to zero
cvThreshHist(hist, maxval/32);
// backproject the thresholded histogram. backprojection should contain higher values for the
// background and lower values for the foreground
cvCalcBackProject(planes, backproject32f, hist);
// convert to 8u type
double min, max;
cvMinMaxLoc(backproject32f, &min, &max);
cvConvertScale(backproject32f, backproject8u, 255.0 / max);
// threshold backprojected image. this gives us the background
cvThreshold(backproject8u, bw, 10, 255, CV_THRESH_BINARY);
// some morphology on background
cvDilate(bw, bw, kernel, 1);
cvMorphologyEx(bw, bw, NULL, kernel, MORPH_CLOSE, 2);
// get the foreground
cvSubRS(bw, cvScalar(255, 255, 255), bw);
cvMorphologyEx(bw, bw, NULL, kernel, MORPH_OPEN, 2);
cvErode(bw, bw, kernel, 1);
// find contours of the foreground
//CvMemStorage* storage = cvCreateMemStorage(0);
//CvSeq* contours = 0;
//cvFindContours(bw, storage, &contours);
//cvDrawContours(im, contours, CV_RGB(255, 0, 0), CV_RGB(0, 0, 255), 1, 2);
// grabcut
Mat color(im);
Mat fg(bw);
Mat mask(bw->height, bw->width, CV_8U);
mask.setTo(GC_PR_BGD);
mask.setTo(GC_PR_FGD, fg);
Mat bgdModel, fgdModel;
grabCut(color, mask, Rect(), bgdModel, fgdModel, GC_INIT_WITH_MASK);
Mat gcfg = mask == GC_PR_FGD;
vector<vector<cv::Point>> contours;
vector<Vec4i> hierarchy;
findContours(gcfg, contours, hierarchy, CV_RETR_LIST, CV_CHAIN_APPROX_SIMPLE, cv::Point(0, 0));
for(int idx = 0; idx < contours.size(); idx++)
{
drawContours(color, contours, idx, Scalar(0, 0, 255), 2);
}
// cleanup ...
UPDATE: We can do the above using the C++ interface as shown below.
const int channels[] = {0, 1, 2};
const int histSize[] = {32, 32, 32};
const float rgbRange[] = {0, 256};
const float* ranges[] = {rgbRange, rgbRange, rgbRange};
Mat hist;
Mat im32fc3, backpr32f, backpr8u, backprBw, kernel;
Mat im = imread("bFly6.jpg");
im.convertTo(im32fc3, CV_32FC3);
calcHist(&im32fc3, 1, channels, Mat(), hist, 3, histSize, ranges, true, false);
calcBackProject(&im32fc3, 1, channels, hist, backpr32f, ranges);
double minval, maxval;
minMaxIdx(backpr32f, &minval, &maxval);
threshold(backpr32f, backpr32f, maxval/32, 255, THRESH_TOZERO);
backpr32f.convertTo(backpr8u, CV_8U, 255.0/maxval);
threshold(backpr8u, backprBw, 10, 255, THRESH_BINARY);
kernel = getStructuringElement(MORPH_ELLIPSE, Size(3, 3));
dilate(backprBw, backprBw, kernel);
morphologyEx(backprBw, backprBw, MORPH_CLOSE, kernel, Point(-1, -1), 2);
backprBw = 255 - backprBw;
morphologyEx(backprBw, backprBw, MORPH_OPEN, kernel, Point(-1, -1), 2);
erode(backprBw, backprBw, kernel);
Mat mask(backpr8u.rows, backpr8u.cols, CV_8U);
mask.setTo(GC_PR_BGD);
mask.setTo(GC_PR_FGD, backprBw);
Mat bgdModel, fgdModel;
grabCut(im, mask, Rect(), bgdModel, fgdModel, GC_INIT_WITH_MASK);
Mat fg = mask == GC_PR_FGD;
I would consider a few options. My assumption is that the camera does not move. I haven't used the images or written any code, so this is mostly from experience.
Rather than just looking for edges, try separating the background using a segmentation algorithm. Mixture of Gaussian can help with this. Given a set of images over the same region (i.e. video), you can cancel out regions which are persistent. Then, new items such as instruments will pop out. Connected components can then be used on the blobs.
I would look at segmentation algorithms to see if you can optimize the conditions to make this work for you. One major item is to make sure your camera is stable or you stabilize the images yourself pre-processing.
I would consider using interest points to identify regions in the image with a lot of new material. Given that the background is relatively plain, small objects such as needles will create a bunch of interest points. The towel should be much more sparse. Perhaps overlaying the detected interest points over the connected component footprint will give you a "density" metric which you can then threshold. If the connected component has a large ratio of interest points for the area of the item, then it is an interesting object.
On this note, you can even clean up the connected component footprint by using a Convex Hull to prune the objects you have detected. This may help situations such as a medical instrument casting a shadow on the towel which stretches the component region. This is a guess, but interest points can definitely give you more information than just edges.
Finally, given that you have a stable background with clear objects in view, I would take a look at Bag-of-Features to see if you can just detect each individual object in the image. This may be useful since there seems to be a consistent pattern to the objects in these images. You can build a big database of images such as needles, gauze, scissors, etc. Then BoF, which is in OpenCV will find those candidates for you. You can also mix it in with other operations you are doing to compare results.
Bag of Features using OpenCV
http://www.codeproject.com/Articles/619039/Bag-of-Features-Descriptor-on-SIFT-Features-with-O
-
I would also suggest an idea to your initial version. You can also skip the contours, whose regions have width and height greater than the half the image width and height.
//take the rect of the contours
Rect rect = Imgproc.boundingRect(contours.get(i));
if (rect.width < inputImageWidth / 2 && rect.height < inputImageHeight / 2)
//then continue to draw or use for next purposes.
I have written a C++ program using OpenCV that can detect and highlight the edges of any object from a live video. But now I don't know how to extract the four corners of the cube from the many edges that are being detected in the video. So I am looking for some help here.
Here is the link of the paper that I am using as a guide for my this project.
http://www.cs.ubc.ca/~andrejk/525project/525report.pdf
You can find the program code for this paper in the link below. It's written in Python. (I am using C++ and I don't know Python)
http://www.cs.ubc.ca/~andrejk/525project/cubefinder.py
According to the paper the next step would be, 'edge segmentation with adaptive threshold.'
Which I don't really understand. And also I don't know how to extract the corners of the cube then.
The short summary of the method that I have used is as following.
1. Input from webcam
2. Apply Laplacian filter
3. Apply Hough Line Transform.
I get the following result.
Code
using namespace std;
using namespace cv;
Mat laplacianFilter(Mat image)
{
Mat hImage;
GaussianBlur(image,hImage,Size(3,3),0,0,BORDER_DEFAULT);
cvtColor(hImage,hImage,CV_RGB2GRAY);
Laplacian(hImage,hImage,CV_16SC1,3,1,0,BORDER_DEFAULT);
convertScaleAbs(hImage,hImage,1,0);
return hImage;
}
Mat hghTransform(Mat image, Mat &image2)
{
Mat lImage;
Canny(image,image,50,200,3);
cvtColor(image,lImage,CV_GRAY2BGR);
vector<Vec4i> lines;
HoughLinesP(image, lines, 1, CV_PI/180, 50, 50, 10 );
for( size_t i = 0; i < lines.size(); i++ )
{
Vec4i l = lines[i];
line( image2, Point(l[0], l[1]), Point(l[2], l[3]), Scalar(0,255,0), 3, CV_AA);
}
return lImage;
}
int main()
{
int c;
VideoCapture cap(0);
Mat image;
Mat image2;
namedWindow("hghtransform");
namedWindow("laplacianfilter");
namedWindow("cannyOutput");
while(1)
{
cap>>image;
cap>>image2;
//Output
imshow("laplacianfilter",laplacianFilter(image));
imshow("cannyOutput",hghTransform(laplacianFilter(image),image2));
imshow("hghtransform",image2);
c=waitKey(33);
if(c==27)
return 0;
}
return 0;
}
Adaptive threshold will give you a clear line of edges which enables you to get 9 squares of a rubik side properly.
You can see a decent comparison of global and adaptive threshold here:
here: https://sites.google.com/site/qingzongtseng/adaptivethreshold
original image:
global threshold:
adaptive threshold:
For the corner, I am not sure whether it's stated in the paper, but I would do something like:
==> finding area like 1, 2, 3, 4 for upper-left, upper-right, lower-left, and lower-right corner respectively
==> with a template matching algorithm.
hope it helps.
note: you might want to have a background with less noise there. =)
I have an Image that I would like to zoom into and view at high detail. It is of unknown size and mostly black and white with some text on it. When I zoom in the text becomes unreadable and I thought it was to do with not having enough pixels/texels to display so I upscaled the image by a factor of 2. Now that I have scaled it, it is still unreadable.
Then I started to use OpenCV with :
void resizeFirst(){
Mat src = imread( "../floor.png", 1 );
Mat tmp;
resize(src,tmp,Size(),3,3,INTER_CUBIC);
Mat dst = tmp.clone();
Size s(3,3);
//blur( tmp, dst, s, Point(-1,-1) );//homogeneous
GaussianBlur(tmp, dst, s, 3);//gaussian
//medianBlur ( tmp, dst, 5 );//median
//bilateralFilter ( tmp, dst, 5, 5*2, 5/2 );
addWeighted(tmp, 1.5, dst, -0.5, 0, dst);
imwrite("sharpenedImage.png",dst);
}
void blurFirst(){
Mat src = imread( "../floor.png", 1 );
Size s(3,3);
Mat dst;
GaussianBlur(src, dst, s, 3);//gaussian
addWeighted(src, 2, dst, -1, 0, dst);
Mat tmp;
resize(dst,tmp,Size(),3,3,INTER_CUBIC);
imwrite("sharpenedImage0.png",tmp);
}
and the output is better but the image still isnt sharp. Does anyone have any ideas on how to keep text sharp when zooming into an image?
EDIT: below are sample images.
The first one is the smaller res original and the second I resized and tried to do gaussian sharpening as per below
Resize function offers different interpolation methods
INTER_NEAREST nearest-neighbor interpolation
INTER_LINEAR bilinear interpolation (used by default)
INTER_AREA resampling using pixel area relation. It may be the preferred method for image decimation, as it gives moire-free results. But when the image is zoomed, it is similar to the INTER_NEAREST method
INTER_CUBIC bicubic interpolation over 4x4 pixel neighborhood
INTER_LANCZOS4 Lanczos interpolation over 8x8 pixel neighborhood
try all the interpolation methods and use the one that suits you the most. The resize function will however change the aspect ratio of your image.