OpenCV camera calibration with chessboard of different colours - opencv

A doubt came to my mind this morning: does the findChessboardCorners OpenCV function work with a chessboard of different colours, for example blue?
If it's not the case, do you think that a quite straightforward thresholding would do the trick?

You can't pass coloured images to the findChessboardCorners because it only takes a greyscale image as #api55 pointed out in his comment.
You might be worth taking a look at the checkchessboard code provided here
// does a fast check if a chessboard is in the input image. This is a workaround to
// a problem of cvFindChessboardCorners being slow on images with no chessboard
// - src: input binary image
// - size: chessboard size
// Returns 1 if a chessboard can be in this image and findChessboardCorners should be called,
// 0 if there is no chessboard, -1 in case of error
int checkChessboardBinary(const cv::Mat & img, const cv::Size & size)
{
CV_Assert(img.channels() == 1 && img.depth() == CV_8U);
Mat white = img.clone();
Mat black = img.clone();
int result = 0;
for ( int erosion_count = 0; erosion_count <= 3; erosion_count++ )
{
if ( 1 == result )
break;
if ( 0 != erosion_count ) // first iteration keeps original images
{
erode(white, white, Mat(), Point(-1, -1), 1);
dilate(black, black, Mat(), Point(-1, -1), 1);
}
vector<pair<float, int> > quads;
fillQuads(white, black, 128, 128, quads);
if (checkQuads(quads, size))
result = 1;
}
return result;
}
With the main loop being:
CV_IMPL
int cvFindChessboardCorners( const void* arr, CvSize pattern_size,
CvPoint2D32f* out_corners, int* out_corner_count,
int flags )
is the main implementation of this method. In here they
Use cvCheckChessboard to determine if a chessboard is in the image
Convert to binary (B&W) and dilate to split the corners apart Use
icvGenerateQuads to find the squares.
So in answer to your question, as long as there is sufficient contrast in your image after you convert it to greyscale it will likely work, I would imagine a greyscaled blue and white image would be good enough, if it was a light aqua or yellow or something you might struggle without more processing

Related

how to get Matrix(ROI) in openCV using by line that inclined

I already tried to search about openCV ROI function, but All of them used rectangle roi function.
I want to get roi using by inclined line that get from hough transform function.
My situation is next :
I have multiple vertical lines(little inclined) that output from hough transform function.
i want to get image(Matrix) between vertical lines.
enter image description here
i want to get divided matrix in my image (For example, A image, B image, C image etc.. )
Is there ROI function that used line in openCV?
or
any another method?
I think you need to use contours to define your roi. If it is not a perfect square you can not use the ROI function, because this is always a perfect square (not even a rotated square)
int main()
{
enum hierIdx { H_NEXT = 0, H_PREVIOUS, H_FIRST_CHILD, H_PARENT };
cv::Mat img = cv::imread("example_image.jpg", cv::IMREAD_UNCHANGED);
// convert RGB to gray scale image
cv::Mat imgGrs;
cv::cvtColor(img, imgGrs, cv::COLOR_RGB2GRAY);
// because it was a .jpg the grey values are messed up
// we fix it by thresholding at 128
cv::threshold(imgGrs, imgGrs, 128, 255, cv::THRESH_BINARY);
imgGrs = ~imgGrs;
// now create contours (we need the hierarchy to find the inner shapes)
std::vector<std::vector<cv::Point> > contours;
std::vector<cv::Vec4i> hierarchy;
cv::findContours(imgGrs.clone(), contours, hierarchy, CV_RETR_TREE, CV_CHAIN_APPROX_SIMPLE);
//cv::drawContours(img, contours, -1, cv::Scalar(255, 0, 0), 1);
int iLen = (int)hierarchy.size();
int idxChild = -1;
// find first child of master
for (int i = 0; i < iLen; i++){
if (hierarchy[i][H_PARENT] < 0) {
idxChild = hierarchy[i][H_FIRST_CHILD];
break;
}
}
// used for erosion of mask
cv::Mat element = cv::getStructuringElement(cv::MORPH_ELLIPSE, cv::Size(3, 3));
while (idxChild >= 0)
{
// create image to use as mask for section
cv::Mat mask = cv::Mat::zeros(imgGrs.size(), CV_8U);
cv::drawContours(mask, contours, idxChild, cv::Scalar(255), CV_FILLED);
// make masker 1 pixel smaller so we wont see the outer contours
cv::erode(mask, mask, element);
// ok nu we create a singled out part we want
cv::Mat part = imgGrs & mask;
// Crop it to the AOI rectangle
cv::Rect aoi = cv::boundingRect(contours[idxChild]);
part = part(aoi);
// part is now the aoi image you asked for
// proceed to next AOI
idxChild = hierarchy[idxChild][H_NEXT];
}
return 0;
}

Removing paper folding artifacts

I need to recognize some handwriting on text written with soft pen. Using OpenCV, different thresholding methods, bilateral filtering etc, I get quite good results extracting text from paper. But I also get artifacts from folding:
I cannot change way how paper is handled or photographed before it will be processed. After thresholding same paper looks like this:
I want to remove these artifacts. Biggest trouble for me is situation when some character like "T" happens to be on this line. Horizontal part of "T" may nicely fit to this line.
What I do now: I can detect if there is a standalone line. If something is few pixels tall and very wide, I eliminate it.
I have been reading a lot of information about shadow elimination (because I assume problem is shadow). But they all expect to work in other context - surveillance video feed or image with color background.
Any ideas?
UPDATE:
Was working on ideas based on similar works: http://ivrgwww.epfl.ch/alumni/fredemba/papers/FFICPR06.pdf
Test input
Output of test code:
Source code:
#include "opencv2/opencv.hpp"
using namespace std;
using namespace cv;
int filt1_trackbar=13;
int filt2_trackbar=49;
int filt3_trackbar=6;
int main( int argc, char** argv ) {
Mat src, shadow;
src = imread( argv[1], 1 );
if( !src.data ) {
return -1;
}
Mat histImage1( src.rows, src.cols, CV_8UC3, Scalar(127,127,127) );
Mat histImage2( src.rows, src.cols, CV_8UC3, Scalar(127,127,127) );
int cn = src.channels();
uint8_t* pixelPtr = (uint8_t*)src.data;
for(int i=0 ; i< src.rows;i++) {
for(int j=0 ; j< src.cols;j++) {
Scalar_<uint8_t> bgrPixel;
bgrPixel.val[0] = pixelPtr[i*src.cols*cn + j*cn + 0]; // B
bgrPixel.val[1] = pixelPtr[i*src.cols*cn + j*cn + 1]; // G
bgrPixel.val[2] = pixelPtr[i*src.cols*cn + j*cn + 2]; // R
if(bgrPixel.val[2] !=0 ) { // avoid division by zero
float a= 100.0*(((float)bgrPixel.val[0] / (float)bgrPixel.val[2])); // B/R
float b= 100.0*(((float)bgrPixel.val[1] / (float)bgrPixel.val[2])); // G/R
if(!isinf(a) && !isinf(b)) {
histImage1.at<Vec3b>(i,j)=Vec3b(a,a,a);
histImage2.at<Vec3b>(i,j)=Vec3b(b,b,b);
}
}
}
}
addWeighted(histImage1, 2.0, histImage2, -1.0, 0, shadow);
Mat hsv1,hsv2;
cvtColor(shadow, hsv1, CV_BGR2HSV);
cvtColor(src, hsv2, CV_BGR2HSV);
vector<Mat> channels1;
vector<Mat> channels2;
split(hsv1, channels1);
split(hsv2, channels2);
addWeighted(channels1[2], 0.5, channels2[2], 0.5, 0, channels1[2]);
insertChannel(channels1[2],hsv2,2);
Mat unshadow;
cvtColor(hsv2,unshadow, CV_HSV2BGR);
namedWindow( "src", WINDOW_NORMAL);
namedWindow( "shadow", WINDOW_NORMAL);
namedWindow( "unshadow", WINDOW_NORMAL);
imshow("src", src);
imshow("shadow", shadow);
imshow("unshadow", unshadow);
imwrite("shadow.png", shadow);
imwrite("unshadow.png", unshadow);
waitKey(0);
return 0;
}
It did improve image but not good enough in my opinion. I was impressed it worked at all on such grayscale context. Maybe someone can spot something wrong?
I will write an "Answer" because it is too much for a comment:
Shadow removal is (in my experience) not easy, you might be interested in this Paper: "Fredembach and Finlayson - Simple Shadow Removal"
Another idea i got a while back while working on a similar problem (i haven't tried it myself):
You basically want to identify big (in comparison to the characters) regions on your image and treat them differently. If you would know the shadow regions you could for example make the pages more uniform by brightening up the darker regions. The Question is how you can obtain this large regions.
You could first colour the dark writing in the same clour as the surrounding paper. Afterwards you could use the Bilateral Filter of OpenCV to get large uniform colour patches. You could identify the borders with a contour detection and you'd know where the paper differs in colour (caused by the shadows).
Hopefully this post shines a new light on your problem and gives you some ideas.

iOS + Tesseract Ocr + OpenCV

I wrote a digital OCR for ios.
I have a test image png with two digits 5 and 4.
I find the contours. How do I transfer the contour one at tesseract?
init tesseract:
tess = new tesseract::TessBaseAPI();
tess->Init([dataPath cStringUsingEncoding:NSUTF8StringEncoding], "eng");
tess->SetPageSegMode(tesseract::PSM_SINGLE_CHAR); //<-- !!!!
tess->tesseract::TessBaseAPI::SetVariable("tessedit_char_whitelist", "0123456789");
Function for detect contours:
- (std::vector<std::vector<cv::Point> >)findSquaresInImage:(cv::Mat)_image {
std::vector<std::vector<cv::Point> > squares;
cv::Mat pyr, timg, gray0(_image.size(), CV_8U), gray;
int thresh = 50, N = 11;
cv::pyrDown(_image, pyr, cv::Size(_image.cols/2, _image.rows/2));
cv::pyrUp(pyr, timg, _image.size());
std::vector<std::vector<cv::Point> > contours;
int ch[] = {0, 0};
mixChannels(&timg, 1, &gray0, 1, ch, 1);
for( int l = 0; l < N; l++ ) {
if( l == 0 ) {
cv::Canny(gray0, gray, 0, thresh, 5);
cv::dilate(gray, gray, cv::Mat(), cv::Point(-1,-1));
}
else {
gray = gray0 >= (l+1)*255/N;
}
cv::findContours(gray, contours, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE);
std::vector<cv::Point> approx;
CvRect rec1;
std::string str;
std::map<int,IplImage*> pic_list;
for( size_t i = 0; i < contours.size(); i++ )
{
rec1 = cv::boundingRect(contours[i]);
if (rec1.height > 0.5*gray.rows && rec1.width < 0.756*gray.cols) {
NSLog(#"%d %d %d %d", rec1.width, rec1.height, rec1.x, rec1.y);
cv::approxPolyDP(cv::Mat(contours[i]), approx, arcLength(cv::Mat(contours[i]), true)*0.02, true);
squares.push_back(approx);
}
}
}
return squares; }
function for draw contours:
cv::Mat debugSquares( std::vector<std::vector<cv::Point> > squares, cv::Mat image ) {
for ( int i = 0; i< squares.size(); i++ ) {
// draw contour
cv::drawContours(image, squares, i, cv::Scalar(255,0,0), 1, 8, std::vector<cv::Vec4i>(), 0, cv::Point());
// draw bounding rect
cv::Rect rect = boundingRect(cv::Mat(squares[i]));
cv::rectangle(image, rect.tl(), rect.br(), cv::Scalar(0,255,0), 2, 8, 0);
// draw rotated rect
cv::RotatedRect minRect = minAreaRect(cv::Mat(squares[i]));
cv::Point2f rect_points[4];
minRect.points( rect_points );
for ( int j = 0; j < 4; j++ ) {
cv::line( image, rect_points[j], rect_points[(j+1)%4], cv::Scalar(0,0,255), 1, 8 ); // blue
}
}
return image;
}
method for btn Click:
- (IBAction)onMath:(id)sender {
UIImage *image = [UIImage imageNamed:#"test1.png"];
cv::Mat iMat = [self cvMatFromUIImage:image];
std::vector<std::vector<cv::Point> > sq = [self findSquaresInImage:iMat];
cv::Mat hui = debugSquares(sq, iMat);
image = [self UIImageFromCVMat:hui];
self.imView.image = image;
}
image after:
link to project on github: https://github.com/MaxPatsy/iORC
Can you check this answer here
I described some tips for preparing images for Tesseract here: Using tesseract to recognize license plates
In your example, there are several things going on...
You need to get the text to be black and the rest of the image white (not the reverse). That's what character recognition is tuned on. Grayscale is ok, as long as the background is mostly full white and the text mostly full black; the edges of the text may be gray (antialiased) and that may help recognition (but not necessarily - you'll have to experiment)
One of the issues you're seeing is that in some parts of the image, the text is really "thin" (and gaps in the letters show up after thresholding), while in other parts it is really "thick" (and letters start merging). Tesseract won't like that :) It happens because the input image is not evenly lit, so a single threshold doesn't work everywhere. The solution is to do "locally adaptive thresholding" where a different threshold is calculated for each neighbordhood of the image. There are many ways of doing that, but check out for example:
Adaptive gaussian thresholding in OpenCV with cv2.adaptiveThreshold(...,cv2.ADAPTIVE_THRESH_GAUSSIAN_C,...)
Local Otsu's method
Local adaptive histogram equalization
Another problem you have is that the lines aren't straight. In my experience Tesseract can handle a very limited degree of non-straight lines (a few percent of perspective distortion, tilt or skew), but it doesn't really work with wavy lines. If you can, make sure that the source images have straight lines :) Unfortunately, there is no simple off-the-shelf answer for this; you'd have to look into the research literature and implement one of the state of the art algorithms yourself (and open-source it if possible - there is a real need for an open source solution to this). A Google Scholar search for "curved line OCR extraction" will get you started, for example:
Text line Segmentation of Curved Document Images
Lastly: I think you would do much better to work with the python ecosystem (ndimage, skimage) than with OpenCV in C++. OpenCV python wrappers are ok for simple stuff, but for what you're trying to do they won't do the job, you will need to grab many pieces that aren't in OpenCV (of course you can mix and match). Implementing something like curved line detection in C++ will take an order of magnitude longer than in python (* this is true even if you don't know python).
Good luck!

Thresholding for a colour in opencv

I am trying to set up my programme to threshold for a colour (in BGR format). I have not fully decided which colour I will be looking for yet. I would also like the program to record how many pixels it has detected of that colour. My code so far is below but it is not working.
#include "cv.h"
#include "highgui.h"
int main()
{
// Initialize capturing live feed from the camera
CvCapture* capture = 0;
capture = cvCaptureFromCAM(0);
// Couldn't get a device? Throw an error and quit
if(!capture)
{
printf("Could not initialize capturing...\n");
return -1;
}
// The two windows we'll be using
cvNamedWindow("video");
cvNamedWindow("thresh");
// An infinite loop
while(true)
{
// Will hold a frame captured from the camera
IplImage* frame = 0;
frame = cvQueryFrame(capture);
// If we couldn't grab a frame... quit
if(!frame)
break;
//create image where threshloded image will be stored
IplImage* imgThreshed = cvCreateImage(cvGetSize(frame), 8, 1);
//i want to keep it BGR format. Im not sure what colour i will be looking for yet. this can be easily changed
cvInRangeS(frame, cvScalar(20, 100, 100), cvScalar(30, 255, 255), imgThreshed);
//show the original feed and thresholded feed
cvShowImage("thresh", imgThreshed);
cvShowImage("video", frame);
// Wait for a keypress
int c = cvWaitKey(10);
if(c!=-1)
{
// If pressed, break out of the loop
break;
}
cvReleaseImage(&imgThreshed);
}
cvReleaseCapture(&capture);
return 0;
}
To threshold for a color,
1) convert the image to HSV
2) Then apply cvInrangeS
3) Once you got threshold image, you can count number of white pixels in it.
Try this tutorial to track yellow color: Tracking colored objects in OpenCV
I can tell how to do it in both Python and C++ and both with and without converting to HSV.
C++ Version (Converting to HSV)
Convert the image into an HSV image:
// Convert the image into an HSV image
IplImage* imgHSV = cvCreateImage(cvGetSize(img), 8, 3);
cvCvtColor(img, imgHSV, CV_BGR2HSV);
Create a new image that will hold the threholded image:
IplImage* imgThreshed = cvCreateImage(cvGetSize(img), 8, 1);
Do the actual thresholding using cvInRangeS
cvInRangeS(imgHSV, cvScalar(20, 100, 100), cvScalar(30, 255, 255), imgThreshed);
Here, imgHSV is the reference image. And the two cvScalars represent the lower and upper bound of values that are yellowish in colour. (These bounds should work in almost all conditions. If they don't, try experimenting with the last two values).
Consider any pixel. If all three values of that pixel (H, S and V, in that order) lie within the stated ranges, imgThreshed gets a value of 255 at that corresponding pixel. This is repeated for all pixels. So what you finally get is a thresholded image.
Use countNonZero to count the number of white pixels in the thresholded image.
Python Version (Without converting to HSV):
Create the lower and upper boundaries of the range you are interested in, in Numpy array format (Note: You need to use import numpy as np)
lower = np.array((a,b,c), dtype = "uint8")
upper = np.array((x,y,z), dtype = "uint8")
In the above (a,b,c) is the lower bound and (x,y,z) is the upper bound.
2.Get the mask for the pixels that satisfy the range:
mask = cv2.inRange(image, lower, upper)
In the above, image is the image on which you want to work.
Count the number of white pixels that are present in the mask using countNonZero:
yellowpixels = cv2.countNonZero(mask)
print "Number of Yellow pixels are %d" % (yellowpixels)
Sources:
http://srikanthvidyasagar.blogspot.com/2016/01/tracking-colored-objects-in-opencv.html
http://www.pyimagesearch.com/2014/08/04/opencv-python-color-detection/
count number of black pixels in an image in Python with OpenCV

OpenCV: Incorrect contour around blobs

I'm trying to draw contours around blobs in a binary image, however, sometimes, openCV draws a single contour around two distinct blobs. below is an example. How can i solve this issue?
Here it should draw two bounding boxes for the blob on the right and separately for the one of the left. I agree they are close but enough distance in between them. I'm only drawing External contours instead of the tree or list. I'm also using cvFindNextContour(contourscanner) as this is a easier implementation for my case.
Thanks
EDIT:
Image displayed in the "output" window is from a different function which does just image subtraction. Image displayed in the "contours" window is in the function pplfind(). "output" image is passed to img_con().
IplImage* img_con(IplImage* image){
int ppl;
CvMemStorage* memstr = cvCreateMemStorage();
IplImage* edges = cvCreateImage(cvGetSize(image),8,1);
cvCanny(image,edges,130,255);
CvContourScanner cscan = cvStartFindContours(image,memstr,sizeof(CvContour),CV_RETR_EXTERNAL,CV_CHAIN_APPROX_NONE,cvPoint(0,0));
ppl = pplfind(cscan,cvGetSize(image));
if (ppl !=0 )
printf("Estimated number of people: %d\n",ppl);
cvEndFindContours(&cscan);
cvClearMemStorage(memstr);
return edges;
}
int pplfind(CvContourScanner cscan, CvSize frSize){
ofstream file; char buff[50];
file.open("box.txt",ofstream::app);
int ppl =0;
CvSeq* c;
IplImage *out = cvCreateImage(frSize,8,3);
while (c = cvFindNextContour(cscan)){
CvRect box = cvBoundingRect(c,1);
if ((box.height > int(box.width*1.2))&&(box.height>20)){//&&(box.width<20)){//
ppl++;
cvRectangle(out,cvPoint(box.x,box.y),cvPoint(box.x+box.width,box.y+box.height),CV_RGB(255,0,50),1);
cvShowImage("contours",out);
//cvWaitKey();
}
//printf("Box Height: %d , Box Width: %d ,People: %d\n",box.height,box.width,ppl);
//cvWaitKey(0);
int coord = sprintf_s(buff,"%d,%d,%d\n",box.width,box.height,ppl);
file.write(buff,coord);
}
file.close();
cvReleaseImage(&out);
return ppl;
}
I've never used cvFindNextContour, but running cvFindContours with CV_RETR_EXTERNAL on your image seems to work fine:
I use OpenCV + Python, so this code might not be useful for you, but for the sake of completeness here it goes:
contours = cv.findContours(img, cv.CreateMemStorage(0), mode=cv.CV_RETR_EXTERNAL)
while contours:
(x,y,w,h) = cv.BoundingRect(contours)
cv.Rectangle(colorImg, (x,y), (x+w,y+h), cv.Scalar(0,255,255,255))
contours = contours.h_next()
Edit: you asked how to draw only those contours with certain properties; it would be something like this:
contours = cv.findContours(img, cv.CreateMemStorage(0), mode=cv.CV_RETR_EXTERNAL)
while contours:
(x,y,w,h) = cv.BoundingRect(contours)
if h > w*1.2 and h > 20:
cv.Rectangle(colorImg, (x,y), (x+w,y+h), cv.Scalar(0,255,255,255))
contours = contours.h_next()

Resources