OpenCV Multiple marker detection? - opencv

I've been working on detecting fiducial markers in scenes. An example of my fiducial marker is here:
http://tinypic.com/view.php?pic=4r6k3q&s=8#.VNgsgzVVK1F
I have been able to detect a single fiducial marker in a scene very well. What is the methodology for detecting multiple fiducial markers in a scene? Doing feature detection, extraction, and then matching is great for finding a single match, but it seems to be the wrong method for detecting multiple matches since it would be difficult to determine which features belong to which marker?
The fiducial markers would be the same, and would not be in a known location in the scene.
Update:
Below is some sample code. I was trying to match the first fiducial marker with x number of keypoints, and then use the remaining keypoints to match the second marker. However, this is not robust at all. Does anybody have any suggestions?
OrbFeatureDetector detector;
vector<KeyPoint> keypoints1, keypoints2;
detector.detect(im1, keypoints1);
detector.detect(im2, keypoints2);
Mat display_im1, display_im2;
drawKeypoints(im1, keypoints1, display_im1, Scalar(0,0,255));
drawKeypoints(im2, keypoints2, display_im2, Scalar(0,0,255));
SiftDescriptorExtractor extractor;
Mat descriptors1, descriptors2;
extractor.compute( im1, keypoints1, descriptors1 );
extractor.compute( im2, keypoints2, descriptors2 );
BFMatcher matcher;
vector< DMatch > matches1, matches2;
matcher.match( descriptors1, descriptors2, matches1 );
sort (matches1.begin(), matches1.end());
matches2 = matches;
int numElementsToSave = 50;
matches1.erase(matches1.begin()+numElementsToSave,matches1.end());
matches2.erase(matches2.begin(),matches2.begin()+numElementsToSave);
Mat match_im1, match_im2;
drawMatches( im1, keypoints1, im2, keypoints2,
matches1, match_im1, Scalar::all(-1), Scalar::all(-1),
vector<char>(), DrawMatchesFlags::NOT_DRAW_SINGLE_POINTS );
drawMatches( im1, keypoints1, im2, keypoints2,
matches2, match_im2, Scalar::all(-1), Scalar::all(-1),
vector<char>(), DrawMatchesFlags::NOT_DRAW_SINGLE_POINTS );

I have never tried it before, but here you have a good explanation about the detection of multiple occurrences:
This tutorial shows how enabling multiple detection of the same
object. To enable multiple detection, the parameter
General->multiDetection should be checked. The approach is as
following:
As usual, we match all features between the objects and the scene.
For an object which is in the scene two times (or more), it should have twice of matched features. We apply a RANSAC algorithm to find a
homography. The inliers should belong to only one occurrence of the
object, all others considered as outliers. We redo the homography
process on the outliers, then find another homography… we do this
process until no homography can be computed.
It may happens that a homography can be found superposed on a previous one using the outliers. You could set
Homography->ransacReprojThr (in pixels) higher to accept more
inliers in the homographies computed, which would decrease the chance
of superposed detections. Another way is to ignore superposed
homographies on a specified radius with the parameter General->multiDetectionRadius (in pixels).
For more information see the page below:
https://code.google.com/p/find-object/wiki/MultiDetection

I developed a semi-automatic algorithm to detect multiple markers (interest points) from image using findContours method on a binary image (my markers are white on a green surface then I limit my search to area constraint as I know how big is each marker in each frame. of course this got some false positives but it was good enough. I couldn't see the picture in your post as tinypic is blocked here for some reason. But you can use the matchShape opencv function to eliminate the bad contours.
here is the part of code I made for this.
Mat tempFrame;
cvtColor(BallFrame, tempFrame, COLOR_BGR2GRAY);
GaussianBlur(tempFrame, tempFrame, Size(15, 15), 2, 2); // remove noise
Mat imBw;
threshold(tempFrame, imBw, 220, 255, THRESH_BINARY); // High threshold to get better results
std::vector<std::vector<Point> > contours;
std::vector<Vec4i> hierarchy;
findContours(imBw, contours, hierarchy, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_SIMPLE);
Point2f center;
float radius = 0.0;
for (int i = 0; i < contours.size(); i++)
{
double area = contourArea(contours[i]);
if (area > 1 && area < 4000) {
minEnclosingCircle(contours[i], center, radius);
if (radius < 50) // eliminate wide narrow contours
{
// You can use `matchShape` here to match your marker shape
}
}
}
I hope this will help

Related

Color band on SURF descriptors

I'm reading this paper for obtaining better VLAD descriptors. The main difference is the use of the so called CSURF which is (quoting the paper):
In order to extract CSURF, the image
is first transformed to grayscale and
interest points are computed using the
standard SURF algorithm. Then, instead
of computing the SURF descriptor of
each interest point on the intensity
channel, CSURF computes three SURF
descriptors, one on each color band.
How could I implement this in OpenCV? All the descriptors that I've seen so far (SIFT, SURF, ETC) are computed on the gray scale, how could I use SURF to describe keypoints based on one color band (Red, Green or Blue)?
UPDATE: IS THIS SOLUTION CORRECT?
Mat img = imread( src, CV_LOAD_IMAGE_COLOR );
Mat grey;
cvtColor(img,gray,CV_BGR2GRAY);
int minHessian = 400;
Ptr<SURF> detector = SURF::create( minHessian );
std::vector<KeyPoint> keypoints;
detector->detect( grey, keypoints );
vector<Mat> spl;
Mat blueDesc, greenDesc, redDesc;
split(img,spl);
detector->detectAndCompute( spl[0], Mat(), keypoints, blueDesc, true);
detector->detectAndCompute( spl[1], Mat(), keypoints, greenDesc, true);
detector->detectAndCompute( spl[2], Mat(), keypoints, redDesc, true);

OpenCV contour evolution

I have a contour that I would like to "snap" to edges in an image. That is, some thing like Intelligent Scissors, but for the whole contour at the same. A user has provided a rough sketch of the outline of an object, and I'd like to clean it up by "pushing" each point on the contour to the nearest point in an edge image.
Does something like this exist in OpenCV?
You can mimic active contours using cv::grabCut as suggested. You choose the radius of attraction (how far from the original position the curve can evolve), and by using dilated and eroded images, you define the unknown region around the contour.
// cv::Mat img, mask; // contour on mask as filled polygon
if ( mask.size()!=img.size() )
CV_Error(CV_StsError,"ERROR");
int R = 32; // radius of attraction
cv::Mat strel = cv::getStructuringElement(cv::MORPH_ELLIPSE, cv::Size(2*R+1,2*R+1) );
cv::Mat gc( mask.size(), CV_8UC1, cv::Scalar(cv::GC_BGD) );
cv::Mat t;
cv::dilate( mask, t, strel );
gc.setTo( cv::GC_PR_BGD, t );
gc.setTo( cv::GC_PR_FGD, mask ); // 3
cv::erode( mask, t, strel );
gc.setTo( cv::GC_FGD, t ); // 1
cv::grabCut( img, gc, cv::Rect(), cv::Mat(), cv::Mat(), 2 );
gc &= 0x1; // either foreground or probably foreground
gc *= 255; // so that you see it
What you may loose, is the topology of the contour. Some processing required there. Also, you cannot control the curvature or smoothness of the contour and it's not really contour evolution in sense.
Only if you are interested, ITK geodesic active contour might be what you are looking for http://www.itk.org/Doxygen/html/classitk_1_1GeodesicActiveContourLevelSetImageFilter.html

The best and quickest method for detecting quadratic shape in image using OpenCV?

In last few days I'm looking for good and quick method for finding quadratic shape in image.
For example, take a look at attached image.
I want to find the edges of white screen part (the TV screen in this case).
I can replace the white canvas with whatever I want, e.g. QR code, some texture, etc. - just looking for the coordinates of that shape.
Other features of the shape:
Only one shape should be detected.
Perspective transform should be used.
The languages is not that important, but I want to use OpenCV for this.
These are good algorithms that have been implemented in OpenCV:
Harris corner detector as GoodFeatureToTrackDetector
GoodFeaturesToTrackDetector harris_detector (1000, 0.01, 10, 3, true);
vector<KeyPoint> keypoints;
cvtColor (image, gray_image, CV_BGR2GRAY);
harris_detector.detect (gray_image, keypoints);
Fast corner detector as FeatureDetector::create("FAST") and FASTX
Ptr<FeatureDetector> feature_detector = FeatureDetector::create("FAST");
vector<KeyPoint> keypoints;
cvtColor (image, gray_image, CV_BGR2GRAY);
feature_detector->detect (gray_image, keypoints);
Or
FASTX (gray_image, keypoints, 50, true, FastFeatureDetector::TYPE_9_16);
SIFT (Scale Invariant Feature Transform) as FeatureDetector::create("SIFT")
Ptr<FeatureDetector> feature_detector = FeatureDetector::create("SIFT");
vector<KeyPoint> keypoints;
cvtColor (image, gray_image, CV_BGR2GRAY);
feature_detector->detect (gray_image, keypoints);
Update for perspective transform (you must know 4 points before haned):
Point2f source [4], destination [4];
// Assign values to source and destination points.
perspective_matrix = getPerspectiveTransform( source, destination );
warpPerspective( image, result, perspective_matrix, result.size() );

Get the SIFT descriptor for specified point using OpenCV

I want get the SIFT feature for specified points. These points is gotten by hand not by KeyPoint Detector. My question is: I only know the position of the points but have no idea about the size and angle value. How should I set this value?
Here is my code:
int main()
{
Mat img_object = imread("img/test.jpg", 0);
SiftDescriptorExtractor extractor;
Mat descriptors;
std::vector<KeyPoint> keypoints;
// set keypoint position and size: should I set
// size parameter to 32 for 32x32 patch?
KeyPoint kp(50, 60, 32);
keypoints.push_back(kp);
extractor.compute( img_object, keypoints, descriptors );
return 0;
}
Should I set the size param of KeyPoint to 32 for 32x32 patch. Is this implementation reasonable?
Usually, keypoint detectors work on a local neighbourhood around a point. This is the size field of OpenCV's KeyPoint class. The angle field is the dominant orientation of the keypoint (this could be set to -1, note).
OpenCV KeyPoint class
Another reference here.

Having some difficulty in image stitching using OpenCV

I'm currently working on Image stitching using OpenCV 2.3.1 on Visual Studio 2010, but I'm having some trouble.
Problem Description
I'm trying to write a code for stitching multiple images derived from a few cameras(about 3~4), i,e, the code should keep executing image stitching until I ask it to stop.
The following is what I've done so far:
(For simplification, I'll replace some part of the code with just a few words)
1.Reading frames(images) from 2 cameras (Currently I'm just working on 2 cameras.)
2.Feature detection, descriptor calculation (SURF)
3.Feature matching using FlannBasedMatcher
4.Removing outliers and calculate the Homography with inliers using RANSAC.
5.Warp one of both images.
For step 5., I followed the answer in the following thread and just changed some parameters:
Stitching 2 images in opencv
However, the result is terrible though.
I just uploaded the result onto youtube and of course only those who have the link will be able to see it.
http://youtu.be/Oy5z_7LeaMk
My code is shown below:
(Only crucial parts are shown)
VideoCapture cam1, cam2;
cam1.open(0);
cam2.open(1);
while(1)
{
Mat frm1, frm2;
cam1 >> frm1;
cam2 >> frm2;
//(SURF detection, descriptor calculation
//and matching using FlannBasedMatcher)
double max_dist = 0; double min_dist = 100;
//-- Quick calculation of max and min distances between keypoints
for( int i = 0; i < descriptors_1.rows; i++ )
{
double dist = matches[i].distance;
if( dist < min_dist ) min_dist = dist;
if( dist > max_dist ) max_dist = dist;
}
(Draw only "good" matches
(i.e. whose distance is less than 3*min_dist ))
vector<Point2f> frame1;
vector<Point2f> frame2;
for( int i = 0; i < good_matches.size(); i++ )
{
//-- Get the keypoints from the good matches
frame1.push_back( keypoints_1[ good_matches[i].queryIdx ].pt );
frame2.push_back( keypoints_2[ good_matches[i].trainIdx ].pt );
}
Mat H = findHomography( Mat(frame1), Mat(frame2), CV_RANSAC );
cout << "Homography: " << H << endl;
/* warp the image */
Mat warpImage2;
warpPerspective(frm2, warpImage2,
H, Size(frm2.cols, frm2.rows), INTER_CUBIC);
Mat final(Size(frm2.cols*3 + frm1.cols, frm2.rows),CV_8UC3);
Mat roi1(final, Rect(frm1.cols, 0, frm1.cols, frm1.rows));
Mat roi2(final, Rect(2*frm1.cols, 0, frm2.cols, frm2.rows));
warpImage2.copyTo(roi2);
frm1.copyTo(roi1);
imshow("final", final);
What else should I do to make the stitching better?
Besides, is it reasonable to make the Homography matrix fixed instead of keeping computing it ?
What I mean is to specify the angle and the displacement between the 2 cameras by myself so as to derive a Homography matrix that satisfies what I want.
Thanks. :)
It sounds like you are going about this sensibly, but if you have access to both of the cameras, and they will remain stationary with respect to each other, then calibrating offline, and simply applying the transformation online will make your application more efficient.
One point to note is, you say you are using the findHomography function from OpenCV. From the documentation, this function:
Finds a perspective transformation between two planes.
However, your points are not restricted to a specific plane as they are imaging a 3D scene. If you wanted to calibrate offline, you could image a chessboard with both cameras, and the detected corners could be used in this function.
Alternatively, you may like to investigate the Fundamental matrix, which can be calculated with a similar function. This matrix describes the relative position of the cameras, but some work (and a good textbook) will be required to extract them.
If you can find it, I would strongly recommend having a look at Part II: "Two-View Geometry" in the book "Multiple View Geometry in computer vision", by Richard Hartley and Andrew Zisserman, which goes through the process in detail.
I have been working lately on image registration. My algorithm takes two images, calculates the SURF features, find correspondences, find homography matrix and then stitch both images together, I did it with the next code:
void stich(Mat base, Mat target,Mat homography, Mat& panorama){
Mat corners1(1, 4,CV_32F);
Mat corners2(1,4,CV_32F);
Mat corners(1,4,CV_32F);
vector<Mat> planes;
/* compute corners
of warped image
*/
corners1.at<float>(0,0)=0;
corners2.at<float>(0,0)=0;
corners1.at<float>(0,1)=0;
corners2.at<float>(0,1)=target.rows;
corners1.at<float>(0,2)=target.cols;
corners2.at<float>(0,2)=0;
corners1.at<float>(0,3)=target.cols;
corners2.at<float>(0,3)=target.rows;
planes.push_back(corners1);
planes.push_back(corners2);
merge(planes,corners);
perspectiveTransform(corners, corners, homography);
/* compute size of resulting
image and allocate memory
*/
double x_start = min( min( (double)corners.at<Vec2f>(0,0)[0], (double)corners.at<Vec2f> (0,1)[0]),0.0);
double x_end = max( max( (double)corners.at<Vec2f>(0,2)[0], (double)corners.at<Vec2f>(0,3)[0]), (double)base.cols);
double y_start = min( min( (double)corners.at<Vec2f>(0,0)[1], (double)corners.at<Vec2f>(0,2)[1]), 0.0);
double y_end = max( max( (double)corners.at<Vec2f>(0,1)[1], (double)corners.at<Vec2f>(0,3)[1]), (double)base.rows);
/*Creating image
with same channels, depth
as target
and proper size
*/
panorama.create(Size(x_end - x_start + 1, y_end - y_start + 1), target.depth());
planes.clear();
/*Planes should
have same n.channels
as target
*/
for (int i=0;i<target.channels();i++){
planes.push_back(panorama);
}
merge(planes,panorama);
// create translation matrix in order to copy both images to correct places
Mat T;
T=Mat::zeros(3,3,CV_64F);
T.at<double>(0,0)=1;
T.at<double>(1,1)=1;
T.at<double>(2,2)=1;
T.at<double>(0,2)=-x_start;
T.at<double>(1,2)=-y_start;
// copy base image to correct position within output image
warpPerspective(base, panorama, T,panorama.size(),INTER_LINEAR| CV_WARP_FILL_OUTLIERS);
// change homography to take necessary translation into account
gemm(T, homography,1,T,0,T);
// warp second image and copy it to output image
warpPerspective(target,panorama, T, panorama.size(),INTER_LINEAR);
//tidy
corners.release();
T.release();
}
Any question I will try

Resources