I would like to map one triangle inside an OpenCV Mat to another one, pretty much like warpAffine does (check it here), but for triangles instead of quads, in order to use it in a Delaunay triangulation.
I know one is able to use a mask, but I'd like to know if there's a better solution.
I have copied the above image and the following C++ code from my post Warp one triangle to another using OpenCV ( C++ / Python ). The comments in the code below should provide a good idea what is going on. For more details and for python code you can visit the above link. All the pixels inside triangle tri1 in img1 are transformed to triangle tri2 in img2. Hope this helps.
void warpTriangle(Mat &img1, Mat &img2, vector<Point2f> tri1, vector<Point2f> tri2)
{
// Find bounding rectangle for each triangle
Rect r1 = boundingRect(tri1);
Rect r2 = boundingRect(tri2);
// Offset points by left top corner of the respective rectangles
vector<Point2f> tri1Cropped, tri2Cropped;
vector<Point> tri2CroppedInt;
for(int i = 0; i < 3; i++)
{
tri1Cropped.push_back( Point2f( tri1[i].x - r1.x, tri1[i].y - r1.y) );
tri2Cropped.push_back( Point2f( tri2[i].x - r2.x, tri2[i].y - r2.y) );
// fillConvexPoly needs a vector of Point and not Point2f
tri2CroppedInt.push_back( Point((int)(tri2[i].x - r2.x), (int)(tri2[i].y - r2.y)) );
}
// Apply warpImage to small rectangular patches
Mat img1Cropped;
img1(r1).copyTo(img1Cropped);
// Given a pair of triangles, find the affine transform.
Mat warpMat = getAffineTransform( tri1Cropped, tri2Cropped );
// Apply the Affine Transform just found to the src image
Mat img2Cropped = Mat::zeros(r2.height, r2.width, img1Cropped.type());
warpAffine( img1Cropped, img2Cropped, warpMat, img2Cropped.size(), INTER_LINEAR, BORDER_REFLECT_101);
// Get mask by filling triangle
Mat mask = Mat::zeros(r2.height, r2.width, CV_32FC3);
fillConvexPoly(mask, tri2CroppedInt, Scalar(1.0, 1.0, 1.0), 16, 0);
// Copy triangular region of the rectangular patch to the output image
multiply(img2Cropped,mask, img2Cropped);
multiply(img2(r2), Scalar(1.0,1.0,1.0) - mask, img2(r2));
img2(r2) = img2(r2) + img2Cropped;
}
You should use the getAffineTransform to find the transform, and use warpAffine to apply it
Related
I have a contour that I would like to "snap" to edges in an image. That is, some thing like Intelligent Scissors, but for the whole contour at the same. A user has provided a rough sketch of the outline of an object, and I'd like to clean it up by "pushing" each point on the contour to the nearest point in an edge image.
Does something like this exist in OpenCV?
You can mimic active contours using cv::grabCut as suggested. You choose the radius of attraction (how far from the original position the curve can evolve), and by using dilated and eroded images, you define the unknown region around the contour.
// cv::Mat img, mask; // contour on mask as filled polygon
if ( mask.size()!=img.size() )
CV_Error(CV_StsError,"ERROR");
int R = 32; // radius of attraction
cv::Mat strel = cv::getStructuringElement(cv::MORPH_ELLIPSE, cv::Size(2*R+1,2*R+1) );
cv::Mat gc( mask.size(), CV_8UC1, cv::Scalar(cv::GC_BGD) );
cv::Mat t;
cv::dilate( mask, t, strel );
gc.setTo( cv::GC_PR_BGD, t );
gc.setTo( cv::GC_PR_FGD, mask ); // 3
cv::erode( mask, t, strel );
gc.setTo( cv::GC_FGD, t ); // 1
cv::grabCut( img, gc, cv::Rect(), cv::Mat(), cv::Mat(), 2 );
gc &= 0x1; // either foreground or probably foreground
gc *= 255; // so that you see it
What you may loose, is the topology of the contour. Some processing required there. Also, you cannot control the curvature or smoothness of the contour and it's not really contour evolution in sense.
Only if you are interested, ITK geodesic active contour might be what you are looking for http://www.itk.org/Doxygen/html/classitk_1_1GeodesicActiveContourLevelSetImageFilter.html
I want to find the corner position of an blurred image with a corner inside it. like the following example:
I can make sure that only one corner is inside the image, and I assume that
the corner is part of a black and white chessboard.
How can I detect the cross position with openCV?
Thanks!
Usually you can determine the corner using the gradient:
Gx = im[i][j+1] - im[i][j-1]; Gy = im[i+1][j] – im[i-1][j];
G^2 = Gx^2 + Gy^2;
teta = atan2 (Gy, Gx);
As your image is blurred, you should compute the gradient at a larger scale:
Gx = im[i][j+delta] - im[i][j-delta]; Gy = im[i+ delta][j] – im[i- delta][j];
Here is the result that I obtained for delta = 50:
The gradient norm (multiplied by 20)
gradient norm http://imageshack.us/scaled/thumb/822/xdpp.jpg
The gradient direction:
gradient direction http://imageshack.us/scaled/thumb/844/h6zp.jpg
another solution
#include <opencv2/opencv.hpp>
using namespace cv;
int main()
{
Mat img=imread("c:/data/corner.jpg");
Mat gray;
cvtColor(img,gray,CV_BGR2GRAY);
threshold(gray,gray,100,255,CV_THRESH_BINARY);
int step=15;
std::vector<Point> points;
for(int i=0;i<gray.rows;i+=step)
for(int j=0;j<gray.cols;j+=step)
if(gray.at<uchar>(i,j)==255)
points.push_back(Point(j,i));
//fit a rotated rectangle
RotatedRect box = minAreaRect(Mat(points));
//circle(img,box.center,2,Scalar(255,0,0),-1);
//invert it,fit again and get average of centers(may not be needed if a 'good' threshold is found)
Point p1=Point(box.center.x,box.center.y);
points.clear();
gray=255-gray;
for(int i=0;i<gray.rows;i+=step)
for(int j=0;j<gray.cols;j+=step)
if(gray.at<uchar>(i,j)==255)
points.push_back(Point(j,i));
box = minAreaRect(Mat(points));
Point p2=Point(box.center.x,box.center.y);
//circle(img,p2,2,Scalar(0,255,0),-1);
circle(img,Point((p1.x+p2.x)/2,(p1.y+p2.y)/2),3,Scalar(0,0,255),-1);
imshow("img",img);
waitKey();
return 0;
}
Rather than work right away at a ridiculously large scale, as suggested by others, I recommend downsizing first (which has the effect of deblurring), do one pass of Harris to find the corner, then upscale its position and do a pass of findCornerSubpix at full resolution with a large window (large enough to encompass the obvious saddle point of the intensity).
In this way you get the best of both worlds: fast detection to initialize the refinement, and accurate refinement given the original imagery.
See also this other relevant answer
When finding a reference image in a scene using SURF, I would like to crop the found object in the scene, and "straighten" it back using warpPerspective and the reversed homography matrix.
Meaning, let's say I have this SURF result:
Now, I would like to crop the found object in the scene:
and "straighten" only the cropped image with warpPerspective using the reversed homography matrix. The result I'm aiming at is that I'll get an image containing, roughly, only the object, and some distorted leftovers from the original scene (as the cropping is not a 100% the object alone).
Cropping the found object, and finding the homography matrix and reversing it are simple enough. Problem is, I can't seem to understand the results from warpPerspective. Seems like the resulting image contains only a small portion of the cropped image, and in a very large size.
While researching warpPerspective I found that the resulting image is very large due to the nature of the process, but I can't seem to wrap my head around how to do this properly. Seems like I just don't understand the process well enough. Would I need to warpPerspective the original (not cropped) image and than crop the "straightened" object?
Any advice?
try this.
given that you have the unconnected contour of your object (e.g. the outer corner points of the box contour) you can transform them with your inverse homography and adjust that homography to place the result of that transformation to the top left region of the image.
compute where those object points will be warped to (use the inverse homography and the contour points as input):
cv::Rect computeWarpedContourRegion(const std::vector<cv::Point> & points, const cv::Mat & homography)
{
std::vector<cv::Point2f> transformed_points(points.size());
for(unsigned int i=0; i<points.size(); ++i)
{
// warp the points
transformed_points[i].x = points[i].x * homography.at<double>(0,0) + points[i].y * homography.at<double>(0,1) + homography.at<double>(0,2) ;
transformed_points[i].y = points[i].x * homography.at<double>(1,0) + points[i].y * homography.at<double>(1,1) + homography.at<double>(1,2) ;
}
// dehomogenization necessary?
if(homography.rows == 3)
{
float homog_comp;
for(unsigned int i=0; i<transformed_points.size(); ++i)
{
homog_comp = points[i].x * homography.at<double>(2,0) + points[i].y * homography.at<double>(2,1) + homography.at<double>(2,2) ;
transformed_points[i].x /= homog_comp;
transformed_points[i].y /= homog_comp;
}
}
// now find the bounding box for these points:
cv::Rect boundingBox = cv::boundingRect(transformed_points);
return boundingBox;
}
modify your inverse homography (result of computeWarpedContourRegion and inverseHomography as input)
cv::Mat adjustHomography(const cv::Rect & transformedRegion, const cv::Mat & homography)
{
if(homography.rows == 2) throw("homography adjustement for affine matrix not implemented yet");
// unit matrix
cv::Mat correctionHomography = cv::Mat::eye(3,3,CV_64F);
// correction translation
correctionHomography.at<double>(0,2) = -transformedRegion.x;
correctionHomography.at<double>(1,2) = -transformedRegion.y;
return correctionHomography * homography;
}
you will call something like
cv::warpPerspective(objectWithBackground, output, adjustedInverseHomography, sizeOfComputeWarpedContourRegionResult);
hope this helps =)
I am new in OpenCV so please to be lenient.
I am doing an Android application to recognize the squares/rectangles and crop them. Function which looks for the squares/rectangles puts the found objects to vector> squares. I just wonder how to crop the picture according to the data in points stored in vector> squares and how to compute an angle on which the picture should be rotated. Thank you for any help
This post is citing from OpenCV QA: Extract a RotatedRect area.
There's a great article by Felix Abecassis on rotating and deskewing images. This also shows you how to extract the data in the RotatedRect:
http://felix.abecassis.me/2011/10/opencv-rotation-deskewing/
You basically only need cv::getRotationMatrix2D to get the rotation matrix for the affine transformation with cv::warpAffine and cv::getRectSubPix to crop the rotated image. The relevant lines in my application are:
// This is the RotatedRect, I got it from a contour for example...
RotatedRect rect = ...;
// matrices we'll use
Mat M, rotated, cropped;
// get angle and size from the bounding box
float angle = rect.angle;
Size rect_size = rect.size;
// thanks to http://felix.abecassis.me/2011/10/opencv-rotation-deskewing/
if (rect.angle < -45.) {
angle += 90.0;
swap(rect_size.width, rect_size.height);
}
// get the rotation matrix
M = getRotationMatrix2D(rect.center, angle, 1.0);
// perform the affine transformation on your image in src,
// the result is the rotated image in rotated. I am doing
// cubic interpolation here
warpAffine(src, rotated, M, src.size(), INTER_CUBIC);
// crop the resulting image, which is then given in cropped
getRectSubPix(rotated, rect_size, rect.center, cropped);
There are lots of useful posts around, I'm sure you can do a better search.
Crop:
cropping IplImage most effectively
Rotate:
OpenCV: how to rotate IplImage?
Rotating or Resizing an Image in OpenCV
Compute angle:
OpenCV - Bounding Box & Skew Angle
Altought this question is quite old, I think there is the need for an answer that is not expensive as rotating the whole image (see #bytefish's answer). You will need a bounding rect, for some reason rotatedRect.boundingRect() didn't work for me, so I had to use Imgproc.boundingRect(contour). This is OpenCV for Android, the operations are almost the same for other environments:
Rect roi = Imgproc.boundingRect(contour);
// we only work with a submat, not the whole image:
Mat mat = image.submat(roi);
RotatedRect rotatedRect = Imgproc.minAreaRect(new MatOfPoint2f(contour.toArray()));
Mat rot = Imgproc.getRotationMatrix2D(rotatedRect.center, rotatedRect.angle, 1.0);
// rotate using the center of the roi
double[] rot_0_2 = rot.get(0, 2);
for (int i = 0; i < rot_0_2.length; i++) {
rot_0_2[i] += rotatedRect.size.width / 2 - rotatedRect.center.x;
}
rot.put(0, 2, rot_0_2);
double[] rot_1_2 = rot.get(1, 2);
for (int i = 0; i < rot_1_2.length; i++) {
rot_1_2[i] += rotatedRect.size.height / 2 - rotatedRect.center.y;
}
rot.put(1, 2, rot_1_2);
// final rotated and cropped image:
Mat rotated = new Mat();
Imgproc.warpAffine(mat, rotated, rot, rotatedRect.size);
I need some help on Augmented Reality.
I have develop a small application.NOw I want to use shape detection algorithm or specially circle detection algorithm.I want that after my camera get open It should only detect circles and if it gets circles it should get replaced with some corresponding image.
I hope you understood what I want to do.
To add shape detection algorithm for (circle), you can consider using circle detection with Hough Transform from OpenCV. Taken from OpenCV tutorial website, here are some snippets:
// Loads an image
cv::Mat src = cv::imread( filename, cv::IMREAD_COLOR );
cv::Mat gray;
cv::cvtColor(src, gray, cv::COLOR_BGR2GRAY);
cv::medianBlur(gray, gray, 5);
cv::vector<Vec3f> circles;
cv::HoughCircles(gray, circles, cv::HOUGH_GRADIENT, 1,
gray.rows/16, // change this value to detect circles with different distances to each other
100, 30, 1, 30 // change the last two parameters
// (min_radius & max_radius) to detect larger circles
);
for( size_t i = 0; i < circles.size(); i++ )
{
cv::Vec3i c = circles[i];
cv::Point center = cv::Point(c[0], c[1]);
// circle center
cv::circle( src, center, 1, cv::Scalar(0,100,100), 3, cv::LINE_AA);
// circle outline
int radius = c[2];
cv::circle( src, center, radius, cv::Scalar(255,0,255), 3, cv::LINE_AA);
}
OpenCV can do the task as you mentioned, and is compatible for AR application.