I tried to set a ROI with angle in an image. At first, I thought that using a mask would give me the same result as setting the ROI in my IplImage structure. Then, I would just use cvResize the same way I did when I used ROI, but this time for a mask.
However, this is obviously not this easy, because of the angle.
Is there any way to copy the inside of any rectangle, at any angle, into a new IplImage, which would be of the size of this very rectangle?
CvSeq* approximatedContour = cvApproxPoly(currentContour,
sizeof(CvContour),
0,
CV_POLY_APPROX_DP,
8);
// Circonscrire le polygone trouve dans un rectangle
etiquetteBox = cvMinAreaRect2(approximatedContour);
CvPoint2D32f boxPoints[4];
CvPoint2D32f* c1 = (&cvPoint2D32f(0,0),
&cvPoint2D32f(200,0),
&cvPoint2D32f(0,200),
&cvPoint2D32f(200,200));
CvMat* mmat = cvCreateMat(3,3,CV_32FC1);
cvBoxPoints(etiquetteBox, boxPoints);
IplImage* mask = cvCreateImage(cvSize(in->width,in->height), IPL_DEPTH_8U, 1);
IplImage* ROIimg = cvCreateImage(cvSize(in->width,in->height), IPL_DEPTH_8U, 1);
drawBox(mask,etiquetteBox,target_color[3]);
cvAnd(thresImg,mask,ROIimg,mask);
if(voirSeuillage)
cvCvtColor(ROIimg,in,CV_GRAY2BGR); //ROIimg is OK here!
mmat = cvGetPerspectiveTransform(boxPoints,c1,mmat);
cvWarpPerspective(ROIimg,thresImgResized,mmat); // here I get a full black image!
Doing this, as kindly suggested by Banthar, I get a full black image instead of what is delimited by boxPoints in ROIimg, what's wrong with this code?
After applying answer:
Here is what I do now :
double angle = 0.;
// TODO adaptive angle compensation
if(abs(etiquetteBox.angle) > 30)
angle = etiquetteBox.angle + 270.;
else
angle = etiquetteBox.angle - 270.;
CvPoint2D32f boxPoints[4];
CvPoint2D32f c1[] = {cvPoint2D32f(0,0),
cvPoint2D32f(20,0),
cvPoint2D32f(20,20),
cvPoint2D32f(0,20)};
CvMat* mmat = cvCreateMat(3,3,CV_32FC1);
cvBoxPoints(etiquetteBox, boxPoints);
Point center = Point(10,10);
//warp the image to fit the polygon into the 20x20 image
mmat = cvGetPerspectiveTransform(boxPoints,c1,mmat);
cvWarpPerspective(thresImg,thresImgResized,mmat);
//rotate the image because the inconsistent angle of etiquetteBox
// from a frame to the next ...
//it would be very cool to find a way to fix this...
CvMat rot_mat = getRotationMatrix2D( center, angle,1.0);
cvWarpAffine(thresImgResized,rotatedIm,&rot_mat);
It is still not quite what I want, because the object is rotating into the 20x20 rotatedImg; in thresImgResized, after cvWarpPerspective, the object is well segmented, BUT it is reversed because the inconsistency of the angle of etiquetteBox (-0 degrees in a frame, -90 in the next, depending on how I hold the object to be detected), which I get this way:
cvFindContours(dilImage,
contoursStorage,
&contours,
sizeof(CvContour),
CV_RETR_LIST,
CV_CHAIN_APPROX_TC89_KCOS);
// Trouver des polygones
CvSeq* currentContour = contours;
while(currentContour != 0 && !etiquette)
{
CvSeq* approximatedContour = cvApproxPoly(currentContour,
sizeof(CvContour),
0,
CV_POLY_APPROX_DP,
9);
// Circonscrire le polygone trouve dans un rectangle
etiquetteBox = cvMinAreaRect2(approximatedContour);
I don't know how to fix this, but, at least, it is better than setting my IplImage ROI, because I compensate the etiquetteBox's angle switch from -0 to -90 degrees in consecutive frames.
You are using the wrong brackets in definition of c1. Try this:
CvPoint2D32f c1[] = {
cvPoint2D32f(0,200),
cvPoint2D32f(0,0),
cvPoint2D32f(200,0),
cvPoint2D32f(200,200),
};
Related
I already tried to search about openCV ROI function, but All of them used rectangle roi function.
I want to get roi using by inclined line that get from hough transform function.
My situation is next :
I have multiple vertical lines(little inclined) that output from hough transform function.
i want to get image(Matrix) between vertical lines.
enter image description here
i want to get divided matrix in my image (For example, A image, B image, C image etc.. )
Is there ROI function that used line in openCV?
or
any another method?
I think you need to use contours to define your roi. If it is not a perfect square you can not use the ROI function, because this is always a perfect square (not even a rotated square)
int main()
{
enum hierIdx { H_NEXT = 0, H_PREVIOUS, H_FIRST_CHILD, H_PARENT };
cv::Mat img = cv::imread("example_image.jpg", cv::IMREAD_UNCHANGED);
// convert RGB to gray scale image
cv::Mat imgGrs;
cv::cvtColor(img, imgGrs, cv::COLOR_RGB2GRAY);
// because it was a .jpg the grey values are messed up
// we fix it by thresholding at 128
cv::threshold(imgGrs, imgGrs, 128, 255, cv::THRESH_BINARY);
imgGrs = ~imgGrs;
// now create contours (we need the hierarchy to find the inner shapes)
std::vector<std::vector<cv::Point> > contours;
std::vector<cv::Vec4i> hierarchy;
cv::findContours(imgGrs.clone(), contours, hierarchy, CV_RETR_TREE, CV_CHAIN_APPROX_SIMPLE);
//cv::drawContours(img, contours, -1, cv::Scalar(255, 0, 0), 1);
int iLen = (int)hierarchy.size();
int idxChild = -1;
// find first child of master
for (int i = 0; i < iLen; i++){
if (hierarchy[i][H_PARENT] < 0) {
idxChild = hierarchy[i][H_FIRST_CHILD];
break;
}
}
// used for erosion of mask
cv::Mat element = cv::getStructuringElement(cv::MORPH_ELLIPSE, cv::Size(3, 3));
while (idxChild >= 0)
{
// create image to use as mask for section
cv::Mat mask = cv::Mat::zeros(imgGrs.size(), CV_8U);
cv::drawContours(mask, contours, idxChild, cv::Scalar(255), CV_FILLED);
// make masker 1 pixel smaller so we wont see the outer contours
cv::erode(mask, mask, element);
// ok nu we create a singled out part we want
cv::Mat part = imgGrs & mask;
// Crop it to the AOI rectangle
cv::Rect aoi = cv::boundingRect(contours[idxChild]);
part = part(aoi);
// part is now the aoi image you asked for
// proceed to next AOI
idxChild = hierarchy[idxChild][H_NEXT];
}
return 0;
}
I'm unsing cv::undistort but it crops the image. I'd like to have all the undistorted image, so that the undistorted size is bigger then the original one, like this:
I think I need to use cv::getOptimalNewCameraMatrix but I had no luck with my trials.. some help?
Just for the record:
You should use cv::getOptimalNewCameraMatrix and set the alpha parameter to 1. Alpha 0 only shows valid points on the image, alpha 1 shows all the original points as well as black regions. cv::getOptimalNewCameraMatrix aslo gives you a ROI to crop the result from cv::undistort.
This code would do the trick:
void loadUndistortedImage(std::string fileName, Mat & outputImage,
Mat & cameraMatrix, Mat & distCoeffs) {
Mat image = imread(fileName, CV_LOAD_IMAGE_GRAYSCALE);
// setup enlargement and offset for new image
double y_shift = 60;
double x_shift = 70;
Size imageSize = image.size();
imageSize.height += 2*y_shift;
imageSize.width += 2*x_shift;
// create a new camera matrix with the principal point
// offest according to the offset above
Mat newCameraMatrix = cameraMatrix.clone();
newCameraMatrix.at<double>(0, 2) += x_shift; //adjust c_x by x_shift
newCameraMatrix.at<double>(1, 2) += y_shift; //adjust c_y by y_shift
// create undistortion maps
Mat map1;
Mat map2;
initUndistortRectifyMap(cameraMatrix, distCoeffs, Mat(),
newCameraMatrix, imageSize, CV_16SC2, map1, map2);
//remap
remap(image, outputImage, map1, map2, INTER_LINEAR);
}
See http://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html
and http://docs.opencv.org/2.4/modules/imgproc/doc/geometric_transformations.html
the best would be subclass OpenCV's class and overload the method undistort(), in order to access all the images you need.
For my project I am using parts of the next code: link.
To track objects of a specific color I implemented this method:
My question is: How can I calculate the distance to the tracked colored objects?
Thank you in advance!
*The application calls the method for the left and right frame. This is not efficient...
**I need to calculate detectedObject.Zcor
DetectedObject Detect(IplImage *frame)
{
//Track object (left frame and right frame)
//Calculate average position
//Show X,Y,Z coordinate and detected color
color_image = frame;
imgThreshold = cvCreateImage(cvSize(color_image->width,color_image->height), IPL_DEPTH_8U, 1);
cvInitFont(&font, CV_FONT_HERSHEY_PLAIN, 1, 1, 0, 1.4f, CV_AA);
imgdraw = cvCreateImage(cvGetSize(color_image),8,3);
cvSetZero(imgdraw);
cvFlip(color_image, color_image, 1);
cvSmooth(color_image, color_image, CV_GAUSSIAN, 3, 0);
threshold = getThreshold(color_image);
cvErode(threshold, threshold, NULL, 3);
cvDilate(threshold, threshold, NULL, 10);
imgThreshold = cvCloneImage(threshold);
storage = cvCreateMemStorage(0);
contours = cvCreateSeq(0, sizeof(CvSeq), sizeof(CvPoint), storage);
cvFindContours(threshold, storage, &contours, sizeof(CvContour), CV_RETR_CCOMP, CV_CHAIN_APPROX_NONE, cvPoint(0,0));
final = cvCreateImage(cvGetSize(color_image),8,3);
for(; contours!=0; contours = contours->h_next)
{
CvRect rect = cvBoundingRect(contours, 0);
cvRectangle(color_image,
cvPoint(rect.x, rect.y),
cvPoint(rect.x+rect.width, rect.y+rect.height),
cvScalar(0,0,255,0),
2,8,0);
string s = to_string(rect.x) + "," + to_string(rect.y);
char const* pchar = s.c_str();
cvPutText(frame, pchar, cvPoint(rect.x, rect.y), &font, cvScalar(0,0,255,0));
detectedObject.Xcor = rect.x;
detectedObject.Ycor = rect.y;
}
cvShowImage("Threshold", imgThreshold);
cvAdd(final,imgdraw,final);
detectedObject.Zcor = 0;
return detectedObject;
}
For depth estimation you will need a calibrated stereo pair (known camera matrices for both the left and the right cameras). Then, using the camera matrices and corresponding points/contours in the stereo pair, you can compute depth.
I am new in OpenCV so please to be lenient.
I am doing an Android application to recognize the squares/rectangles and crop them. Function which looks for the squares/rectangles puts the found objects to vector> squares. I just wonder how to crop the picture according to the data in points stored in vector> squares and how to compute an angle on which the picture should be rotated. Thank you for any help
This post is citing from OpenCV QA: Extract a RotatedRect area.
There's a great article by Felix Abecassis on rotating and deskewing images. This also shows you how to extract the data in the RotatedRect:
http://felix.abecassis.me/2011/10/opencv-rotation-deskewing/
You basically only need cv::getRotationMatrix2D to get the rotation matrix for the affine transformation with cv::warpAffine and cv::getRectSubPix to crop the rotated image. The relevant lines in my application are:
// This is the RotatedRect, I got it from a contour for example...
RotatedRect rect = ...;
// matrices we'll use
Mat M, rotated, cropped;
// get angle and size from the bounding box
float angle = rect.angle;
Size rect_size = rect.size;
// thanks to http://felix.abecassis.me/2011/10/opencv-rotation-deskewing/
if (rect.angle < -45.) {
angle += 90.0;
swap(rect_size.width, rect_size.height);
}
// get the rotation matrix
M = getRotationMatrix2D(rect.center, angle, 1.0);
// perform the affine transformation on your image in src,
// the result is the rotated image in rotated. I am doing
// cubic interpolation here
warpAffine(src, rotated, M, src.size(), INTER_CUBIC);
// crop the resulting image, which is then given in cropped
getRectSubPix(rotated, rect_size, rect.center, cropped);
There are lots of useful posts around, I'm sure you can do a better search.
Crop:
cropping IplImage most effectively
Rotate:
OpenCV: how to rotate IplImage?
Rotating or Resizing an Image in OpenCV
Compute angle:
OpenCV - Bounding Box & Skew Angle
Altought this question is quite old, I think there is the need for an answer that is not expensive as rotating the whole image (see #bytefish's answer). You will need a bounding rect, for some reason rotatedRect.boundingRect() didn't work for me, so I had to use Imgproc.boundingRect(contour). This is OpenCV for Android, the operations are almost the same for other environments:
Rect roi = Imgproc.boundingRect(contour);
// we only work with a submat, not the whole image:
Mat mat = image.submat(roi);
RotatedRect rotatedRect = Imgproc.minAreaRect(new MatOfPoint2f(contour.toArray()));
Mat rot = Imgproc.getRotationMatrix2D(rotatedRect.center, rotatedRect.angle, 1.0);
// rotate using the center of the roi
double[] rot_0_2 = rot.get(0, 2);
for (int i = 0; i < rot_0_2.length; i++) {
rot_0_2[i] += rotatedRect.size.width / 2 - rotatedRect.center.x;
}
rot.put(0, 2, rot_0_2);
double[] rot_1_2 = rot.get(1, 2);
for (int i = 0; i < rot_1_2.length; i++) {
rot_1_2[i] += rotatedRect.size.height / 2 - rotatedRect.center.y;
}
rot.put(1, 2, rot_1_2);
// final rotated and cropped image:
Mat rotated = new Mat();
Imgproc.warpAffine(mat, rotated, rot, rotatedRect.size);
as the title says i'm trying to find the number of non-zero pixels in a certain area of a cv::Mat, namely within a RotatedRect.
For a regular Rect one could simply use countNonZeroPixels on a ROI. However ROIs can only be regular (non rotated) rectangles.
Another idea was to draw the rotated rectangle and use that as a mask. However openCV neither supports the drawing of rotated rectangles nor does countNonZeroPixels accept a mask.
Does anyone have a solution for how to elegantly solve this ?
Thank you !
Ok, so here's my first take at it.
The idea is to rotate the image reverse to the rectangle's rotation and than apply a roi on the straightened rectangle.
This will break if the rotated rectangle is not completely within the image
You can probably speed this up by applying another roi before rotation to avoid having to rotate the whole image...
#include <highgui.h>
#include <cv.h>
// From http://stackoverflow.com/questions/2289690/opencv-how-to-rotate-iplimage
cv::Mat rotateImage(const cv::Mat& source, cv::Point2f center, double angle)
{
cv::Mat rot_mat = cv::getRotationMatrix2D(center, angle, 1.0);
cv::Mat dst;
cv::warpAffine(source, dst, rot_mat, source.size());
return dst;
}
int main()
{
cv::namedWindow("test1");
// Our rotated rect
int x = 300;
int y = 350;
int w = 200;
int h = 50;
float angle = 47;
cv::RotatedRect rect = cv::RotatedRect(cv::Point2f(x,y), cv::Size2f(w,h), angle);
// An empty image
cv::Mat img = cv::Mat(cv::Size(640, 480), CV_8UC3);
// Draw rotated rect as an ellipse to get some visual feedback
cv::ellipse(img, rect, cv::Scalar(255,0,0), -1);
// Rotate the image by rect.angle * -1
cv::Mat rotimg = rotateImage(img, rect.center, -1 * rect.angle);
// Set roi to the now unrotated rectangle
cv::Rect roi;
roi.x = rect.center.x - (rect.size.width / 2);
roi.y = rect.center.y - (rect.size.height / 2);
roi.width = rect.size.width;
roi.height = rect.size.height;
cv::imshow("test1", rotimg(roi));
cv::waitKey(0);
}
A totally different approach might be to rotate your image (in opposite direction), and still use the rectangular ROI in combination with countNonZeroPixels. The only problem will be that you have to rotate your image around a pivot of the center of the ROI...
To make it clearer, see attached example:
To avoid rotation in similar task I iterate over each pixel in RotatedRect with such function:
double filling(Mat& img, RotatedRect& rect){
double non_zero = 0;
double total = 0;
Point2f rect_points[4];
rect.points( rect_points );
for(Point2f i=rect_points[0];norm(i-rect_points[1])>1;i+=(rect_points[1]-i)/norm((rect_points[1]-i))){
Point2f destination = i+rect_points[2]-rect_points[1];
for(Point2f j=i;norm(j-destination)>1;j+=(destination-j)/norm((destination-j))){
if(img.at<uchar>(j) != 0){
non_zero+=1;
}
total+=1;
}
}
return non_zero/total;
}
It's looks like usual iteration over rectangle, but on each step we add unit 1px vector to current point in direction to destination.
This loop NOT iterate over all points and skip a few pixels, but it was okay for my task.
UPD: It much better to use LineIterator to iterate:
Point2f rect_points[4];
rect.points(rect_points);
Point2f x_start = rect_points[0];
Point2f x_end = rect_points[1];
Point2f y_direction = rect_points[3] - rect_points[0];
LineIterator x = LineIterator(frame, x_start, x_end, 4);
for(int i = 0; i < x.count; ++i, ++x){
LineIterator y = LineIterator(frame, x.pos(), x.pos() + y_direction, 4);
for(int j=0; j < y_count; j++, ++y){
Vec4b pixel = frame.at<Vec4b>(y.pos);
/* YOUR CODE HERE */
}
}