Related
I want to project a single point (-1450,-1660) on an image
I am using opencv 4.0.1 c++
I have the camera matrix and distortion coefficient
and my code is
vector <Point3f> inputpoints;
Point3f myPoint;
myPoint.x = -1450;
myPoint.y = -1660;
myPoint.z = 0;
inputpoints.push_back(myPoint);
vector<Point2f> outputpoints;
vector<Point3f> tvec;
tvec.push_back(Point3f(0, 0, 0));
vector<Point3f> rvec;
rvec.push_back(Point3f(0, 0, 0));
double mydata[9] = { 3.3202343554882879e+02, 1., 6.4337059696010670e+02, 0, 3.3196938477610536e+02, 5.3844814394773562e+02, 0., 0., 1. };
Mat mycameraMatrix = Mat(3, 3, CV_64F, mydata);
double mydata2[4] = { -1.1129472191078109e-03, 4.9443845791693870e-02,
-7.2244333582166609e-03, -1.7309984187889034e-03 };
Mat mydiscoff = Mat{ 4,1, CV_64F ,mydata2 };
Mat newCamMat1= Mat(3, 3, CV_64F);
cv::fisheye::projectPoints(inputpoints, rvec, tvec, mycameraMatrix, mydiscoff, outputpoints);
when I run the program I get this exception
OpenCV(4.0.1) Error: Assertion failed (mtype == type0 || (CV_MAT_CN(mtype) == CV_MAT_CN(type0) && ((1 << type0) & fixedDepthMask) != 0)) in cv::debug_build_guard::_OutputArray::create, file c:\build\master_winpack-build-win64-vc15\opencv\modules\core\src\matrix_wrap.cpp, line 1395
I changed the type of camera matrix and distortion coefficient to CV_32f but I still got the same error , I am a very beginner in openCV ..so can any one tell me what caused this exception?
I know the rvec should be 3*3 but I just followed someone else code who wrote that can be written in this way
okay the problem was that projectpoints and fisheye::projectpoints differ in the order of parameters ..so I was putting the order which belongs to projectpoints
I have written a C++ program using OpenCV that can detect and highlight the edges of any object from a live video. But now I don't know how to extract the four corners of the cube from the many edges that are being detected in the video. So I am looking for some help here.
Here is the link of the paper that I am using as a guide for my this project.
http://www.cs.ubc.ca/~andrejk/525project/525report.pdf
You can find the program code for this paper in the link below. It's written in Python. (I am using C++ and I don't know Python)
http://www.cs.ubc.ca/~andrejk/525project/cubefinder.py
According to the paper the next step would be, 'edge segmentation with adaptive threshold.'
Which I don't really understand. And also I don't know how to extract the corners of the cube then.
The short summary of the method that I have used is as following.
1. Input from webcam
2. Apply Laplacian filter
3. Apply Hough Line Transform.
I get the following result.
Code
using namespace std;
using namespace cv;
Mat laplacianFilter(Mat image)
{
Mat hImage;
GaussianBlur(image,hImage,Size(3,3),0,0,BORDER_DEFAULT);
cvtColor(hImage,hImage,CV_RGB2GRAY);
Laplacian(hImage,hImage,CV_16SC1,3,1,0,BORDER_DEFAULT);
convertScaleAbs(hImage,hImage,1,0);
return hImage;
}
Mat hghTransform(Mat image, Mat &image2)
{
Mat lImage;
Canny(image,image,50,200,3);
cvtColor(image,lImage,CV_GRAY2BGR);
vector<Vec4i> lines;
HoughLinesP(image, lines, 1, CV_PI/180, 50, 50, 10 );
for( size_t i = 0; i < lines.size(); i++ )
{
Vec4i l = lines[i];
line( image2, Point(l[0], l[1]), Point(l[2], l[3]), Scalar(0,255,0), 3, CV_AA);
}
return lImage;
}
int main()
{
int c;
VideoCapture cap(0);
Mat image;
Mat image2;
namedWindow("hghtransform");
namedWindow("laplacianfilter");
namedWindow("cannyOutput");
while(1)
{
cap>>image;
cap>>image2;
//Output
imshow("laplacianfilter",laplacianFilter(image));
imshow("cannyOutput",hghTransform(laplacianFilter(image),image2));
imshow("hghtransform",image2);
c=waitKey(33);
if(c==27)
return 0;
}
return 0;
}
Adaptive threshold will give you a clear line of edges which enables you to get 9 squares of a rubik side properly.
You can see a decent comparison of global and adaptive threshold here:
here: https://sites.google.com/site/qingzongtseng/adaptivethreshold
original image:
global threshold:
adaptive threshold:
For the corner, I am not sure whether it's stated in the paper, but I would do something like:
==> finding area like 1, 2, 3, 4 for upper-left, upper-right, lower-left, and lower-right corner respectively
==> with a template matching algorithm.
hope it helps.
note: you might want to have a background with less noise there. =)
I am trying to rotate the output of Hue-histogram which is (5*1) into another image which should have (1*5) dimension.
I am using the following code but i am getting segmentation fault:
MatND hist_input;
calcHist( &hsv_input, 1, channels, Mat(), hist_input, 1, histSize, ranges, true, false );
normalize( hist_input, hist_input, 0, 1, NORM_MINMAX, -1, Mat() );
MatND rotated_histogram;
rotated_histogram.create( 0, hist_input.rows, CV_32FC1);
for (int i=0; i<hist_input.rows; i++)
{
rotated_histogram.at<float>(0,i) = hist_input.at<float>(i,0);
}
I think it is easier to use the cv::MantND::t() function. It is returning the transpose, so it is turning it for you.
I would add an observation too: the at is very slow, so if you are not doing this for test it is ok, but I suggest you to use ptr for applications
I am trying to get the pose of the camera with the help of solvePNP() from OpenCV.
After running my program I get the following errors:
OpenCV Error: Assertion failed (npoints >= 0 && npoints == std::max(ipoints.checkVector(2, CV_32F), ipoints.checkVector(2, CV_64F))) in solvePnP, file /opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_tarballs_ports_graphics_opencv/opencv/work/OpenCV-2.4.2/modules/calib3d/src/solvepnp.cpp, line 55
libc++abi.dylib: terminate called throwing an exception
I tried to search how to solve these errors, but I couldn't resolve it unfortunately!
Here is my code, all comment/help is much appreciated:
enum Pattern { NOT_EXISTING, CHESSBOARD, CIRCLES_GRID, ASYMMETRIC_CIRCLES_GRID };
void calcBoardCornerPositions(Size boardSize, float squareSize, vector<Point3f>& corners,
Pattern patternType)
{
corners.clear();
switch(patternType)
{
case CHESSBOARD:
case CIRCLES_GRID:
for( int i = 0; i < boardSize.height; ++i )
for( int j = 0; j < boardSize.width; ++j )
corners.push_back(Point3f(float( j*squareSize ), float( i*squareSize ), 0));
break;
case ASYMMETRIC_CIRCLES_GRID:
for( int i = 0; i < boardSize.height; i++ )
for( int j = 0; j < boardSize.width; j++ )
corners.push_back(Point3f(float((2*j + i % 2)*squareSize), float(i*squareSize), 0));
break;
}
}
int main(int argc, char* argv[])
{
float squareSize = 50.f;
Pattern calibrationPattern = CHESSBOARD;
//vector<Point2f> boardCorners;
vector<vector<Point2f> > imagePoints(1);
vector<vector<Point3f> > boardPoints(1);
Size boardSize;
boardSize.width = 9;
boardSize.height = 6;
vector<Mat> intrinsics, distortion;
string filename = "out_camera_xml.xml";
FileStorage fs(filename, FileStorage::READ);
fs["camera_matrix"] >> intrinsics;
fs["distortion_coefficients"] >> distortion;
fs.release();
vector<Mat> rvec, tvec;
Mat img = imread(argv[1], CV_LOAD_IMAGE_GRAYSCALE); // at kell adnom egy kepet
bool found = findChessboardCorners(img, boardSize, imagePoints[0], CV_CALIB_CB_ADAPTIVE_THRESH);
calcBoardCornerPositions(boardSize, squareSize, boardPoints[0], calibrationPattern);
boardPoints.resize(imagePoints.size(),boardPoints[0]);
//***Debug start***
cout << imagePoints.size() << endl << boardPoints.size() << endl << intrinsics.size() << endl << distortion.size() << endl;
//***Debug end***
solvePnP(Mat(boardPoints), Mat(imagePoints), intrinsics, distortion, rvec, tvec);
for(int i=0; i<rvec.size(); i++) {
cout << rvec[i] << endl;
}
return 0;
}
EDIT (some debug info):
I debugged it row by row. I stepped into all of the functions. I am getting the Assertion failed in SolvePNP(...). You can see below what I see when I step into the solvePNP function. First it jumps over the first if statement /if(vec.empty())/, and goes into the second if statement /if( !copyData )/, there when it executes the last line /*datalimit = dataend = datastart + rows*step[0]*/ jumps back to the first if statement and returns => than I get the Assertion failed error.
template<typename _Tp> inline Mat::Mat(const vector<_Tp>& vec, bool copyData)
: flags(MAGIC_VAL | DataType<_Tp>::type | CV_MAT_CONT_FLAG),
dims(2), rows((int)vec.size()), cols(1), data(0), refcount(0),
datastart(0), dataend(0), allocator(0), size(&rows)
{
if(vec.empty())
return;
if( !copyData )
{
step[0] = step[1] = sizeof(_Tp);
data = datastart = (uchar*)&vec[0];
datalimit = dataend = datastart + rows*step[0];
}
else
Mat((int)vec.size(), 1, DataType<_Tp>::type, (uchar*)&vec[0]).copyTo(*this);
}
Step into the function in a debugger and see exactly which assertion is failing. ( Probably it requires values in double (CV_64F) rather than float. )
OpenCVs new "inputarray" wrapper issuppsoed to allow you to call functions with any shape of mat, vector of points, etc - and it will sort it out. But a lot of functions assume a particular inut format or have obsolete assertions enforcing a particular format.
The stereo/calibration systems are the worst for requiring a specific layout, and frequently succesive operations require a different layout.
The types don't seem right, at least in the code that worked for me I used different types(as mentioned in the documentation).
objectPoints – Array of object points in the object coordinate space, 3xN/Nx3 1-channel or 1xN/Nx1 3-channel, where N is the number of points. vector can be also passed here.
imagePoints – Array of corresponding image points, 2xN/Nx2 1-channel or 1xN/Nx1 2-channel, where N is the number of points.
vector can be also passed here.
cameraMatrix – Input camera matrix A = \vecthreethree{fx}{0}{cx}{0}{fy}{cy}{0}{0}{1} .
distCoeffs – Input
vector of distortion coefficients (k_1, k_2, p_1, p_2[, k_3[, k_4,
k_5, k_6]]) of 4, 5, or 8 elements. If the vector is NULL/empty, the
zero distortion coefficients are assumed.
rvec – Output rotation vector (see Rodrigues() ) that, together with tvec , brings points from the model coordinate system to the
camera coordinate system.
tvec – Output translation vector.
useExtrinsicGuess – If true (1), the function uses the provided rvec and tvec values as initial
approximations of the rotation and translation vectors, respectively,
and further optimizes them.
Documentation from here.
vector<Mat> rvec, tvec should be Mat rvec, tvec instead.
vector<vector<Point2f> > imagePoints(1) should be vector<Point2f> imagePoints(1) instead.
vector<vector<Point3f> > boardPoints(1) should be
vector<Point3f> boardPoints(1) instead.
Note: I encountered the exact same problem, and this worked for me(It is a little bit confusing since calibrateCamera use vectors). Haven't tried it for imagePoints or boardPoints though.(but as it is documented in the link above, vector,vector should work, I thought I'd better mention it), but for rvec,trec I tried it myself.
I run in exactly the same problem with solvePnP and opencv3. I tried to isolate the problem in a single test case. I seams passing a std::vector to cv::InputArray does not what is expected. The following small test works with opencv 2.4.9 but not with 3.2.
And this is exactly the problem when passing a std::vector of points to solvePnP and causes the assert at line 63 in solvepnp.cpp to fail !
Generating a cv::mat out of the vector list before passing to solvePnP works.
//create list with 3 points
std::vector<cv::Point3f> vectorList;
vectorList.push_back(cv::Point3f(1.0, 1.0, 1.0));
vectorList.push_back(cv::Point3f(1.0, 1.0, 1.0));
vectorList.push_back(cv::Point3f(1.0, 1.0, 1.0));
//to input array
cv::InputArray inputArray(vectorList);
cv::Mat mat = inputArray.getMat();
cv::Mat matDirect = cv::Mat(vectorList);
LOG_INFO("Size vector: %d mat: %d matDirect: %d", vectorList.size(), mat.checkVector(3, CV_32F), matDirect.checkVector(3, CV_32F));
QVERIFY(vectorList.size() == mat.checkVector(3, CV_32F));
Result opencv 2.4.9 macos:
TestObject: OpenCV
Size vector: 3 mat: 3 matDirect: 3
Result opencv 3.2 win64:
TestObject: OpenCV
Size vector: 3 mat: 9740 matDirect: 3
I faced the same issue. In my case, (in python) converted the input array type as float.
It worked fine afterwards.
I am doing some detection work using OpenCV, and I need to use the distance transform. Except the distance transform function in opencv gives me an image that is exactly the same as the image I use as source. Anyone know what I am doing wrong? Here is the portion of my code:
cvSetData(depthImage, m_rgbWk, depthImage->widthStep);
//gotten openCV image in "depthImage"
IplImage *single_channel_depthImage = cvCreateImage(cvSize(320, 240), 8, 1);
cvSplit(depthImage, single_channel_depthImage, NULL, NULL, NULL);
//smoothing
IplImage *smoothed_image = cvCreateImage(cvSize(320, 240), 8, 1);
cvSmooth(single_channel_depthImage, smoothed_image, CV_MEDIAN, 9, 9, 0, 0);
//do canny edge detector
IplImage *edges_image = cvCreateImage(cvSize(320, 240), 8, 1);
cvCanny(smoothed_image, edges_image, 100, 200);
//invert values
IplImage *inverted_edges_image = cvCreateImage(cvSize(320, 240), 8, 1);
cvNot(edges_image, inverted_edges_image);
//calculate the distance transform
IplImage *distance_image = cvCreateImage(cvSize(320, 240), IPL_DEPTH_32F, 1);
cvZero(distance_image);
cvDistTransform(inverted_edges_image, distance_image, CV_DIST_L2, CV_DIST_MASK_PRECISE, NULL, NULL);
In a nutshell, I grad the image from the kinect, turn it into a one channel image, smooth it, run the canny edge detector, invert the values, and then I do the distance transform. But the transformed image looks exactly the same as the input image. What's wrong?
Thanks!
I believe the key here is that they look the same. Here is a small program I wrote to show the difference:
#include <opencv2/core/core.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/imgproc/imgproc.hpp>
#include <iostream>
using namespace std;
using namespace cv;
int main(int argc, char** argv)
{
Mat before = imread("qrcode.png", 0);
Mat dist;
distanceTransform(before, dist, CV_DIST_L2, 3);
imshow("before", before);
imshow("non-normalized", dist);
normalize(dist, dist, 0.0, 1.0, NORM_MINMAX);
imshow("normalized", dist);
waitKey();
return 0;
}
In the non-normalized image, you see this:
which doesn't really look like it changed anything, but the distance steps are very small compared to the overall range of values [0, 255] (due to imshow converting the image from 32-bit float to 8-bits for display), we can't see the differences, so let's normalize it...
Now we get this:
The values themselves should be correct, but when displayed you will need to normalize the image to see the difference.
EDIT :
Here is a small 10x10 sample from the upper-left corner of the dist matrix show that the values are in fact different:
[10.954346, 10.540054, 10.125763, 9.7114716, 9.2971802, 8.8828888, 8.4685974, 8.054306, 7.6400146, 7.6400146;
10.540054, 9.5850525, 9.1707611, 8.7564697, 8.3421783, 7.927887, 7.5135956, 7.0993042, 6.6850128, 6.6850128;
10.125763, 9.1707611, 8.2157593, 7.8014679, 7.3871765, 6.9728851, 6.5585938, 6.1443024, 5.730011, 5.730011;
9.7114716, 8.7564697, 7.8014679, 6.8464661, 6.4321747, 6.0178833, 5.6035919, 5.1893005, 4.7750092, 4.7750092;
9.2971802, 8.3421783, 7.3871765, 6.4321747, 5.4771729, 5.0628815, 4.6485901, 4.2342987, 3.8200073, 3.8200073;
8.8828888, 7.927887, 6.9728851, 6.0178833, 5.0628815, 4.1078796, 3.6935883, 3.2792969, 2.8650055, 2.8650055;
8.4685974, 7.5135956, 6.5585938, 5.6035919, 4.6485901, 3.6935883, 2.7385864, 2.324295, 1.9100037, 1.9100037;
8.054306, 7.0993042, 6.1443024, 5.1893005, 4.2342987, 3.2792969, 2.324295, 1.3692932, 0.95500183, 0.95500183;
7.6400146, 6.6850128, 5.730011, 4.7750092, 3.8200073, 2.8650055, 1.9100037, 0.95500183, 0, 0;
7.6400146, 6.6850128, 5.730011, 4.7750092, 3.8200073, 2.8650055, 1.9100037, 0.95500183, 0, 0]
I just figured this one out.
The OpenCV distanceTransform
Calculates the distance to the closest zero pixel for each pixel of
the source image.
and so it expects your edges image to be negative.
All you need to do is to negate your edges image:
edges = 255 - edges;
You can print this values using this code before normalize function:
for(int x=0; x<10;x++)
{
cout<<endl;
for(int y=0; y<10;y++)
cout<<std::setw(10)<<dist.at<float>(x, y);
}
Mat formats
Input: CV_8U
Dist: CV_32F
Normalized: CV_8U
normalize(Mat_dist, Mat_norm, 0, 255, NORM_MINMAX, CV_8U);
If you want to visualize the result, you need to scale the normalization to 0 ... 255 and not to 0 ... 1 or everything will seem black. Using imshow(); on a scaled to 0 ... 1 image will work but may cause problmes in the next processing steps. Al least it did in my case.