Currently I am working on 3D image visualizing project using C# and emgucv.net. On that project following steps already done with 2 images of same scene(a little different in rotation and translation),
Feature detection(SURF), matching and calculate homography
calculate fundamental matrix
calculate essential matrix using above fundamental and camera intrinsic matrices
finally calculate the Rotational and translational matrices
Also I have obtain 4 possible answers for transformational matrix(3X4 [R|T]) using different combinations of R and T by changing its sign. Now I want to select the correct transformation matrix from those 4 answers. Before that I want to check either one of the answer is correct. So I have to re-project the points of second image using "Camera intrinsic matrix" and each one of "Transformation matrix". After that I can compare with resultant points with the second image points to confirm the result(translational matrix).
My question is, How to re-combine translational matrix(rotational[3X3] and translational[3X1] matrix ) and camera intrinsic matrix to project points into image points using emgucv.net?
OR any alternative method to confirm the transformational matrix that I obtain?
Thanks in advance for any help.
Related
Hi i am a beginner in computer vision and i wish to know what exactly is the difference between a homography and affine tranformation, if you want to find the translation between two images which one would you use and why?. From papers and definitions I found online, I am yet to find the difference between them and where one is used instead of the other.
Thanks for your help.
A picture is worth a thousand words:
I have set it down in the terms of a layman.
Homography
A homography, is a matrix that maps a given set of points in one image to the corresponding set of points in another image.
The homography is a 3x3 matrix that maps each point of the first image to the corresponding point of the second image. See below where H is the homography matrix being computed for point x1, y1 and x2, y2
Consider the points of the images present below:
In the case above, there are 4 homography matrices generated.
Where is it used?
You may want to align the above depicted images. You can do so by using the homography.
Here the second image is mapped with respect to the first
Another application is Panoramic Stitching
Visit THIS BLOG for more
Affine transformation
An affine transform generates a matrix to transform the image with respect to the entire image. It does not consider certain points as in the case of homography.
Hence in affine transformation the parallelism of lines is always preserved (as mentioned by EdChum ).
Where is it used?
It is used in areas where you want to alter the entire image:
Rotation (self understood)
Translation (shifting the entire image by a certain length either to top/bottom or left/right)
Scaling (it is basically shrinking or blowing up an image)
See THIS PAGE for more
I ran Bouguet's calibration toolbox (http://www.vision.caltech.edu/bouguetj/calib_doc/htmls/example.html) on Matlab and have the parameters from the calibration (intrinsic [focal lengths and principal point offsets] and extrinsic [rotation and translations of the checkerboard with respect to the camera]).
Feature coordinate points of the checkerboard on my images are also known.
I want to obtain rectified images so that I can make a disparity map (for which I have the code for) from each pair of rectified images.
How can I go about doing this?
The documentation is here. At the end, it reads "Add these values as constants to your program, call the initUndistortRectifyMap and the remap function to remove distortion and enjoy distortion free inputs with cheap and low quality cameras".
Once your cameras are rectified, you may be interested in the class StereoVar or StereoBM to get the disparity map. Use reprojectImageTo3D once you are done if you want to check that your results look fine in 3D.
If fully calibrated use: http://link.springer.com/article/10.1007/s001380050120#page-1 Both cameras have the same orientation, share the same R.
First row of the new R is the baseline = subtraction of both camera centers. Second row cross product of baseline with old left z-axis (3 row R_old_left). Third row cross product of the first two rows.
Warp images with H_left=P_new(1:3,1:3)*P_old_left(1:3,1:3)^-1 and H_right=P_new(1:3,1:3)*P_old_right(1:3,1:3)^-1.
Rectified left pixel coordinates are u_new=(h11*u+h12*v+h13)/(h31*u+h32*v+h33), v=(h21*u+h22*v+h23)/(h31*u+h32*v+h33), same with the right ones
I have two images that are taken from different positions. The 2nd camera is located to the right, up and backward with respect to 1st camera.
So I think there is a perspective transformation between the two views and not just an affine transform since cameras are at relatively different depths. Am I right?
I have a few corresponding points between the two images. I think of using these corresponding points to determine the transformation of each pixel from the 1st to the 2nd image.
I am confused by the functions findFundamentalMat and findHomography. Both return a 3x3 matrix. What is the difference between the two?
Is there any condition required/prerequisite to use them (when to use them)?
Which one to use to transform points from 1st image to 2nd image? In the 3x3 matrices, which the functions return, do they include the rotation and translation between the two image frames?
From Wikipedia, I read that the fundamental matrix is a relation between corresponding image points. In an SO answer here, it is said the essential matrix E is required to get corresponding points. But I do not have the internal camera matrix to calculate E. I just have the two images.
How should I proceed to determine the corresponding point?
Without any extra assumption on the world scene geometry, you cannot affirm that there is a projective transformation between the two views. This is only true if the scene is planar. A good reference on that topic is the book Multiple View Geometry in Computer Vision by Hartley and Zisserman.
If the world scene is not planar, you should definitely not use the findHomography function. You can use the findFundamentalMat function, which will provide you an estimation of the fundamental matrix F. This matrix describes the epipolar geometry between the two views. You may use F to rectify your images in order to apply stereo algorithms to determine a dense correspondence map.
I assume you are using the expression "perspective transformation" to mean "projective transformation". To the best of my knowledge, a perspective transformation is a world to image mapping, not an image to image mapping.
The Fundamental matrix has the relation
x'Fu = 0
with x in one image and u in the other iff x and u are projections of the same 3d point.
Also
l = Fu
defines a line (lx' = 0) where the correponding point of u must be on, so it can be used to confine the searchspace for the correspondences.
A Homography maps a point on one projection of a plane to another projection of the plane.
x = Hu
There are only two cases where the transformation between two views is a projective transformation (ie a homography): either the scene is planar or the two views were generated by a camera rotating around its center.
I have 4 coplanar points in object coordinates and the correspoinding image points (on image plane). I want to compute the relative translation and rotation of the object plane with respect to the camera.
FindExtrinsicCameraParams2 is supposed to be the solution. But I'm having troubles with using it. Errors keep on showing when compiling
Has anyone successfully used this function in OpenCV?? Could I have some comments or sample code to use this function??
Thank you!
I would use the OpenCV function FindHomography() as it is simpler and you can converto easily from homography to extrinsic parameters.
You have to call the function like this
FindHomography(srcPoints, dstPoints, H, method, ransacReprojThreshold=3.0, status=None)
method is CV_RANSAC. If you pass more than 4 points, RANSAC will select the best 4-point set to satisfy the model.
You will get the homography in H, and if you want to convert it to extrinsic parameters you should do what I explain in this post.
Basically, the extrinsics matrix (Pose), has the first, second and fourth columns equal tp homography. The third column is redundant because it is the crossproduct of columns one and two.
After several days testing OpenCV functions related to 3D calibration, getting over all the errors, awkward output numbers, I finally get the correct outputs for these functions including findHomography, solvePnP (new version of FindExtrinsicCameraParams) and cvProjectPoints. Some of the tips have been discussed in use OpenCV cvProjectPoints2 function.
These tips are also applied for the error in this post. Specifically, in this post, my violation is passing float data to CV_64F Mat. All done now!!
You can use CalibrateCamera2.
objectPts - your 4 coplanar points
imagePts - your corresponding image points.
This method will compute instrinsic matrix and distortion coefficients, which tell you how the objectPts have been projected as the imagePts on to the camera's imaging plane.
There are no extrinsic parameters to compute here since you are using only 1 camera. If you used 2 cameras, then you are looking at computing Extrinsic Matrix using StereoCalibrate.
Ankur
I'm trying to learn OpenCV. I've a question regarding homography and epipolar geometry.
Suppose I've calculated homography using cvFindHomography() function using two static images' matched feature points taken with two cameras from two different view points.
Is it wrong if I reuse homography matrix to detect corresponding points in camera 1(right) from the image taken by camera2(left)(because I know that x' = H.x where x' is left images' 2d homogenous feature point, x is right images' 2d corresponding homogenous feature point and H is homography matrix) where the 2d points in camera1 and camera2 were not used to calculate homography matrix?
What I mean to ask is can I reuse calculated homography matrix of those two cameras to find corresponding points for any images that is not used to calculate homography matrix?
Does it matter which image I use when it was once determined by fixed images? or do i need to calculate it every time?
You can use homography to project points from one image to another as long as cameras don't move anymore and the scene doesn't change.
I understand that those cameras (calibrated) take the pictures and then you work with those two pictures all the time. Allright, if you calculate homography, then you can project all the points you want from both images. You will get some error, of course, but this is due to noise in the images and non-linearities that affect to linear method used by findhomography.
If you keep capturing images with the cameras then you have to compute homography again for every new pair of images, because you don't know a priori how the scene will change.