I am trying to understand the concept of homography. It gives features but I can't get that how does it calculate features from images?
A homography is nothing but a mapping between points on one surface or plane to points on another. In the case of computer vision, it is a matrix specifying the transformation between two views of images, for example.
A homography can be estimated by identifying the keypoints in both the images and then estimating the transformation between the two views. There are many keypoint descriptors available that help in identifying these keypoints.
In camera calibration, an extrinsic matrix is computed by capturing different views of an object of known geometry, like a Chessboard from which the corner points are detected. The matrix is estimated by mathematically solving for the detected points from the many different views captured.
A details derivation of the estimation and solving to obtain the homography matrix can be found in this book. :)
Related
I have two images obtained by a calibrated camera from two different poses. I also have correspondences of 2D points between the images. Some of the points have depth information, so I also know their 3D coordinates. I want to calculate the relative pose between the images.
I know I can compute a fundamental matrix or an essential matrix from the 2D points. I also know PnP can find the pose with 2D-to-3D correspondences and that it's also doable getting just correspondences of 3D points. However, I don't know any algorithm that takes advantage of all the available information. Is there any?
There is only one such algorithm: Bundle Adjustment - everything else is a hack. Get your initial estimates separately, use any "reasonable & simple" hacky way of merging them to get an initial estimate, then byte the bullet and bundle. If you are coding in C++, Google's Ceres is my recommended B.A. library.
What is the minimum number of chessboard image pairs in order to mathematically calibrate and rectify two cameras ? One pair is considered as a single view of the chessboard by each camera, ending with a left and right image of the same scene. As far as I know we need just one pair for a stereo system, as the stereo calibration seeks the relations between the tow cameras.
Stereo calibration seeks not only the rotation and translation between the two cameras, but also the intrinsic and distortion parameters of each camera. You need at least two images to calibrate each camera separately, just to get the intrinsics. If you have already calibrated each camera separately, then, yes, you can use a single pair of checkerboard images to get R and t. However, you will not get a very good accuracy.
As a rule of thumb, you need 10-20 image pairs. You need enough images to cover the field of view, and to have a good distribution of 3D orientations of the board.
To calibrate a stereo pair of cameras, you first calibrate the two cameras separately, and then you do another joint optimization of the parameters of both cameras plus the rotation and translation between them. So one pair of images will simply not work.
Edit:
The camera calibration algorithm used in OpenCV, Caltech Calibration Toolbox, and the Computer Vision System Toolbox for MATLAB is based on the work by Zhengyou Zhang. His paper explains it better than I ever could.
The crux of the issue here is that the points on the chessboard are co-planar, which is a degenerate configuration. You simply cannot solve for the intrinsics using just one view of a planar board. You need more than one view, with the board in different 3-D orientations. Views where the boards are in parallel planes do not add any information.
"One image with 3 corners give us 6 pieces of information can be used to solve both intrinsic and distortion. "
I think that this is your main error. These corners are not independent. A pattern with a 100x100 chessboard pattern does not provide more information than a 10x10 pattern in your perfect world as the points are on the same plane.
If you have a single view of a chessboard, a closer distance to the board can be compensated by the focus so that you are not (even in your perfect world) able to calibrate your camera's intrinsic AND extrinsic parameters.
I'm working on OpenCV in visual studio 2012 , i tried to get Homography matrix to Stitch 2 images in opencv so really i don't know the steps to follow to do'it :
First you should be aware that two point correspondences, which you illustrate in your image, are not sufficient for homography estimation. A homography has 7 free parameters - there are 8 matrix elements, but a homography is a homogeneous quantity. This means you can scale the matrix as you like, which reduces the free parameters by one. To estimate 7 parameters you need at least four corresponding points.
I think understanding what you do is more important than blindly calling OpenCv methods, so please read about the real algorithms. The simplest (but non-optimal) method for homography computation is DLT, which amounts te rearranging the equation y = Hx for four or more points in order to solve for the components of H in a least squares sense. This is a nice explanation of the details:
https://engineering.purdue.edu/kak/courses-i-teach/ECE661.08/solution/hw4_s1.pdf
The principal academic reference is Multiple View Geometry for Computer Vision by Heartley and Zisserman.
As pointed out in the comments, the OpenCv docs are here:
http://docs.opencv.org/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html#findhomography
To establish an homography between two images you need at least 4 points.
Each one has two coordinates, giving you two equations / constraints.
An homography has 8 degrees of freedom (it's a 3x3 full-rank matrix with 9 elements, -1 d.o.f. because of scale since it operates on homogeneous coordinates).
Note that only images of planar surfaces, or images taken by a camera rotating around its center can be related by an homography (in this latter case, the planar surface is the plane at infinity).
This page shows how to compute an homography with feature extraction and RANSAC.
I am dealing with the problem, which concerns the camera calibration. I need calibrated cameras to realize measurements of the 3D objects. I am using OpenCV to carry out the calibration and I am wondering how can I predict or calculate a volume in which the camera is well calibrated. Is there a solution to increase the volume espacially in the direction of the optical axis? Does the procedure, in which I increase the movement range of the calibration target in 'z' direction gives sufficient difference?
I think you confuse a few key things in your question:
Camera calibration - this means finding out the matrices (intrinsic and extrinsic) that describe the camera position, rotation, up vector, distortion, optical center etc. etc.
Epipolar Rectification - this means virtually "rotating" the image planes so that they become coplanar (parallel). This simplifies the stereo reconstruction algorithms.
For camera calibration you do not need to care about any volumes - there aren't volumes where the camera is well calibrated or wrong calibrated. If you use the chessboard pattern calibration, your cameras are either calibrated or not.
When dealing with rectification, you want to know which areas of the rectified images correspond and also maximize these areas. OpenCV allows you to choose between two extremes - either making all pixels in the returned areas valid and cutting out pixels that don't fit into the rectangular area or include all pixels even with invalid ones.
OpenCV documentation has some nice, more detailed descriptions here: http://opencv.willowgarage.com/documentation/camera_calibration_and_3d_reconstruction.html
i want to find a position of a point with opencv. i calibrated two cameras using cvCalibrateCamera2. so i know both intrinsic and extrinsic parameters. I read that with a known intrinsic and extrinsic parameters, i can reconstruct 3d by triangulation easily. Is there a function in opencv to achive this.I think cvProjectPoint2 may be useful but i don t understand what exactly. So how i can find 3d position of a point.
Thanks.
You first have to find disparities. There are two algorithms implemented in OpenCV - block matching (cvFindStereoCorrespondenceBM) and graph cuts (cvFindStereoCorrespondenceGC). The latter one gives better results but is slower. After disparity detection you can reproject the disparities to 3D using cvReprojectImageTo3D. This gives you distances for each point of the input images that is in both camera views.
Also note that the stereo correspondence algorithms require a rectified image pair (use cvStereoRectify, cvInitUndistortRectifyMap and cvRemap).