Consider the following problem:
I have the original image A saved as "A.png".
Moreover, I also have camera video feed that shows (possibly with some perspective transformation) an image of A, denoted Va, with some level of radial distortion.
I also have an homography from A to Va and its inverse.
How could I undistort Va? Note that I do not want to undo the perspective transformation, just remove the radial distortion from Va.
Example:
I have a fully mapped and undistorted reference image (including real world size)
an image from a video frame (left image)
and an homography and its inverse between those two
In our use case, the left image would have radial distortion but we would like to remove it without applying a simple backprojection (this would create artifacts)
Undistortion is the process to transform a distorted image (e.g. image with a fisheye pattern) to undistorted version.
In this case your video frames do not suffer from distortion. And if you have already determined homography matrix, you need to apply perspective transformation.
You might need to invert the homography matrix in case you need to invert the transformation direction.
Related
My setup has checkerboard charts with known world coordinates present in each image that I use to stitch images together (in a 2D plane) and to find my P-matrix. However, I am stuck on finding a general approach into combining all my images into a spherical image.
Known:
Ground truth correspondence points in each image
camera calibration parameters (camera matrix, distortion coefficients)
homography between images
world-image plane matrix: P = K[R | t] for each image. However I think this matrix's estimation isn't that great.
real world coordinates of ground truthed points
camera has almost only rotation, minimal translation
I know openGL well enough to do the spherical/texture wrapping once I can stitch the images into a cubemap format
Unknown:
Spherical image
image cubemap
I have two images, one is the result of applying an affine transform to the other.
I can register them using homography by extracting the points using the ORB_create function in OpenCV.
However, I want to calculate the Affine matrix needed for this transformation.
is there any way of doing it simply by having the two images?
Detect a rotated rectangle and use its corners to get your transformation matrix
Use : getPerspectiveTransform or getAffineTransform
Edit: regarding rotated rectangle detection :
Please check this Opencv tutorial on how to find contours and detect rotated rectangles Creating Bounding rotated boxes and ellipses for contours
I have two images.
Say one is a 10x10 which we call trainImage and then there is another queryImage which is the same chessboard photographed using a phone camera. Now I have to find the position of camera in (x,y,z) coordinates. Using openCV and feature detection I have been able to identify the chessboard object in photographed object, but how to go ahead with calculating the transformations on chessboard so that I can eventually calculate the position of camera. Any pointers to start looking upon will also be really appreciated. Thanks.
Edit:
Reframing the problem statement again, I have two images trainImage and queryImage. I need to find the position of camera i.e. (x,y,z) if we assume that trainImage is at (0,0,0) in queryImage. I did some reading to find this I need rvec(rotation vector) and tvec(translation vector).
When I use findHomography() function on two images I get a 3x3 homgraphy matrix using which I can find the pixels points(x,y) in queryImage by multiplying to pixel points(x,y) in trainImage. How can I use this homographyMatrix for calculating tvec and rvec.
In my case, i use four sets of points to do the Bird's Eye Projection.But i forgot to do the camera calibration first!
So i want to know is the result is same doing Camera calibration before Bird's Eye Projection and after Bird's Eye Projection in OpenCV?
Can you give me some advice?Thank you very much.
Can you specify what calibration do you refer to? There are generally 2 kinds of camera parameters you can estimate during calibration - intrinsic and extrinsic.
Intrinsic parameters can be for simplicity assumed 'fixed' for particular camera, which includes lens and sensor. Those parameters typically include focal length, sensor's dimensions, and distortion coefficients.
Extrinsic parameters are 'dynamic', and typically refer to camera position and orientation.
Now, if you represent those as some abstract transformations - they don't commute, which means you can't change their order. So, if you want to apply homography to an image - you have to undistort it first, because generally homography maps plane to another plane, and after distortion your planes will be messed up.
But on the other hand, once you apply one transform, you can estimate how much of other transform you have 'left to do'. This is OK for linear stuff, but turns ugly if you warp distorted image using homography and THEN try to undistort it.
Tl,Dr - perform intrinsic calibration and undistortion first, since it is easier and they are fixed for camera, then apply your transformations.
I want to project a point in 3D space into 2D image coordinates. I have the calibrated intrinsics and extrinsics of the camera I'm using. I have the camera matrix K and distortion coefficients D. However, I want the projected image coordinates to be of the undistorted image.
From my research, I found two ways to do this.
Use opencv's getOptimalNewCameraMatrix function to compute a new undistorted image's camera matrix K'. Then use this K' in opencv's projectPoints function, with the distortion parameters set to 0, to get the projected point.
Use projectPoints function using the raw camera matrix K, along with the distortion coefficients D in this function and get the projected point.
Should the output of both methods match?
I think that there is something missing in your thought.
Camera matrix K and dist. coefficent D are the parameters for make the undistortion (if your lens is distorting the image like in a fisheye). They are what is called intrinsic camera parameters.
If we change terms from computer vision to computer graphics, those parameters are the one you use for defining the frustum of the view, and, for example, they are used for getting the focal length of the camera.
But they are not enough to do the projection stuff.
For the projection, if you think in a computer graphics term (like opengl, for instance) you need to have the model-view-projection matrix. The model matrix is the matrix that specifies the position of the object in the world. The view matrix specifies the position of the camera, and the projection matrix specify the frustum (focal angle, perspective distortion, etc).
If you want to know how to transform the points of the model from 3d to 2d (or viceversa) you need the projection and the view matrixes (you have the model matrix because you have the 3d points from which you want to start). And in computer vision the view matrix is called estrinsic parameters.
So, you need the estrinsic parameters too, that are the position of the camera in the world. That is, for instance, those parameters are the rvec and tvec that cv:: projectPoints needs.
If you want to compute them, they are exactly the output of cv::solvePnP that do the opposite of what you want to do: from some known 3d points coupled with the known 2d projection on them on the camera screen, this function gives you the estrinsic parameters (from which you can get the view matrix for some opengl-opencv-augmented-reality-whatever application via cv::Rodrigues).
Last note: while the instrinsic parameters are fixed in all the pictures you shoot with a camera (while you don't change the focal length of course), the estrinisc parameters changes every time you move the camera for take a new picture from a different view point (that is: this changes the perspective point of view, so the 3D-2D projection you want to find)
Hope could help!