I've 4 ps3eye cameras. And I've calibrated camera1 and camera2 using cvStereoCalibrate() function of OpenCV library
using a chessboard pattern by finding the corners and passing their 3d coordinates into this function.
Also I've calibrated camera2 and camera3 using another set of chessboard images viewed by camera2 and camera3.
Using the same method I've calibrated camera3 and camera4.
So now I've extrinsic and intrinsic parameters of camera1 and camera2,
extrinsic and intrinsic parameters of camera2 and camera3,
and extrinsic and intrinsic parameters of camera3 and camera4.
where extrinsic parameters are matrices of rotation and translation and intrinsic are matrices of focus length and principle point.
Now suppose there's a 3d point(world coordinate)(And I know how to find 3d coordinates from stereo cameras) that is viewed by camera3 and camera4 which is not viewed by camera1 and camera2.
The question I've is: How do you take this 3d world coordinate point that is viewed by camera3 and camera4 and transform it with respect to camera1 and camera2's
world coordinate system using rotation, translation, focus and principle point parameters?
OpenCV's stereo calibration gives you only the relative extrinsic matrix between two cameras.
Acording to its documentation, you don't get the transformations in world coordinates (i.e. in relation to the calibration pattern ). It suggests though to run a regular camera calibration on one of the images and at least know its transformations. cv::stereoCalibrate
If the calibrations were perfect, you could use your daisy-chain setup to derive the world transformation of any of the cameras.
As far as I know this is not very stable, because the fact that you have multiple cameras should be considered when running the calibration.
Multi-camera calibration is not the most trivial of problems. Have a look at:
Multi-Camera Self-Calibration
GML C++ Camera Calibration Toolbox
I'm also looking for a solution to this, so if you find out more regarding this and OpenCV, let me know.
Related
Knowing the rotations and translations of two cameras in world coordinates (relative to some known point), how would I calibrate my stereo system?
In OpenCV the normal approach is to use a calibration pattern in front of both cameras to get point correspondences. These points are used in stereoCalibrate which calculates the rotation matrix R and translation vector T (and the fundamental matrix F). In the next step the stereo rectification can be done to row-align images of both cameras with stereoRectify. stereoRectify needs R and T to calculate the homographies for the perspective transform of the images and also calculates the Q-matrix for translating disparity to depth.
Giving the situation that R and T in the world coordinate system are already known (known is the rotation around the z-Axis (floor-ceiling or yaw angle in aeronomy) and the rotation around the axis perpendicular to the camera view (pitch angle)), in which coordinate system should they be given to stereoRectify? What I mean with that is that there is the coordinate system of Camera1, of Camera2, and the (or one) world coordinate system.
The computation of the essential matrix E can be done with R * S where S is the skew-symmetric matrix of T and the fundamental matrix F with M_r.inv().t() * E * M_l.inv() following LearningOpenCV 3 from Kaehler and Bradski (M_r and M_l are the camera intrinsics of the right and left camera respectively). Here the question on R and T is the same. Is it the rotation from one camera to the other in world coordinates or e.g. in the coordinate system of one camera?
A sketch of the involved coordinate systems can be found here:
How is the camera coordinate system in OpenCV oriented?, however it is still unclear for me how exactly R and T should be calculated.
The question is not terribly clear, but...
IIUC you know the extrinsic parameters of both cameras, ergo their relative pose, but not the intrinsic ones. Therefore you still need to calibrate the cameras' intrinsics.
Knowing the relative pose of the cameras simply allows you to calibrate the intrinsics of the two cameras independently. Whether this is a simplification for your procedure or not depends on your particular setup.
Note that, unless you have inferred the extrinsics you have from a separate, image-based procedure, you should hardly trust their values - especially if they are derived by some sort of CAD model of your rig. The reason is that, unless your cameras have quite low resolution, pixel-level accuracy is likely to be much finer than what the manufacturing tolerances of your rig would account for.
I'm currently trying to discover the 3D position of a projector within a real world coordinate system. The origin of such a system is, for example, the corner of a wall. I've used Open Frameworks addon called ofxCvCameraProjectorCalibration
that is based on OpenCV functions, namely calibrateCamera and stereoCalibrate methods. The application output is the following:
camera intrisic matrix (distortion coeficients included);
projector intrisic matrix (distortion coeficients included);
camera->projector extrinsic matrix;
My initial idea was, while calibration the camera, place the chessboard pattern at the corner of the wall and extract the extrinsic parameters ( [RT] matrix ) for that particular calibration.
After calibrating both camera and projector do I have all the necessary data to discover the position of the projector in real world coordinates? If so, what's the matrix manipulation required to get it?
How do you stereo cameras so that the output of the triangulation is in a real world coordinate system, that is defined by known points?
OpenCV stereo calibration returns results based on the pose of the left hand camera being the reference coordinate system.
I am currently doing the following:
Intrinsically calibrating both the left and right camera using a chess board. This gives the Camera Matrix A, and the distortion coefficients for the camera.
Running stereo calibrate, again using the chessboard, for both cameras. This returns the extrinsic parameters, but they are relative to the cameras and not the coordinate system I would like to use.
How do I calibrate the cameras in such a way that known 3D point locations, with their corresponding 2D pixel locations in both images provides a method of extrinsically calibrating so the output of triangulation will be in my coordinate system?
Calculate disparity map from the stereo camera - you may use cvFindStereoCorrespondenceBM
After finding the disparity map, refer this: OpenCv depth estimation from Disparity map
If I have a camera which is already calibrated, so that I already know distortion coefficients, and the camera matrix. And that I have a set of points that all are in a plane, and I know the realworld metrics and pixel-location of those points, I have constructed a homography.
Given this homography, camera matrix and distortion coefficients, how can I find the camera pose in the easiest way? Prefferable by using openCV.
Can I for instance use the "DecomposeProjectionMatrix()" function?
It accepts only a 3x4 projection matrix, but I have a simple 3x3 homography
In this older post you have a method for that. It is a mathematical conversion that gives you the pose matrix, which is translation and rotation.
So I have a depth map and the extrinsics and intrinsics of the camera.I want to get back the 3D points and the surface normals .I am using the functionReprojectImageTo3D.In the stereo rectify function to find Q how do I get the The rotation matrix
between
the 1st and the 2nd cameras’ coordinate systems? I have individual rotation matrix and translation vector but how do I get it for "between the cameras?"
.Also this would give me the 3D points .Is there a method to generate the surface normals?
Given that you have the extrinsic matrix of both cameras, can't you simply take the inverse extrinsic matrix of camera 1, multiplied by the extrinsic matrix of camera 2?
Also, for a direct relation between the two cameras, take a look at the Fundamental Matrix (or, more specific, the Essential matrix). See if you can find a copy of the book Multiple View Geometry by Hartley and Zisserman.
As for the surface normals, you can compute those yourself by computing crossproducts on the corners of triangles. However, you then first need the reconstructed 3D point cloud.