Open cv stereo calibration with FOH 120°, getting "bad" rms
I'm currently working on a Project in which I have to calibrate a pair of cameras to rectify the images for a stream.
For the calibration I'm using a chessboard pattern.
I'm doing the calibration by following those steps:
I calibrate the master and slave camera independently with 30 images and save the intrinsics and extrinsics
Then I do the stereo calibration on both cameras using 10 images and feeding the intrinsics found by mono calibration.
Now the results I get are not really satisfying. Rms from mono calibration is in the range from 1.2 to 5 and the rms from the stereo calibration is in the range from 1.7 to 3.
What confuses me is the fact that opencv has a namespace for fisheye calibration but I'm not sure if I should use that because my cameras are not fisheye but wide angle.
I tried to set a lot of different flags in the cv::calibrate and cv::stereoCalibrate functions but that didn't lead me to a satisfying output.
Has anyone some experience in this field? Id be happy to share some code but I would need to know what part of it would be interesting, because I implemented it in a custom gstreamer plugin which makes it difficult to just share the whole code.
Related
I have a vehicle with two cameras, left and right. Is there a difference between me calibrating each camera separately vs me performing "stereo calibration" ? I am asking because I noticed in the OpenCV documentation that there is a stereoCalibrate function, and also a stereo calibration tool for MATLAB. If I do separate camera calibration on each and then perform a depth calculation using the undistorted images of each camera, will the results be the same ?
I am not sure what the difference is between the two methods. I performed normal camera calibration for each camera separately.
For intrinsics, it doesn't matter. The added information ("pair of cameras") might make the calibration a little better though.
Stereo calibration gives you the extrinsics, i.e. transformation matrices between cameras. That's for... stereo vision. If you don't perform stereo calibration, you would lack the extrinsics, and then you can't do any depth estimation at all, because that requires the extrinsics.
TL;DR
You need stereo calibration if you want 3D points.
Long answer
There is a huge difference between single and stereo camera calibration.
The output of single camera calibration are intrinsic parameters only (i.e. the 3x3 camera matrix and a number of distortion coefficients, depending on the model used). In OpenCV this is accomplished by cv2.calibrateCamera. You may check my custom library that helps reducing the boilerplate.
When you do stereo calibration, its output is given by the intrinsics of both cameras and the extrinsic parameters.
In OpenCV this is done with cv2.stereoCalibrate. OpenCV fixes the world origin in the first camera and then you get a rotation matrix R and translation vector t to go from the first camera (origin) to the second one.
So, why do we need extrinsics? If you are using a stereo system for 3D scanning then you need those (and the intrinsics) to do triangulation, so to obtain 3D points in the space: if you know the projection of a general point p in the space on both cameras, then you can calculate its position.
To add something to what #Christoph correctly answered before, the intrinsics should be almost the same, however, cv2.stereoCalibrate may improve the calculation of the intrinsics if the flag CALIB_FIX_INTRINSIC is not set. This happens because the system composed by two cameras and the calibration board is solved as a whole by numerical optimization.
I want to verify an algorithm about camera calibration. But using the pictures taken by myself is not convincing. Are there any canonical image libraries for camera calibration simulation?
Look at the page A Flexible New Technique for Camera Calibration by Zhengyou Zhang; in the section "Experimental data and result for camera calibration", you will find five images like this one , five sets of image coordinates (like for example this one https://www.microsoft.com/en-us/research/wp-content/uploads/2016/12/imagepointsone.txt) and the result of the calibration here: https://www.microsoft.com/en-us/research/wp-content/uploads/2016/12/completecalibration.txt
I do not know whether it is canonical or not, for sure Zhengyou Zhang is well known for his work related to camera calibration and his article:
ZHANG, Zhengyou. A flexible new technique for camera calibration. IEEE Transactions on pattern analysis and machine intelligence, 2000, 22.11: 1330-1334.
is highly cited.
You may also have a look at Camera Calibration Toolbox for Matlab by Jean-Yves Bouguet, his code is the basis for the OpenCV algorithms but I do not know whether there are images for accuracy and correctness testing.
DISCLAIMER: Apologies for this very large question, as it could take a lot of your time.
I have a stereo setup consisting of two webcams these cameras possess auto-focus technology. The stereo setup is in canonical configuration where the cameras are separated by 10 cm distance.
I am using the stereo_calib.cpp program for stereo calibration, provided by the OpenCV sample programs.
Initially, I have captured the chessboard images by my stereo rig as the way it is shown in the sample cpp stereo images, and then tried to calibrate the setup, but the stereo rectification was either completely blank or the undistorted left and right images are tilted about 40 degrees.
As this was the case, then I have captured a set of 17 stereo pair chessboard images by keeping the Z distance constant, without any rotation, at this point the stereo images were correctly rectified in the process of stereo calibration. This Working Set contains the images of chessboard taken by the stereo setup along with the program and the image of how well the rectification has been achieved.
Later, when I was trying to calibrate the stereo setup again (as the cameras in the stereo setup was disturbed), with another new set of images of the chessboard taken by my stereo rig, the program is unable to rectify the stereo images. I am providing the Non Working Set where you can check the images taken by the stereo setup along with the images of the rectification.
As a picture is worth a thousand words.
You gotta see the output images of the provided program, which let you know much more than what I could say in my own words.
I am trying to find some new ways of stereo face recognition techniques.
Any help in this regard is highly appreciated.
And adding to this, I also need some existing techniques by which I could kick start my experimentation on new ways of face recognition using the stereo information.
Say I plan to use OpenCV for 3D reconstruction using a stereo approach...and I do not have any special stereo camera but only webcams.
1.)How do I build a cheap stereo setup using a set of web cams?
2.)Is it possible to snap two images using web cams and convert them to stereo using openCV API?
I will use the stereo algorithm from the link below
Stereo vision with OpenCV
Using this approach I want to create a detailed mapping of an indoor environment.
(I would not like to use any projects like Insight3D which cannot be used for commercial purposes without distributing the source code)
You can find here a lot of resources including tutorials and stereo vision cameras
Firstly, ensure that your web cams don't have any inbuilt autofocus technology. As the cameras should have fixed focal length.
1) Align the cameras in canonical configuration with varying baseline distance. Then calibrate them using opencv's stereo_calib.cpp program. Usually, the distance will be 20-60cms. For some web cameras even 10cm will give you better results. If rms error and reprojection error are less than 0.5 then you could consider that the stereo setup is ready.
2) Yes, it is possible to capture stereo images from the setup which I just mentioned. Check out this link for capturing images from cameras.
OpenCV provides better algorithms from which one can do wonders with 3D vision.
Stereo is better suited for indoor environment as it is very sensitive to lighting variations.
I am trying to perform a 3D surface reconstruction from a stereo configuration with OpenCV example files. I have created a stereo camera from 2 web cams.
I have obtained the calibration parameters using stereo_calib.cpp ( https://code.ros.org/trac/opencv/browser/trunk/opencv/samples/cpp/stereo_calib.cpp?rev=4086 ) and generated a point cloud with stereo_match.cpp ( https://code.ros.org/trac/opencv/browser/trunk/opencv/samples/c/stereo_match.cpp?rev=2614 ). The resulting point cloud, opened with MeshLab doesn't resemble the original scene at all ( http://img707.imageshack.us/i/snapshot01u.png/ ).
What am I missing here?
Steps to recreate:
stereo_calib.cpp (on the same folder where the images are, with no arguments so it assumes default)
stereo_match left01.jpg right01.jpg -i intrinsics.yml -e extrinsics.yml -p cloud.asc
import cloud.asc on MeshLab
Thanks
The description and the image you've posted do not give a lot of information. But from my experience with 3D reconstruction it seem to be that the extrinsic or intrinsic parameters are not correct.
Thus, something had gone wrong with your calibration. You have to get the intrinsic calibration of both cameras and the extrinsic between your cameras.
The extrinsic parameters represent the relative position and orientation between the cameras. Thus, if they are incorrect anything can happen, mostly the point cloud is extremely degenerated.
The intrinsic parameters describe how the 3D points are projected onto the 2D image. Thus, if you have incorrect intrinsic parameters your point cloud is deformed with a linear transformation (assumed that there is a negligible radial/tangential distortion or distortion of a higher order, respectively).
There is a lot of good literature covering the topic of 3D reconstruction and stereo. One of the most important of them is the book "Multiple View Geometry" from Richard Hartley and Andrew Zisserman.