I'm working on depth reconstruction from disparity map. I use OpenCV to calibrate my stereo camera, then undistort and rectify the images. I use LibELAS to compute the disparity map.
My question is: According to OpenCV document (https://docs.opencv.org/3.1.0/dd/d53/tutorial_py_depthmap.html), the depth is computed by depth = Baseline*focal_length/disparity. But according to middlebury dataset (http://vision.middlebury.edu/stereo/data/scenes2014/), the depth is computed by depth = baseline * focal_length / (disparity + doffs). The "doffs" is "x-difference of principal points, doffs = cx1 - cx0".
What does the "doffs" mean ? How can I get the "doffs" from OpenCV calibration ?
The OpenCV-Calibration gives you the intrinsic matrices for both of your cameras. These are 3x3 Matrices with the following style: (from doc)
fx 0 cx
0 fy cy
0 0 1
cx and cy are the coordiantes of your principle point. From there you can calculate doffs as stated by you. With ideal cameras these parameters are the center of the image. But in real cameras they differ in a few pixels.
Related
I'm using a physical camera in Unity where I set the focal length f and sensor size sx and sy. Can these parameters and image resolution be used to create a camera calibration matrix? I probably need the focal length in terms of pixels and the cx and cy parameters that denote the deviation of the image plane center from the camera's optical axis. Is cx = w/2 and cy = h/2 correct in this case (w: width, h: height)?
I need the calibration matrix to compute a homography in OpenCV using the camera pose from Unity.
Yes, that's possible. I have done that with multiple different camera models( pinhole model, fisheye lens, polynominal lens model, etc).
Calibrate your camera with opencv and put the calibration parameters to the shader. You need to write a custom shader. Have a look at my previous question.
Camera lens distortion in OpenGL
You don't need homography here.
#Tuebel gave me a nice piece of code and I have successfully adapted it to real camera models.
The hardest part will be managing the difference between opengl camera coordinate and opencv camera coordinate. The camera calibration parameters are of course calibrated based on the opencv camera coordinate.
I work on a project that I need the focal length to calculate the distance from the image. so is there any way to get is automatically form camera properties
You can calibrate your camera using OpenCV. See this tutorial for details. As a result you'll get the camera matrix in the form of
fx 0 cx
0 fy cy
0 0 1
where:
fx, fy focal length of the camera in x and y direction in pixels
cx, cy principal point (the point that all rays converge) coordinates in pixels
then if you know the physical diameters of the sensor you can call calibrationMatrixValues function to get focal length of the camera in real world units (e.g. millimeters).
The proper way is to do camera calibration which is not easy but well documented.
https://learnopencv.com/camera-calibration-using-opencv/
In the past I have used a desktop-application by BoofCV.
Tutorial Camera Calibration - BoofCV
The tutorial will guide you step by step.
It involves:
Downloading their app.
printing a checkerboard and sticking it on a hard surface which is fun.
Following the calibration process.
Remember: this has to be done only once (for each camera).
I'm trying to estimate depth from a stereo system with two cameras. The simple equation that I use is:
Depth = (Base line * Focal lenght) / (Pixel disparity * Pixel size)
but i can't find Pixel disparity and Pixel size
how to find pixel disparity , pixel size?
Thank you.
You can get pixel size form spec sheet of your camera sensor. Alternatively, pixel size is not required if you have calibrated your camera, so that calibrated focal length will be in pixels.
So you can modify your formula as:
Depth (in cm) = Baseline(in cm) * Focal Length(in pixels) / Disparity (in Pixels)
For getting pixel disparity, you can use OpenCV Block Matching and Semi-Global Block Matching techniques Calib 3D Docs. There are many more accurate disparity estimation algorithms were published.
http://docs.opencv.org/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html#calibratecamera
I used cv::calibrateCamera method with 9*6 chessboard pattern.
Now I am getting rvecs and tvecs corresponding to each pattern,
Can somebody explain the format of rvecs and tvecs?
As far as I have figured out it is each one is 3*1 matrix.
and OpenCV documentation suggests to see Rodrigues function.
http://en.wikipedia.org/wiki/Rodrigues'_rotation_formula
As far rodrigues is concerned it is way to rotate a vector
around a given axis with angle theta.
but for this we need four values unit Vector(ux,uy,uz) and the angle. but openCV seem to use only 3 values.
OpenCV rodrigues documentation refer the below link http://docs.opencv.org/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html#void Rodrigues(InputArray src, OutputArray dst, OutputArray jacobian)
says that it will convert 3*1 matrix to 3*3 rotation matrix.
Is this matrix same as which we use 3D graphics.
can I convert it to 4*4 matrix and use it for transformations like below
M4X4 [
x x x 0
x x x 0
x x x 0
0 0 0 1
]
x : are the values from output 3by3 matrix of rodrigues function.
Is the relationship valid:
Vout = M4X4 * Vin;
using the matrix above.
The 3x1 rotation vector can express a rotation matrix by defining an axis of rotation via the direction that the vector points and an angle via the magnitude of the vector. Using the opencv function Rodrigues(InputArray src, OutputArray dst) you can obtain a rotation matrix which fits the function you describe.
I have a pair of matched 2D features extracted from rectified stereo image. Using cvPerspectiveTransform function in OpenCV, I attempted to reconstruct those features in 3D. The result is not consistent with the actual object dimension in real world. I realize there is a function in Matlab calibration toolbox that converts 2D stereo features into 3D point cloud. Nevertheless, the features are lifted from original images.
If I want to work with rectified images, is it possible to reconstruct the 3D locations based on 2D feature locations and disparity information.
If you know the focal length (f) and the baseline width (b, the distance of the projection axis of both cameras) as well as the disparity (d) in a rectified stereo image pair, you can calculate the distance (Z) with the following formula:
Z = f*(b/d);
This follows from the following equations:
x_l = f*(X/Z); // projecting a 3D point onto the left image
x_r = f*((X+b)/Z); // projecting the same 3D point onto the right image
d = x_r - x_l = f * (b/Z); // calculating the disparity
Solving the last equation for Z should lead to the formula given above.