I try to construct 3D point cloud and measure real sizes or distances of objects using stereo camera. The cameras are stereo calibrated, and I find 3D points using reprojection matrix Q and disparity.
My problem is the calculated sizes are changing depending the distance from cameras. I calculate the distances between two 3D points, it has to be constant, but when object gets closer to the camera, distance increasing.
Am i missing something? The 3D coordinates have to be in camera coordinates, not in pixel coordinates. So it seems inaccurate to me. Any idea?
You didn't mention how far apart your cameras are - the baseline. If they are very close together compared with the distance of the point that you are measuring, a slight inaccuracy in your measurement can lead to a big difference in the computed distance.
One way you can check if this is the problem is by testing with only lateral movement of the camera.
Related
When we compute the pose of the camera with respect to a primitive like a marker or a 3D model..etc, the origin of that primitive is usually precisly known like the origin of a chessboard or a marker (in blue).
Now the question is where is the origin of the camera (in black)? The vector translation of the pose is expressed with respect to which reference? How can we determine where it is?
The optical center is meant to be on the optical axis (ideally it projects to the center of the image), at a distance of the sensor equal to the focal length, which can be expressed in pixel units (knowing the pixel size).
You can see where the optical axis lies (it is the symmetry axis of the lens), but the optical center is somewhere inside the camera.
OpenCV uses the pinhole camera model to model cameras. The origin of the 3D coordinate system used in OpenCV, for camera calibration and other purposes, is the camera itself, or more specifically, the pinhole of the camera model. It is the point where all light rays that enter the camera converge to a point, and is also called the "centre of projection".
Real cameras with lenses do not actually have a pinhole. But by analysing images taken with the camera, it is possible to calculate a pinhole model which models the real camera's optics very closely. That is what OpenCV does when it calibrates your camera. As #Yves Daoust said, the pinhole of this model (and hence the 3D coordinate origin) will be a 3D point somewhere inside your camera (or possibly behind it, depending on its focal length), but it is not possible for OpenCV to say exactly where it is relative to your camera's body, because OpenCV knows nothing about the physical size or shape of your camera or its sensor.
Even if you knew exactly where the origin is relative to your camera's body, it probably would not be of much use, because you can't take any physical measurements with respect to a point that is located inside your camera without taking it apart! Really, you can do everything you need to do in OpenCV without knowing this detail.
I have filmed an object on aruco board from two positions with the same camera. I've undistorted images and calibrated camera before work. I take one red point on one shot and calculate 3d line, which corresponds to this point 3D space, and then project it to another image:
The problem is that there is some discrepancy (~5px-15px) between line and point on another image. I also observed the same problem with opengl-generated images, so it doesn't seem to be a problem of my camera. I use this piece of code to detect board position:
MarkerMapPoseTracker MSPoseTracker; // tracks the pose of the marker map
MSPoseTracker.setParams(camParam, theMarkerMapConfig);
MSPoseTracker.estimatePose(ret.markers);
Is it possible to increase tolerance somehow? I've also found function which has some sort of tolerance parameter:
bool estimatePose(Marker& m, const CameraParameters& cam_params, float markerSize, float minErrorRatio = 4 /*tau_e in paper*/)
but I don't know how to pass this parameter to MSPoseTracker.estimatePose. How can I improve precision, if I believe its possible to do it at least theory?
I have ended up with brute-force solution, which I have implemented in lack of time. I've taken coordinates of markers borders on the calibration board with getMarker3DInfo, found coordinates of the points being projected with cv::projectPoints, then found a homography between these two groups of four points. This allowed me to calculate L2 norm between original photo and projected calibration board. Then I have connected bobyqa algorithm, which is opensource GNU optimisation libraty, specialised for multidimensional optimisation with no derivatives for heavy-calculating discrepancy function. This gives pretty good result:
I calculated Disparity map in c# (Emgu). The attached file1 is left and right image, and disparity map. The noise of disparity map is high. How can i decrease noise in disparity map?
Thanks.
If you want better result in disparity map you should have good camera calibration. The more accurate camera calibration lead to more accurate disparity map.
As tiziran pointed out, good (stereo) calibration is important. Since you usually calibrate each camera retrieving its whole projection matrix, it's difficult to say which of its parameters is the most important.
Stereo calibration involves also the determination of the rotation and traslation of the second camera with respect to the first camera.
In your case, some other things have to be considered:
A) in general noise depends on window correlation size and on the correlation method. Several method exists. The bigger the correlation size, the lower the noise but also the lower the precision.
B) To have disparity, points have to be seen from both your cameras. Half of each image is out the field of view of the other camera, so it would be useless (and tipically noise occurs in areas where disparity couldn't be computated). I think in this case there is too much distance / rotation between the cameras (it does not helps).
C) It is hard to have good disparity where there is no texture at all, or where the texture dimension is too bigger than the correlation window size. In your images there are zones with uniform white color and no texture
D) I think that is hard to have good disparity where the carpet is out of focus. It is my personal consideration but this fact does not help you for sure.
I'm using stereo vision to obtain 3D reconstruction. I'm using opencv library.
I've implemented my code this way:
1) Stereo Calibration
2) undistort and Rectification of image pair
3) disparity map - using SGBM
4) 3D coordinates calculating depht map - unsing reprojectImageTo3D();
Results:
-Good disparity map, and good 3D reconstruction
-Bad 3D coordinates values, the distances don't corresponde to the reality.
The 3D distances, the distante between camera and object, have 10 mm error and increse with distance. I,ve used various baselines and i get always error.
When i compare the extrinsic parameter, vector T, output of "stereoRectify" the baseline match.
So i dont know where the problem is.
Can someone help me please, thanks in advance
CAlibration:
http://textuploader.com/ocxl
http://textuploader.com/ocxm
Ten mm error can be reasonable for stereo vision solutions, all depending of course on the sensor sensitivity, resolution, baseline and the distance to the object.
The increasing error with respect to the object's distance is also typical to the problem - the stereo correspondence essentially performs triangulation between the two video sensors to the object, and the larger the distance is the derivative of the angle between the video sensors to the object translates to larger distance on the depth axis, which means larger error. Good example is when the angle between the video sensors to the object is almost right, which means that any small positive error in estimating it will throw the estimated depth to infinity.
The architecture you selected looks good. You can try increasing the sensors resolution, or maybe dig in to the calibration process which has a lot of room for tuning in the openCV library - making sure only images taken with the chessboard being static are selected, choose higher variety of the different poses of the chessboard, adding images until the registration between the two images drops below the maximal error you can allow, etc.
I am dealing with the problem, which concerns the camera calibration. I need calibrated cameras to realize measurements of the 3D objects. I am using OpenCV to carry out the calibration and I am wondering how can I predict or calculate a volume in which the camera is well calibrated. Is there a solution to increase the volume espacially in the direction of the optical axis? Does the procedure, in which I increase the movement range of the calibration target in 'z' direction gives sufficient difference?
I think you confuse a few key things in your question:
Camera calibration - this means finding out the matrices (intrinsic and extrinsic) that describe the camera position, rotation, up vector, distortion, optical center etc. etc.
Epipolar Rectification - this means virtually "rotating" the image planes so that they become coplanar (parallel). This simplifies the stereo reconstruction algorithms.
For camera calibration you do not need to care about any volumes - there aren't volumes where the camera is well calibrated or wrong calibrated. If you use the chessboard pattern calibration, your cameras are either calibrated or not.
When dealing with rectification, you want to know which areas of the rectified images correspond and also maximize these areas. OpenCV allows you to choose between two extremes - either making all pixels in the returned areas valid and cutting out pixels that don't fit into the rectangular area or include all pixels even with invalid ones.
OpenCV documentation has some nice, more detailed descriptions here: http://opencv.willowgarage.com/documentation/camera_calibration_and_3d_reconstruction.html