I was trying to do a lens calibration using OpenCV for a camera with variable zoom and focus (a broadcast camera). I managed to acquire decent parameters for a lens (focal length, k1, k2), however, I stopped at the nodal offset.
As I understand the nodal point of a lens is the point at which light rays converge. This is causing shift of a object from a camera in Z-coordinate. Basically, when I do cv::SolvePnP with my know parameters the distance from a object to camera is not exactly the same as it it in a world. For example, I completely zoomed in with a camera and put it in the focus. OpenCV can estimate that the pattern is roughly 3 meters away from a camera but when I measure it with laser measuring tool, it's 1.8 meters. This is not the case when you put lens wide because nodal offset is really small.
Question is, is there any method to measure nodal offset of the camera using a pattern and without measuring the distance of a pattern from a camera?
What I tried
I used the pan, tilt and roll tripod that can provide the rotation of the camera. I have put pattern in front of a camera and captured it several times with different angles. I was hoping I can see some difference in position when I transform pattern using a rotation from a tripod.
Also I noticed that the Unreal engine is estimating a nodal offset using a pattern by placing a CG object and aligning it with a video [link]. However I thought there is a different way how to achieve that without having a CG object.
Related
I have a multi-camera system where the field of views are mostly non-overlapping. I have been researching on methods to calibrate the camera extrinsics and the first thing I'm going to try is to take a picture of a chessboard at a known location and use solvePnP from OpenCV to find the extrinsic rotation and translation vectors for each camera separately (following the method described in the answer here).
My problem is, this method uses only one measurement and as every measurement it is prone to errors. I assume that by taking multiple measurements, either by changing the position or the orientation of the chessboard, the accuracy can be improved. But what would be the best way to combine the rotation and translation obtained from the different measurements? A simple average?
In theory I would think that an option could be using solvePnP on all the points at the same time. Since I am calculating extrinsics the camera can't be moved so I would have to change to position and/or orientation of the board for each picture and measure the 3D points positions as accurately as possible each time.
I'm also wondering if using two chessboards in the same picture would be a possible solution, even if OpenCV doesn't seem to support multiple chessboard detection.
Is there a better way to measure extrinsics or anything that I'm missing?
I use single-camera calibration with checkerboard and I used one fix position of the camera to do the calibration. Now my question is if I use the same position but change the height of the camera then do I need to do calibration again? If no then will I get the same result by using the different height of the camera?
In my case, I changed the height of the camera but the position of the camera was the same. And I got a different result when I changed height. So I was wondering that may I need to do again calibration of the camera or not?
please help me out.
Generally speaking, and to achieve greatest accuracy, you will need to recalibrate the camera whenever it is moved. However, if the lens mount is rigid enough w.r.t the sensor, you may get away with only updating the extrinsic calibration, especially if your accuracy requirements are modest.
To see why this is the case notice that, unless you have a laboratory-grade rig holding and moving the camera, you can't just change the height only. With a standard tripod, for example, there will in general be a motion in all three axes amounting to a significant fraction of the sensor's size, which will be reflected in visible motion of several pixel with respect to the scene.
Things get worse / more complicated when you also add rotation to re-orient the field of view, since a mechanical mount will not, in general, rotate the camera around its optical center (i.e. the exit pupil of the lens), and therefore every rotation necessarily comes with an additional translation.
In your specific case, since you are only interested in measurements on a plane, and therefore can compute everything using homographies, refining the extrinsic calibration amounts to just recomputing the world-to-image scale. This can easily be achieved by taking one or more images of objects of known size on the plane - a calibration checkerboard is just such an object.
When we compute the pose of the camera with respect to a primitive like a marker or a 3D model..etc, the origin of that primitive is usually precisly known like the origin of a chessboard or a marker (in blue).
Now the question is where is the origin of the camera (in black)? The vector translation of the pose is expressed with respect to which reference? How can we determine where it is?
The optical center is meant to be on the optical axis (ideally it projects to the center of the image), at a distance of the sensor equal to the focal length, which can be expressed in pixel units (knowing the pixel size).
You can see where the optical axis lies (it is the symmetry axis of the lens), but the optical center is somewhere inside the camera.
OpenCV uses the pinhole camera model to model cameras. The origin of the 3D coordinate system used in OpenCV, for camera calibration and other purposes, is the camera itself, or more specifically, the pinhole of the camera model. It is the point where all light rays that enter the camera converge to a point, and is also called the "centre of projection".
Real cameras with lenses do not actually have a pinhole. But by analysing images taken with the camera, it is possible to calculate a pinhole model which models the real camera's optics very closely. That is what OpenCV does when it calibrates your camera. As #Yves Daoust said, the pinhole of this model (and hence the 3D coordinate origin) will be a 3D point somewhere inside your camera (or possibly behind it, depending on its focal length), but it is not possible for OpenCV to say exactly where it is relative to your camera's body, because OpenCV knows nothing about the physical size or shape of your camera or its sensor.
Even if you knew exactly where the origin is relative to your camera's body, it probably would not be of much use, because you can't take any physical measurements with respect to a point that is located inside your camera without taking it apart! Really, you can do everything you need to do in OpenCV without knowing this detail.
I have a problem in which i have a stationary video camera in a room and several videos from it, i need to transform the image coordinates into world coordinates.
What i know:
1. all the measurements of the room.
2. 16 image coordinates and their respected world coordinates.
The problem i encounter:
At first i thought i just need to create a geometric transformation (According to http://xenia.media.mit.edu/~cwren/interpolator/), but i have a problem since the edge of the room are distorted in the image, and i cant calibrate the camera because i can't get a hold of the room or the camera.
Is there anyway i can overcome those difficulties and measure the distance in the room with some accuracy?
Thanks
You can calibrate the distortion of the camera by extracting first the edges of your room and then finding the best set of distortion parameters (that will minimize edge distortion).
There are few works that implement this approach though:
you can find a skeleton of distortion estimation procedure in R. Szeliski's book, but without an implementation;
alternatively, you can find a method + implementation (+ an online demo where you can upload your images) on IPOL.
Regarding the perspective distortion, after removing the lens distortion just proceed with the link that you have found by applying this method to the image of the four corners of the room floor.
This will give you the mapping between an image pixel and a ground pixel (and thus the object world coordinate, assuning you only want the X-Y coordinates). If you need the height measurement, then you need to find an object with a known height in your images to calibrate it too.
I am tracking a moving vehicle with a stereo camera system. In both images I use background segmentation to get only the moving parts in the pictures, then put a rectangle around the biggest object.
Now I want to get the 3D coordinates of the center of the rectangle. The identified centers in the two 2D pictures are almost correlating points (I know not exactly). I did a stereo calibration with MATLAB, so I have the intrinsic parameters of both cameras and the extrinsic parameters of the stereo system.
OpenCV doesn't provide any function for doing this as far as I know and to be honest reading Zisserman didn't really help me, but maybe I am just blind to the obvious.
This should work:
1. For both camera's, compute a ray from your camera origin through the rectangle's center.
2. Convert the rays to world coordinates.
3. Compute the intersection between the two rays (or the closest point, in case they do not exactly intersect)