I'm using a physical camera in Unity where I set the focal length f and sensor size sx and sy. Can these parameters and image resolution be used to create a camera calibration matrix? I probably need the focal length in terms of pixels and the cx and cy parameters that denote the deviation of the image plane center from the camera's optical axis. Is cx = w/2 and cy = h/2 correct in this case (w: width, h: height)?
I need the calibration matrix to compute a homography in OpenCV using the camera pose from Unity.
Yes, that's possible. I have done that with multiple different camera models( pinhole model, fisheye lens, polynominal lens model, etc).
Calibrate your camera with opencv and put the calibration parameters to the shader. You need to write a custom shader. Have a look at my previous question.
Camera lens distortion in OpenGL
You don't need homography here.
#Tuebel gave me a nice piece of code and I have successfully adapted it to real camera models.
The hardest part will be managing the difference between opengl camera coordinate and opencv camera coordinate. The camera calibration parameters are of course calibrated based on the opencv camera coordinate.
I am using opencv to calibrate my webcam. So, what I have done is fixed my webcam to a rig, so that it stays static and I have used a chessboard calibration pattern and moved it in front of the camera and used the detected points to compute the calibration. So, this is as we can find in many opencv examples (https://docs.opencv.org/3.1.0/dc/dbb/tutorial_py_calibration.html)
Now, this gives me the camera intrinsic matrix and a rotation and translation component for mapping each of these chessboard views from the chessboard space to world space.
However, what I am interested in is the global extrinsic matrix i.e. once I have removed the checkerboard, I want to be able to specify a point in the image scene i.e. x, y and its height and it gives me the position in the world space. As far as I understand, I need both the intrinsic and extrinsic matrix for this. How should one proceed to compute the extrinsic matrix from here? Can I use the measurements that I have already gathered from the chessboard calibration step to compute the extrinsic matrix as well?
Let me place some context. Consider the following picture, (from https://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html):
The camera has "attached" a rigid reference frame (Xc,Yc,Zc). The intrinsic calibration that you successfully performed allows you to convert a point (Xc,Yc,Zc) into its projection on the image (u,v), and a point (u,v) in the image to a ray in (Xc,Yc,Zc) (you can only get it up to a scaling factor).
In practice, you want to place the camera in an external "world" reference frame, let's call it (X,Y,Z). Then there is a rigid transformation, represented by a rotation matrix, R, and a translation vector T, such that:
|Xc| |X|
|Yc|= R |Y| + T
|Zc| |Z|
That's the extrinsic calibration (which can be written also as a 4x4 matrix, that's what you call the extrinsic matrix).
Now, the answer. To obtain R and T, you can do the following:
Fix your world reference frame, for example the ground can be the (x,y) plane, and choose an origin for it.
Set some points with known coordinates in this reference frame, for example, points in a square grid in the floor.
Take a picture and get the corresponding 2D image coordinates.
Use solvePnP to obtain the rotation and translation, with the following parameters:
objectPoints: the 3D points in the world reference frame.
imagePoints: the corresponding 2D points in the image in the same order as objectPoints.
cameraMatris: the intrinsic matrix you already have.
distCoeffs: the distortion coefficients you already have.
rvec, tvec: these will be the outputs.
useExtrinsicGuess: false
flags: you can use CV_ITERATIVE
Finally, get R from rvec with the Rodrigues function.
You will need at least 3 non-collinear points with corresponding 3D-2D coordinates for solvePnP to work (link), but more is better. To have good quality points, you could print a big chessboard pattern, put it flat in the floor, and use it as a grid. What's important is that the pattern is not too small in the image (the larger, the more stable your calibration will be).
And, very important: for the intrinsic calibration, you used a chess pattern with squares of a certain size, but you told the algorithm (which does kind of solvePnPs for each pattern), that the size of each square is 1. This is not explicit, but is done in line 10 of the sample code, where the grid is built with coordinates 0,1,2,...:
objp[:,:2] = np.mgrid[0:7,0:6].T.reshape(-1,2)
And the scale of the world for the extrinsic calibration must match this, so you have several possibilities:
Use the same scale, for example by using the same grid or by measuring the coordinates of your "world" plane in the same scale. In this case, you "world" won't be at the right scale.
Recommended: redo the intrinsic calibration with the right scale, something like:
objp[:,:2] = (size_of_a_square*np.mgrid[0:7,0:6]).T.reshape(-1,2)
Where size_of_a_square is the real size of a square.
(Haven't done this, but is theoretically possible, do it if you can't do 2) Reuse the intrinsic calibration by scaling fx and fy. This is possible because the camera sees everything up to a scale factor, and the declared size of a square only changes fx and fy (and the T in the pose for each square, but that's another story). If the actual size of a square is L, then replace fx and fy Lfx and Lfy before calling solvePnP.
I am trying to undistort a barrel/radial distortion from an image. When I see the equations they do not require the focal length of the camera. But the openCV API initundistortrectifymap requires them in form of the camera intrinsic matrix. Why so ? Anyway to do it without them? Because I understand the undistort is common for various distortion corrections.
The focal length is essential in distortion removal -since it provides info on the intrinsic parameters of the camera- and it is fairly simple to add it to the camera matrix. Just remember that you have to convert it from millimeters to pixels. This is done to ensure that the pixels are rectangular. For the conversion you need to know the sensor's height and width in millimeters, the horizontal (Sh) and vertical (Sv) number of pixels of the sensor and the focal length in millimeters. The conversion is done using the following equations:
fx = (f(mm) x Sh(px))/sensorwidth(mm)
fy = (f(mm) x Sv(px))/sensorheight(mm)
More on the camera matrix elements can be found here.
I have sucessfully calibrated an analog camera using opencv. The ouput focal length and principal points are in pixels.
I know in digital cameras you can easily multiply the size of the pixel in the sensor by the focal length in pixels and get the focal length in mm (or whatever).
How can I do with this analog camera to get the focal length in mm?
The lens manufacturers usually write focal length on the lens. Even the name of the lens contains it, e.g. "canon lens 1.8 50mm".
If not, you can try to measure it manually.
Get lens apart from the camera. Take a small well illuminated object, place it in 1-3 meters in from of lens and sheet of paper back from it. Get sharp and focused image of the object on the paper.
Now measure following:
a - distance from lens to the object;
y - object size;
y' - object image size on the paper;
f = a/(1+y/y') - focus distance.
If your output is in pixels, you must be digitizing the analog input at some point. You just need to figure out the size of the pixel that you are creating.
For example, if you are scanning film in, then you use the pixel size of the scanner.
I am trying to run the cvPOSIT algorithm to map points that are projected using an Optoma PK301 with a the Kinect's depth camera. I have already determined the intrinsic parameters of the projector by calibrating it using the Projector-Camera Calibration Toolbox (http://code.google.com/p/procamcalib/) in Matlab. Would I be able to use these intrinsic parameters (in particular the focal lengths fc) to determine the actual focal length of the projector to feed into the POSIT function in OpenCV?
Ok, the quick answer to this is that the POSIT algorithm assumes that the focal length in both the x and y axes are the same (as it would be on an ideal camera). For the purposes of the POSIT algorithm, just take the average of the two: (fx + fy)/2.