I have some rendered images from Maya, What I like to have is extrinsic and intrinsic matrices as you would get from the calibration of a real camera. I've extracted the rotation and translation but I can't figure out the intrinsic matrix. I found is the projection matrix but it doesn't work the same way. I want to be able to convert between world coordinate to pixel coordinate like this: https://docs.opencv.org/2.4/modules/calib3d/doc/camera_calibration_and_3d_reconstruction.html
In case someone else is looking for an answer, I found this link useful: https://github.com/chloelle/MayaCamera. It converts Maya camera attributes to OpenCV camera matrices.
Related
I'm trying to use ARCore poses in Colmap for 3D reconstruction.
I realize that the pose returned from ARCore is in OpenGL coordinate system.I'm trying to convert it to OpenCV coordinate system as required by colmap.
These are the steps I've done:
Convert the quaternion to normalized quaternions and then to a rotation matrix (3x3).
Convert the second and third columns to negative by multiplying it with [[1,0,0],[0,-1,0],[0,0,-1]] array. (Since OpenGL uses column major).
Transpose the rotation matrix (3x3) to get row major.
Convert it back to quaternions.
Still after doing this, the camera positions are wrong in reconstructions.
After a bit of reading, I thought it might be because Arcore is returning Model transform which transform points from model to world coordinates. But Colmap requires poses in world to camera coordinates. I was wondering I might be able to solve it if I convert the ARcore poses to world to camera coordinates first before doing everything else.
Is this the correct method? If yes, how do I get the View matrix? If not, what am I doing wrong here?
What I have
I'm generating images using the standard perspective camera in unity. The camera is aiming to the ground plane (in unity it's the xz-plane), see image. From this I need to remove the perspective so all crop rows are parallel to each other.
Methode
The warpPerspective() function from openCV can be used to remove perspective from an image. All information is known such as, field of view, rotation, position, ... and thus I know how a 3D point maps on the 2D plane and visa versa. The problem is OpenCV uses an other system. In openCV should be a 3X3 matrix and the transformation matrix from unit is a 4X4 matrix. Is there a conversion between the two? Or should I think of another strategy?
EDIT
I can not use the orthographic camera in unity.
Fixed
Solved the issue by constructing a ray from the camera origin through each pixel and looking for an intersection with the ground plane. After this I discretised the ground plane in a grid with the same resolution of the original image. Points that map to the same cell are accumulated
I you cannot use the unity's orthographic camera, what I would try to imitate the c++ code from the examples from your link in open CV documentation. Another approach can be to try to obtain the projection matrix of the points you want the projection to be removed by multiplying by the inverse matrix (the inverse of the transformation matrix of that point). A matrix multiplied by its inverse is the identitiy so the projection transformation would be removed. I think that should be possible, you can dig on that you can obtain/change the projection matrix checking this. The point would be to undo the projection transformation. Then you would need to obtain the according othographic projection matrix and apply it to obtain the positions you're after. That should be the same thing that the unity's orthographic camera does.
To understand the projection matrix to the lowest level this source is awesome.
I think that In the camera component you just need to change the projection from prespective to orthographic:
I want to convert the pixel coordinate into real world coordinate. And I found that the ARKit API provide a function in ARCamera call viewMatrix()
Returns a transform matrix for converting from world space to camera
space
It this function can obtain extrinsic matrix for the camera?
This may help:
self.sceneView.session.currentFrame?.camera.transform
The position and orientation of the camera in world coordinate space.
.transform documentation
You can directly extract the eulerAngles from this, but will have to parse the translation yourself.
How come you manually want to project pixels into world positions? (The transform alone isn't going to help you there obviously).
In my case, i use four sets of points to do the Bird's Eye Projection.But i forgot to do the camera calibration first!
So i want to know is the result is same doing Camera calibration before Bird's Eye Projection and after Bird's Eye Projection in OpenCV?
Can you give me some advice?Thank you very much.
Can you specify what calibration do you refer to? There are generally 2 kinds of camera parameters you can estimate during calibration - intrinsic and extrinsic.
Intrinsic parameters can be for simplicity assumed 'fixed' for particular camera, which includes lens and sensor. Those parameters typically include focal length, sensor's dimensions, and distortion coefficients.
Extrinsic parameters are 'dynamic', and typically refer to camera position and orientation.
Now, if you represent those as some abstract transformations - they don't commute, which means you can't change their order. So, if you want to apply homography to an image - you have to undistort it first, because generally homography maps plane to another plane, and after distortion your planes will be messed up.
But on the other hand, once you apply one transform, you can estimate how much of other transform you have 'left to do'. This is OK for linear stuff, but turns ugly if you warp distorted image using homography and THEN try to undistort it.
Tl,Dr - perform intrinsic calibration and undistortion first, since it is easier and they are fixed for camera, then apply your transformations.
If I have a camera which is already calibrated, so that I already know distortion coefficients, and the camera matrix. And that I have a set of points that all are in a plane, and I know the realworld metrics and pixel-location of those points, I have constructed a homography.
Given this homography, camera matrix and distortion coefficients, how can I find the camera pose in the easiest way? Prefferable by using openCV.
Can I for instance use the "DecomposeProjectionMatrix()" function?
It accepts only a 3x4 projection matrix, but I have a simple 3x3 homography
In this older post you have a method for that. It is a mathematical conversion that gives you the pose matrix, which is translation and rotation.