I need to translate the data from kinect 2 joint orientation so it will be in same format as kinect 1 BoneOrientation.
I've tried to understand from documentation what extacly the Joint orientation quaternions represent but with no success. I've tried comparing kinect 1 and kinect 2 and it looks like kinect 2 gives absolute rotation. but starting from the limbs (hands, shoulders, legs) there is a great difference.
When trying to use this data in my XNA avatar it looks like the shoulders and elbow are twisted.
Need some help to understand how to translate from kinect 2 to kinect 1.
Related
I'm trying to use ARCore poses in Colmap for 3D reconstruction.
I realize that the pose returned from ARCore is in OpenGL coordinate system.I'm trying to convert it to OpenCV coordinate system as required by colmap.
These are the steps I've done:
Convert the quaternion to normalized quaternions and then to a rotation matrix (3x3).
Convert the second and third columns to negative by multiplying it with [[1,0,0],[0,-1,0],[0,0,-1]] array. (Since OpenGL uses column major).
Transpose the rotation matrix (3x3) to get row major.
Convert it back to quaternions.
Still after doing this, the camera positions are wrong in reconstructions.
After a bit of reading, I thought it might be because Arcore is returning Model transform which transform points from model to world coordinates. But Colmap requires poses in world to camera coordinates. I was wondering I might be able to solve it if I convert the ARcore poses to world to camera coordinates first before doing everything else.
Is this the correct method? If yes, how do I get the View matrix? If not, what am I doing wrong here?
I'm working on a stereo vision system based on openCV which current return correct 3d coordinates, but in the wrong perspective.
I have program a function which give me the camera-3d-coordinate and the expected real-world-coordinate from a cheesboard, but I didn't find out how to generate a transformation matrix from this data.
All possible functions I found in OpenCV doesn't work because they work with 2d-coordinates on a image, and not with the calculated 3d coordinates.
Check out this answer. It is for 2D points, but if You expand T to 3 + 1 element and make R 3x3, it will work.
Here is code that uses this method.
I am trying to find the position of the camera in (x,y,z). R1 and R2 are the two reference points on the floor. R1' and R2' are the points shown in a image plane. 2492 pixels is the width of the viewing I was able to find the distance between R1 and R2 (0.98m), also between R1' and R2' (895.9pixels). The angle at which the camera is 69 degrees. The camera is placed on the left side of the reference points.
I am trying to do a mathematical model. If anyone could please help me with this would be much appreciated.
I think that the problem, as you described it, is too ambiguous to be solved.
Your problem is quite similar to the general PnP problem, whose objective is to estimate the relative pose between an object and a camera, based on a set of N known 3D points and their projections in the image. From what I know, the P3P problem, i.e. the PnP problem for 4 3D points and their 4 projections (see this website for a description), is the best we can solve. FYI, OpenCV implements the solvePnP function, which does that for N>=4.
Comparatively, in your problem, you do know the viewing angle of the camera, but you only have the distance between 2 3D points and the distance between their 2 projections. I do not think this can be solved as is. However, there may be a way if you look for hidden additional constraints (e.g. the camera center and 2 3D points are coplanar, etc) or if you intentionally add some (e.g. use more points).
I'm using openCV the calibrateCamera function to calibrate my camera. I started from the tutorial implementation, but there seems something wrong.
The camera is looking down on a table and i use a chessboard with an area that covers about 1/2 or 1/4 of my total image. Since I aim to track a flat object that slides over this table, I also slide my chessboard over this table.
So my first question is: is it ok that i move my chessboard over this table? Or do I have to make some 3D movements in order to get some good result?
Because I was wondering: how does the function guesses the distance between the table and the camera? He has only a guess of his focal point, and he has only one "eye", so there is no depth vision.
My second question: how does the bloody thing work? :p Can anyone show me some implementation of this function?
Thx!
the camera calibration needs a seed of points to calculate the camera matrix and the the position of the central point of the camera, and the distortion matrices , if you want to use a chessboard you have to take in consideration its dimension(I never used the circles function because the detection of chessboard is easier ) , the dimension of the chessboard should be pair X unpair number so you can get a correct rotation matrix ! the calibration function needs minimum 8x set of chessboardCorners and ( I use 30 tell 50) it depends on how precise you want to be .the return value of the calibration function is the re-projection error this should be near to zero if the calibration is good.
the cameraCalibration take a the size of used chessboards ( you can use different chessboardSize) and the dimension ( in mm or cm or even m etc.. ) your result will depend on your given dimension.
By the way after getting the chessboardCorners you have to refines them with the function CornerSubPix you can set how good the refinement is in the function parameter.
In the internet you can find a lot docs about this subject.
http://www.ics.uci.edu/~majumder/vispercep/cameracalib.pdf
I hope it helps !
regarding the chessboard positions, I got best results with 25-30 images
First I do 3 -4 images that show the chessboard at different distances full frame half 1/3 1/4
then I make sure to go to each corner, each center of each edge plus 4 rotation on each axis XYZ. When using a 640x480 sensor my reprojection error was mostly around 0.1 or even better
here a few links that got me in the right direction:
How to verify the correctness of calibration of a webcam?
I got a task:
We have a system working where a camera does a halfcircle around a human head. We know the camera matrix and the rotation/translation of every frame. (Distortion and more... but I want first to work without these parameters)
My task is that I have only the Camera Matrix, which is constant over this move, and the images (more than 100). Now I have to get the translation and rotation from frame by frame and compare it with the rotation and translation in real world (from the system which I have but only for compare, I have too prove it!)
First steps I did so far:
use the robustMatcher from the OpenCV Cookbook - works finde - 40-70 Matches each frame - visible looks it very good!
I get the fundamentalMatrix with getFundamental(). I use the robust Points from robustMatcher and RANSAC.
When I got the F i can get the Essentialmatrix E with my CameraMatrix K like this:
cv::Mat E = K.t() * F * K; //Found at the Bible HZ Chapter 9.12
Now we need to extract R and t out of E with SVD. By the way camera1 position is just zero because we have to start somewhere.
cv::SVD svd(E);
cv::SVD svd(E);
cv::Matx33d W(0,-1,0, //HZ 9.13
1,0,0,
0,0,1);
cv::Matx33d Wt(0,1,0,//W^
-1,0,0,
0,0,1);
cv::Mat R1 = svd.u * cv::Mat(W) * svd.vt; //HZ 9.19
cv::Mat R2 = svd.u * cv::Mat(Wt) * svd.vt; //HZ 9.19
//R1 or R2???
R = R1; //R2
//t=+u3 or t=-u3?
t = svd.u.col(2); //=u3
This is my actual status!
My plans are:
triangulate all points to get 3D points
Join frame i with frame i++
Visualize my 3D points them somehow!
Now my Questions are:
is this robust matcher dated? is there a other method?
Is it wrong to use this points as descriped at my second step? Must they be converted with distortion or something?
What R and t is this i extract here? Is it the rotation and translation between camera1 and camera2 with point of view from camera1?
When I read at the bible or papers or elsewhere i find that there are 4 possibilities how R and t can be!
´P′ = [UWV^T |+u3] oder [UWV^T |−u3] oder [UW^TV^T |+u3] oder [UW^TV^T |−u3]´
P´ is the projectionmatrix of the second image.
That means t could be - or + and R could be total different?!
I found out that I should calculate one point into 3D and find out if this point is infront of both cameras, then I have found the correct matrix!
I found some of this code at the internet and he just said this no further calculating:
cv::Mat R1 = svd.u * cv::Mat(W) * svd.vt
and
t = svd.u.col(2); //=u3
Why is this correct? If it isn't - how would I do this triangulation in OpenCV?
I compared this translation to the translation which is given to me. (First i had to transfer the translation and rotation in relationship to camera1 but I got this now!) But its not the same. The values of my program are just lets call it jumping from plus too minus. But it should be more constant because the camera is moving in a constant circle.
I am sure that some axes may be switched. I know that the translation is only from -1 till 1 but I thought I could extract a factor from my results to my comparevalues and then it should be similiar.
Does somebody have done something like this before?
Many people doing a camera calibration by using a chessboard, but I can't use this method to get the extrinsic parameters.
I know that visual sfm can do this somehow. (At youtube is a video where someone walks around a tree and get from these pictures a reconstruction of this tree using visual sfm)
This is pretty the same what I have to do.
Last question:
Does somebody know an easy way to visualize my 3D Points? I prefere MeshLab. Some experience with that?
Many people doing a camera calibration by using a chessboard, but I can't use this method to get the extrinsic parameters.
A chess board or checker board is used to find the internal/intrinsic matrix/parameters, not the extrinsic parameters. You're saying you have got the internal matrix already, I suppose that's what you meant by
We know the camera matrix and ...
Those videos you have seen on youtube have done the same, the camera is already calibrated, that is the internal matrix is known.
is this robust matcher dated? is there a other method?
I don't have that book so cant see the code and answer this.
Is it wrong to use this points as descriped at my second step? Must they be converted with distortion or something?
You need to cancel the radial distortion first, see undistortPoints.
What R and t is this i extract here? Is it the rotation and translation between camera1 and camera2 with point of view from camera1?
R is the orientation of the second camera in the first camera's coordinate system. And T is position of the second camera in that coordinate system. These have several usages.
When I read at the bible or papers or elsewhere i find that there are 4 possibilities how ....
Read the relevant section of the bible, this is very well explained there, triangulation is naive method, a better approach is explained there.
Does somebody know an easy way to visualize my 3D Points?
To see them in Meshlab a very easy way is to save the coordinate of the 3D points in a PLY file, this is an extremely simple format and supported by Meshlab and almost all other 3D model viewers.
In this article "An Efficient Solution to the Five-Point Relative Pose Problem", Nistér explain a very good method to determine which of the four configurations it the correct one (talking about R and T).
I've tried the robust matcher and I think is quiet good. The problems that has this matcher is that is really slow because it uses SURF, maybe you should try with others detectors and extractors to improve the speed.I also believe that the function in OpenCV that calculates the fundamental matrix does not need the Ransac parameter because the methods rate and symmetry do a great job removing the outliers, you should try the 8-point parameter.
OpenCV has the function triangulate, this only needs two projection Matrices, points that are in the first and the second image. Check the calib3d module.