I'm working on a stereo vision system based on openCV which current return correct 3d coordinates, but in the wrong perspective.
I have program a function which give me the camera-3d-coordinate and the expected real-world-coordinate from a cheesboard, but I didn't find out how to generate a transformation matrix from this data.
All possible functions I found in OpenCV doesn't work because they work with 2d-coordinates on a image, and not with the calculated 3d coordinates.
Check out this answer. It is for 2D points, but if You expand T to 3 + 1 element and make R 3x3, it will work.
Here is code that uses this method.
Related
I'm trying to use ARCore poses in Colmap for 3D reconstruction.
I realize that the pose returned from ARCore is in OpenGL coordinate system.I'm trying to convert it to OpenCV coordinate system as required by colmap.
These are the steps I've done:
Convert the quaternion to normalized quaternions and then to a rotation matrix (3x3).
Convert the second and third columns to negative by multiplying it with [[1,0,0],[0,-1,0],[0,0,-1]] array. (Since OpenGL uses column major).
Transpose the rotation matrix (3x3) to get row major.
Convert it back to quaternions.
Still after doing this, the camera positions are wrong in reconstructions.
After a bit of reading, I thought it might be because Arcore is returning Model transform which transform points from model to world coordinates. But Colmap requires poses in world to camera coordinates. I was wondering I might be able to solve it if I convert the ARcore poses to world to camera coordinates first before doing everything else.
Is this the correct method? If yes, how do I get the View matrix? If not, what am I doing wrong here?
What I have
I'm generating images using the standard perspective camera in unity. The camera is aiming to the ground plane (in unity it's the xz-plane), see image. From this I need to remove the perspective so all crop rows are parallel to each other.
Methode
The warpPerspective() function from openCV can be used to remove perspective from an image. All information is known such as, field of view, rotation, position, ... and thus I know how a 3D point maps on the 2D plane and visa versa. The problem is OpenCV uses an other system. In openCV should be a 3X3 matrix and the transformation matrix from unit is a 4X4 matrix. Is there a conversion between the two? Or should I think of another strategy?
EDIT
I can not use the orthographic camera in unity.
Fixed
Solved the issue by constructing a ray from the camera origin through each pixel and looking for an intersection with the ground plane. After this I discretised the ground plane in a grid with the same resolution of the original image. Points that map to the same cell are accumulated
I you cannot use the unity's orthographic camera, what I would try to imitate the c++ code from the examples from your link in open CV documentation. Another approach can be to try to obtain the projection matrix of the points you want the projection to be removed by multiplying by the inverse matrix (the inverse of the transformation matrix of that point). A matrix multiplied by its inverse is the identitiy so the projection transformation would be removed. I think that should be possible, you can dig on that you can obtain/change the projection matrix checking this. The point would be to undo the projection transformation. Then you would need to obtain the according othographic projection matrix and apply it to obtain the positions you're after. That should be the same thing that the unity's orthographic camera does.
To understand the projection matrix to the lowest level this source is awesome.
I think that In the camera component you just need to change the projection from prespective to orthographic:
I am currently working on a camera 3D realsense camera that detection and calculate the box or boxes dimension.
I am new in computer vision. I first worked on i just work on detection objects detection with color or without color to get a basic understanding. Using C++ and openCV, I want to managed to get the corners (and their x y z pixel coordinates) of the square using smoothing (remove noise), edge detection (canny function), lines detection (Hough transform) and lines intersection (mathematical calculation) on an simplified picture (uniform background).
Now is my question: do you have any direction/recommendation/advice/literature about dimension calculation of box. https://www.youtube.com/watch?v=l-i2E7aZY6A
i am using c++ and opencv with Intel realsens 3D camera.
thanks in advance((-_-))
Once you have the colour image pixel coordinates of the box you can obtain the real-world coordinates (also known as vertices or camera coordinates in the documentation) using methods in the projection interface, then simple pythagoras to calculate the distance between the points in mm.
If you have no experience with RealSense I'd recommend reading the documentation and looking through the sample apps included with the SDK.
With PCL (Point Cloud Library) you can find planes (or spheres and other surfaces), then refine the result with 2D image processing (eg. edge detection).
http://pointclouds.org/
https://www.youtube.com/watch?v=VD044WAHEe4
Localization of an object specified in the image.
I am working on the project of computer vision to find the distance of an object using stereo images.I followed the following steps using OpenCV to achieve my objective
1. Calibration of camera
2. Surf matching to find fundamental matrix
3. Rotation and Translation vector using svd as method is described in Zisserman and Hartley book.
4. StereoRectify to get the projection matrix P1, P2 and Rotation matrices R1, R2. The Rotation matrices can also be find using Homography R=CameraMatrix.inv() H Camera Matrix.
Problems:
i triangulated point using least square triangulation method to find the real distance to the object. it returns value in the form of [ 0.79856 , .354541 .258] . How will i map it to real world coordinates to find the distance to an object.
http://www.morethantechnical.com/2012/01/04/simple-triangulation-with-opencv-from-harley-zisserman-w-code/
Alternative approach:
Find the disparity between the object in two images and find the depth using the given formula
Depth= ( focal length * baseline ) / disparity
for disparity we have to perform the rectification first and the points must be undistorted. My rectification images are black.
Please help me out.It is important
Here is the detail explanation of how i implemented the code.
Calibration of Camera using Circles grid to get the camera matrix and Distortion coefficient. The code is given on the Github (Andriod).
2.Take two pictures of a car. First from Left and other from Right. Take the sub-image and calculate the -fundmental matrix- essential matrix- Rotation matrix- Translation Matrix....
3.I have tried to projection in two ways.
Take the first image projection as identity matrix and make a second project 3x4d through rotation and translation matrix and perform Triangulation.
Get the Projection matrix P1 and P2 from Stereo Rectify to perform Triangulation.
My object is 65 meters away from the camera and i dont know how to calculate this true this based on the result of triangulation in the form of [ 0.79856 , .354541 .258]
Question: Do i have to do some extra calibration to get the result. My code is not based to know the detail of geometric size of the object.
So you already computed the triangulation? Well, then you have points in camera coordinates, i.e. in the coordinate frame centered on one of the cameras (the left or right one depending on how your code is written and the order in which you feed your images to it).
What more do you want? The vector length (square root of the sum of the square coordinates) of those points is their estimated distance from the same camera. If you want their position in some other "world" coordinate system, you need to give the coordinate transform between that system and the camera - presumably through a calibration procedure.
I have a small cube with n (you can assume that n = 4) distinguished points on its surface. These points are numbered (1-n) and form a coordinate space, where point #1 is the origin.
Now I'm using a tracking camera to get the coordinates of those points, relative to the camera's coordinate space. That means that I now have n vectors p_i pointing from the origin of the camera to the cube's surface.
With that information, I'm trying to compute the affine transformation matrix (rotation + translation) that represents the transformation between those two coordinate spaces. The translation part is fairly trivial, but I'm struggling with the computation of the rotation matrix.
Is there any build-in functionality in OpenCV that might help me solve this problem?
Sounds like cvGetPerspectiveTransform is what you're looking for; cvFindHomograpy might also be helpful.
solvePnP should give you the rotation matrix and the translation vector. Try it with CV_EPNP or CV_ITERATIVE.
Edit: Or perhaps you're looking for RQ decomposition.
Look at the Stereo Camera tutorial for OpenCV. OpenCV uses a planar chessboard for all the computation and sets its Z-dimension to 0 to build its list of 3D points. You already have 3D points so change the code in the tutorial to reflect your list of 3D points. Then you can compute the transformation.