I am looking for an efficient way to calculate the position of an oject on a surface based on an image taken from a certain perspective.
Let me explain a little further.
There is an object on a rectangular flat surface.
I have a picture taken of this setup with the camera positioned at one of the corners of the surface area at a rather low angle.
On the picture I will thus see a somewhat distorted, diamond-shaped view of the surface area and somewhere on it the object.
Through some image processing I do have the coordinates of the object on the picture but now have to calculate the actual position of the object on the surface.
So I do know that the center of the object is at the pixel-coordinates (x/y) on the picture and I know the coordinates of the 4 reference points that represent the corners of the area.
How can I now calculate the "real world" position of the object most efficiently (x and y coordinates on the surface)?
Any input is highly appreciated since I have worked so hard on this I can't even think straight anymore.
Best regards,
Tom
You have to find a perspective transformation.
Here you may find an explanation and code in Matlab
HTH!
How good is your linear algebra? A perspective transformation can be described by a homography matrix. You can estimate that matrix using the four corner points, invert it and the calculate the world coordinates of every pixel in your image.
Or you can just let OpenCV do that for you.
Related
For a project, I need to store circles detected on some photos. The problem is that some of these photos are taken from an angle, meaning the circles are ellipses. Is it possible to somehow turn the ellipses into circles?
I thought of rectifying the ellipse, then transforming the rectangle to a square. Indeterminate problem comes to my mind, meaning there are too many possible variations for my approach, and the results are different for each approach.
To find perspective transform, you need to have 4 pairs of corresponding coordinates: points at distorted picture and their ideal positions after correction of perspective.
In this case you can calculate matrix of perspective transform with getPerspectiveTransform function and apply it to correct all the picture. Example
So I have been tinkering a little bit with opencv and I want to be able to use a camera image to get the position of certain objects that are lying flat on plane. These objects are simple shapes such as circles squares etc. They all have the same height of 5cm. To be able to relate real world points to pixels on the camera I painted 4 white squares on the plane with known distances between them.
So the steps I have been taking are:
Initialization:
Calibrate my camera using a checkerboard image and save the calibration data.
Get the input image. call cv::undistort with the calibration data for my camera.
Find the center points of the 4 squares in the image and pass that data and the real world coordinates of the squares to the cv::solvePnP function. Save the rvec and tvec return parameters.
Warp the perspective of the image so you can get a top down view from the image. This is essentially following this tutorial: https://docs.opencv.org/3.4.1/d9/dab/tutorial_homography.html
Use the resulting image to again find the 4 white squares and then calculate a "pixels per meter" translation constant which can relate a certain amount of difference in pixels between points to the real world distance on the plane where the 4 squares are.
Finding object, This is done after initialization:
Get the input image. call cv::undistort with the calibration data for my camera.
Warp the perspective of the image so you can get a top down view from the image. This is the same as step 4 during initialisation.
Find the centerpoint of the object to detect.
Since the centerpoint of the object is on a higher plane then where I calibrated I use the following formula to correct this(d = is the pixel offset from the center of the image. camHeight is the cameraHeight I measured by using a tape measure. h is height of the object):
d = x - (h * (x / camHeight))
So here for an illustration how I got this formule:
But still the coordinates are not matching up...
So I am wondering at all if this is the correct. Specifically I have the following questions:
Is using cv::undistort before using cv::solvenPnP correct? cv::solvePnP also takes the camera calibration data as input so I'm not sure if I have to pass an undistorted image to it or not.
Similar to 1. During Finding object I call cv::undistort -> cv::warpPerspective. Is this undistort necessary here?
Is my calculation to correct for the parallel planes in step 4 correct? I feel like I am missing something but I can't see what. One thing I am wondering is whether I can get the camera height from opencv once solvePnp is done.
I am a newbie to CV so If anything else is totally wrong please also point it out to me.
Thank you for reading this wall of text!
Given a photo containing a circle, for example this photo of a fountain:
is it possible to define the 3D position and rotation of the fountain in relation to the camera?
I realise we have to define the scale, so lets say the fountain is 2m wide (the diameter of the circle consisting of the inner rim of the fountain is 2m).
So assuming the circle is a perfect circle, and defining the diameter to 2m, is it possible to determine how the circle and the camera relate spatially? I dont know any camera matrix or anything, the only information i have is the picture.
I specifically want to determine the 3D coordinates of a given pixel on the rim of the fountain.
What would be the math and/or OpenCV code to do this?
Circle with perspective is an ellipse. So you basicly you need an ellipse detector.
This algorithm should work:
Detect all ellipses in the given image.
Filter ellipses that you think they are not a circles in origin. (This is not possible using just 1 Camera so you have to depend on previous knowledge. Something like that you knows that you are taking a photo for a circle).
mmm I stopped typing here and bring a paper&pen and started figuring how to estimate the Homography and it is not that easy! you should deal with the circle a special case of an ellipse and then try to construct a linear system of equations. However, I made quick googling :
https://www.researchgate.net/publication/265212988_Homography_estimation_using_one_ellipse_correspondence_and_minimal_additional_information
http://www.macs.hw.ac.uk/bmvc2006/papers/306.pdf
Seems very interesting topic, I am going to spare sometimes on it later!
I need to find the size or coordinates of a rectangle that is displayed as a quadrilateral in a 3D image. The quadrilateral is on a plane that lines up with 3d world vanishing points. To clarify, the quadrilateral IS a rectangle in the 3D world, and that's the rectangle I want the size of.
I do not need to get all the textures and make a new image. I also do not know the coordinates of the target rectangle as required by the homography (perspective transformation) solutions I've seen, because I don't know the aspect ratio it's supposed to have.
I've read through this thread: proportions of a perspective-deformed rectangle and the guy seemed to find an algorithm that works. However I've read other research papers that claim to calculate a homography yet they don't say how they did it. Also it seems such a basic function there would be something in the existing openCV library.
Thanks.
I would like to measure the displacement of an object between two images. The displacement can be anything in the image plane. The result should give the displacement, if possible in sub pixel accuracy.
There are some assumptions, which should make it easier, but didn't help me so far:
the camara objective is virtualy distortion free (telecentric) and oriented perpendicular to the object plane
the object plane never changes
the flat marker object (could be known image, e.g. a play card) is always in the object plane, so it isn't scaled or warped -> only rotational and translational changing.
My first approach was to take the feature recognition example from EmguCV, find the first object in the first image, take the relevant piece of that picture, use it now as template and search it in the second image. This did work, but a little unsatisfactory. There was scaling and warpping in the homography matrix (probably because of some points, that where assigned wrong) and the placing accuracy was quite bad.
I tried this once with the demo of the commercial image processing software Halcon and it worked like a charm in sub pixel accuracy. There you can do some sort of least square fit of a template to the image you are searching the object in. The result is an affine transform matrix and very precise.
Is there something comparable in EmguCV/OpenCV?
Thank you in advance!
Edit:
Found the solution in EmguCV in the function
CameraCalibration.EstimateRigidTransform(PointF[] src, PointF[] dest, bool fullAffine);
with fullAffine set to false. My problem before was, that I was using
Features2DToolbox.GetHomographyMatrixFromMatchedFeatures();
from the matching example.
Found the solution in EmguCV in the function
CameraCalibration.EstimateRigidTransform(PointF[] src, PointF[] dest, bool fullAffine);
with fullAffine set to false. My problem before was, that I was using
Features2DToolbox.GetHomographyMatrixFromMatchedFeatures();
from the matching example.
The only problem left was the small scaling still produced by EstimateRigidTransform, but I was able to calculate it out of the result.