Using python Segmentation-Models package, I have trained a model and now I can localize the object on the image. I would like to get the reference point into world coordinate system.
As it is seen one part of the object is kind of straight line. How can I get the middle point of this line and map it into world coordinates. I could maybe use Gabor filter or something like that but the object could be rotated, so the line is has not always same slope.
I am just looking for a tip to start off. Any idea to reference this object somehow and to map into World coordinate system ?
Related
I am trying to figure out how to roughly project the geographic position of an annotated object in an image?
The Setup
A picture with a known object in it. i.e. we know the width/height.
A bounding box highlighting where that object is in frame. X,Y,Width,Height.
The precise longitude and latitude of the camera that took the picture. The Origin.
The heading of the camera.
The focal length of the camera.
The camera sensor size.
The height of the camera off the ground.
Can anyone point me toward a solution for roughly projecting the objects location from the image origin location, given those data points?
The solution is simple if you assume an ellipsoidal surface for the Earth. If you need to use a Digital Terrain Model (DTM) things will get quickly more complicated. For example, your object may be visible in the image but occluded on the DTM because of various sources of error. In the following I assume you work with the ellipsoid.
Briefly, what you need to do is backproject the vertices of the image bounding box, obtaining four vectors (rays) in camera coordinates. You then transform them into Earth-Centered Earth-Fixed (ECEF) coordinates and solve for the intersection of the rays with the WGS-72 (or WGS-84) ellipsoid as explained here.
I recommend using the nvector library to help with this kind of calculations. You didin't specify the language you work with, but there are ports of nvector to many common languages, including Python and Matlab.
I'm aware of the existence of SCNLookAtConstraint in SceneKit. This method, however, is not what I'm looking for.
What I need is able to point the camera at any given moment to a point (not a node) in space.
Having a X, Y, Z in world coordinates, I need the camera to rotate so that it points towards this point.
The points will change over time (meaning different points are being passed to a method, I need to 'jump' from one to the other).
I'd rather skip calculating eulerAngles, some solutions I have seen are based on trig functions and cos(angle) in the denominator. The user is basically free to rotate by dragging the cameraNode.
Any method I'm missing for this in SceneKit?
It seems to me that using a node as the target is the simplest way to go. If you create an empty node (no geometry or camera or light) it won't be visible, and you'll be able to use the highest available abstractions that the framework offers you. Move the node from point to point, with animation or with abrupt jumps.
The GLKMatrix4 methods may be of interest here, specifically GLKMatrix4MakeLookAt.
Given an eye position (camera), a position you wish to look at, and an up vector you will be able to create a transformation matrix (as a GLKMatrix4). The SceneKit function SCNMatrix4FromGLKMatrix4 will then translate this to a SCNMatrix4 that you can set as the transform of your camera node.
TBH, I would probably go with Hal's solution.
Currently, I desperately try to detect an object (robot) based on 2D laser scans (of another robot). In the following two pictures, the blue arrow corresponds to the pose of the laser scanner and points towards the object, that I would like to detect.
one side of the object
two sides of the object
Since it is basically a 2D picture, my first approach was to to look for some OpenCV implementations such as HoughLinesP or LSDDetector in order to detect the lines. Unfortunately, since the focus of OpenCV is more on "real" images with "real" lines, this approach does not really work with the point clouds, as far as I have understood it correctly. Another famous library is the point-cloud library, which on the other hand focus more on 3D point clouds.
My current approach would be to segment the laser scans and then use some iterative closest point (ICP) C++ implementation to find a 2D point cloud template in the laser scans. Since I am not that familiar with object detection and all that nice stuff, I am quite sure that there are some more sophisticated solutions...
Do you have any suggestions?
Many thanks in advance :)
To get lines from points, you could try RANSAC.
You would iteratively fit lines to the points, then remove points corresponding to the new line and repeat until there is not enough points or the support is too low or something like that.
Hope it helps.
I am working on a project where I need to figure out real world distance of an object with reference to a fixed point using an image.
I have detected the object in a 2D image, using SURF. My object is inside a box now. What will give me the position of the centroid of the object. How can I use this to find out the real word distance?
If I plan to incorporate stereo vision, triangulating the centroid of the object, what is the difference between the distance I obtain here and in the previous method?
On a single image, probably the best starting point to learn about estimating metrical properties is Antonio Criminisi's work (2000 vintage, but still very relevant): http://www.cs.illinois.edu/~dhoiem/courses/vision_spring10/sources/criminisi00.pdf
Does anyone know how to locate the coordinates of the moving object? I have found some examples online about tracking the objects by using optical flow, but I only got some tracked points on the moving objects. May I just draw rectangle around the each moving object instead? Is there a way to get the coordinates of each moving object? Appreciate any help in advance. Thanks!
Fit a rectangle to the points you get with optical flow and you can consider the centre of the fitted rectangle as a fair estimate of 2D trajectory of the whole moving body..
u can use the Moments operator
first calculate the contour size....
and just add this code block
Moments moment = moments((cv::Mat)contours[index]);
area = moment.m00;//m00 gives the area
x = moment.m10/area;//gives the x coordinate
y = moment.m01/area; //gives y coordiante
where the contours is the output of the findcontours(),
It is pretty hard to tell the coordinates of an object only from a couple of points on it. You can use moments (here is a tutorial) tu get a quite stable point describing where is Your object.
You may also do some additional work, like segmentation using tracked points to get the contour of tracked object, which should make it even easier to find its mass centre. Went overboard with ths.
There is also a tracking method called CAMSHIFT which returns a rectangle bounding the tracked object.
If You know precisely what are You tracking, and can make sure that some known points on tracked object are tracked, and You are able to recognise them, than You can use POSIT to determine the object's 3D coordinates and orientation. Take a glance at ArUco to get the idea about what I'm talking about.
To get the 3D position from previous methods, You can use stereo vision, and use centre of mass from both cameras to compute the coordinates.