How to find the coordinates of moving objects to draw rectangle - opencv

Does anyone know how to locate the coordinates of the moving object? I have found some examples online about tracking the objects by using optical flow, but I only got some tracked points on the moving objects. May I just draw rectangle around the each moving object instead? Is there a way to get the coordinates of each moving object? Appreciate any help in advance. Thanks!

Fit a rectangle to the points you get with optical flow and you can consider the centre of the fitted rectangle as a fair estimate of 2D trajectory of the whole moving body..

u can use the Moments operator
first calculate the contour size....
and just add this code block
Moments moment = moments((cv::Mat)contours[index]);
area = moment.m00;//m00 gives the area
x = moment.m10/area;//gives the x coordinate
y = moment.m01/area; //gives y coordiante
where the contours is the output of the findcontours(),

It is pretty hard to tell the coordinates of an object only from a couple of points on it. You can use moments (here is a tutorial) tu get a quite stable point describing where is Your object.
You may also do some additional work, like segmentation using tracked points to get the contour of tracked object, which should make it even easier to find its mass centre. Went overboard with ths.
There is also a tracking method called CAMSHIFT which returns a rectangle bounding the tracked object.
If You know precisely what are You tracking, and can make sure that some known points on tracked object are tracked, and You are able to recognise them, than You can use POSIT to determine the object's 3D coordinates and orientation. Take a glance at ArUco to get the idea about what I'm talking about.
To get the 3D position from previous methods, You can use stereo vision, and use centre of mass from both cameras to compute the coordinates.

Related

How to find the location of an object with respect to another object in an image?

We are doing a project in which we are detecting (using YOLOv4) runway debris using a fixed camera mounted at a pole on the side of the runway. We want to find out the position of the object with respect to the runway surface. How can we find out the distance of the object from the runway sides?
I would recommend using reliable sensors such as "light curtains" to make sure there is no debris on a runway. AI can fail, especially if things are hard to see.
As for mapping image coordinates to world coordinates on a plane, look into "homography". OpenCV has getPerspectiveTransform. it's easiest if you pick a big rectangle of known dimensions in your scene (runway), mark the corners, find them in your picture, note those points too, and those four pairs you give to getPerspectiveTransform. now you have a homography matrix that will transform coordinates back and forth (you can invert it).
check out the tutorials section in OpenCV's documentation for more on homographies.

Object projection in geographic coordinate system

I am trying to figure out how to roughly project the geographic position of an annotated object in an image?
The Setup
A picture with a known object in it. i.e. we know the width/height.
A bounding box highlighting where that object is in frame. X,Y,Width,Height.
The precise longitude and latitude of the camera that took the picture. The Origin.
The heading of the camera.
The focal length of the camera.
The camera sensor size.
The height of the camera off the ground.
Can anyone point me toward a solution for roughly projecting the objects location from the image origin location, given those data points?
The solution is simple if you assume an ellipsoidal surface for the Earth. If you need to use a Digital Terrain Model (DTM) things will get quickly more complicated. For example, your object may be visible in the image but occluded on the DTM because of various sources of error. In the following I assume you work with the ellipsoid.
Briefly, what you need to do is backproject the vertices of the image bounding box, obtaining four vectors (rays) in camera coordinates. You then transform them into Earth-Centered Earth-Fixed (ECEF) coordinates and solve for the intersection of the rays with the WGS-72 (or WGS-84) ellipsoid as explained here.
I recommend using the nvector library to help with this kind of calculations. You didin't specify the language you work with, but there are ports of nvector to many common languages, including Python and Matlab.

LookAt camera method? NOT SCNLookAtConstraint

I'm aware of the existence of SCNLookAtConstraint in SceneKit. This method, however, is not what I'm looking for.
What I need is able to point the camera at any given moment to a point (not a node) in space.
Having a X, Y, Z in world coordinates, I need the camera to rotate so that it points towards this point.
The points will change over time (meaning different points are being passed to a method, I need to 'jump' from one to the other).
I'd rather skip calculating eulerAngles, some solutions I have seen are based on trig functions and cos(angle) in the denominator. The user is basically free to rotate by dragging the cameraNode.
Any method I'm missing for this in SceneKit?
It seems to me that using a node as the target is the simplest way to go. If you create an empty node (no geometry or camera or light) it won't be visible, and you'll be able to use the highest available abstractions that the framework offers you. Move the node from point to point, with animation or with abrupt jumps.
The GLKMatrix4 methods may be of interest here, specifically GLKMatrix4MakeLookAt.
Given an eye position (camera), a position you wish to look at, and an up vector you will be able to create a transformation matrix (as a GLKMatrix4). The SceneKit function SCNMatrix4FromGLKMatrix4 will then translate this to a SCNMatrix4 that you can set as the transform of your camera node.
TBH, I would probably go with Hal's solution.

Getting 3D coordinates with two known correlating points in OpenCV

I am tracking a moving vehicle with a stereo camera system. In both images I use background segmentation to get only the moving parts in the pictures, then put a rectangle around the biggest object.
Now I want to get the 3D coordinates of the center of the rectangle. The identified centers in the two 2D pictures are almost correlating points (I know not exactly). I did a stereo calibration with MATLAB, so I have the intrinsic parameters of both cameras and the extrinsic parameters of the stereo system.
OpenCV doesn't provide any function for doing this as far as I know and to be honest reading Zisserman didn't really help me, but maybe I am just blind to the obvious.
This should work:
1. For both camera's, compute a ray from your camera origin through the rectangle's center.
2. Convert the rays to world coordinates.
3. Compute the intersection between the two rays (or the closest point, in case they do not exactly intersect)

Finding real world distance using an image

I am working on a project where I need to figure out real world distance of an object with reference to a fixed point using an image.
I have detected the object in a 2D image, using SURF. My object is inside a box now. What will give me the position of the centroid of the object. How can I use this to find out the real word distance?
If I plan to incorporate stereo vision, triangulating the centroid of the object, what is the difference between the distance I obtain here and in the previous method?
On a single image, probably the best starting point to learn about estimating metrical properties is Antonio Criminisi's work (2000 vintage, but still very relevant): http://www.cs.illinois.edu/~dhoiem/courses/vision_spring10/sources/criminisi00.pdf

Resources