Are there functions within OpenCV that will 'track' a gradually changing curve without following sharply divergent crossing lines? Ex: If one were attempting to track individual outlines of two crossed boomerangs, is there an easy way to follow the curved line 'through' the intersection where the two boomerangs cross?
This would require some kind of inertial component that would continue a 'virtual' line when the curve was interrupted by the other crossed boomerang, and then find the continuation of the original line on the opposite side.
This seems simple, but it sounds so complicated when trying to explain it. :-) It does seem like a scenario that would occur often (attempting to trace an occluded object). Perhaps part of a third party library or specialized project?
I believe I have found an approach to this. OpenCV's approxPolyDP finds polygons to approximate the contour. It is relatively easy to track angles between the polygon's sides (as opposed to finding continuous tangents to curves). When an 'internal' angle is found where the two objects meet, it should be possible to match with a corresponding internal angle on the opposite side.
Ex: When two bananas/boomerangs/whatever overlap, the outline will form a sort of cross, with four points and four 'internal angles' (> 180 degrees). It should be possible to match the coordinates of the four internal angles. If their corresponding lines (last known trajectory before overlap) are close enough to parallel, then that indicates overlapping objects rather than one more complex shape.
approxPolyDP simplifies this to geometry and trig. This should be a much easier solution than what I had previously envisioned with continuous bezier curves and inertia. I should have thought of this earlier.
Related
Thanks for taking your time to read this.
We have fixed stereo pairs of cameras looking into a closed volume. We know the dimensions of the volume and have the intrinsic and extrinsic calibration values
for the camera pairs. The objective being to be able to identify the 3d positions of multiple duplicate objects accurately.
Which naturally leads to what is described as the correspondence problem in litrature. We need a fast technique to match ball A from image 1 with Ball A from image 2 and so on.
At the moment we use the properties of epipolar geomentry (Fundamental matrix) to match the balls from different views in a crude way and works ok when the objects are sparse,
but gives a lot of false positives if the objects are densely scattered. Since ball A in image 1 can lie anywhere on the epipolar line going across image 2, it leads to mismatches
when multiple objects lie on that line and look similar.
Is there a way to re-model this into a 3d line intersection problem or something? Since the ball A in image 1 can only take a bounded limit of 3d values, Is there a way to represent
it as a line in 3d? and do a intersection test to find the closest matching ball in image 2?
Or is there a way to generate a sparse list of 3d values which correspond to each 2d grid of pixels in image 1 and 2, and do a intersection test
of these values to find the matching objects across two cameras?
Because the objects can be identical, OpenCV feature matching algorithms like FLANN, ORB doesn't work.
Any ideas in the form of formulae or code is welcome.
Thanks!
Sak
You've set yourself quite a difficult task. Because one point can occlude another in a view, it's not generally possible even to count the number of points. If each view has two points, but those points fall on the same epipolar line on the other view, then you can count anywhere between 2 and 4 points.
Assuming you want to minimize the points, this starts to look like Minimum Vertex Cover in a dense bipartite graph, with each edge representing the association of a point from each view, and the weight of each edge taken from the registration error of associating the corresponding points (vertices) from each view. MVC is, of course, NP-hard, and if you treat the problem as a general MVC problem then you'll never do better than O(n^2) because that's how many edges there are to examine.
Your particular MVC problem might have structure that can be exploited to perform a more efficient approximation. In particular, I might suggest calculating the epipolar lines in one view, ordering them by angle from the epipole, and similarly sorting the points in that view from the epipole. You can then iterate over the two sorted lists roughly in parallel, greedily associating each point with a nearby epipolar line. Then you can do the same in the other view, but only looking at points in that view which had not yet been associated during the previous pass. I think that a more regimented and provably optimal approach might be possible with dynamic programming (particularly if you strictly bound the registration error) which wouldn't require the second pass, but I can't sketch it out offhand.
For different types of objects it's easy- to find the match using sum-of-absolute-differences. For similar objects, the idea(s) could lead to publish a good paper. Anyway here's one quick algorithm:
detect the two balls in first image (using object detection methods).
divide the image into two segments cantaining two balls.
repeat steps 1 & 2 for second image also.
the direction of segments in two images should give correspondence of the two balls.
Try this, it should work for two balls.
Currently, I desperately try to detect an object (robot) based on 2D laser scans (of another robot). In the following two pictures, the blue arrow corresponds to the pose of the laser scanner and points towards the object, that I would like to detect.
one side of the object
two sides of the object
Since it is basically a 2D picture, my first approach was to to look for some OpenCV implementations such as HoughLinesP or LSDDetector in order to detect the lines. Unfortunately, since the focus of OpenCV is more on "real" images with "real" lines, this approach does not really work with the point clouds, as far as I have understood it correctly. Another famous library is the point-cloud library, which on the other hand focus more on 3D point clouds.
My current approach would be to segment the laser scans and then use some iterative closest point (ICP) C++ implementation to find a 2D point cloud template in the laser scans. Since I am not that familiar with object detection and all that nice stuff, I am quite sure that there are some more sophisticated solutions...
Do you have any suggestions?
Many thanks in advance :)
To get lines from points, you could try RANSAC.
You would iteratively fit lines to the points, then remove points corresponding to the new line and repeat until there is not enough points or the support is too low or something like that.
Hope it helps.
I have a problem with detecting if contour is a simple curve or line. I need to filter as many object as I can and leave only interesting objects for me. But it's not a goal to filter all unwanted objects. I have more logic to detecting required objects later but it cost a lot of time. So I'm looking for some low cost solution for filtering unwanted object before main algorithm take the part.
I have two examples from my application.
So I want to remove the yellow marked objects. And leave the green marked object. I don't circle everything, because they are so many :]. My idea was that if I can detect if the line is starting and ending in the corners of bounding box, but I dont'k know how can I detect where the line starts and ends. Or If I'm able to detect starting and ending points of contour I can decide it by their distance.
I use canny edge detection and find contours. And I have some basic filtering of object depending on their size.
I'm out of ideas and stuck at this part. I will be glad for your help or ideas.
I found that HoughLinesP do the job.
My goal is to create a CGPath that matches the outline of a SKSpriteNode.
This would be useful in creating glows/outlines of SKSpriteNodes as well as a path for physics.
One thought I have had, but I have not really worked much at all with CIImage, so I don't know if there is a way to access/modify images on a pixel level.
Then maybe I would be able to port something like this to Objective-C :
http://www.sakri.net/blog/2009/05/28/detecting-edge-pixels-with-marching-squares-algorithm/
Also very open to other approaches that make this process automated as opposed to me creating shape paths for every sprite I make for physics or outline/glow effects.
What you're looking for is called a contour tracing algorithm. Moore neighbor tracing is popular and works well for images and tilemaps. But do check out the alternatives because they may better fit your purposes.
AFAIK marching squares and contour tracing are closely related, if not the same (class of) algorithms.
An implementation for tilemaps (to create physics shapes from tiles) is included in Kobold Kit. The body of the algorithm is in the traceContours method of KKTilemapLayerContourTracer.m.
It looks more complex than it really is, on the other hand it takes a while to wrap your head around it because it is a "walking" algorithm, meaning the results of prior steps is used in the current step to make decisions.
The KK implementation also includes a few minor fixes specifically for tilemaps (ie two or more horizontally or vertically connected tiles become a single line instead of dividing the line into tile-sized segments). It was also created with a custom point array structure, and when I ported it to SK I decided it would be easier to continue with that and only at the end convert the point arrays to CGPath objects.
You can make certain optimizations if you can safely assume that the shape you're trying to trace is not going to touch the borders, and there can not be any tiles that are only connected diagonally. All of this becomes clearer when you're actually implementing the algorithm for your own purposes.
But as far as a ready-made, fits-all-purposes solution goes: there ain't none.
How can I implement the A* algorithm on a gridless 2D plane with no nodes or cells? I need the object to maneuver around a relatively high number of static and moving obstacles in the way of the goal.
My current implementation is to create eight points around the object and treat them as the centers of imaginary adjacent squares that might be a potential position for the object. Then I calculate the heuristic function for each and select the best. The distances between the starting point and the movement point, and between the movement point and the goal I calculate the normal way with the Pythagorean theorem. The problem is that this way the object often ignores all obstacle and even more often gets stuck moving back and forth between two positions.
I realize how silly mu question might seem, but any help is appreciated.
Create an imaginary grid at whatever resolution is suitable for your problem: As coarse grained as possible for good performance but fine-grained enough to find (desirable) gaps between obstacles. Your grid might relate to a quadtree with your obstacle objects as well.
Execute A* over the grid. The grid may even be pre-populated with useful information like proximity to static obstacles. Once you have a path along the grid squares, post-process that path into a sequence of waypoints wherever there's an inflection in the path. Then travel along the lines between the waypoints.
By the way, you do not need the actual distance (c.f. your mention of Pythagorean theorem): A* works fine with an estimate of the distance. Manhattan distance is a popular choice: |dx| + |dy|. If your grid game allows diagonal movement (or the grid is "fake"), simply max(|dx|, |dy|) is probably sufficient.
Uh. The first thing that come to my mind is, that at each point you need to calculate the gradient or vector to find out the direction to go in the next step. Then you move by a small epsilon and redo.
This basically creates a grid for you, you could vary the cell size by choosing a small epsilon. By doing this instead of using a fixed grid you should be able to calculate even with small degrees in each step -- smaller then 45° from your 8-point example.
Theoretically you might be able to solve the formulas symbolically (eps against 0), which could lead to on optimal solution... just a thought.
How are the obstacles represented? Are they polygons? You can then use the polygon vertices as nodes. If the obstacles are not represented as polygons, you could generate some sort of convex hull around them, and use its vertices for navigation. EDIT: I just realized, you mentioned that you have to navigate around a relatively high number of obstacles. Using the obstacle vertices might be infeasible with to many obstacles.
I do not know about moving obstacles, I believe A* doesn't find an optimal path with moving obstacles.
You mention that your object moves back and fourth - A* should not do this. A* visits each movement point only once. This could be an artifact of generating movement points on the fly, or from the moving obstacles.
I remember encountering this problem in college, but we didn't use an A* search. I can't remember the exact details of the math but I can give you the basic idea. Maybe someone else can be more detailed.
We're going to create a potential field out of your playing area that an object can follow.
Take your playing field and tilt or warp it so that the start point is at the highest point, and the goal is at the lowest point.
Poke a potential well down into the goal, to reinforce that it's a destination.
For every obstacle, create a potential hill. For non-point obstacles, which yours are, the potential field can increase asymptotically at the edges of the obstacle.
Now imagine your object as a marble. If you placed it at the starting point, it should roll down the playing field, around obstacles, and fall into the goal.
The hard part, the math I don't remember, is the equations that represent each of these bumps and wells. If you figure that out, add them together to get your final field, then do some vector calculus to find the gradient (just like towi said) and that's the direction you want to go at any step. Hopefully this method is fast enough that you can recalculate it at every step, since your obstacles move.
Sounds like you're implementing The Wumpus game based on Norvig and Russel's discussion of A* in Artifical Intelligence: A Modern Approach, or something very similar.
If so, you'll probably need to incorporate obstacle detection as part of your heuristic function (hence you'll need to have sensors that alert your agent to the signs of obstacles, as seen here).
To solve the back and forth issue, you may need to store the traveled path so you can tell if you've already been to a location and have the heurisitic function examine the past N number of moves (say 4) and use that as a tie-breaker (i.e. if I can go north and east from here, and my last 4 moves have been east, west, east, west, go north this time)