I have to make a bot which has to overcome obstacles autonomously in an arena that will be filled with rocks. The bot has to find its way through this area and reach the end point. I am thinking of using edge detector operators like canny and sobel for this problem.
I want to know whether those will be suitable options for this problem. If so, then after detecting the edges, how can I make the bot find the path, overcoming the rock obstacles?
I am using QT IDE and opencv library.
Since you will be analyzing frames of video, and the robot will be moving most of the time, image differences and optical flow too will be helpful. Edge detection alone might not help a lot, unless the surroundings and obstacles are simple and have known properties. Posting a photo of the scene can help those who want to answer the question.
Yes, canny is a very good edge detector. In fact the opencv implementation uses sobel to get the gradient estimate. You may need to apply a Gaussian filter to the image before edge detection. Edges are good features to look for rocks, but depending on the background other features such as color may also be useful. It probably would be easier if you gather 3D scene information via stereo, or laser scanner, or kinect like sensor. Also consider detecting when you bump into rocks and building up a map of where they are.
You can use contours to detect any object. You can estimate its size by finding the area of the contours. Then you can use moments to find the center of the object.
Related
First of all I'm a total newbie in image processing, so please don't be too harsh on me.
That being said, I'm developing an application to analyse changes in blood flow in extremities using thermal images obtained by a camera. The user is able to define a region of interest by placing a shape (circle,rectangle,etc.) on the current image. The user should then be able to see how the average temperature changes from frame to frame inside the specified ROI.
The problem is that some of the images are not steady, due to (small) movement by the test subject. My question is how can I determine the movement between the frames, so that I can relocate the ROI accordingly?
I'm using the Emgu OpenCV .Net wrapper for image processing.
What I've tried so far is calculating the center of gravity using GetMoments() on the biggest contour found and calculating the direction vector between this and the previous center of gravity. The ROI is then translated using this vector but the results are not that promising yet.
Is this the right way to do it or am I totally barking up the wrong tree?
------Edit------
Here are two sample images showing slight movement downwards to the right:
http://postimg.org/image/wznf2r27n/
Comparison between the contours:
http://postimg.org/image/4ldez2di1/
As you can see the shape of the contour is pretty much the same, although there are some small differences near the toes.
Seems like I was finally able to find a solution for my problem using optical flow based on the Lukas-Kanade method.
Just in case anyone else is wondering how to implement it in Emgu/C#, here's the link to a Emgu examples project, where they use Lukas-Kanade and Farneback's algorithms:
http://sourceforge.net/projects/emguexample/files/Image/BuildBackgroundImage.zip/download
You may need to adapt a few things, e.g. the parameters for the corner detection (the frame.GoodFeaturesToTrack(..) method) , but it's definetly something to start with.
Thanks for all the ideas!
I have to implement a contour detection of full human body (from feet to head, in several poses such as raising hands etc.) using opencv. I managed to compile and run code I found here https://gist.github.com/yoggy/1470956, but it only draws a rectangle around the body, and not the exact contour. Can one help me with identifying and displaying the contour itself?
Thanks!!
I'm afraid the answer to this question is:
There's no algorithm that can do this perfectly.
Computer vision has not developed to that extent yet. Take a look at recent papers in CVPR, PAMI, and you will find that most algorithms are "rectangle", or more specifically, bounding-box based, in terms of human labeling and algorithmic detecting.
It is true that you can find the contours within the bounding-box. However the computer just doesn't know which contour belongs to the specified object.
I suggest you search for "human pose estimation" for further information.
One approach that might work is background subtraction:
http://docs.opencv.org/3.1.0/db/d5c/tutorial_py_bg_subtraction.html
This would work for video but perhaps also for single images in a scenario where you were in a controlled (fixed camera) environment where you had an image of the pose and also and image of the background, with no one present.
You can use the function findCountors within the returned bounding box:
http://docs.opencv.org/doc/tutorials/imgproc/shapedescriptors/find_contours/find_contours.html
For a project of mine, I'm required to process images differences with OpenCV. The goal is to detect an intrusion in a zone.
To be a little more clear, here are the inputs and outputs:
Inputs:
An image of reference
A second image from approximately the same point of view (can be an error margin)
Outputs:
Detection of new objects in the scene.
Bonus:
Recognition of those objects.
For me, the most difficult part of it is to take off small differences (luminosity, camera position margin error, movement of trees...)
I already read a lot about OpenCV image processing (subtraction, erosion, threshold, SIFT, SURF...) and have some good results.
What I would like is a list of steps you think is the best to have a good detection (humans, cars...), and the algorithms to do each step.
Many thanks for your help.
Track-by-Detect, human tracker:
You apply the Hog detector to detect humans.
You draw a respective rectangle as foreground area on the foreground mask.
You pass this mask to "The OpenCV Video Surveillance / Blob Tracker Facility"
You can, now, group the passing humans based on their blob.{x,y} values into public/restricted areas.
I had to deal with this problem the last year.
I suggest an adaptive background-foreground estimation algorithm which produced a foreground mask.
On top of that, you add a blob detector and tracker, and then calculate if an intersection takes place between the blobs and your intrusion area.
Opencv comes has samples of all of these within the legacy code. Ofcourse, if you want you can also use your own or other versions of these.
Links:
http://opencv.willowgarage.com/wiki/VideoSurveillance
http://experienceopencv.blogspot.gr/2011/03/blob-tracking-video-surveillance-demo.html
I would definitely start with a running average background subtraction if the camera is static. Then you can use findContours() to find the intruding object's location and size. If you want to detect humans that are walking around in a scene, I would recommend looking at using the built-in haar classifier:
http://docs.opencv.org/doc/tutorials/objdetect/cascade_classifier/cascade_classifier.html#cascade-classifier
where you would just replace the xml with the upperbody classifier.
i want detect the important ROI Element in a picture. (i want to get the position)
I've tested a reverted SeamCarving-Method. I hoped, that the most importand Area in a picture have the most energylevel. I've generated one vertical and one horizontal Seam and took the intersection. But this method don't seem to be perfect.
Some examples:
good detection:
good detection http://img713.imageshack.us/img713/2928/seamcastle.jpg
good detection http://img39.imageshack.us/img39/9584/seamente.jpg
good detection http://img193.imageshack.us/img193/2693/seamwuffi.jpg
near aceptable;
good detection http://img440.imageshack.us/img440/7459/seamflower.jpg
worse detection:
good detection http://img836.imageshack.us/img836/5766/seamsun.jpg (maybe the point is a good result. It's the point with the max. energylevel in this picture)
good detection http://img507.imageshack.us/img507/2750/seambluesky1.jpg
Have anyone a idea to detect roi's more better?
greeting,
desire
I think the key terms you are looking for is: Saliency Detection, Salient Object Detection, etc
Perhaps these papers will point you in the right direction:
A Model of Saliency-Based Visual Attention for Rapid Scene Analysis (PDF)
Simulating Human Saccadic Scanpaths on Natural Images (PDF)
Salient Object Detection by Composition (PDF)
Saliency Filters: Contrast Based Filtering for Salient Region Detection (Web)
I'm trying to make my OpenCV-based fiducial marker detection more robust when the user moves the camera (phone) violently. Markers are ArTag-style with a Hamming code embedded within a black border. Borders are detected by thresholding the image, then looking for quads based on the found contours, then checking the internals of the quads.
In general, decoding of the marker is fairly robust if the black border is recognized. I've tried the most obvious thing, which is downsampling the image twice, and also performing quad-detection on those levels. This helps with camera defocus on extreme nearground markers, and also with very small levels of image blur, but doesn't hugely help the general case of camera motion blur
Is there available research on ways to make detection more robust? Ideas I'm wondering about include:
Can you do some sort of optical flow tracking to "guess" the positions of the marker in the next frame, then some sort of corner detection in the region of those guesses, rather than treating the rectangle search as a full-frame thresholding?
On PCs, is it possible to derive blur coeffiients (perhaps by registration with recent video frames where the marker was detected) and deblur the image prior to processing?
On smartphones, is it possible to use the gyroscope and/or accelerometers to get deblurring coefficients and pre-process the image? (I'm assuming not, simply because if it were, the market would be flooded with shake-correcting camera apps.)
Links to failed ideas would also be appreciated if it saves me trying them.
Yes, you can use optical flow to estimate where the marker might be and localise your search, but it's just relocalisation, your tracking will have broken for the blurred frames.
I don't know enough about deblurring except to say it's very computationally intensive, so real-time might be difficult
You can use the sensors to guess the sort of blur you're faced with, but I would guess deblurring is too computational for mobile devices in real time.
Then some other approaches:
There is some really smart stuff in here: http://www.robots.ox.ac.uk/~gk/publications/KleinDrummond2004IVC.pdf where they're doing edge detection (which could be used to find your marker borders, even though you're looking for quads right now), modelling the camera movements from the sensors, and using those values to estimate how an edge in the direction of blur should appear given the frame-rate, and searching for that. Very elegant.
Similarly here http://www.eecis.udel.edu/~jye/lab_research/11/BLUT_iccv_11.pdf they just pre-blur the tracking targets and try to match the blurred targets that are appropriate given the direction of blur. They use Gaussian filters to model blur, which are symmetrical, so you need half as many pre-blurred targets as you might initially expect.
If you do try implementing any of these, I'd be really interested to hear how you get on!
From some related work (attempting to use sensors/gyroscope to predict likely location of features from one frame to another in video) I'd say that 3 is likely to be difficult if not impossible. I think at best you could get an indication of the approximate direction and angle of motion which may help you model blur using the approaches referenced by dabhaid but I think it unlikely you'd get sufficient precision to be much more help.