I am struggling to get good results detecting objects with a series of images I have. The issue is that they are similar objects with a bit of real world noise (in this case bunches of bananas). Having them overlapping is probably not helping.
My approach has been this tried and tested classic:
- Convert to grayscale
- Blur the image slightly
- Apply Canny edge detection
- Dilate the image
- Find the contours
Right now I have been trying to do a Canny edge detection but the edges aren't quite right and so when I find the contours I get all sorts of funny shapes (and only sometimes the bananas).
I have also tried Sobel edge detection with even poorer results.
example pic before contour detection:
example pic after contour detection:
Can somebody please give me some suggestions on how I can refine this?
Related
I am am currently working on a method to extract colors from a macbeth color chart. So far I have had moderate success by using thresholding and then extracting square contours. Sadly through, colors that are too close to each other either mix together or do no get detected.
The code in it's current form:
<script src="https://pastebin.com/embed_js/mNi0TcDE"></script>
The image before any processing
After thresholding, you can see that there are areas where lines are incomplete due to too small differences in color. I have tried to use dilation to midigate these issues and it does work to a degree. But not enough to detect all squares.
Image after thresholding
This results in the following contours being detected
Detected contours
I have tried using:
Hough lines, sadly no lines here detected.
Centroids of contours, but I was unable to find a way to use centroids to draw lines and detect the centers of the missing contours
Corner detection, corners where found. But I was unsuccessful in finding a real way to put them to use.
Can anyone point me in the right direction?
Thaks in advance,
Emil
Hum, if your goal is color calibration, you really do not need to detect the squares in their entirety. A 10x10 sample near the center of the image of each physical square will give you 100 color samples, which is plenty for any reasonable calibration procedure.
There are many ways to approach this problem. If you can guarantee that the chart will cover the image, you could even just do k-means clustering, since you know in advance the exact number of clusters you seek.
If you insist on using geometry, I'd do template matching in scale+angle space - it is reasonable to assume that the chart will be mostly facing, and only slightly rotated, so you only need to estimate scale and a small rotation about the axis orthogonal to the chart.
The problem is to detect a known rectangular object in an image.
Which of the following is computationally less expensive:
Finding homography - For finding homography, we use the template of the known object to do feature matching.
Contour detection - We try to detect the biggest contour in the image. In this particular case we assume that the biggest contour will correspond to the known rectangular object we are trying to find.
In both the cases we do perspective transform after detecting the object to set the perspective.
NOTE: We are using Open-CV functions to find the homography and detecting contour.
You should try finding the biggest contour. It's the simplest and will be far faster. You needs to detect Canny edges then find contours and find the one with the biggest area. However, it can fail if contours are unclear or if there is a bigger object as it doesn't consider shape. You can also apply both of your ideas to get better results.
EDIT:
To reply your comment, you have Canny edge + find contours + find biggest against find features + match features
I think that the first combination is less computationally expensive. Moreover, there is a good implementation of squares/rectangle detection here.
However, if the contours of the rectangle are not clear, and if moreover the rectangle is highly textured, you should get better results with features matching.
I'm playing with Eye Gaze estimation using a IR Camera. So far i have detected the two Pupil Center points as follows:
Detect the Face by using Haar Face cascade & Set the ROI to Face.
Detect the Eyes by using Haar Eye cascade & Name it as Left & Right Eye Respectively.
Detect the Pupil Center by thresholding the Eye region & found the pupil center.
So far I've tried to find the gaze direction by using the Haar Eye Boundary region. But this Haar Eye rect is not always showing the Eye Corner points. So the results was poor.
Then I've tried to tried to detect Eye Corner points using GFTT, Harriscorners & FAST but since I'm using NIR Camera the Eye Corner points are not clearly visible & so i cant able to get the exact corner positions.So I'm stuck here!
What else is the best feature that can be tracked easily from face? I heard about Flandmark but i think that is also will not work in IR captured images.
Is there any feature that can be extracted easily from the face images? Here I've attached my sample output image.
I would suggest flandmark, even if your intuition is the opposite - I've used it in my master thesis (which was about head pose estimation, a related topic). And if the question is whether it will work with the example image you've provided, I think it might detect features properly - even on a gray scaled image. I think in the flandmark they probably convert to image to grayscale before applying a detector (like the haar detector works). Moreover, It surprisingly works with the low resolution images what is an advantage too (especially when you're saying eye corners are not clearly visible). And flandmark can detect both eye corners, mouth corners and nose tip (actually I will not rely on the last one, from my experience detecting nose tip on the single image is quite noisy, however works fine with an image sequence with some filtering e.g. average, Kalman). If you decide to use that technique and it works, please let us know!
I want check if forehead is visible in given facial image or is covered by hairs. For this, I need to get boundary of hairs that are falling on the forehead. I tried to use Sobel operator and the dilation to get the boundary but what I am getting is only the boundary around whole face and not the boundary of hairs falling on forehead. I am using otsu's algorithm to threshold the image. Background in my image is white and hair color is black.
Can you suggest how can I get the boundary for hairs on forehead? I know grabcut works but it takes more time to extract the hair portion.
Thank You!
use face haarcascade to detect face.
use eye cascade to detect one or both eyes .
expand the region of face from top .
estimate for had using eye point and face position.
this is the simplest way to detect forehead ...
Since you already have forehead region, i have couple of alternative suggestions.
Use canny edge detector. If the skin colours is different from hair , it should work.
If above is not enough, use local binary pattern on the forehead region. This along with the canny edge image should do it for you.
I have been working with OpenCV for a fairly short time, and have performed Canny Edge Detection on an image, and also performed dilation after that to further separate the object (in my case a square) from the background.
My problem, now is to identify graspable regions in 2D using an algorithm that requires me to handle co-ordinates of the points in those edges. Is there any way I can use OpenCV to get the co-ordinates of the corners so I can find the equation of the lines forming the edge of the square? I know the size of the square. My problem involves 2D co-ordinate geometry, and hence the need for co-ordinates.
I can provide the image after edge detection and dilation if need be. Help would be appreciated a lot.
Just offering a second method - not guaranteed to work.
Step 1: extract connected component and their contours. This can be applied after the Canny Edge Detection step.
FindContours
Step 2: If the contours are fairly good approximation of a square, you can use their bounding box directly.
BoundingRect - if the rectangles are always upright (not rotated)
MinAreaRect - if the rectangles are rotated.
The reason for the disclaimer is that it only works on very clean results, without any broken edges or gaps in the Canny edges, etc. If you need a more robust way of finding rectangles, Hough transform will be necessary.
You could use the corner detectors provided in OpenCV like Harris or Corner Eigenvalues. Here's an example of that along with full-fledged code.
In case other features are also throwing up corners you may need to go in for connected component analysis.