I would like to find approximately object on my depth map.
For now my process is the following :
1. Normalization of the depth
2. Threshold to get only the closest object
3. Gaussian blur
4. Canny edge detection
5. Contour detection
However, I'm not able to find the box around my object. Actually, I don't know if it's possible with this kind of depth map...
I have three object on the table : a box of food and two mugs.
I would like to find approximately a box around my object.
Is there a way to do it just by image processing ?
Any help would be appreciated.
Thank you very much in advance.
You can do this using OpenCV. Have a look at the following solution.
I used the depth map provided in the question as my input image. I performed binary threshold of the gray scale image of the depth map
ret,th = cv2.threshold(gray,127,255, 1)
and obtained the following:
Now in order to fill the gaps in the image, I performed morphological close operation
kernel = np.ones((15,15),np.uint8)
dilate = cv2.morphologyEx(th, cv2.MORPH_CLOSE, kernel, 3)
Then I found contours using:
contours,hierarchy = cv2.findContours(dilate,2,1)
and drew them using:
cv2.drawContours(img, contours, -1, (0,255,0), 3)
finally obtained this:
Hope this is what you were looking for :)
Related
I'm dealing with an image and I need your help. After a lot of image processing I get this from a microscopic image. This is my pre-final thresholded image:
As you can see there's a big C in the upper left corner. This should not be an open blob, it must be a closed one.
How can I achieve that with out modifying the rest? I was thinking on applying Convex Hull to that contour, but I don't know how to apply to that and only that contour, with out even touching the others.
I mean, maybe there's a "meassure" I can use to isolate this contour from the rest. Maybe a way to tell how convex/cancave it is or how big is the "hole" it delimits.
In further works maybe appears some other unclosed contours that I'll need to close, so don't focus it this particular case I'll need something I can use or adapt to other similar cases.
Thanks in advance!
While Jeru's answer is correct on the part where you want to close the contour once you have identified it, I think OP also wants to know how he can automatically identify the "C" blob without having to find out manually that's it the 29th contour.
Hence, I propose a method to identify it : compute the centroids of each shape and check if this centroid is inside the shape. It should be the case for blobs (circle) but not for "C"s.
img=cv2.imread(your_image,0)
if img is None:
sys.exit("No input image") #good practice
res=np.copy(img) #just for visualisation purposes
#finding the connectedComponents (each blob)
output=cv2.connectedComponentsWithStats(img,8)
#centroid is sort of the "center of mass" of the object.
centroids=output[3]
#taking out the background
centroids=centroids[1:]
#for each centroid, check if it is inside the object
for centroid in centroids:
if(img[int(centroid[1]),int(centroid[0])]==0):
print(centroid)
#save it somewhere, then do what Jeru Luke proposes
#an image with the centroids to visualize
res[int(centroid[1]), int(centroid[0])]=100
This works for your code (I tried it out), but caveat, may not work for every "C form" especially if they are "fatter", as their centroid could well be inside them. I think there may be a better measure for convexity as you say, at least looking for such a measure seems the right direction for me.
Maybe you can try something like computing ConvexHull on all your objects (without modifying your input image), than measure the ratio between the object's area and the "convex hull around it"'s area, and if that ratio is under a certain threshold then you classify it as a "C" shape and modify it accordingly.
I have a solution.
First, I found and drew contours on the threshold image given by you.
In the image, I figured out that the 29th contour is the one with the C. Hence I colored every contour apart from the 29th contour with black. The contour having the C alone was in white.
Code:
#---- finding all contours
contours, iji = cv2.findContours(thresh,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
#---- Turning all contours black
cv2.drawContours(im1, contours, -1, (0,0,0), -1)
#---- Turning contour of interest alone white
cv2.drawContours(im1, contours, 29, (255, 255, 255), -1)
You are left with the blob of interest
Having isolated the required blob, I then performed morphological closing using the ellipse kernel for a certain number of iterations.
#---- k = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(30,30))
#---- closing = cv2.morphologyEx(opening, cv2.MORPH_CLOSE, k)
#---- cv2.imshow("closed_img", closing)
The ball is now in your court! I learnt something as well! Have fun.
I am new to OpenCV and I was trying to extract the region bound by largest contour. It may be a simple question, but I am not able to figure it out. I tried googling too, without any luck.
I would:
Use contourArea() to find the largest closed contour.
Use boundingRect() to get the bounds of that contour.
Draw the contour using drawContours() (with thickness set to -1 to
fill the contour) and use this as a mask.
Use use the mask to set all pixels in the original image not in the
ROI to (0,0,0).
Use the bounding rectangle to extract just that area from the
original image.
Here is well explained what do you want do develop.
Basically you have to:
apply threshold to a copy of the original image;
use findContours -> output is:
vector<vector<Point>>
that stores contours;
iterate on contours to find the largest.
I have this project where I need (on iOS) to detect simple geometric shapes inside an image.
After searching the internet I have concluded that the best tool for this is OpenCV. The thing is that up until two hours ago I had no idea what OpenCV is and I have never even remotely did anything involving image processing. My main experience is JS/HTML,C#,SQL,Objective-C...
Where do I start with this?
I have found this answer that I was able to digest and by reading already other stuff, I understand that OpenCV should return an Array of shapes with the points/corners, is this true? Also how will it represent a circle or a half circle?
Also what about the shape orientation?
Do you know of any Demo iOS project that can demonstrate a similar functionality?
If you have only these regular shapes, there is a simple procedure as follows :
Find Contours in the image ( image should be binary as given in your question)
Approximate each contour using approxPolyDP function.
First, check number of elements in the approximated contours of all the shapes. It is to recognize the shape. For eg, square will have 4, pentagon will have 5. Circles will have more, i don't know, so we find it. ( I got 16 for circle and 9 for half-circle.)
Now assign the color, run the code for your test image, check its number, fill it with corresponding colors.
Below is my example in Python:
import numpy as np
import cv2
img = cv2.imread('shapes.png')
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
ret,thresh = cv2.threshold(gray,127,255,1)
contours,h = cv2.findContours(thresh,1,2)
for cnt in contours:
approx = cv2.approxPolyDP(cnt,0.01*cv2.arcLength(cnt,True),True)
print len(approx)
if len(approx)==5:
print "pentagon"
cv2.drawContours(img,[cnt],0,255,-1)
elif len(approx)==3:
print "triangle"
cv2.drawContours(img,[cnt],0,(0,255,0),-1)
elif len(approx)==4:
print "square"
cv2.drawContours(img,[cnt],0,(0,0,255),-1)
elif len(approx) == 9:
print "half-circle"
cv2.drawContours(img,[cnt],0,(255,255,0),-1)
elif len(approx) > 15:
print "circle"
cv2.drawContours(img,[cnt],0,(0,255,255),-1)
cv2.imshow('img',img)
cv2.waitKey(0)
cv2.destroyAllWindows()
Below is the output:
Remember, it works only for regular shapes.
Alternatively to find circles, you can use houghcircles. You can find a tutorial here.
Regarding iOS, OpenCV devs are developing some iOS samples this summer, So visit their site : www.code.opencv.org and contact them.
You can find slides of their tutorial here : http://code.opencv.org/svn/gsoc2012/ios/trunk/doc/CVPR2012_OpenCV4IOS_Tutorial.pdf
The answer depends on the presence of other shapes, level of noise if any and invariance you want to provide for (e.g. rotation, scaling, etc). These requirements will define not only the algorithm but also required pre-procesing stages to extract features.
Template matching that was suggested above works well when shapes aren't rotated or scaled and when there are no similar shapes around; in other words, it finds a best translation in the image where template is located:
double minVal, maxVal;
Point minLoc, maxLoc;
Mat image, template, result; // template is your shape
matchTemplate(image, template, result, CV_TM_CCOEFF_NORMED);
minMaxLoc(result, &minVal, &maxVal, &minLoc, &maxLoc); // maxLoc is answer
Geometric hashing is a good method to get invariance in terms of rotation and scaling; this method would require extraction of some contour points.
Generalized Hough transform can take care of invariance, noise and would have minimal pre-processing but it is a bit harder to implement than other methods. OpenCV has such transforms for lines and circles.
In the case when number of shapes is limited calculating moments or counting convex hull vertices may be the easiest solution: openCV structural analysis
You can also use template matching to detect shapes inside an image.
I am looking for the right set of algorithms to solve this image processing problem:
I have a distorted binary image containing a distorted rectangle
I need to find a good approximation of the 4 corner points of this rectangle
I can calculate the contour using OpenCV, but as the image is distorted it will often contain more than 4 corner points.
Is there a good approximation algorithm (preferably using OpenCV operations) to find the rectangle corner points using the binary image or the contour description?
The image looks like this:
Thanks!
Dennis
Use cvApproxPoly function to eliminate number of nodes of your contour, then filter out those contours that have too many nodes or have angles which much differ from 90 degrees. See also similar answer
little different answer, see
http://opencv.willowgarage.com/documentation/cpp/camera_calibration_and_3d_reconstruction.html
Look at the opencv function ApproxPoly. It approximates a polygon from a contour.
Try Harris Corner Detector. There is example in OpenCV package. You need to play with params for your image.
And see other OpenCV algorithms: http://www.comp.leeds.ac.uk/vision/opencv/opencvref_cv.html#cv_imgproc_features
I would try generalised Hough Transform it is a bit slow but deals well with distorted/incomplete shapes.
http://en.wikipedia.org/wiki/Hough_transform
This will work even if you start with some defects, i.e. your approxPolly call returns pent/hexagons. It will reduce any contour, transContours in example, to a quad, or whatever poly you wish.
vector<Point> cardPoly;// Quad storage
int PolyLines = 0;//PolyPoly counter ;)
double simplicity = 0.5;//Increment of adjustment, lower numbers may be more precise vs. high numbers being faster to cycle.
while(PolyLines != 4)//Adjust this
{
approxPolyDP(transContours, Poly, simplicity, true);
PolyLines = Poly.size();
simplicity += 0.5;
}
I'm trying to use OpenCV to "parse" screenshots from the iPhone game Blocked. The screenshots are cropped to look like this:
I suppose for right now I'm just trying to find the coordinates of each of the 4 points that make up each rectangle. I did see the sample file squares.c that comes with OpenCV, but when I run that algorithm on this picture, it comes up with 72 rectangles, including the rectangular areas of whitespace that I obviously don't want to count as one of my rectangles. What is a better way to approach this? I tried doing some Google research, but for all of the search results, there is very little relevant usable information.
The similar issue has already been discussed:
How to recognize rectangles in this image?
As for your data, rectangles you are trying to find are the only black objects. So you can try to do a threshold binarization: black pixels are those ones which have ALL three RGB values less than 40 (I've found it empirically). This simple operation makes your picture look like this:
After that you could apply Hough transform to find lines (discussed in the topic I referred to), or you can do it easier. Compute integral projections of the black pixels to X and Y axes. (The projection to X is a vector of x_i - numbers of black pixels such that it has the first coordinate equal to x_i). So, you get possible x and y values as the peaks of the projections. Then look through all the possible segments restricted by the found x and y (if there are a lot of black pixels between (x_i, y_j) and (x_i, y_k), there probably is a line probably). Finally, compose line segments to rectangles!
Here's a complete Python solution. The main idea is:
Apply pyramid mean shift filtering to help threshold accuracy
Otsu's threshold to get a binary image
Find contours and filter using contour approximation
Here's a visualization of each detected rectangle contour
Results
import cv2
image = cv2.imread('1.png')
blur = cv2.pyrMeanShiftFiltering(image, 11, 21)
gray = cv2.cvtColor(blur, cv2.COLOR_BGR2GRAY)
thresh = cv2.threshold(gray, 0, 255, cv2.THRESH_BINARY_INV + cv2.THRESH_OTSU)[1]
cnts = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
cnts = cnts[0] if len(cnts) == 2 else cnts[1]
for c in cnts:
peri = cv2.arcLength(c, True)
approx = cv2.approxPolyDP(c, 0.015 * peri, True)
if len(approx) == 4:
x,y,w,h = cv2.boundingRect(approx)
cv2.rectangle(image,(x,y),(x+w,y+h),(36,255,12),2)
cv2.imshow('thresh', thresh)
cv2.imshow('image', image)
cv2.waitKey()
I wound up just building on my original method and doing as Robert suggested in his comment on my question. After I get my list of rectangles, I then run through and calculate the average color over each rectangle. I check to see if the red, green, and blue components of the average color are each within 10% of the gray and blue rectangle colors, and if they are I save the rectangle, if they aren't I discard it. This process gives me something like this:
From this, it's trivial to get the information I need (orientation, starting point, and length of each rectangle, considering the game window as a 6x6 grid).
The blocks look like bitmaps - why don't you use simple template matching with different templates for each block size/color/orientation?
Since your problem is the small rectangles I would start by removing them.
Since those lines are much thinner than the borders of the rectangles I would start by applying morphological operations on the image.
Using a structural element that looks like this:
element = [ 1 1
1 1 ]
should remove lines that are less than two pixels wide. After the small lines are removed the rectangle finding algorithm of OpenCV will most likely do the rest of the job for you.
The erosion can be done in OpenCV by the function cvErode
Try one of the many corner detectors like harris corner detector. also it is in general a good idea to try that at multiple resolutions : so do some preprocessing of of varying magnification.
It appears that you want some sort of color dominated square then you can suppress the other colors, by first using something like cvsplit .....and then thresholding the color...so only that region remains....follow that with a cropping operation ...I think that could work as well ....