I have some really simple images, from which I would like to extract the longest contour.
An example image would be like this one:
I am using the exact same sample code from OpenCV's tutorial page. With one differenc I set the threshold to a fix number, namely 100.
The main line is this one:
cv::findContours(cannyOutput, contours, hierarchy, CV_RETR_TREE, CV_CHAIN_APPROX_SIMPLE, cv::Point(0, 0));
After I call the above function I iterate through the found contours and check which one is the longest, then I save the longest one. Under longest I mean which one has the most points.
In some cases, like in the above example image the longest contour is doubled. To make it more understandable what I mean under "doubled" here is a visualized image of the found contour:
So I tried to figure it out myself why this is happening by trying to understand the OpenCV docs of findContour, but I still can't understand the real reason.
What I manged to achieve, if I change to CV_RETR_EXTERNAL from CV_RETR_TREE, I don't get the doubled contour.
So my questions would be:
What is the reason behind the doubled contour and why does CV_RETR_EXTERNAL solve the problem?
Getting the contour which has the most points doesn't necessarily mean, that it is the longest, right? Due to CV_CHAIN_APPROX_SIMPLE flag. Would CV_CHAIN_APPROX_NONE solve this problem for example?
Q: What is the reason behind the doubled contour and why does CV_RETR_EXTERNAL solve the problem?
A: OpenCV findCountours standard mode is CV_RETR_LIST, which outputs, for a line, as in your case, the inner and outer contour. CV_RETR_EXTERNAL, as described in the docs, will outputs only the "extreme outer contours". Note that the outer contour does not mean the longest one. I would recommend you to loop through all the contours given by the CV_RETR_LIST mode and do your calculation.
Q: Getting the contour which has the most points doesn't necessarily mean, that it is the longest, right? Due to CV_CHAIN_APPROX_SIMPLE flag. Would CV_CHAIN_APPROX_NONE solve this problem for example?
A: The first question is true, if your findCountours method is different than CV_CHAIN_APPROX_NONE. It is also true that CV_CHAIN_APPROX_NONE will solve this problem as it will "store absolutely all the contour points", but you can also sum all the distances between the points if you prefer to use any other method.
Related
I'm trying to extract the geometries of the papers in the image below, but I'm having some trouble with grabbing the contours. I don't know which threshold algorithm to use (here I used static threshold = 10, which is probably not ideal.
And as you can see, I can get the correct number of images, but I can't get the proper bounds using this method.
Simply applying Otsu just doesn't work, it doesn't capture the geometries.
I assume I need to apply some edge detection, but I'm not sure what to do once I apply Canny or some other.
I also tried sobel in both directions (+ve and -ve in x and y), but unsure how to extract these contours from there.
How do I grab these contours?
Below is some previews of the images in the process of the final convex hull results.
**Original Image** **Sharpened**
**Dilate,Sharpen,Erode,Sharpen** **Convex Of Approximated Polygons Hulls (which doesn't fully capture desired regions)**
Sorry in advance about the horrible formatting, I have no idea how to make images smaller or title them nicely in SOF
I am planning to use Opencv on Raspberry pi 3 with camera to count lines in the following image
It will be used in machine which produce threads. If one (or more) will be lost it will stop the machine.
Now i am wondering how to do that...?
I will do a loop to capture the images
I will crop the images to see only the part with lines
I will convert it to black&white
how to count them ? in a loop - check pixel value change? Or there is a better/faster idea?
Thank you for advice!
EDIT
P.S.
I used cv2.findContours (answer from Jeru Luke).
I've putted an A4 sheet with black lines in front of camera. I works ok in while loop BUT... i have 43 lines on sheet. When camera detects some differences i wrote the results to file. Sometimes i have 710,800,67 etc.
Pls look on file with values i have https://www.dropbox.com/s/jnn4w8mq3rrtppo/bledy.txt?dl=0
lines....The error is permanet for some few secounds. Tere is noting wronge when i have 43,43,43,43,44,43,43,43 (the one is wrong) because i watch few values before i put error. But when the are hundreds of bad values i have no idea...
I have something relatively simpler. It does not involve any for loops hence requires less time. I used the concept of counting the contours in the image after finding an appropriate threshold. I found the perfect threshold through trial-and-error.
I have the approach in python:
import cv2
path = 'C:/Users/Desktop/stack/contour/'
img = cv2.imread(path + 'lines.png', 0)
cv2.imshow('original Image', img)
ret, thresh = cv2.threshold(img, 80, 255, cv2.THRESH_BINARY_INV)
cv2.imshow('thresh1', thresh)
_, contours, hierarchy = cv2.findContours(thresh, cv2.RETR_EXTERNAL, cv2.CHAIN_APPROX_SIMPLE)
print('Number of lines:', len(contours))
cv2.waitKey(0)
cv2.destroyAllWindows()
Note:
As you can see there are no for loops involved. There is no need to count the number of pixel changes as well. Each presumed line becomes a contour. Using `len(contours) you get the number of lines present.
Using Hough Line transform would work well only If the lines are straight. Since the lines in the provided image are slanted it won't find perfect lines. This statement is more emphasized in the comments by #MarkSetchell.
Use Hough Lines Transform to detect the lines and just count the number of countours you find.
Here there's a tutorial for your problem (since you didn't specify the Language, this is in python).
OpenCV Tutorial Hough Lines
I'm dealing with an image and I need your help. After a lot of image processing I get this from a microscopic image. This is my pre-final thresholded image:
As you can see there's a big C in the upper left corner. This should not be an open blob, it must be a closed one.
How can I achieve that with out modifying the rest? I was thinking on applying Convex Hull to that contour, but I don't know how to apply to that and only that contour, with out even touching the others.
I mean, maybe there's a "meassure" I can use to isolate this contour from the rest. Maybe a way to tell how convex/cancave it is or how big is the "hole" it delimits.
In further works maybe appears some other unclosed contours that I'll need to close, so don't focus it this particular case I'll need something I can use or adapt to other similar cases.
Thanks in advance!
While Jeru's answer is correct on the part where you want to close the contour once you have identified it, I think OP also wants to know how he can automatically identify the "C" blob without having to find out manually that's it the 29th contour.
Hence, I propose a method to identify it : compute the centroids of each shape and check if this centroid is inside the shape. It should be the case for blobs (circle) but not for "C"s.
img=cv2.imread(your_image,0)
if img is None:
sys.exit("No input image") #good practice
res=np.copy(img) #just for visualisation purposes
#finding the connectedComponents (each blob)
output=cv2.connectedComponentsWithStats(img,8)
#centroid is sort of the "center of mass" of the object.
centroids=output[3]
#taking out the background
centroids=centroids[1:]
#for each centroid, check if it is inside the object
for centroid in centroids:
if(img[int(centroid[1]),int(centroid[0])]==0):
print(centroid)
#save it somewhere, then do what Jeru Luke proposes
#an image with the centroids to visualize
res[int(centroid[1]), int(centroid[0])]=100
This works for your code (I tried it out), but caveat, may not work for every "C form" especially if they are "fatter", as their centroid could well be inside them. I think there may be a better measure for convexity as you say, at least looking for such a measure seems the right direction for me.
Maybe you can try something like computing ConvexHull on all your objects (without modifying your input image), than measure the ratio between the object's area and the "convex hull around it"'s area, and if that ratio is under a certain threshold then you classify it as a "C" shape and modify it accordingly.
I have a solution.
First, I found and drew contours on the threshold image given by you.
In the image, I figured out that the 29th contour is the one with the C. Hence I colored every contour apart from the 29th contour with black. The contour having the C alone was in white.
Code:
#---- finding all contours
contours, iji = cv2.findContours(thresh,cv2.RETR_TREE,cv2.CHAIN_APPROX_SIMPLE)
#---- Turning all contours black
cv2.drawContours(im1, contours, -1, (0,0,0), -1)
#---- Turning contour of interest alone white
cv2.drawContours(im1, contours, 29, (255, 255, 255), -1)
You are left with the blob of interest
Having isolated the required blob, I then performed morphological closing using the ellipse kernel for a certain number of iterations.
#---- k = cv2.getStructuringElement(cv2.MORPH_ELLIPSE,(30,30))
#---- closing = cv2.morphologyEx(opening, cv2.MORPH_CLOSE, k)
#---- cv2.imshow("closed_img", closing)
The ball is now in your court! I learnt something as well! Have fun.
I'm attempting to wrap my head around the basics of CV. The bit that initially got me interested was template matching (it was mentioned in a Pycon talk unrelated to CV), so I figured I'd start there.
I started with this image:
Out of which I want to detect Mario. So I cut him out:
I understand the concept of sliding the template around the image to see the best fit, and following a tutorial, I'm able to find mario with the following code:
def match_template(img, template):
s = time.time()
img_size = cv.GetSize(img)
template_size = cv.GetSize(template)
img_result = cv.CreateImage((img_size[0] - template_size[0] + 1,
img_size[1] - template_size[1] + 1), cv.IPL_DEPTH_32F, 1)
cv.Zero(img_result)
cv.MatchTemplate(img, template, img_result, cv.CV_TM_CCORR_NORMED)
min_val, max_val, min_loc, max_loc = cv.MinMaxLoc(img_result)
# inspect.getargspec(cv.MinMaxLoc)
print min_val
print max_val
print min_loc
print max_loc
cv.Rectangle(img, max_loc, (max_loc[0] + template.width, max_loc[1] + template.height), cv.Scalar(120.), 2)
print time.time() - s
cv.NamedWindow("Result")
cv.ShowImage("Result", img)
cv.WaitKey(0)
cv.DestroyAllWindows()
So far so good, but then I came to realize that this is incredibly fragile. It will only ever find Mario with that specific background, and with that specific animation frame being displayed.
So I'm curious, given that Mario will always have the same Mario-ish attributes, (size, colors) is there a technique with which I could find him regardless of whether his currect frame is standing still, or one of the various run cycle sprites? Kind of like fuzzy matching that you can do on strings, but for images.
Maybe since he's the only red thing, there is a way of simply tracking the red pixels?
The whole other issue is removing the background from the template. Maybe that would help the MatchTemplate function find Mario even though he doesn't exactly match the tempate? As of now, I'm not entirely sure how that would work ( I see that there is a mask param in MatchTemplate, but I'll have to investigate further)
My main question is whether or not template matching is the way to go about detecting an image that is mostly the same, but varies (like when he's walking), or is there another technique I should look into?
Update:
Attempts at matching other Marios
Going off of mmgp's suggestion that it should be workable for matching other things, I ran a couple of tests.
I used this as the template to match:
And then took a couple of screen shots to test the matching against.
For the first, I successfully find Mario, and get a max value of 1.
However, trying to find jumping Mario results in a complete misfire.
Now granted, the mario in the template, and the mario in the scene is facing opposite directions, as well as being different animation frames, but I would think they still match a lot more than anything else in the image -- if only for the colors alone. But it targets the platform as being the closest match to the template.
Note that the max value for this one was 0.728053808212.
Next I tried a scene without mario to see what would happen.
But oddly enough, I get the exact result as the image with jumping mario -- right down to the similarity value: 0.728053808212. Mario being in the picture is just as accurate as him not being in the picture.
Really strange! I don't know the actual details of the underlying algorithm, but I'd imagine, from a standard deviation perspective, the boxes in the scene that at least match the Red in template Mario's suit would be closer to the mean distance than a blue platform, right? So, it's extra confusing that it's not even in the general area of where I would expect it to be.
I'm guessing this is user error on my end, or maybe just a misunderstanding.
Why would a scene with a similar Mario have as much of a match as a scene with no Mario at all?
No method is infallible, but template matching do have a good chance to work there. It might require some pre-processing, and until there is a larger sample (a short video for example) to demonstrate the possible problems, there isn't much point in trying more advanced methods simply because some library implement them for you -- especially if you don't know under which conditions they are expected to work.
For instance, here are the results I get using template matching (red rectangles) -- all them are using the template http://i.stack.imgur.com/EYs9B.png, even the last one:
To achieve that I started by considering only the red channel of both the template and the input image. From that we easily calculate the internal morphological gradient and only then perform the matching. In order to not get a rectangle when Mario is not present, it is needed to set a minimum threshold for the matching. Here is the template and one of the images after these two transformations:
And here is some sample code to achieve that:
import sys
import cv2
import numpy
img = cv2.imread(sys.argv[1])
img2 = img[:,:,2]
img2 = img2 - cv2.erode(img2, None)
template = cv2.imread(sys.argv[2])[:,:,2]
template = template - cv2.erode(template, None)
ccnorm = cv2.matchTemplate(img2, template, cv2.TM_CCORR_NORMED)
print ccnorm.max()
loc = numpy.where(ccnorm == ccnorm.max())
threshold = 0.4
th, tw = template.shape[:2]
for pt in zip(*loc[::-1]):
if ccnorm[pt[::-1]] < threshold:
continue
cv2.rectangle(img, pt, (pt[0] + tw, pt[1] + th),
(0, 0, 255), 2)
cv2.imwrite(sys.argv[2], img)
I expect it to fail in more varied situations, but there are a couple of easy adjustments to be done.
Template matching doesn't always give good results. you should look into Keypoints matching.
Step1: Find Keypoints
Let's assume that you managed to cut out Mario or get ROI image of mario. Make this image your template image. Now, find keypoints in the main image and also in the template. So now you have two sets of keypoints. One for the image and other for Mario(template).
You can use SIFT, SURF, ORB depending on your preferences.
[EDIT]:
This is what I got using this method with SIFT and flann based knn matching. I haven't done the bounding box part.
Since your template is very small, SIFT and SURF would not give many keypoints. But to get good number of feature points, you could try Harris Corner detector. I applied Harris corner on the image and I got pretty good points on Mario.
Step2: Match Keypoints
If you have used SIFT or SURF, you'd have descriptors of both the image and the template. Match these keypoints using KNN or some other efficient matching algorithm. If you are using OpenCV, I'd suggest you to look into flannbased matcher. After matching the keypoints, you would want to filter out the incorrect matches. You can do this by K- nearest neighbors and depending upon the distance of the nearest match you can further filter out keypoints. You can further filter your matches using Forward-Backward Error.
Forward-Backward Error Estimation:
Match template keypoints to the image keypoints This will give you a set of matches.
Match the image keypoints to the template keypoints. This will give you another set of matches.
Common set of both these set will filter out incorrect matches.
[EDIT]:
If you are using Harris Corner detector, you'd get only points and not keypoints. You can either convert them into keypoints or write your own brute force mathcer. It's not that difficult.
Step3: Estimation
After filtering the keypoints, you'd have a cluster of keypoints near your object (in this case, Mario) and few scattered keypoints. To eliminate these scattered keypoints, you could use clustering. DBSCAN clustering will help you get a good cluster of points.
step4: Bounding Box Estimation
Now you have a cluster of keypoints. Using k-means, you should try to find the center of the cluster. Once you obtain the center of the cluster, you can estimate the bounding box.
I hope this helps.
[EDIT]
Trying to match points using Harris Corners. After filtering Harris corners, I'm using brute force method to match the points. some better algorithm might give you better results.
I am tryig to find a way to determine if a contour is closed or not,
but I am usign findContours and not cvFindContours so I dont have the flags..
Any idea how to do it?
By the way, I was asked to find the number of loops in the contour,
(meaning how many times he crosses himself).
Is it possible that a single contour will have loops?
If so, any idea of how to find how many there are?
Thanks,
Tamir.
i think that you can't detect the contours which have intersections using cvFindContours. if this function return the contours which are have a intersection than you can be sure that this contour is a loop. If the contour have 1 intersection for example imagine the contour corresponded to number "8" than cvFindContours return 3 contours, the 2 circles and the large outlier. I think you must use the graph theory for this task. create graph where the vertex is the pixels which lie in the contour, and the edges of graph is the neighbor pixels in the image. than you can find the all loops in the graph.