I am trying to detect elliptical kernels, in OpenCV using C++. I have tried obtaining Canny edges, and then using fitEllipse() function on the edges. Though this finds ellipses, the accuracy is horrid when the image is noisy, or if there are many edges.
I have realised that the way to go is detecting ellipses, and not fitting them. Maybe something like Hough circles, but for ellipses? I also do not know the length the ellipses could be, as it varies between images.
Can anybody help me get started on that? All related answers are very vague, and I just want pointers on where to start.
As you already got, you don't need ellipse fitting, but ellipse detection.
You can find in my other answer two papers with C++ code available. I'll report here for completeness:
L. Libuda, I. Grothues, K.-F. Kraiss, Ellipse detection in digital image
data using geometric features, in: J. Braz, A. Ranchordas, H. Arajo,
J. Jorge (Eds.), Advances in Computer Graphics and Computer Vision,
volume 4 of Communications in Computer and Information Science,
Springer Berlin Heidelberg, 2007, pp. 229-239. link, code
M. Fornaciari, A. Prati, R. Cucchiara,
"A fast and effective ellipse detector for embedded vision applications", Pattern Recognition, 2014 link, code
It's also fairly easy to port to OpenCV this matlab script with the implementation of the two papers:
"A New Efficient Ellipse Detection Method" (Yonghong Xie Qiang , Qiang Ji / 2002)
"Randomized Hough Transform for Ellipse Detection with Result Clustering" (CA Basca, M Talos, R Brad / 2005)
Another very interesting algorithm is:
Dilip K. Prasad, Maylor K.H. Leung and Siu-Yeung Cho, “Edge curvature and convexity based ellipse detection method,” Pattern Recognition, 2012.
Matlab code can be found here
Directly applying Hough Transform is not feasible for an ellipse, since you'll work in a 5 dimensional parameter space. This will be really slow, and not accurate. So a lot of algorithms have been proposed. Among the ones mentioned here:
Xie and Ji approach is quite famous and very easy (but still slow)
Basca et al. approach is faster, but may be not accurate
Prasad et al. approach is fast, and very accurate.
Libuda et al. approach is very fast and accurate
Fornaciari et al. approach is the fastest, but may be inaccurate in particular cases.
For the up-to-date bibliography for ellipse detection, you can refer to this page.
Related
I am working on a project that aims to build a program which automatically gives a relatively accurate detection of pupil region in eye pictures. I am currently using simplecv in Python, given that Python is easier to experiment with. Since I just started, the eye pictures I am working with are fairly standardized. However, the size of iris and pupil as well as the color of iris can vary. And the position of the eye can shift a little among pictures. Here's a picture from wikipedia that is similar to the pictures I am using:
"MyStrangeIris.JPG" by Epicstessie is licensed under CC BY-SA 3.0
I have tried simple thresholding. Since different eyes have different iris colors, a fixed thresholding would not work on all pictures.
In addition, I tried simplecv's build-in sobel and canny edge detection, it's not working especially for eyes with darker iris. I also doubt that sobel or canny alone can solve the problem, given sometimes there are noises on the edge of the pupil (e.g., reflection of eyelash)
I have entry-level knowledge about image processing and machine learning. Right now, I am thinking about three possibilities:
Do a regression on the threshold value base on some variables
Make a specific mask only for edge detection for the pupil
classification on each pixel (this looks like lots of work to build the training set)
Am I on the right track? I would like to reach out to anyone with more experience on this type of problem. Any tips/suggestions are more than welcome. Thanks!
I think that for start you should put aside the machine learning. You have so much more to try in "regular" computer vision.
You need to try and describe a model for your problem. A good way to do this is to sit and think how you as a person detect iris. For example, i can think of:
It is near the center of image.
It is is Brown/green/blue circle, with distinct black center, surrounded by mostly white ellipse.
You have a skin color around the white ellipse.
It can't be too small or too large (depends on your images..)
After you build your model, try to find better ways to find these features. Hard to point on specific stuff, but you can start from: HSV color space, Correlation, Hough transform, Morphological operations..
Only after you feel you have exhausted all conventional tools, start thinking on features extraction and machine learning..
And BTW, because you are not the first person that try to detect iris, you can look at other projects for ideas.
I have written a small matlab code for image (link you have provided), function which i have used is hough transform for circle detection, which has also implemented in opencv, so porting will not create problem, i just want to know that i am on write way or not.
my result and code is as follows:
clc
clear all
close all
im = imresize(imread('irisdet.JPG'),0.5);
gray = rgb2gray(im);
Rmin = 50; Rmax = 100;
[centersDark, radiiDark] = imfindcircles(gray,[Rmin Rmax],'ObjectPolarity','dark');
figure,imshow(im,[])
viscircles(centersDark, radiiDark,'EdgeColor','b');
Input Image:
Result of Algorithm:
Thank You
Not sure about iris classification, but I've done written digit recognition from photos. I would recommend tuning up the contrast and saturation, then use a k-nearest neighbour algorithm to classify your images. Depending on your training set, you can get as high as 90% accuracy.
I think you are on the right track. Do image preprocessing to make classification easier, then train an algorithm of your choice. You would want to treat each image as one input vector though, instead of classifying each pixel!
I think you can try Active Shape Modelling or if you want a really feature rich modelling and do not care about the time it takes execute the algorithm you can try Active appearance modelling. You might want to look into these papers for better understanding:
Active Shape Models: Their Training and Application
Statistical Models of Appearance for Computer Vision - In Depth
At the end of the introduction to this instructive kaggle competition, they state that the methods used in "Viola and Jones' seminal paper works quite well". However, that paper describes a system for binary facial recognition, and the problem being addressed is the classification of keypoints, not entire images. I am having a hard time figuring out how, exactly, I would go about adjusting the Viola/Jones system for keypoint recognition.
I assume I should train a separate classifier for each keypoint, and some ideas I have are:
iterate over sub-images of a fixed size and classify each one, where an image with a keypoint as center pixel is a positive example. In this case I'm not sure what I would do with pixels close to the edge of the image.
instead of training binary classifiers, train classifiers with l*w possible classes (one for each pixel). The big problem with this is that I suspect it will be prohibitively slow, as every weak classifier suddenly has to do l*w*original operations
the third idea I have isn't totally hashed out in my mind, but since the keypoints are each parts of a greater part of a face (left, right center of an eye, for example), maybe I could try to classify sub-images as just an eye, and then use the left, right, and center pixels (centered in the y coordinate) of the best-fit subimage for each face-part
Is there any merit to these ideas, and are there methods I haven't thought of?
however, that paper describes a system for binary facial recognition
No, read the paper carefully. What they describe is not face specific, face detection was the motivating problem. The Viola Jones paper introduced a new strategy for binary object recognition.
You could train a Viola Jones style Cascade for eyes, another for a nose, and one for each keypoint you are interested in.
Then, when you run the code - you should (hopefully) get 2 eyes, 1 nose, etc, for each face.
Provided you get the number of items you expected, you can then say "here are the key points!" What takes more work is getting enough data to build a good detector for each thing you want to detect, and gracefully handling false positives / negatives.
I ended up working on this problem extensively. I used "deep learning," aka several layers of neural networks. I used convolutional networks. You can learn more about them by checking out these demos:
http://cs.stanford.edu/people/karpathy/convnetjs/demo/mnist.html
http://deeplearning.net/tutorial/lenet.html#lenet
I made the following changes to a typical convolutional network:
I did not do any down-sampling, as any loss of precision directly translates to a decrease in the model's score
I did n-way binary classification, with each pixel being classified as a keypoint or non-keypoint (#2 in the things I listed in my original post). As I suspected, computational complexity was the primary barrier here. I tried to use my GPU to overcome these issues, but the number of parameters in the neural network were too large to fit in GPU memory, so I ended up using an xl amazon instance for training.
Here's a github repo with some of the work I did:
https://github.com/cowpig/deep_keypoints
Anyway, given that deep learning has blown up in popularity, there are surely people who have done this much better than I did, and published papers about it. Here's a write-up that looks pretty good:
http://danielnouri.org/notes/2014/12/17/using-convolutional-neural-nets-to-detect-facial-keypoints-tutorial/
I was reading about median filters in John C. Russ - The Image Processing Handbook, Sixth Edition, and in his book he describes the possible shape in the neighbouring pixels, which could be, for example a cross or square.
So I came across OpenCV medianBlur and was unsure about the pattern used. I would assume it is squared, but not sure. So what is the pattern implemented in OpenCV?
You are correct. It is squared.
For a project I've to detect a pattern and track it in space despite rotation, noise, etc.
It's highlighted with IR light and recorded with an IR camera:
Picture: https://i.stack.imgur.com/RJuVS.png
As on this picture it will be only very simple shape and we can choose which one we're gonna use.
I need direction on how to process a recognition of these shapes please.
What I do currently is thresholding and erosion to get a cleaner shape and then a contour detection and a polygon approximation.
What should I do then? I tried hu-moments but it wasn't good at all.
Could you please give me a global approach to recognize and track such pattern in space?
Can you choose which shape to project?
if so I would recomend using few concentric circles. Then using hough transform for circles you can easily find the center of the shape even when tracking is extremly hard (large movement/low frame rate).
If you must use rectangular shape then there is a good open source which does that. It is part of a project to read street signs and auto-translate them.
Here is a link: http://code.google.com/p/signfinder/
This source is not large and it would be easy to cut out the relevant part.
It uses "good features to track" of openCV in module CornerFinder.
Hope it helped
It is possible, you need following steps: thresholding image, some morphological enhancement,
blob extraction and normalization of blob size, blobs shape analysis, comparison of analysis results with pattern that you want to track.
There is many methods for blobs shape analysis. Simple methods: geometric dimensions, area, perimeter, circularity measurement; bit quads and others (for example, William K. Pratt "Digital Image Processing", chapter 18). Complex methods: spacial moments, template matching, neural networks and others.
In any event, it is very hard to answer exactly without knowledge of pattern shapes that you want to track )
hope it helped
I"m using a Thermal camera for a project and I'm a little stumped so as to how to think about calculating intrinsics for it. The usual camera's would determine different points on a chessboard or something similar, but the thermal camera won't really be able to differentiate between those points. Does anyone have any insight on what the intrinsics for thermal cameras would really look like?
Cheers!
EDIT - In addition to the great suggestions I currently have, I'm also considering using aluminum foil on the whites to create a thermal difference. Let me know what you think of this idea as well.
This might or might not work, depending on the accuracy you need:
Use a chessboard pattern and shine a really strong light at it. The black squares will likely get hotter than the white squares, so you might be able to see the pattern in the thermal image.
Put small lightbulbs on the edges of a chessboard pattern, light them, wait until they become hot, use your thermal camera on it.
This problem is addressed in A Mask-Based Approach for the Geometric Calibration of Thermal Infrared Cameras, which basically advocates placing an opaque mask with checkerboard squares cut out of it in front of a radiating source such as a computer monitor.
Related code can be found in mm-calibrator.
If you have a camera that is also sensitive to the visible light end of the spectrum (i.e. most IR cameras - which is what most Thermography is based on after all) then simply get a IR cut-off filter and fit this to front of the cameras lens (you can get some good c-mount based ones). Calibrate as normal to the fixed optics then remove the filter. Intrinsics should be the same - since optical properties are the same (for most purposes).
Quoting from the section 5 Image fusion
in GADE, Rikke; MOESLUND, Thomas B. Thermal cameras and applications: a survey. Machine Vision and Applications, 2014, 25.1: 245-262. (freely downloadable in June 2014):
The standard chessboard method for geometric calibration, correction
of lens distortion, and alignment of the cameras relies on colour
difference, and cannot be used for thermal cameras without some
changes. Cheng et al. [30] and Prakash et al. [146] reported that when
heating the board with a flood lamp, the difference in emissivity of
the colours will result in an intensity difference in the thermal
image. However, a more crisp chessboard pattern can be obtained by
constructing a chessboard of two different materials, with large
difference in thermal emissivity and/or temperature [180]. This
approach is also applied in [68] using a copper plate with milled
checker patterns in front of a different base material, and in [195]
with a metal wire in front of a plastic board. When these special
chessboards are heated by a heat gun, hairdryer or similar, a clear
chessboard pattern will be visible in the thermal image, due to the
different emissivity of the materials. At the same time, it is also
visible in the visual image, due to colour difference. Figure 12 shows
thermal and RGB pictures from a calibration test. The chessboard
consists of two cardboard sheets, where the white base sheet has been
heated right before assembling the board.
[30] CHENG, Shinko Y.; PARK, Sangho; TRIVEDI, Mohan M. Multiperspective thermal ir and video arrays for 3d body tracking and driver activity analysis. In: Computer Vision and Pattern Recognition-Workshops, 2005. CVPR Workshops. IEEE Computer Society Conference on. IEEE, 2005. p. 3-3.
[146] PRAKASH, Surya, et al. Robust thermal camera calibration and 3D mapping of object surface temperatures. In: Defense and Security Symposium. International Society for Optics and Photonics, 2006. p. 62050J-62050J-8.
[180] VIDAS, Stephen, et al. A mask-based approach for the geometric calibration of thermal-infrared cameras. Instrumentation and Measurement, IEEE Transactions on, 2012, 61.6: 1625-1635.
[68] HILSENSTEIN, V. Surface reconstruction of water waves using thermographic stereo imaging. In: Image and Vision Computing New Zealand. 2005. p. 102-107.
[195] NG, Harry, et al. Acquisition of 3D surface temperature distribution of a car body. In: Information Acquisition, 2005 IEEE International Conference on. IEEE, 2005. p. 5 pp.
You may want to consider running a thermal resistor wire on the lines of the pattern (you also need a power source).
You can drill holes in a metal plate and then heat the plate, hopefully the holes will be colder than the plate and will appear as circles in the image.
Then you can use OpenCV (>2.0) to find circle centers http://docs.opencv.org/doc/tutorials/calib3d/camera_calibration/camera_calibration.html#cameracalibrationopencv
See also the function findCirclesGrid.