I have a problem with the pupil center detection. I trained a CNN to give me the pupil center location but it is not always at the center.
How can I make good processing and have the ellipse fitting algorithm detect the center?
The process is this. I cut the face on a picture with dlib then I make the prediction and after I get the results I want to predict the center.
Here are two examples of the cnn prediction. Any guidance will be appreciated .
Direct radial rays from the center you found. Compute intensity gradient along each ray. Maximal gradients will your points on edge of iris. Then use fit ellipse.
From those pictures, it appears that the variable occlusion of the iris is what is throwing off your center find. What may help is being more specific about just the edge between iris and eye white (and not with eyelid). To do this I would (but there may be better ways). Drop a point inside the iris blob and project a grid of radially spaced vectors outward looking for the first dark to light transition above a minimum contrast. For each ray measure the contrast of the edge. The contrast should be almost exactly the same for all iris to eyewhite transitions and will have variance with the eyelid. Perform whatever type of data clustering you prefer to isolate the chunk of only pupil to eyewhite edges and then only feed those edge points into the ellipse center find.
Related
I have a problem at hand where I need to detect/predict the coordinates of the hinge point or axis of rotation point using image processing. The image is as shown below:
I've used a method where I started with tracking the circular movement (in an arc) of a few feature points in an RoI around the default hinge coordinates (entered manually) in a configuration file. This circular motion of these tracked points happens around the vertical axis which passes through the hinge point. Now, I tracked these points from their initial position until the connecting bar made a particular angle (15°/20°) with the y-axis, I drew secants between these different positions (start and end positions) of the same point and drew its perpendicular bisector, which will ideally pass through the centre of the (concentric) circles, which is the ideal hinge point.
Eg:
y_intercepts calculated for each point
H0 (322, 42)
H1 (322, 64) (within tolerance, closest to GT)
H2 (322, 48)
H_avg (322,52)
H_groundtruth (x,y): (322, 61)
We need an accuracy or tolerance of +/- 3 pixels.
Now, the issues we faced in this ideal scenario to practical working of it is:
Different tracked points give different potential hinge points (different dots on the vertical yellow line), (few of which are very close the ground truth(yellow circle)), but their weighted/average (big green circle) goes off the mark. Quite frankly, this is a problem of too many in which we do get the closest potentially to ground truth, but we’re not sure, which of these points is the closest as we’re not to use the default hitch coordinates (entered manually) from config file.
One solution could be to use frameworks already implemented for image registration such as elastix. If you configure it for a rigid registration, you can get the transformation matrix and therefore the center of the rotation.
The problem here is that only one part of your image is moving. Before doing the registration, I would simply mask the region of interest by calculating a mask from the subtraction of the two images, to keep only the part where something actually moved.
Such approach could get a subpixel accuracy. You could also repeat it for multiple angles and average the result. Alternatively to the averaging, you could use the RANSAC algorithm to know which hinge points are off (outliers) and exclude them.
Here is an example how to do a simple rigid transformation with elastix.
I hope this helps!
I intended this as only a comment, but it ended up significantly over the character limit:
The problem from an accuracy perspective (sorry, couldn't resist) seems to be that you're trying to use a planar euclidean geometry technique to solve a projective geometry problem.
Those feature tracks are only circular arcs in 3D world space. They're actually (noisy) elliptical arcs in 2D image pixel space due to the projection.
Your hinge rotation axis isn't a single pixel either, unless your camera's optical axis is directly aligned with the hinge axis. If that's not the case (as the perspective in the photo you added suggests), then your hinge axis is actually a line in pixel space, not a point, and different heights for the different tracks in model space will be 'centered' around different pixels on that line. So asking for +/- 3 pixel hinge 'point' accuracy is unclear, and so is measuring angles in pixel space in general in a way that doesn't account for perspective.
I only mention these details because you seem focused on measuring accurately. Often, those kinds of 2D approximations are fine for many applications, but high accuracy and precision from a single camera (if that's really what you need) requires better 3D scene understanding. (Or you could train a deep network with a bunch of labeled ground truth images and let it figure out the mappings.)
Now maybe you don't need such high accuracy for your application after all. In that case, simple affine geometry techniques like that mentioned in the other answer might work well enough.
I am trying to compare two image of drawings using corner features in the images. Here is a sample image:
Query image:
I used SIFT algorithm to compare images but it did not work because in SIFT we consider a window of 16X16 pixel to extract the features at point of interest but here in this case(drawing objects) we will get only corner points as feature points and SIFT feature descriptor will give very similar feature to all corner points and hence in the feature matching step it will reject the corners because of their close similarity scores.
So i am using below approach to compare the images.
I am using Shi-Tomasi algorithm based function in opencv ie. cv2.goodFeaturesToTrack() to find the corners(feature points) in an image. After finding corners i want to classify them in 4 categories and compare them in two images. Below is corner categories defined as of now which my vary because of huge variations in corner types(angle, no. of lines crossing at corners, irregular pixel variation at corner point):
Corner categories:
Type-1: L-shaped
Type-2: Line intersection
Type-3: Line-curve intersection
type-4: Curve-Curve intersectio
I am trying to solve this using below approach:
=> Take a patch of fixed window size surrounding the corner pixel say a window of 32X32
=> Find the gradient information ie. gradient magnitude and its direction in this window and use this information to classify the corner in above 4 classes.After going through image classification i came to know that Using HOG algorithm image gradient information can be converted to feature vectors.
=> HOG feature vector calculated in above step can be used to train SVM to get a model.
=> This model can be used for new feature point classification.
After implementing above algorithm i am getting poor accuracy.
If there is any other way to classify the corners please suggest.
I am currently working in SIFT, I had generated the difference of Gaussian and the extrema image layers. Can anyone explain to me how to use Hessian matrix to eliminate the low contrast keypoint?
A good keypoint is a corner. This comes from the Harris corner work and the Good features to track (KLT) papers first, then emphasized by the Mikolajczyk and Schmid paper.
Intuitively, a corner is a good feature because it is an intersection of two lines, while a single line segment can be moved along its direction, thus causing a less accurate localization.
A line segment is an edge, i.e., a first order derivative (gradient). A corner is an edge that changes its direction abruptly. This is measured by a second order derivative, hence the use of the Hessian matrix that contains the values of the directional second derivatives.
I am working on Face detection using Core Image. I m facing a problem when giving the points on face boundaries. Actually I ve to give the points on face boundaries and make those points in a movable state.
Please share your ideas with me..
Thanks in Advance...
I will suggest you four ways. I have not done them in IOS. I implemented them in java. For face boundary detection you can use following methods:
Skin color Thresholding followed by chin curve estimate and shrinking and growing algorithm.
Canny Edge Detection followed by four region divide and getting longest face edge map.
Adaptive active contour model (Snake algorithm).
Hair region detection followed by chin curve esimate and removing all the region except hair and chin. (Doesn't work for bald.)
If there are shadows in the face use Snake algorithm. If the faces in the images are clear then go for Skin Color Thresholding. Canny Edge Detection can give u very slow performance.
For further clarification read this : FACE REGION DETECTION
I'm trying to implement user-assisted edge detection using OpenCV.
Assume you have an image in which we need to find a polygonal shape. For the sake of discussion, let's say we need to find the top of a rectangular table in a picture. The user will click on the four corners of the table to help us narrow things down. Connecting those four points gives us a polygon, or four vectors.
But the user is not very accurate when clicking on those corners. So I'd like to use edge information from the image to increase the accuracy.
I'm using a Canny edge detector with a fairly high treshold to determine important edges in my image. (more precisely, I'm scaling down, blurring, converting to grayscale, then run Canny). How can I compute whether a vector aligns with an edge in my image? If I have a way to compute "alignment", my overal algorithm comes down to perturbating the location of the four edge points, computing the total "alignment" of my polygon with the edges in the image, until I find an optimum.
What is a good way to define and compute this "alignment" metric?
You may want to try to use FindContours to detect your table or any other contour. Then build a contour also from the user input points. After this you can read about Contour Moments by which you can compare contours. You can compare all the contours from the image with the one built from the user points and then select the closest match.