I'm trying to compute a motion histogram of the region around surf descriptor, and i don't know if there is a method to compute the size of the histogram (how many bins) and the size of the patch (the region of the neighborhood around the interest point)?
Is there any relation between the surf scale and the histogram?
Thanks a lot
Size(number of bins) of the motion histogram is always the same - number of the directions you divide motion. You specify it.
Scale is one of the parameters of the detected SURF keypoint, you can simply access it. About relation -it is the same question as "is there any relation between size of the patch and its movement?". Depends on your data.
Related
When using OpenCV for example, algorithms like SIFT or SURF are often used to detect keypoints. My question is what actually are these keypoints?
I understand that they are some kind of "points of interest" in an image. I also know that they are scale invariant and are circular.
Also, I found out that they have orientation but I couldn't understand what this actually is. Is it an angle but between the radius and something? Can you give some explanation? I think I need what I need first is something simpler and after that it will be easier to understand the papers.
Let's tackle each point one by one:
My question is what actually are these keypoints?
Keypoints are the same thing as interest points. They are spatial locations, or points in the image that define what is interesting or what stand out in the image. Interest point detection is actually a subset of blob detection, which aims to find interesting regions or spatial areas in an image. The reason why keypoints are special is because no matter how the image changes... whether the image rotates, shrinks/expands, is translated (all of these would be an affine transformation by the way...) or is subject to distortion (i.e. a projective transformation or homography), you should be able to find the same keypoints in this modified image when comparing with the original image. Here's an example from a post I wrote a while ago:
Source: module' object has no attribute 'drawMatches' opencv python
The image on the right is a rotated version of the left image. I've also only displayed the top 10 matches between the two images. If you take a look at the top 10 matches, these are points that we probably would want to focus on that would allow us to remember what the image was about. We would want to focus on the face of the cameraman as well as the camera, the tripod and some of the interesting textures on the buildings in the background. You see that these same points were found between the two images and these were successfully matched.
Therefore, what you should take away from this is that these are points in the image that are interesting and that they should be found no matter how the image is distorted.
I understand that they are some kind of "points of interest" of an image. I also know that they are scale invariant and I know they are circular.
You are correct. Scale invariant means that no matter how you scale the image, you should still be able to find those points.
Now we are going to venture into the descriptor part. What makes keypoints different between frameworks is the way you describe these keypoints. These are what are known as descriptors. Each keypoint that you detect has an associated descriptor that accompanies it. Some frameworks only do a keypoint detection, while other frameworks are simply a description framework and they don't detect the points. There are also some that do both - they detect and describe the keypoints. SIFT and SURF are examples of frameworks that both detect and describe the keypoints.
Descriptors are primarily concerned with both the scale and the orientation of the keypoint. The keypoints we've nailed that concept down, but we need the descriptor part if it is our purpose to try and match between keypoints in different images. Now, what you mean by "circular"... that correlates with the scale that the point was detected at. Take for example this image that is taken from the VLFeat Toolbox tutorial:
You see that any points that are yellow are interest points, but some of these points have a different circle radius. These deal with scale. How interest points work in a general sense is that we decompose the image into multiple scales. We check for interest points at each scale, and we combine all of these interest points together to create the final output. The larger the "circle", the larger the scale was that the point was detected at. Also, there is a line that radiates from the centre of the circle to the edge. This is the orientation of the keypoint, which we will cover next.
Also I found out that they have orientation but I couldn't understand what actually it is. It is an angle but between the radius and something?
Basically if you want to detect keypoints regardless of scale and orientation, when they talk about orientation of keypoints, what they really mean is that they search a pixel neighbourhood that surrounds the keypoint and figure out how this pixel neighbourhood is oriented or what direction this patch is oriented in. It depends on what descriptor framework you look at, but the general jist is to detect the most dominant orientation of the gradient angles in the patch. This is important for matching so that you can match keypoints together. Take a look at the first figure I have with the two cameramen - one rotated while the other isn't. If you take a look at some of those points, how do we figure out how one point matches with another? We can easily identify that the top of the cameraman as an interest point matches with the rotated version because we take a look at points that surround the keypoint and see what orientation all of these points are in... and from there, that's how the orientation is computed.
Usually when we want to detect keypoints, we just take a look at the locations. However, if you want to match keypoints between images, then you definitely need the scale and the orientation to facilitate this.
I'm not as familiar with SURF, but I can tell you about SIFT, which SURF is based on. I provided a few notes about SURF at the end, but I don't know all the details.
SIFT aims to find highly-distinctive locations (or keypoints) in an image. The locations are not merely 2D locations on the image, but locations in the image's scale space, meaning they have three coordinates: x, y, and scale. The process for finding SIFT keypoints is:
blur and resample the image with different blur widths and sampling rates to create a scale-space
use the difference of gaussians method to detect blobs at different scales; the blob centers become our keypoints at a given x, y, and scale
assign every keypoint an orientation by calculating a histogram of gradient orientations for every pixel in its neighborhood and picking the orientation bin with the highest number of counts
assign every keypoint a 128-dimensional feature vector based on the gradient orientations of pixels in 16 local neighborhoods
Step 2 gives us scale invariance, step 3 gives us rotation invariance, and step 4 gives us a "fingerprint" of sorts that can be used to identify the keypoint. Together they can be used to match occurrences of the same feature at any orientation and scale in multiple images.
SURF aims to accomplish the same goals as SIFT but uses some clever tricks in order to increase speed.
For blob detection, it uses the determinant of Hessian method. The dominant orientation is found by examining the horizontal and vertical responses to Haar wavelets. The feature descriptor is similar to SIFT, looking at orientations of pixels in 16 local neighborhoods, but results in a 64-dimensional vector.
SURF features can be calculated up to 3 times faster than SIFT features, yet are just as robust in most situations.
For reference:
A good SIFT tutorial
An introduction to SURF
I have a set of 100K 64x64 gray patches (that are already aligned, meaning they all have the same orientation) and I would like to extract a SIFT descriptor from each one using OpenCV.
It is clear to me all I need to do is to define a vector with one keypoint kp such that: kp.x=32, kp.y=32.
However, I don't know how to set the kp.size parameter. From going over SIFT's code, it looks as it's doing some non-trivial calculations with that parameter instead of just assuming that it's the size of the patch.
Question 1: what should be the kp.size parameter when extracting SIFT descriptors from patches of size 64x64?
Question 2: what should be the kp.size parameter when extracting SURF descriptors from patches of size 64x64?
If you look at sift original publication, the scale of the keypoint is used to weight the histogram of gradients magnitude and orientations(paragraph 6. The local image descriptor). So in your case, since the grey patches are aligned, it is up to you to decide if you want to weight the contributions of the pixels further from the patch center or not, and select the scale (i.e. the with of the gaussian weighting window) accordingly.
For SURF, it's basically the same principle except that instead of gradient magnitude the response to haar wavelet is use, but you could still weight those responses with a gaussian window.
Also, since you are working with those aligned patches I would advise you not to use the high-level functions of OpenCV, but to simply use/recode the descriptor extraction part, and to apply any weighting you want to compute your patch representation. One reason to do so is that, in the SIFT example, the computation of SIFT descriptors might "add new keypoints" to the one you provided, if the algorithm is "not happy" with the keypoint orientation, it duplicates the keypoint at the same location but different orientation.
Okay. So the SIFT descriptor uses a neighbourhood of 4x4 grids usually, each grid usually being 4x4 pixels. Therefore the neighbourhood in pixels is usually 16x16. The scale/size is the parameter to determine the amount of downsampling/blurring/radius of keypoint. So I would think in your case, this would be 4.
You probably would also know that SIFT keypoints also work on sub-pixel layers. (32,32) would not be the exact center of your image patch, which would actually be (32.5, 32.5) if your image dimensions (x,y) start from 1. If they start from 0, it would be (31.5, 31.5)- as in the case of opencv.
I want to implement a simple noise correction scheme for RGB images. The image contain some garbage pixels at random locations. So, I am thinking of doing this:
Segment the image.
Calculate histograms for each segment.
Analyze the histogram and dump the pixels which are negligible in histogram distribution over a segment.
I am using openCV. I have implemented steps 1 and 2, but I am not able to find out the number of pixels in each histogram bin. Please help
In order to analyze a histogram, you have to make a few assumptions about it. One good assumption is that the histogram will be roughly modeled as noise + gaussian bell curves.
Check this out.
http://en.wikipedia.org/wiki/Root-finding_algorithm
Finding the roots of the derivative function of the histogram will give you the location of the peaks. You can then find the boundaries of each peak, possibly by finding the roots of the second derivative function.
After you identify the location and span of the histogram peaks, you can classify pixels as being signal or noise pixels.
I am working on an image registration method applied to histological images.
I have one question. I would like to use MSER feature detector to detect keypoints on my image. After the MSER contours were retrieved using the MSER function provided by opencv, I calculate the centroid of each contour in order to use it as an interesting point.
If I make a description of the interesting points directly, with a Surf descriptor for example, the size of the descriptor is one, and is not possible to compare them.
Therefore it is necessary to modify the size of the descriptor with a suitable size.
Does anyone have an idea?
Thanks
Tha answer is very late but I hope it helps someone.
MSER returns you regions and not points.
In order to extract descriptors from MSER regions, you have to map the regions to a standard size, say from any elliptical shape to 30x30 pixel circle and then extract descriptor (such as SURF) out of it.
If you use SURF alone then it uses Harris corners as interest points and then places fixed sized windows around it in a scale space.
Matching is done by comparing descriptors usually. You are trying to match keypoints (or interest points) which is different.
Last but not the least, it is unlikely that MSER centroids and SURF interest points can be can occur at the same location since MSER detect homogenous regions and SURF uses Harris Corners. Centroids of MSER can not have corners, so technically speaking, they will always be the outliers for each other.
In MATLAB 2011 onwards, MSER can be combined with SURF descriptors (Included in the Computer Vision System ToolBox).
One other way is to use computer_descriptors.ln shared by: http://www.robots.ox.ac.uk/~vgg/research/affine/descriptors.html
Best Regards
I am using disparity map generation from 2 stereoscopic images .And then I use the normal triangulation formula depth= focallength*baseline/disparity to get the depth .How can I check that the recovered depth is indeed correct? Is there some test for this ? I guess there are some tweak-able parameters like multiplying this depth by some factor etc but again that is more of trial and error.I am looking for something more concrete.How do people in the vision community generally verify the results?
I suggest you verify the depth measured in your images by measuring it in the real world. If there was a way to verify the measurement you did in the images .. in the images then you probably would have used that way to measure depth in the first place.
Measure the distance from your camera to some object in the real world, and measure the size of the object perpendicular to the axis of one of the camera's. Then also measure the distance and size in your images. You use the size measure in the real world combined with the size of the object in pixels in the image to scale the distance you calculate. The result should match the distance you mesaured.