Problem Statement: I have a video of ANSYS Simulation of vortices formed due to flat plate plunging. The video contains vortices (in simpler terms blobs), which are distinguished according to their rotation (based on red or blue colour).
Objective: The vortices are needed to be identified and labelled according to their rotation. For example, vortices having clockwise rotation (red coloured) are labelled as R1, R2, R3, and so on and the anticlockwise rotation (blue coloured) as B1, B2, B3, etc.
Problem: I have been able to identify vortices as either clockwise and anticlockwise using contour detection, but the major issue is of numbering. The identified blob when it goes out of frame the label gets destroyed ie. if the vortex had a label A1 and now goes out of frame this label gets assigned to another vortex in the frame thus loosing the uniqueness.
so your problem is tracking?
keep track of "objects" and their positions. I could call them vortices but I'll keep it general.
in a new frame, find blobs
then assign each of those blobs to the spatially nearest object (within some distance). check to make sure you don't match multiple blobs to the same object... or be aware of the possibility at least. if there's no existing object within some distance, create a new object for this blob.
update each object's position from its assigned blob's current position
take care to cull objects that disappear (i.e. had no assignment).
this is tracking by assignment.
this can be made arbitrarily complicated.
you should keep track of the colors of your objects so you know not to match a red blob to a blue object (vortex).
you could represent your objects using just a centroid. or you could represent them using a contour. then "distance" might involve calculating overlap and maybe even shape (look up fourier descriptors, they're the fourier transform of a polar representation of the contour).
you could assign (and update) a velocity to an object, and predict its next position linearly, and use that to maybe get a more stable prediction.
Related
There is an operation called Shift which is performed after DFT to bring zero-frequency components to the center of the frequency spectrum.
I have two questions regarding this operation:
Why don't/can't DFT automatically center the zero-frequency components?
What happens if we don't perform Shift operation after DFT? I.e. how does it affect our other tasks of image processing?
Can anyone provide me some material to know about this specific operation named Shift?
References:
- fftw shift zero-frequency to the image center
- Why we shift the zero-frequency component to the center of the spectrum?
The DFT, by definition, uses n=0..N-1 and k=0..N-1, where n is the index into the time-domain signal and k the index into the frequency-domain signal. k also corresponds to the frequency. The DFT is defined this way in analogy to Fourier series.
Since the frequency in the DFT is periodic, one can think of k=N-1 to correspond to k=-1 instead. The shift function thus moves the upper half of the frequencies to the left of the origin, so they can be more readily interpreted as negative frequencies. But this is solely a convenience for display, as it brings the frequency-domain signal to a form that we're more familiar with (this is probably because it makes some Fourier analysis easier to explain, and hence text books display it this way, and hence we learn about Fourier by looking at Frequency plots with the origin in the middle).
For most tasks in image processing, we do not need to shift the origin. Again, it is only for display that this is convenient and pretty.
For example, to compute cross-correlation:
cc = ifft( fft(img1) * conj(fft(img2)) )
Here, the top-left pixel of cc is the origin. If img1==img2, the top-left pixel will be the maximum value. If we had an fft function that shifted the origin to the middle, then the cross-correlation image cc would have its origin in the middle also. After finding the peak, we'd need to do some computations to figure out what the shift between img1 and img2 is. (Not that this is complicated, but it shows that shifting is not necessarily advantageous.)
When convolving, one often has a spatial-domain kernel with the origin in the middle (as for example in this recent question). In this case, one must shift the origin to the top-left before computing the DFT. But there is no point in shifting the origin of the frequency-domain signals just to multiply them together and then undo the shift. One can simply directly multiply the signals that have the origin in the top-left corner:
kernel = ifftshift(kernel)
filtered = ifft( fft(img) * fft(kernel) )
Note that there are two different shift functions, often called fftshift and ifftshift. The one shifts the origin from the top-left to the middle, the other shifts the origin from the middle to the top-left. These two functions do exactly the same thing for even-sized signals (images), but differ if the sizes are odd.
I am working on a task where the image I have presents missing data, I wish to obtain the gradient without internal boundary issues.
The idea is to build a height map out of point cloud data (done) and then evaluate the slopes using a gradient function, however the points are sparse and thus the image presents missing data.
The first approach I tried was to use dilation in order to grow the area by some pixels, then apply the gradient filter and finally mask the boundaries to remove fabricated data, but it seems to erodes slopes as well:
On this picture a height map is gererated from a point cloud which in turn comes from a stereo camera system, the camera is facing a steep wall. On the left is the height map and on the right is the dilated map. On the right side it seems the wall has been "pushed back".
What would be the best approach to eliminate the internal border conditions? I thought about dilating the values with a special function that ignores the "non-available pixels" (perhaps represented by 0 or -1) and takes the average of the surrounding available pixel values (if available). Is there such a function in OpenCV?
What is Distance Transform?What is the theory behind it?if I have 2 similar images but in different positions, how does distance transform help in overlapping them?The results that distance transform function produce are like divided in the middle-is it to find the center of one image so that the other is overlapped just half way?I have looked into the documentation of opencv but it's still not clear.
Look at the picture below (you may want to increase you monitor brightness to see it better). The pictures shows the distance from the red contour depicted with pixel intensities, so in the middle of the image where the distance is maximum the intensities are highest. This is a manifestation of the distance transform. Here is an immediate application - a green shape is a so-called active contour or snake that moves according to the gradient of distances from the contour (and also follows some other constraints) curls around the red outline. Thus one application of distance transform is shape processing.
Another application is text recognition - one of the powerful cues for text is a stable width of a stroke. The distance transform run on segmented text can confirm this. A corresponding method is called stroke width transform (SWT)
As for aligning two rotated shapes, I am not sure how you can use DT. You can find a center of a shape to rotate the shape but you can also rotate it about any point as well. The difference will be just in translation which is irrelevant if you run matchTemplate to match them in correct orientation.
Perhaps if you upload your images it will be more clear what to do. In general you can match them as a whole or by features (which is more robust to various deformations or perspective distortions) or even using outlines/silhouettes if they there are only a few features. Finally you can figure out the orientation of your object (if it has a dominant orientation) by running PCA or fitting an ellipse (as rotated rectangle).
cv::RotatedRect rect = cv::fitEllipse(points2D);
float angle_to_rotate = rect.angle;
The distance transform is an operation that works on a single binary image that fundamentally seeks to measure a value from every empty point (zero pixel) to the nearest boundary point (non-zero pixel).
An example is provided here and here.
The measurement can be based on various definitions, calculated discretely or precisely: e.g. Euclidean, Manhattan, or Chessboard. Indeed, the parameters in the OpenCV implementation allow some of these, and control their accuracy via the mask size.
The function can return the output measurement image (floating point) - as well as a labelled connected components image (a Voronoi diagram). There is an example of it in operation here.
I see from another question you have asked recently you are looking to register two images together. I don't think the distance transform is really what you are looking for here. If you are looking to align a set of points I would instead suggest you look at techniques like Procrustes, Iterative Closest Point, or Ransac.
Given a point on a plane A, I want to be able to map to its corresponding point on plane B. I have a set of N corresponding pairs of reference points between the two planes, however, the overall mapping is not a simple affine transform (no homographies for me).
Things I have tried:
For a given point, find the three closest reference points in plane A, compute barrycentric coordinates of that triangle, and then apply that transform to the corresponding reference points in plane B. How it failed: sometimes the three closest points were nearly collinear, so errors were huge. Also, there was no consistency in the mapping when crossing borders. It was very "jittery."
Compute all possible triangles given the N reference points (N^3). Order them by size. For the given point, find the smallest triangle that it's in. This fixes the linearly of the
points problem, but was still extremely jittery and slow.
Start with a triangulated plane A. Iterate through the reference points, adding each one to the reference plane. Every time you add a point it exists in at least one triangle. Break that triangle into three triangles using the new reference point as a vertex. You end up with plane A triangulated so you can map from plane A to plane B with ease. Issues: You can prove that every triangle will have a point that is on the edge of the planes. This results in huge errors if your reference points are far from the edge of the planes.
I feel like this should be a fairly standard problem. Are there standard algorithms/libraries for this?
There you go my friend.. I have used it myslef and can only recommend you give it a try.
Kahn Academy - Matrix transformations
Understanding how we can map one set of vectors to another set. Matrices used to define linear transformations
https://www.khanacademy.org/math/linear-algebra/matrix_transformations
I have a set of points to define a shape. These points are in order and essentially are my "selection".
I want to be able to contract this selection by an arbitrary amount to get a smaller version of my original shape.
In a basic example with a triangle, the points are simply moved along their normal which is defined by the points to the left and the right of the points in question.
Eventually all 3 points will meet and form one point but until that point they will make a smaller and smaller triangle.
For more complex shapes, when moving the individual points inward, they may pass through the outer edge of the shape resulting in weird artifacts. Obviously I'll need to cull these points and remove them from the array.
Any help in exactly how I can do that would be greatly appreciated.
Thanks!
This is just an idea but couldn't you find the center of mass of the object, create a vector from the center to each point, and move each point along this vector?
To find the center of mass would of course involve averaging each x and y coordinate. Getting a vector is as simple a subtracting the point in question with the center point. Normalizing and scaling are common vector operations that can be found with the Google.
EDIT
Another way to interpret what you're asking is you want to erode your collection of points. As in morphology erosion. This is typically applied to binary images but you can slightly modify the concept to work with a collection of points. Essentially, you need to write a function that, given a point, will return true (black) or false (white) depending on if that point is inside or outside the shape defined by your points. You'd have to look up how to do that for shapes that aren't always concave (it's harder but not impossible).
Now, obviously, every single one of your actual points will return false because they're all on the border (by definition). However, you now have a matrix of points around your point of interest that define where is "inside" and where is "outside". Average all of the "inside" points and move your actual point along the vector between itself and towards this average. You could play with different erosion kernels to see what works best.
You could even work with a kernel with floating point weights instead of either/or values which will affect your average calculation proportional to their weights. With this, you could approximate a circular kernel with a low number of points. Try the simpler method first.
Find the selection center (as suggested by colithium)
Map the selection points to the coordinate system with the selection center at (0,0). For example, if the selection center is at (150,150), and a given selection point is at (125,75), the mapped position of the point becomes (-25,-75).
Scale the mapped points (multiply X and Y by something in the range of 0.0..1.0)
Remap the points back to the original coordinate system
Only simple maths required, no need to muck about normalizing vectors.