i'd like to implement a kalman filter using OpenCV to track eye (in particular eye ball). I read something around internet about Kalman Filter. I have to set the state of my filter. What can i use as state? My only available data are 3D coordinates of eye (x,y,z).
You have to understand the Kalman filter first in order to use it. The most human readable intro with examples I have found so far is the SIGGRAPH Course Pack.
UPDATE
I do not know the Kalman filter implementation in OpenCV.
The state of the filter is perhaps the true coordinates of the eye. However, you can only estimate it from the frames (these are the coordinates you write in your question), hence the need for the filter.
To use Kalman filter as a black-box you will still need
an initial estimate of the state
measurement noise covariance R
process noise covariance Q
A reasonable estimate for 1. is the eye coordinates on the first frame.
As for 2. and 3., see 5.1 Parameter Estimation or Tuning in the SIGGRAPH Course pack.
Perhaps the example 4.3 An Example: Estimating a Random Constant will also help to understand how the Kalman filter works and what you need.
Related
I am confused about the wavelet. I understand that wavelet transform is just a bandpass filter centered at the center frequency. However, in PyWavelets, https://pywavelets.readthedocs.io/en/latest/index.html, a wavelet transform is implemented as a filter bank which can break a signal into low and high components. So my question is: how does wavelet, listed on http://wavelets.pybytes.com/wavelet/bior6.8/, play in this game? If they are used as a bandpass filter, how can the signal be broken into two parts instead of just being bandpassed?
A wavelet transform uses the scaling function, represented by a lowpass filter, to approximate the signal on the next level, and the wavelet function, represented by a highpass filter, to encode the difference between the current level and the next.
On the page you mention these are the left and right plots of each wavelet basis, respectively. The site also has a short explanation of the algorithm (via the API reference URL). The picture on Wikipedia makes it a bit clearer I think.
So the first-level detail coefficients come from a highpass filter, and the final-level approximation coefficients come from a lowpass filter. The levels in between are repeated lowpass filtering (and usually subsampling) followed by one highpass filter. Comparable to edge detection after repeated smoothing. In other words, the bandpass property does not happen in a single step, see also this earlier answer.
I'm trying to predict where thrown object will hit a ground, so I figure I need to predict it's path using Kalman filter to estimate future states of the projectile.
The camera I'm using is Kinect v1, I'm using the depth camera to track the object so I get (X,Y,Z) points each 4 frames, It gives me about 5 to 8 points, this video shows my tracked object, I'm trying to achieve similar results to this one.
Is Kalman filter the way to go?, if so, how do I estimate more than one future state since I want to get the future path of the projectile?.
Thank you.
I have a problem. I have read many papers about video stabilization. Almost papers mention about smoothing motion by using Kalman Filter, so it's strong and run in real-time applications.
But there is also another filter strongly, that is particle filter.
But why dont we use Partilce filter in smoothing motion to create stabilized video?
Some papers only use particle filter in estimating global motion between frames (motion estimation part).
It is hard to understand them.
Can anyone explain them for me, please?
Thank you so much.
A Kalman Filter is uni-modal. That means it has one belief along with an error covariance matrix to represent the confidence in that belief as a normal distribution. If you are going to smooth some process, you want to get out a single, smoothed result. This is consistent with a KF. It's like using least squares regression to fit a line to data. You are simplifying the input to one result.
A particle filter is multi-modal by its very nature. Where a Kalman Filter represents belief as a central value and a variance around that central value, a particle filter just has many particles whose values are clustered around regions that are more likely. A particle filter can represent essentially the same state as a KF (imagine a histogram of the particles that looks like the classic bell curve of the normal distribution). But a particle filter can also have multiple humps or really any shape at all. This ability to have multiple simultaneous modes is ideally suited to handle problems like estimating motion, because one mode (cluster of particles) can represent one move, and another mode represents a different move. When presented with this ambiguity, a KF would have to abandon one of the possibilities altogether, but a particle filter can keep on believing both things at the same time until the ambiguity is resolved by more data.
Can anybody help me to explain what is consistency map (confidence map) in computer vision? and is there any formula or built-in function to calculate it?
Thanks in advance.
Confidence map is your resultant knowledge, using a priori information & and an iteration of an algorithm. Prediction & measurement update style visual tracking algorithms are great examples. From wiki (mean-shift):
The mean shift algorithm can be used for visual tracking. The simplest
such algorithm would create a confidence map in the new image based on
the color histogram of the object in the previous image, and use mean
shift to find the peak of a confidence map near the object's old
position. The confidence map is a probability density function on the
new image, assigning each pixel of the new image a probability, which
is the probability of the pixel color occurring in the object in the
previous image. A few algorithms, such as ensemble tracking,
CAMshift, expand on this idea.
As far as I know, the term consistency map, on the other hand, is used in image registration problems, in general. A voxel needs to be "photo-consistent" to be rendered(modeled) in 3d. From wiki again (photo-consistency):
In computer vision photo-consistency determines whether a given voxel
is occupied. A voxel is considered to be photo consistent when its
color appears to be similar to all the cameras that can see it. Most
voxel coloring or space carving techniques require using photo
consistency as a check condition in Image-based modeling and rendering
applications.
EDIT: OpenCV has built-in kalman filter, mean-shift and camshift algorithms. It also has a 3D rendering API
Does anyone know how signal processing can help in regard to reduce motion blur? or can you point me to some resources, books or papers?
If you mean a naive Motion Blur Kernel, You can build a paramerized model of the kernel (Radius and Angle).
Based on those parameters you'll see its Spectrum has special shape and more importantly special places where it has zeros.
Now, given an image and assuming it doesn't have zeros in its spectrum you can look at its spectrum and infer the parameters of the "Motion Blur Kernel".
Using this parameters build the spectrum of the kernel and use Wiener Filter to restore the original image.
This is a naive method bu where the assumptions are right, it works pretty well.