I'm trying to figure out how to detect whether a human that I've identified in a video is speaking. I'm using some of the multi-person multi-camera tracking code posted here to detect individuals and I want to determine whether someone identified is speaking at any given time. Is anyone aware of good CV projects that might be able to do this? I've trawled around the action recognition literature a bit but haven't found anything that seems to directly address this. Detection of speaking needs to be done only with video.
There is an implementation of face pose estimation in an open source library.
As you can see from this figure : there are lines around lips.By digging into source code of example you can track movement of lips as you try this example on your own environment you will see that lines covering lips are also moving depending on movement of lips.
I know that have some opencv algorthms ( here and here, for example) that handles effectiveness with stationary background subtraction. But i need do this in a non-stationary background. Thus using Hand-held camera.
I would appreciate if someone could give some tips.
Segmentation of a dynamic scene is a topic of ongoing research in CV... However, for a quick win, you might want to try this rather nice library. It builds beyond OpenCV and provides 37 different algorithms for you to try - perhaps one might work well for your use-case. Good luck!
I am trying to figure out how knocktounlock.com is able to detect "knocks" on the iPhone. I am sure they use the accelerometer to achieve this, however all my tries come up with false flags (if user moves, jumps, etc it sometimes fires)
Basically, I want to be able to detect when a user knocks/taps/smacks their phone (and be able to distinguish that from things that may also give a rise to the accelerometer). So I am looking for sharp high peeks. The device will be in the pocket so the movement of the device will not be very much.
I have tried things like high/low pass (not sure if there would be a better option)
This is a duplicate of this: Detect hard taps anywhere on iPhone through accelerometer But it has not received any answers.
Any help/suggestions would be awesome! Thanks.
EDIT: Looking for more thoughts before I accept the answer below. I did hear back from Knocktounlock and they use the fourth derivative (jounce) to get better values to then analyse. Which is interesting.
I would consider knock on the iPhone to be exactly same as bumping two phones with each other. Check out this Github Repo,
https://github.com/joejcon1/iOS-Accelerometer-visualiser
Build&Run the App on iPhone and check out the spikes on Green line. You can see the value of the spike clearly,
Knocking the iPhone:
As you can see the time of the actual spike is very short when you knock the phone. However the spike patterns are little different in Hard Knock and Soft knock but can be distinguished programmatically.
Now lets see the Accelerometer pattern when iPhone moves in space freely,
As you can see the Spikes are bell shaped that means the it takes a little time for spike value to return to 0.
By these pattern it will be easier to determine the knocking pattern. Good Luck.
Also, This will drain your battery out as the sensor will always be running and iPhone needs to persist connection with Mac via Bluetooth.
P.S.: Also check this answer, https://stackoverflow.com/a/7580185/753603
I think the way to go here is using pattern recognition with accelerometer data.
You could (write and) train a classifier (e.g. K-nearest neighbor) with data you gathered and that has been classified by hand. Neural networks are also an option. However, there will be many different ways to solve that problem. But there is probably no straightforward way for achieving this.
Some papers showing pattern recognition approaches to similar topics (activity, movement), like
http://www.math.unipd.it/~cpalazzi/papers/Palazzi-Accelerometer.pdf
(some more, but I am not allowed to post them with my reputation count. You can search for "pattern recognition accelerometer data")
There is also a master thesis about gesture recognition on the iPhone:
http://klingmann.ch/msc_thesis_marco_klingmann_iphone_gestures.pdf
In general you won't achieve 100% correct classification. Depending on the time/knowledge one has got the result will vary between good-usable and we-could-use-random-classification-instead.
Just a though, but It could be useful to add to the mix the output of the microphone to listen to really short, loud noises at the same time that a possible "knock" movement has been detected.
I am surprised that 4th derivative is needed, intuitively feels to me 3rd ("jerk", the derivative of acceleration) should be enough. It is a big hint what to keep eye on, though.
It seems quite simple to me: collect accelerometer data at high rates, plot on chart, observe. Calculate from that first derivative, plot&observe. Then rinse&repeat, derivative of the last one. Draw conclusions. I highly doubt you will need to do pattern recognition per se, clustering/classifiers/what-have-you - i think you will see very distinct peak on one of your charts, may only need to tune collection rate and smoothing.
It is more interesting to me how come you don't have to be running the KnockToUnlock app for this to work? And if it was running in the background, who left it run there for unlimited time. I dont think accel. qualifies for unlimited background run. And after some pondering, i am guessing the reason is that the app uses Bluetooth to connect Mac as accessory - and as such gets a pass from iOS to run in the background (and suck your battery, shhht)
To solve this problem you need to select the frequency. Tap (knock)
has a very high frequency, so you should chose the frequency of the
accelerometer is not lower than 50 Hz (perhaps even 100 Hz) for
quality tap detection in the case of noise from other movements.
The use of classifiers is necessary, but in order to save battery consumption you should not call a classifier very often.It should write a simple algorithm that would find only taps and situation similar to knoks and report that you program need to call a classifier.
Note the gyro signal, it also responds to knocks, besides the
gyroscope signal not be need separated from the constant component
and the gyroscope signal contains less noise.
That is a good video about the basics of working with smartphones sensors: http://talkminer.com/viewtalk.jsp?videoid=C7JQ7Rpwn2k#.UaTTYUC-2Sp .
Please check the attach image it's a kind of seesaw. But as from image the black bodies have same density. And the horizontal rectangle is attached with the triangle using "Revolute" joint. But still not working any suggestion. in the current situation it's need to be balanced.
Due to tiny imbalances in the layout caused by the limitations of floating point precision etc, it's highly unlikely that this will ever balance in the middle reliably (just like real life). One thing you could try is to give the beam some angular damping, which would make it less easy to swivel around, so it would slow down quicker and sleep earlier. That might be enough to get it to come to rest without falling to one side or the other.
I don't think it is the problem of floating point precision. At least it can't appear that fast. As far as I know Box2D resolves contacts (including resting contacts) one by one. It is much faster than simultaneous contact resolve, but less precise at the same time since resolving one contact create impact on the others.
I would try adding a motor with a small maximum torque to your revolute joint and controlling it's speed to balance the system.
Can anyone point me to a robust motion detection sample/implementation? I know EMGU has a motion detection sample, but its not very good, even small changes in light will be falsely detected as motion. I don't need to track objects. I am looking for a way to detect motion in a video that will not be falsely triggered by changing light conditions.
Have a look at AForge. Everything you should need is there (though you'll need to spend some time putting it all together), and it has a robust community if you need specific help.
I concur with nizmahone. use Aforge:
Here is a link with soem motiond etection in C#:
http://www.codeproject.com/KB/audio-video/Motion_Detection.aspx