I am trying to differentiate between camera motion and tool motion in a surgical video.
I have tried optical flow using opencv farneback and pass the results to an ML model to learn but no success.a major issue is getting good keypoints in case of camera motion. Is there an alternate technique to distinguish between camera motion and tool/tissue movement? Note: camera motion happens only in 10% of the video
I wish I could add a comment (too new to be able to comment), as I don't have a good answer for you.
I think it really depends on the nature of the input image. Can you show some typical input images here?
What is your optical flow result look like? I thought you might get some reasonable results.
Have you tried some motion estimation method, to analyze if there is global movement across different frames, or there is only some local movements?
Related
I have a stationary mono camera which captures a single image frame at some fps.
Assume the camera is not allowed to move,how do I generate a stereo image pair from the obtained single image frame? Is there any algorithms exists for this? If so, are they available in Open-CV?
To get a stereo image, you need a stereo camera, i.e. a camera with two calibrated lenses. So you cannot get a stereo image from a single camera with traditional techniques.
However, with the magic of deep learning, you can obtain the depth image from single camera.
And no, there's no builtin OpenCV function to do that.
The most common use of this kind of techniques is in 3D TVs, which often offer 2D-to-3D conversion, and thus mono to stereo conversion.
Various algorithms are used for this, you can look at this state of the art report.
There is also optical way for this.
If you can add binocular prisms/mirrors to your camera objective ... then you could obtain real stereoscopic image from single camera. That of coarse need access to the camera and setting up the optics. This also introduce some problems like wrong auto-focusing , need for image calibration, etc.
You can also merge Red/Cyan filtered images together to maintain the camera full resolution.
Here is a publication which might be helpful Stereo Panorama with a single Camera.
You might also want to have a look at the opencv camera calibration module and a look at this page.
I'm trying to develop an algorithm for real time tracking moving objects with a single moving camera setup as a project, in OpenCV (C++).
My basic objectives are
Detect motion in an (initially) static frame
Track that moving object (camera to follow that object)
Here is what I have tried already
Salient motion detection using temporal differencing and Optical Flow. (does not compensate for a moving camera)
KLT based feature tracking, but I was not able to segment the moving object features (moving object features got mixed with other trackable features in the image)
Mean shift based tracking (required initialization and is a bit computationally expensive)
I'm now trying to look into the following methods
Histogram of Gradients.
Algorithms that implement camera motion parameters.
Any advice on which direction should I proceed forward to acheive my objective.
Type: 'zdenek kalal predator' to google.com and watch the videos, read the papers that came up. I think it will give you a lot of insight.
I am trying to create a Panorama app for iPhone/iPad.
The image stitching bit is OK, I'm using openCV libraries and the results are pretty acceptable.
But I'm a bit stuck on developing the UI for assisting the user while capturing the panorama.
Most apps (even on Android) would provide user with some sort of a marker that translates/rotates exactly matching the movement of the user's camera.
[I'm using the iOS 7 - default camera's panorama feature as a preliminary benchmark].
However, I'm way off the mark till date.
What I've tried:
I've tried using the accelerometer and gyro data for tracking the marker. With this approach -
I've applied an LPF on the accelerometer data and used simple
Newtonian mechanics (with a carefully tuned damping factor) to
translate the marker on the screen. Problem with this approach: very erratic data. Marker tends to jump and wobble between points. Hard to tell between smooth movement and jerk.
I've tried using a complimentary filter between LPF-ed gyro and
accelerometer data to translate the blob. Problem with this approach: Slightly better than the first approach, but still quite random.
I've also tried using image processing to compute optical flow. I'm
using openCV's
goodFeaturesToTrack(firstMat, cornersA, 30, 0.01, 30);
to get the trackable points from a first image (sampled from camera
picker) and then using calcOpticalFlowPyrLK to get the positions
of these points in the next image.
Problem with this approach: However, the motions vectors obtained from tracking these points are too noisy to compute the resultant
direction of motion accurately.
What I think I should do next:
Perhaps compute the DCT matrix from accelerometer and gyro data and
use some algorithm to filter one output with the other.
Work on the image processing algorithms, use some different techniques
(???).
Use Kalman filter to fuse the state prediction from
accelerometer+gyro with that of the image processing block.
The help that I need:
Can you suggest some easier way to get this job done?
If not, can you highlight any possible mistake in my approach? Does it really have to be this complicated?
Please help.
I am working on an app in iOS that will occur an event if camera detects some changes in image or we can say motion in image. Here I am not asking about face recognition or a particular colored image motion, And I got all result for OpenCV when I searched, And I also found that we can achieve this by using gyroscope and accelerometer both , but how??
I am beginner in iOS.So my question is , Is there any framework or any easy way to detect motion or motion sensing by camera.And How to achieve?
For Example if I move my hand before camera then it will show some message or alert.
And plz give me some useful and easy to understand links about this.
Thanx
If all you want is some kind of crude motion detection, my open source GPUImage framework has a GPUImageMotionDetector within it.
This admittedly simple motion detector does frame-to-frame comparisons, based on a low-pass filter, and can identify the number of pixels that have changed between frames and the centroid of the changed area. It operates on live video and I know some people who've used it for motion activation of functions in their iOS applications.
Because it relies on pixel differences and not optical flow or feature matching, it can be prone to false positives and can't track discrete objects as they move in a frame. However, if all you need is basic motion sensing, this is pretty easy to drop into your application. Look at the FilterShowcase example to see how it works in practice.
I don't exactly understand what you mean here:
Here I am not asking about face recognition or a particular colored
image motion, because I got all result for OpenCV when I searched
But I would suggest to go for opencv as you can use opencv in IOS. Here is a good link which helps you to setup opencv in ios.
There are lot of opencv motion detection codes online and here is one among them, which you can make use of.
You need to convert the UIImage ( image type in IOS ) to cv::Mat or IplImage and pass it to the opencv algorithms. You can convert using this link or this.
How can we detect rapid motion and object simultaneously, let me give an example,....
suppose there is one soccer match video, and i want to detect position of each and every players with maximum accuracy.i was thinking about human detection but if we see soccer match video then there is nothing with human detection because we can consider human as objects.may be we can do this with blob detection but there are many problems with blobs like:-
1) I want to separate each and every player. so if players will collide then blob detection will not help. so there will problem to identify player separately
2) second will be problem of lights on stadium.
so is there any particular algorithm or method or library to do this..?
i've seen some research paper but not satisfied...so suggest anything related to this like any article,algorithm,library,any method, any research paper etc. and please all express your views in this.
For fast and reliable human detection, Dalal and Triggs' Histogram of Gradients is generally accepted as very good. Have you tried playing with that?
Since you mentioned rapid motion changes, are you worried about fast camera motion or fast player/ball motion?
You can do 2D or 3D video stabilization to fix camera motion (try the excellent Deshaker plugin for VirtualDub).
For fast player motion, background subtraction or other blob detection will definitely help. You can use that to get a rough kinematic estimate and use that as an estimate of your blur kernel. This can then be used to deblur the image chip containing the player.
You can do additional processing to establish identify based upon OCRing jersey numbers, etc.
You mentioned concern about lights on the stadium. Is the main issue that it will cast shadows? That can be dealt with by the HOG detector. Blob detection to get blur kernel should still work fine with the shadow.
If you have control over the camera, you may want to reduce exposure times to reduce blur. Denoising techniques can be used to reduce CCD noise that occurs with extreme low light and dense optical flow approaches align the frames and boost the signal back up to something reasonable via adding the denoised frames.