I have a project where I need to use OpenCV to detect an object (Tennis Ball) on a webcam, and for bonus credit, track it when I roll it across the table.
I haven't had much luck finding info on this, since I'm using OpenCV 2.4, C++, and a lot of information is in the older OpenCV version. I've read a lot of about different ways to do it, but I just don't know how to implement it into my code.
Any help would be appreciated, especially on how to integrate a detection/tracking function into my code
Here is my code so far, I think the image detection/tracking code should go after I apply the filters:
//Includes & Namespaces
#include "cv.h"
#include "highgui.h"
#include <iostream>
using namespace cv;
using namespace std;
//Main Function
int main(int, char**)
{
VideoCapture vid(0); //Capture from Webcam
if(!vid.isOpened()) //Error Check for Webcam
{
cout << "Could not open camera" << endl;
return -1;
}
Mat pic; //Create Matrix to store image
namedWindow("video",1); //Open Window
for(;;) //Infinite loop
{
Mat frame; //Create Matrix for a single frame
vid >> frame; //Transfer from webcam to matrix
//Filters
cvtColor(frame, pic, CV_BGR2HSV);
GaussianBlur(pic, pic, Size(7,7), 1.5, 1.5);
/*Image Detection Here */
imshow("Picture", pic); //Show image
if(waitKey(30) >= 0)
break;
}
return 0;
}
Did you try to google your question? There are many info about that.
Simple idea is next: detect your object using color thresholding (it seems that it's yellow or white color) and circle detection. After ball is deteccted you need to just track it using (for example) Lucas-Kanade method.
Here are some guides/manuals:
Tracking colored objects OpenCV
Motion Analysis and Object Tracking
Learning OpenCV
Look at OpenCV's folder samples. There'are many very useful examples. In your situation the best example is samples/cpp/lkdemo.cpp.
Related
I start with the following image:
Using opencv I rotate 45° about the Y axis to get the following:
If I tried a little harder I could get it not to be cropped in the foreground.
Now my question: does opencv have the tools to do the reverse transformation? Could I take the second image and produce the first? (Not concerned about blurred pixels.) Please suggest a method.
Yes.
You already made a homography matrix to produce this picture, right?
Just invert it (H.inv()) or pass the WARP_INVERSE_MAP flag.
No need for all that other stuff.
Yes, its possible. After 45° rotation, there are some regions below and above are missing(not seen). You only can not get those parts back.
By using warpPerspective() and getPerspectiveTransform() together, you can easily get back to the first image. Only thing you need to consider is that you should fid the end points of rotated image. Such as: left_up , right_up , left_down , right_down respectively. Since you didn't specify the language, I used C++ to implement the functions. Here is the output and code:
#include <opencv2/imgproc/imgproc.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <iostream>
#include <fstream>
int main()
{
cv::Mat begin = cv::imread("/ur/img/dir/input.jpg");
cv::Mat output;
cv::Point2f Poly2[4] = {
cv::Point2f(31,9),
cv::Point2f(342,51),
cv::Point2f(28,571),
cv::Point2f(345,525), //points I got from looking in paint.
};
cv::Point2f Points[4] = {
cv::Point2f(0,0),
cv::Point2f(432,0),
cv::Point2f(0,576), //The picture I want to transform to.
cv::Point2f(432,576),
};
cv::Mat Matrix = cv::getPerspectiveTransform( Poly2,Points);
cv::warpPerspective(begin, output, Matrix, cv::Size(432, 576));
cv::imshow("Input", begin);
cv::imshow("Output", output);
cv::imwrite("/home/yns/Downloads/tt2.jpg",output);
cv::waitKey(0);
return 0;
}
I need to determine the position of the wrist in a frame with parts of a human under arm & matching hand.
So far I have isolated the hand & arm and I'm able to draw a polygon & hull curve around it:
I achieve this result by simple binary thresholding and automatic contour fitting.
Based on this I want to extract the location of the wrist. This needs to work for all orientations of the hand/wrist.
However, being fairly new to working with OpenCV it is unclear to me what the best way is to determine/isolate the location of the wrist. I have various ideas for this:
The arm section is a fairly straight. Maybe a simple line detection over the contour polygon might do the job to get straight lines for the under arm.
Somehow split the contour polygon into multiple sections. Basically it's fair to assume that the location of the wrist has the smallest distance between the two arms contouring the under arm. Is there a way to find that point along the polygon and then "cut" or "split" the polygon to get two? From there I'd have one polygon representing a rectangle which should be easy to work with.
Use an approach that iterates along the main axis of the polygon fitted using fitLine(), measuring the distance between two opposing points of the polygon, finding the shortest distance.
Unfortunately I lack the experience to make the correct choice here - or even come up with a better idea.
I'd appreciate any kind of ideas & pointers towards achieving this. I could find a lot of valuable research material when it comes to hand detection & tracking and basic body part matching using Haar cascades. Unfortunately, I couldn't find a way to apply those technologies for my use case.
Here's some raw material (images & videos) to work with: (Google Drive Link!): https://drive.google.com/drive/folders/1hU4hGw5dYtVrcXTq8TYWCWfcLWjT-ZJU?usp=sharing
Approach: I used the advantage of arm side. The thickness of arm is almost same until hitting the hand.
Assumption: I coded by assuming the arm will enter the screen vertically. Otherwise my code may not work. I tried all of the images you shared and its working properly for all.
My steps:
Make a simple segmentation methodology for getting only needed part of the image
Start to count the non-black pixel for each column by beginning from the arm side.
Until hitting a column which is different from the previous column countings, you are still on the arm side. When you hit, you reached the wrist.
Note: I decided the threshold experimentally.
Here are the results and code:
Input Image:
After segmentation:
Output after algorithm:
Code:
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include <iostream>
#include <cstdlib>
using namespace cv;
using namespace std;
int main() {
Mat src, gray, blur_image, threshold_output;
// take input image
src = imread("/ur/image/directory/image_01.jpg", 1);
// convert to grayscale
cvtColor(src, gray, COLOR_BGR2GRAY);
// add blurring to the input image
medianBlur(gray,gray,9);
// Apply a segmentation to arm
for(int i=0; i<gray.rows; i++)
for(int j=0;j<gray.cols; j++)
if(gray.at<uchar>(Point(j,i))<110)
gray.at<uchar>(Point(j,i)) = 0;
//Creat a bgr mat to show the results clearly
Mat copy_gray = gray;
cvtColor(copy_gray,copy_gray,CV_GRAY2BGR);
double sum = 0;
int loop_cnt = 0,enter = 1;
Point first,second;
for(int j=gray.cols-1; j>=0; j--)
{
loop_cnt++;
int counter = 0,ff=1,enter2 = 1;
for(int i=0;i<gray.rows; i++)
{
if(gray.at<uchar>(Point(j,i))!=0 && enter)
{
if(ff)
first = Point(j,i);
counter++;
ff = 0;
}
if(!ff && gray.at<uchar>(Point(j,i))==0 && enter2)
{
second = Point(j,i);
enter2 = 0;
}
}
sum += (double)counter;
double average = sum/(double)(loop_cnt);
if(abs(average-counter)>20.0 && enter)
{
line(copy_gray,Point(j,0),Point(j,500),Scalar(0,255,0),5);
enter = 0;
}
}
int distance = norm(second-first)/2;
circle(copy_gray,Point(first.x,first.y+distance),20,Scalar(0,0,255),5);
imshow("Result",copy_gray);
waitKey(0);
return 0;
}
I am not at all experienced with machine learning or image processing, so I'm hoping someone can give some pointers of first thoughts on this problem:
The image below is an example of a photograph of tomato plant leaf. We have thousands of these. We need to trace the veins and output a graph. We have already had undergraduates trace the veins by hand for a few hundreds, so I presume that this can be a training set for a machine learning approach.
So my question: what types of filters/classifiers immediately come to mind? Is there anything you recommend I read or take a look at?
Our first thought was, look at directional derivatives. Each pixel can be classified as being in an edge or not in an edge at a given angle, and if a pixel is in an for a lot of different angles, then it's probably a blotch and not a vein. Then the parameters of gradient threshold and angle variation allowed can be adjusted by the learning but probably this is not the best way...
Thank you for any help!
Two methods immediately come to mind
a sliding window neural network classifier
identifying a threshold that sets apart dark/light pixels in the image (this could be done using machine learning or perhaps a simple computation) and then doing a flood fill to identify regions in the image.
The second method should be simpler and quicker, so I'd perhaps prototype it first to see if it gives good enough answers.
In any case, my intuition is that it's going to be easier to solve the dual problem - not trying to find edges and nodes of the graph, but finding its faces. From that, you get the graph itself easily.
I did this very simple program to filter the vein regions using opencv. I've added comments to explain the operations. Resulting images for the intermediate steps are saved. Hope it helps.
#include "stdafx.h"
#include <opencv2/core/core.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <opencv2/imgproc/imgproc.hpp>
#include <iostream>
using namespace cv;
using namespace std;
#define INPUT_FILE "wMTjH3L.png"
#define OUTPUT_FOLDER_PATH string("")
#define CONTOUR_AREA_THRESHOLD 30.0
int _tmain(int argc, _TCHAR* argv[])
{
// read image as grayscale
Mat im = imread(INPUT_FILE, CV_LOAD_IMAGE_GRAYSCALE);
imwrite(OUTPUT_FOLDER_PATH + string("gray.jpg"), im);
// smooth the image with a gaussian filter
Mat blurred;
GaussianBlur(im, blurred, Size(3, 3), 1.5);
imwrite(OUTPUT_FOLDER_PATH + string("blurred.jpg"), blurred);
// flatten lighter regions while retaining the darker vein regions using morphological opening
Mat morph;
Mat morphKernel = getStructuringElement(MORPH_ELLIPSE, Size(5, 5));
morphologyEx(blurred, morph, MORPH_OPEN, morphKernel);
imwrite(OUTPUT_FOLDER_PATH + string("morph.jpg"), morph);
// apply adaptive thresholding
Mat adaptTh;
adaptiveThreshold(morph, adaptTh, 255.0, ADAPTIVE_THRESH_GAUSSIAN_C, CV_THRESH_BINARY_INV, 7, 2.0);
imwrite(OUTPUT_FOLDER_PATH + string("adaptth.jpg"), adaptTh);
// morphological closing to merge disjoint regions
Mat morphBin;
Mat morphKernelBin = getStructuringElement(MORPH_ELLIPSE, Size(3, 3));
morphologyEx(adaptTh, morphBin, MORPH_CLOSE, morphKernelBin);
imwrite(OUTPUT_FOLDER_PATH + string("adptmorph.jpg"), morphBin);
// find contours
vector<vector<Point>> contours;
vector<Vec4i> hierarchy;
findContours(morphBin, contours, hierarchy, CV_RETR_CCOMP, CV_CHAIN_APPROX_SIMPLE, Point(0, 0));
// filter contours by region areas and draw
RNG rng(12345);
Mat drawing = Mat::zeros(morphBin.size(), CV_8UC3);
for(int idx = 0; idx >= 0; idx = hierarchy[idx][0])
{
if (contourArea(contours[idx]) > CONTOUR_AREA_THRESHOLD)
{
Scalar color( rand()&255, rand()&255, rand()&255 );
drawContours( drawing, contours, idx, color, CV_FILLED, 8, hierarchy );
}
}
imwrite(OUTPUT_FOLDER_PATH + string("cont.jpg"), drawing);
return 0;
}
The output looks like this for the provided sample image:
I want to do video analysis to detect movement for a certain duration in time. For example, I have a video of my lane outside my house. I want to check whether it remains clean. So, i wanted to detect garbage lying around(and in case it is being cleaned). I tried a lot of sites, they said that I will have to take the video and divide it into frames and XOR the frames and find out the object movement.
I tried to find example code for this and I was unable to find the same. So if anybody has expertise in this field using OpenCV/Xuggler/JavaCV/ or any software and some code, can u please post so i can get going.
My main objective is to develop some software through which i can do realtime tracking of the garbage outside my house and check who is dumping it, and whether it is being cleaned. Is it possible? Any ideas/suggestions/ advice is appreciated. Thanks a lot!
I've tried OpenCV, but have no clue how to split video into frames and apply object detection on it.
Thanks!
This is the code of background Subtraction based technique. If there is any change in the background (which your lane in this case), you would be able to detect it.
You should have a sample code of "Background subtraction" provided by openCV samples.
#include "opencv2/core/core.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include "opencv2/video/background_segm.hpp"
#include "opencv2/highgui/highgui.hpp"
#include <stdio.h>
#include <iostream>
#include <vector>
using namespace std;
using namespace cv;
int main()
{
VideoCapture cap;
bool update_bg_model = true;
cap.open(0);
cv::BackgroundSubtractorMOG2 bg;//(100, 3, 0.3, 5);
bg.set ("nmixtures", 3);
std::vector < std::vector < cv::Point > >contours;
cv::namedWindow ("Frame");
cv::namedWindow ("Background");
Mat frame, fgmask, fgimg, backgroundImage;
for(;;)
{
cap >> frame;
bg.operator()(frame, fgimg);
bg.getBackgroundImage (backgroundImage);
cv::erode (fgimg, fgimg, cv::Mat ());
cv::dilate (fgimg, fgimg, cv::Mat ());
cv::findContours (fgimg, contours, CV_RETR_EXTERNAL, CV_CHAIN_APPROX_NONE);
cv::drawContours (frame, contours, -1, cv::Scalar (0, 0, 255), 2);
cv::imshow ("Frame", frame);
cv::imshow ("Background", backgroundImage);
char k = (char)waitKey(30);
if( k == 27 ) break;
}
return 0;
}
As far as your query about splitting a video into frames is concerned, then you should know that a video is nothing but a collection of several frames. So, whenever you use VideoCapture capture(videoFilename); in openCV, you are capturing a single frame/image from your video.
It is very simple with the aid of google's opencv & javacv libraries.The logic is simply grabb the frames from the webcam continuesly using a thread and perform a substraction between the 2 consequetive frames. If there is any change between the 2 frames , it will produes a white pixel.Black pixel indicates no change between the frames[no motion detected].The complete implementation can be found at here
I am trying to use the goodFeatureToTrack() function with opencv 2.4.3 on an gray image of lena...however I always get a zero size of the vector storing the features as cv::Point2f...I have tried using a zero mask also but in that case the application hangs up..I tried playing with the quality level value from 0.01 to 0.001. However still the size of the vector is zero..any idea?...following is my code..
#include <iostream>
#include <opencv2/core/core.hpp>
#include <opencv2/features2d/features2d.hpp>
#include <opencv2/video/tracking.hpp>
#include <opencv2/highgui/highgui.hpp>
#include <vector>
using namespace cv;
using namespace std;
int main()
{
Mat frameROI;
frameROI = imread("C:\\lena.jpg");
std::vector<cv::Point2f> corners;
cvtColor(frameROI,frameROI,CV_RGB2GRAY);
//Mat mask(frameROI.size(), CV_8UC1);
//mask.setTo(Scalar::all(0));
//goodFeaturesToTrack(frameROI,corners,10,0.001,10,mask,3,false,0.04);
goodFeaturesToTrack(frameROI,corners,10,0.001,10);//AFTER EDIT
cout<<"SIZE OF FEATURE VECTOR = "<<corners.size()<<endl;
imshow("VIDEO ROI",frameROI);
waitKey();
return 0;
}
OUTPUT:
SIZE OF FEATURE VECTOR = 0
EDIT: afte Bob's suggestion I omitted the line for mask and modified the function..but now the application hangs up after the goodFeaturesToTrack function is invoked...Any idea?
By setting the mask to all zeros, you are basically excluding the whole image from the search. You should either remove mask.setTo(Scalar::all(0)); completely (thus leaving the matrix empty) or replace it with mask.setTo(Scalar::all(1)); (that is, to search for features in the whole image; otherwise, you should set the mask with 1's in the region of interest and 0's otherwise).
Following image is what your code returns for me if I remove mask.setTo(Scalar::all(0)); completely and draw the points:
Solved the problem just now....instead of using the pre build libs n dlls....build it with MSVC2008 and now its working fine...the same points indicated by Bob are detected..