opencv- vehicle tracking using optical flow - opencv

Question

I have implemented optical flow to track vehicles on road and it turned out to be very slow.
my code uses the functions:
cvGoodFeaturesToTrack
cvFindCornerSubPix
cvCalcOpticalFlowPyrLK
How do I make this tracking fast and efficient?
My code is:
#include "highgui.h"
#include "cv.h"
#include "cxcore.h"
#include <iostream>
using namespace std;
const int MAX_CORNERS = 500;
int main()
{
CvCapture* capture=cvCreateFileCapture("E:\cam1.avi");
IplImage* img_A;// = cvLoadImage("image0.png", CV_LOAD_IMAGE_GRAYSCALE);
IplImage* img_B;// = cvLoadImage("image1.png", CV_LOAD_IMAGE_GRAYSCALE);
img_A=cvQueryFrame(capture);
IplImage* imgA = cvCreateImage( cvGetSize(img_A), 8, 1 );
IplImage* imgB = cvCreateImage( cvGetSize(img_A), 8, 1 );
cvNamedWindow( "ImageA", CV_WINDOW_AUTOSIZE );
cvNamedWindow( "ImageB", CV_WINDOW_AUTOSIZE );
cvNamedWindow( "LKpyr_OpticalFlow", CV_WINDOW_AUTOSIZE );
while(1)
{
int couter=0;
for(int k=0;k<20;k++)
{
img_B=cvQueryFrame(capture);
}
//cvCvtColor(imgA,imgA,CV_BGR2GRAY);
//cvCvtColor(imgB,imgB,CV_BGR2GRAY);
// Load two images and allocate other structures
/*IplImage* imgA = cvLoadImage("image0.png", CV_LOAD_IMAGE_GRAYSCALE);
IplImage* imgB = cvLoadImage("image1.png", CV_LOAD_IMAGE_GRAYSCALE);*/
CvSize img_sz = cvGetSize( img_A );
int win_size = 10;
IplImage* imgC = cvCreateImage( cvGetSize(img_A), 8, 1 );
cvZero(imgC);
// Get the features for tracking
IplImage* eig_image = cvCreateImage( img_sz, IPL_DEPTH_32F, 1 );
IplImage* tmp_image = cvCreateImage( img_sz, IPL_DEPTH_32F, 1 );
int corner_count = MAX_CORNERS;
CvPoint2D32f* cornersA = new CvPoint2D32f[ MAX_CORNERS ];
cvCvtColor(img_A,imgA,CV_BGR2GRAY);
cvCvtColor(img_B,imgB,CV_BGR2GRAY);
cvGoodFeaturesToTrack( imgA, eig_image, tmp_image, cornersA, &corner_count ,0.05, 5.0, 0, 3, 0, 0.04 );
cvFindCornerSubPix( imgA, cornersA, corner_count, cvSize( win_size, win_size ) ,cvSize( -1, -1 ), cvTermCriteria( CV_TERMCRIT_ITER | CV_TERMCRIT_EPS, 20, 0.03 ) );
// Call Lucas Kanade algorithm
char features_found[ MAX_CORNERS ];
float feature_errors[ MAX_CORNERS ];
CvSize pyr_sz = cvSize( imgA->width+8, imgB->height/3 );
IplImage* pyrA = cvCreateImage( pyr_sz, IPL_DEPTH_32F, 1 );
IplImage* pyrB = cvCreateImage( pyr_sz, IPL_DEPTH_32F, 1 );
CvPoint2D32f* cornersB = new CvPoint2D32f[ MAX_CORNERS ];
/*int jk=0;
for(int i=0;i<imgA->width;i+=10)
{
for(int j=0;j<imgA->height;j+=10)
{
cornersA[jk].x=i;
cornersA[jk].y=j;
++jk;
}
}
*/
cvCalcOpticalFlowPyrLK( imgA, imgB, pyrA, pyrB, cornersA, cornersB, corner_count,
cvSize( win_size, win_size ), 5, features_found, feature_errors,
cvTermCriteria( CV_TERMCRIT_ITER | CV_TERMCRIT_EPS, 20, 0.3 ), 0 );
// Make an image of the results
for( int i=0; i < corner_count; i++ )
{
if( features_found[i]==0|| feature_errors[i]>550 )
{
//printf("Error is %f/n",feature_errors[i]);
continue;
}
//printf("Got it/n");
CvPoint p0 = cvPoint( cvRound( cornersA[i].x ), cvRound( cornersA[i].y ) );
CvPoint p1 = cvPoint( cvRound( cornersB[i].x ), cvRound( cornersB[i].y ) );
cvLine( imgC, p0, p1, CV_RGB(255,0,0), 2 );
cout<<p0.x<<" "<<p0.y<<endl;
}
cvShowImage( "LKpyr_OpticalFlow", imgC );
cvShowImage( "ImageA", imgA );
cvShowImage( "ImageB", imgB );
//cvCopyImage(imgB,imgA);
delete[] cornersA;
delete[] cornersB;
cvWaitKey(33);
}
return 0;
}

Answer 1

I might be going a bit over the line here but I would suggest you to check out OpenTLD. OpenTLD (aka Predator) is one of the most efficient tracking algorithm. Zdenek Kalal has implemented OpenTLD in MATLAB. George Nebehay has made a very efficient C++ OpenCV port of OpenTLD.
It's very easy to install and tracking is really efficient.
OpenTLD uses Median Flow Tracker to track and implements PN learning algorithm. In this YouTube Video, Zdenek Kalal shows the use of OpenTLD.
If you just want to implement a Median Flow Tracker, follow this link https://github.com/gnebehay/OpenTLD/tree/master/src/mftracker
If you want to use it in Python, I have made a Median Flow Tracker and also made a Python port of OpenTLD. But python port isn't much efficient.

Answer 2

First of all to track a car you have to somehow detect it (using color segmentation/background subtraction for example). When car is detected you have to track it (track some points on it) using cvCalcOpticalFlowPyrLK. I didn't find code that responces for car detection.
Take a look at this and this articles. Your idea should be the same.
Also your code is a bit wrong. For example why do you call cvGoodFeaturesToTrack in the main loop? You have to call it once - before loop to detect good features to track. But this will also detect non-cars.
Take a look at default OpenCV example: OpenCV/samples/cpp/lkdemo.cpp.