Related
I am working on an object detection code and I chose the edge orientation histogram as a descriptor for the matching.
I am facing a problem in the back projected histogram as i don't seem to have a good matching , the back projected image is mostly white, which means that i cannot use meanshift or so for detection of the object.
Please help me regarding this matter:
here is what i've done so far:
Take an initial ROI (the target needed to be detected in the video stream).
convert ROI to grayscale
apply sobel operator for both x, y derivatives.
calculate orientation using opencv phase function (from derivative x and derivative y)
make a histogram of the generated orientation. with the following specs:
(range : 0 to 2 PI) , (single channel) , (256 bins)
normalize the histogram
the code for doing these steps is the following:
Mat ROI_grad_x, ROI_grad_y , ROI_grad , ROI_gray;
Mat ROI_abs_grad_x, ROI_abs_grad_y;
cvtColor(ROI, ROI_gray, CV_BGR2GRAY);
/// Gradient X
Sobel( ROI_gray, ROI_grad_x, CV_16S, 1, 0, 3 );
/// Gradient Y
Sobel( ROI_gray, ROI_grad_y, CV_16S, 0, 1, 3 );
convertScaleAbs( ROI_grad_x, ROI_abs_grad_x );
convertScaleAbs( ROI_grad_y, ROI_abs_grad_y );
addWeighted( ROI_abs_grad_x, 0.5, ROI_abs_grad_y, 0.5, 0, ROI_grad );
Mat ROI_orientation = Mat::zeros(ROI_abs_grad_x.rows, ROI_abs_grad_y.cols, CV_32F); //to store the gradients
Mat ROI_orientation_norm ;
ROI_grad_x.convertTo(ROI_grad_x,CV_32F);
ROI_grad_y.convertTo(ROI_grad_y,CV_32F);
phase(ROI_grad_x, ROI_grad_y, ROI_orientation , false);
Mat ROI_orientation_hist ;
float ROI_orientation_range[] = {0 , CV_PI};
const float *ROI_orientation_histRange[] = {ROI_orientation_range};
int ROI_orientation_histSize =256;
//calcHist( &ROI_orientation, 1, 0, Mat(), ROI_orientation_hist, 1, &ROI_orientation_histSize, &ROI_orientation_histRange , true, false);
calcHist( &ROI_orientation, 1, 0, Mat(), ROI_orientation_hist, 1, &ROI_orientation_histSize, ROI_orientation_histRange , true, false);
normalize( ROI_orientation_hist, ROI_orientation_hist, 0, 255, NORM_MINMAX, -1, Mat() );
then , and for each camera frame captured , I do the following steps:
convert to grayscale
apply sobel operator for both x derivative and y derivative.
compute orientation using phase opencv function (using the 2 derivatives mentioned above)
back project the histogram onto the orientation frame matrix to get the matches.
the code used for this part is the following :
Mat grad_x, grad_y , grad;
Mat abs_grad_x, abs_grad_y;
/// Gradient X
Sobel( frame_gray, grad_x, CV_16S, 1, 0, 3 );
/// Gradient Y
Sobel( frame_gray, grad_y, CV_16S, 0, 1, 3 );
convertScaleAbs( grad_x, abs_grad_x );
convertScaleAbs( grad_y, abs_grad_y );
addWeighted( abs_grad_x, 0.5, abs_grad_y, 0.5, 0, grad );
///======================
Mat orientation = Mat::zeros(abs_grad_x.rows, abs_grad_y.cols, CV_32F); //to store the gradients
Mat orientation_norm ;
grad_x.convertTo(grad_x,CV_32F);
grad_y.convertTo(grad_y,CV_32F);
phase(grad_x, grad_y, orientation , false);
Mat EOH_backProj ;
calcBackProject( &orientation, 1, 0, ROI_orientation_hist, EOH_backProj, ROI_orientation_histRange, 1, true );
So , what seems to be the problem in my approach ?!
Thanks alot.
I'm currently trying to rectify stereo cameras to create a disparity map. Unfortunately, I'm having trouble getting past the stereo rectification step because I keep receiving the error
"OpenCV Error: Bad argument in unknown function, file ..\..\..\modules\core\src\matrix.cpp, line 697."
The process is complicated by the fact that I'm not the one one who calibrated the cameras, nor do I have access to the cameras used to record the videos. I was given all of the calibration parameters (intrinsics, distortion coefficients, rotation matrix, and translation vector). As you can see, I've tried to turn these directly into CvMats and use them that way, but I get an error when I try to actually use them.
Thanks in advance.
CvMat li, lm, ri, rm, r, t, Rl, Rr, Pl, Pr;
double init_li[3][3] =
{ {477.984984743, 0, 316.17458671},
{0, 476.861945645, 253.45073026},
{0, 0 ,1} };
double init_lm[5] = {-0.117798518453, 0.147554949385, -0.0549082041898, 0, 0};
double init_ri[3][3] =
{{478.640315323, 0, 299.957994781},
{0, 477.898896505, 251.665771947},
{0, 0, 1}};
double init_rm[5] = {-0.10884732532, 0.12118405303, -0.0322073237741, 0, 0};
double init_r[3][3] =
{{0.999973709051976, 0.00129700728791757, -0.00713435189275776},
{-0.00132096594266573, 0.999993501087837, -0.00335452397041856},
{0.00712995468519435, 0.00336386001267643, 0.99996892361313}};
double init_t[3] = {-0.0830973040641153, -0.00062704210860633, 1.4287643345188e-005};
cvInitMatHeader(&li, 3, 3, CV_64FC1, init_li);
cvInitMatHeader(&lm, 5, 1, CV_64FC1, init_lm);
cvInitMatHeader(&ri, 3, 3, CV_64FC1, init_ri);
cvInitMatHeader(&rm, 5, 1, CV_64FC1, init_rm);
cvInitMatHeader(&r, 3, 3, CV_64FC1, init_r);
cvInitMatHeader(&t, 3, 1, CV_64FC1, init_t);
cvInitMatHeader(&Rl, 3,3, CV_64FC1);
cvInitMatHeader(&Rr, 3,3, CV_64FC1);
cvInitMatHeader(&Pl, 3,4, CV_64FC1);
cvInitMatHeader(&Pr, 3,4, CV_64FC1);
//frame is a cv::MAT holding the first frame of the video.
CvSize imageSize = frame.size();
imageSize.width /= 2;
//IT BREAKS HERE
cvStereoRectify(&li, &ri, &lm, &rm, imageSize, &r, &t, &Rl, &Rr, &Pl, &Pr);
so, you've been bitten by the c-api ? why don't you just turn your back on it ?
use the c++ api whenever possible, don't start learning opencv with the old(1.0), deprecated api, please !
double init_li[9] =
{ 477.984984743, 0, 316.17458671,
0, 476.861945645, 253.45073026,
0, 0 ,1 };
double init_lm[5] = {-0.117798518453, 0.147554949385, -0.0549082041898, 0, 0};
double init_ri[9] =
{ 478.640315323, 0, 299.957994781,
0, 477.898896505, 251.665771947,
0, 0, 1};
double init_rm[5] = {-0.10884732532, 0.12118405303, -0.0322073237741, 0, 0};
double init_r[9] =
{ 0.999973709051976, 0.00129700728791757, -0.00713435189275776,
-0.00132096594266573, 0.999993501087837, -0.00335452397041856,
0.00712995468519435, 0.00336386001267643, 0.99996892361313};
double init_t[3] = {-0.0830973040641153, -0.00062704210860633, 1.4287643345188e-005};
cv::Mat li(3, 3, CV_64FC1, init_li);
cv::Mat lm(5, 1, CV_64FC1, init_lm);
cv::Mat ri(3, 3, CV_64FC1, init_ri);
cv::Mat rm(5, 1, CV_64FC1, init_rm);
cv::Mat r, t, Rl, Rr, Pl, Pr; // note: no initialization needed.
//frame is a cv::MAT holding the first frame of the video.
cv::Size imageSize = frame.size();
imageSize.width /= 2;
//IT won't break HERE
cv::stereoRectify(li, ri, lm, rm, imageSize, r, t, Rl, Rr, Pl, Pr);
// no need ever to release or care about anything
Ok, so I figured out the answer. The problem was that I had only initialized headers for Rl, Rr, Pl, and Pr, but no memory was allocated for the data itself. I was able to fix it as follows:
double init_Rl[3][3];
double init_Rr[3][3];
double init_Pl[3][4];
double init_Pr[3][4];
cvInitMatHeader(&Rl, 3,3, CV_64FC1, init_Rl);
cvInitMatHeader(&Rr, 3,3, CV_64FC1, init_Rr);
cvInitMatHeader(&Pl, 3,4, CV_64FC1, init_Pl);
cvInitMatHeader(&Pr, 3,4, CV_64FC1, init_Pr);
Although, I have a theory that I might have been able to use cv::stereoRectify with cv::Mats as parameters, which would have made life much easier. I don't know if cv::stereoRectify exists, but it seems that versions of many of the other c functions are in the cv namespace. In case it's hard to tell, I'm very new to OpenCV.
I'm trying to get an implementation of the Kalman Filter with on OpenCV, so I've followed the following example of tracking of rotating point (OpenCV 2.4.3):
http://fossies.org/unix/misc/OpenCV-2.4.3.tar.gz:a/OpenCV-2.4.3/samples/cpp/kalman.cpp
I am using OpenCV 2.4.2, and QtCreator 4.7.4. All compiles fine, but when I execute the program crashes at the end of this code:
int main(int, char**)
{
help();
Mat img(500, 500, CV_8UC3);
KalmanFilter KF(2, 1, 0);
Mat state(2, 1, CV_32F); /* (phi, delta_phi) */
Mat processNoise(2, 1, CV_32F);
Mat measurement = Mat::zeros(1, 1, CV_32F);
char code = (char)-1;
for(;;)
{
randn( state, Scalar::all(0), Scalar::all(0.1) );
KF.transitionMatrix = *(Mat_<float>(2, 2) << 1, 1, 0, 1);
setIdentity(KF.measurementMatrix);
setIdentity(KF.processNoiseCov, Scalar::all(1e-5));
setIdentity(KF.measurementNoiseCov, Scalar::all(1e-1));
setIdentity(KF.errorCovPost, Scalar::all(1));
randn(KF.statePost, Scalar::all(0), Scalar::all(0.1));
for(;;)
{
Point2f center(img.cols*0.5f, img.rows*0.5f);
float R = img.cols/3.f;
double stateAngle = state.at<float>(0);
Point statePt = calcPoint(center, R, stateAngle);
Mat prediction = KF.predict(); //it crashes here
...
I also tried this tutorial (OpenCV 2.2), and it also crashed at the same point: the .predict() function:
http://www.morethantechnical.com/2011/06/17/simple-kalman-filter-for-tracking-using-opencv-2-2-w-code/comment-page-1/#comment-34618
QtCreator doesn't show me the error code, so I can't figure out what's going on here. I've checked other posts and they say that the problem may be that the matrixes aren't of the same size, but I doubt that since this example works for other people who tried it...
Anyone could give me a hint on how to solve this?
Thank you very much in advance!!
Im trying to create my own sobel edge detection based off of the gx and gy matrices on three channels i have in my code below.
[[0,1,2],
[-1,0,1],
[-2,-1,0]]
and
[-2,-1,0],
[-1,0,1],
[0,1,2]]
I edited the variables j and i in my code further down but it is not working, how can i create a sobel edge detection on those three channels
#include <opencv2/core/core.hpp>
#include <opencv2/highgui/highgui.hpp>
void salt(cv::Mat &image, int n) {
int i,j;
for (int k=0; k<n; k++) {
// rand() is the MFC random number generator
i= rand()%image.cols;
j= rand()%image.rows;
if (image.channels() == 1) { // gray-level image
image.at<uchar>(j,i)= 255;
} else if (image.channels() == 3) { // color image
image.at<cv::Vec3b>(j,i)[0]= 255;
image.at<cv::Vec3b>(j-1,i-1)[1]= 255;
image.at<cv::Vec3b>(j,i-1)[2]= 255;
}
}
}
int main()
{
srand(cv::getTickCount()); // init random number generator
cv::Mat image= cv::imread("space.jpg",0);
salt(image,3000);
cv::namedWindow("Image");
cv::imshow("Image",image);
cv::imwrite("salted.bmp",image);
cv::waitKey(5000);
return 0;
}
I'm a little confused by the question, because the question relates to sobel filters, but you provided a function that adds noise to an image.
To start with, here is the Sobel function, which will call the classic sobel functions (that will calculate dx and dy gradients).
Secondly, there is the more generic filter2D which will let you apply an arbitrary kernel (like the one you created in the question).
Lastly, if you want to apply a different kernel in each channel or band, you can do as the filter2D documentation implies, and call split on an image, and then call filter2D on each channel, and then combine the values into a single band image using the matrix operators.
The most complicated thing I think you could be asking is how to find the locations of that salt you added to the image, and the answer would be to make a kernel for each band like so:
band 0:
[[ 0, 0, 0],
[ 0, 1, 0],
[ 0, 0, 0]]
band 1:
[[ 1, 0, 0],
[ 0, 0, 0],
[ 0, 0, 0]]
band 2:
[[ 0, 1, 0],
[ 0, 0, 0],
[ 0, 0, 0]]
Be sure to put the anchor in the center of the kernel (1,1).
I'm trying to achieve background subtraction in openCV 2.2 using the cv namespace (Qt4.7). I have the following code which compiles fine but when running the program breaks because one mat doesn't equal the other but I can't find out where it is and I'm currently going through the API reference to try and find it.
cvtColor( mcolImage, mcolImage, CV_BGR2RGB);
cvtColor( mcolImage, gscaleImage, CV_RGB2GRAY);
acc = Mat(Size(440,320), CV_32FC3);
accSQ = Mat(Size(440,320), CV_32FC3);
//we accumulate into a Mat to get an frames average
Mat avg;
accumulateWeighted(gscaleImage, acc, 3.0, Mat());
accumulateSquare(gscaleImage, accSQ, Mat());
multiply(acc, acc, avg, 1);
Mat sigma, sigmaSQRT;
subtract(accSQ, avg, sigmaSQRT, Mat());
sqrt(sigmaSQRT, sigma); //Holds the standard deviation
Mat fgImage; //hold the foreground image
cv::absdiff(avg,gscaleImage, fgImage);
//GaussianBlur(gscaleImage, gscaleImage, Size(7,7), 2, 2 );
Mat buff ;
//convert to black and white
threshold(fgImage, buff, 75, THRESH_BINARY, 100);
dilate(buff, buff, Mat(3, 3, CV_8UC1), Point(-1, -1), 1, BORDER_CONSTANT, Scalar(1.0, 1.0, 1.0, 0));
erode(buff, buff, Mat(3, 3, CV_8UC1), Point(-1, -1), 1, BORDER_CONSTANT, Scalar(1.0, 1.0, 1.0, 0));
//rectangle(gscaleImage, cvPoint(100, 300), cvPoint(200, 100), cvScalar(255, 255, 255, 0), 1);
QImage colImagetmp((uchar*)mcolImage.data, mcolImage.cols, mcolImage.rows, mcolImage.step,
QImage::Format_RGB888 ); //Colour
QImage gscaleImagetmp ((uchar*)gscaleImage.data, gscaleImage.cols, gscaleImage.rows, gscaleImage.step,
QImage::Format_Indexed8); //Greyscale. I hope
QImage bwImagetmp((uchar*)buff.data, buff.cols, buff.rows, buff.step,
QImage::Format_Indexed8);
//Setup a colour table for the greyscale image
QVector<QRgb> colorTable;
for (int i = 0; i < 256; i++) colorTable.push_back(qRgb(i, i, i));
bwImagetmp.setColorTable(colorTable);
gscaleImagetmp.setColorTable(colorTable);
ui.intDisplay->setPixmap(QPixmap::fromImage(bwImagetmp));
ui.bwDisplay->setPixmap(QPixmap::fromImage(gscaleImagetmp));
ui.colDisplay->setPixmap( QPixmap::fromImage(colImagetmp ));
Thanks for the help in advanced.
Edit:
After going through the code I found that the absdiff(avg, gscaleImage, fgImage); is where the program is crashing. I think it maybe crashing on the second parameter but not sure.
I solved it (I think) by declaring a new temporary Mat and converting that specifically (using avg.convert() ) to match the gscaleImage type and size.