I'm trying to achieve background subtraction in openCV 2.2 using the cv namespace (Qt4.7). I have the following code which compiles fine but when running the program breaks because one mat doesn't equal the other but I can't find out where it is and I'm currently going through the API reference to try and find it.
cvtColor( mcolImage, mcolImage, CV_BGR2RGB);
cvtColor( mcolImage, gscaleImage, CV_RGB2GRAY);
acc = Mat(Size(440,320), CV_32FC3);
accSQ = Mat(Size(440,320), CV_32FC3);
//we accumulate into a Mat to get an frames average
Mat avg;
accumulateWeighted(gscaleImage, acc, 3.0, Mat());
accumulateSquare(gscaleImage, accSQ, Mat());
multiply(acc, acc, avg, 1);
Mat sigma, sigmaSQRT;
subtract(accSQ, avg, sigmaSQRT, Mat());
sqrt(sigmaSQRT, sigma); //Holds the standard deviation
Mat fgImage; //hold the foreground image
cv::absdiff(avg,gscaleImage, fgImage);
//GaussianBlur(gscaleImage, gscaleImage, Size(7,7), 2, 2 );
Mat buff ;
//convert to black and white
threshold(fgImage, buff, 75, THRESH_BINARY, 100);
dilate(buff, buff, Mat(3, 3, CV_8UC1), Point(-1, -1), 1, BORDER_CONSTANT, Scalar(1.0, 1.0, 1.0, 0));
erode(buff, buff, Mat(3, 3, CV_8UC1), Point(-1, -1), 1, BORDER_CONSTANT, Scalar(1.0, 1.0, 1.0, 0));
//rectangle(gscaleImage, cvPoint(100, 300), cvPoint(200, 100), cvScalar(255, 255, 255, 0), 1);
QImage colImagetmp((uchar*)mcolImage.data, mcolImage.cols, mcolImage.rows, mcolImage.step,
QImage::Format_RGB888 ); //Colour
QImage gscaleImagetmp ((uchar*)gscaleImage.data, gscaleImage.cols, gscaleImage.rows, gscaleImage.step,
QImage::Format_Indexed8); //Greyscale. I hope
QImage bwImagetmp((uchar*)buff.data, buff.cols, buff.rows, buff.step,
QImage::Format_Indexed8);
//Setup a colour table for the greyscale image
QVector<QRgb> colorTable;
for (int i = 0; i < 256; i++) colorTable.push_back(qRgb(i, i, i));
bwImagetmp.setColorTable(colorTable);
gscaleImagetmp.setColorTable(colorTable);
ui.intDisplay->setPixmap(QPixmap::fromImage(bwImagetmp));
ui.bwDisplay->setPixmap(QPixmap::fromImage(gscaleImagetmp));
ui.colDisplay->setPixmap( QPixmap::fromImage(colImagetmp ));
Thanks for the help in advanced.
Edit:
After going through the code I found that the absdiff(avg, gscaleImage, fgImage); is where the program is crashing. I think it maybe crashing on the second parameter but not sure.
I solved it (I think) by declaring a new temporary Mat and converting that specifically (using avg.convert() ) to match the gscaleImage type and size.
Related
I work with the Segmentation of Rocks and have been using the Image Segmentation with Watershed Algorithm from Opencv(https://opencv-python-tutroals.readthedocs.io/en/latest/py_tutorials/py_imgproc/py_watershed/py_watershed.html).
As some background on my input, I get a depth Image as Input image and need to perform segmentation on it.
I have tried to change the kernels for the Laplacian.
I have made a few tweaks to the code. cv::adaptivethreshhold seems to show better than cv::threshold in my case.
Even changing the kernels doesn't improve the result. Hence, I have left the kernels unchanged.
I also referred to questions relating to my probelm :
-> Image Sharpening Using Laplacian Filter
-> How to perform segmentation on complex images like rocks (in c++)
const cv::Mat image = rrlib::coviroa::AccessImageAsMat(*in_depth_img);
cv::Mat depth_img_opencv;
cv::extractChannel(image, depth_img_opencv,0);
cv::Mat dst;
cv::resize(depth_img_opencv,dst, cv::Size(image.cols*3,image.rows*3));
cv::imshow("Depth Image", dst);
cv::waitKey(200);
cv::Mat src;
cv::cvtColor(depth_img_opencv, src,cv::COLOR_GRAY2BGR);
cv:: Mat dst1;
cv::resize(src, dst1, cv::Size(image.cols*3,image.rows*3));
cv::imshow("Color Image", dst1);
cv::waitKey(200);
cv::Mat kernel = (cv::Mat_<float>(3,3) <<
1, 1, 1,
1, -8, 1,
1, 1, 1);
cv::Mat imgLaplacian;
cv::Mat sharp = src;
cv::filter2D(sharp, imgLaplacian, CV_32F, kernel);
src.convertTo(sharp, CV_32F);
cv::Mat imgResult = sharp - imgLaplacian;
imgResult.convertTo(imgResult, CV_8UC3);
src = imgResult;
cv:: Mat dst2;
cv::resize(src, dst2, cv::Size(image.cols*3,image.rows*3));
cv::imshow("Sharpened Image", dst2);
cv::waitKey(200);
cv::Mat bw;
cv::cvtColor(src, bw, CV_BGR2GRAY);
cv::imshow("BGR2GRAY", bw);
cv::waitKey(200);
cv::adaptiveThreshold(bw, bw, 255, CV_ADAPTIVE_THRESH_GAUSSIAN_C , CV_THRESH_BINARY, 7, 0);
cv:: Mat dst3;
cv::resize(bw, dst3, cv::Size(image.cols*3,image.rows*3));
cv::imshow("Binary Image", dst3);
cv::waitKey(200);
cv::Mat dist;
cv::distanceTransform(bw, dist, CV_DIST_L2, 3);
cv::normalize(dist, dist, 0, 1., cv::NORM_MINMAX);
cv:: Mat dst4;
cv::resize(dist, dst4, cv::Size(image.cols*3,image.rows*3));
cv::imshow("Distance Transform", dst4);
cv::waitKey(200);
// Threshold to obtain the peaks
dist.convertTo(dist, CV_8UC1);
cv::adaptiveThreshold(dist, dist, 255, CV_ADAPTIVE_THRESH_GAUSSIAN_C, CV_THRESH_BINARY, 7, 0);
// Dilate a bit the dist image
cv::Mat kernel1 = cv::Mat::ones(3, 3, CV_8UC1);
// Thickens obects in an image
cv::dilate(dist, dist, kernel1);
cv:: Mat dst5;
cv::resize(dist, dst5, cv::Size(image.cols*3,image.rows*3));
cv::imshow("Peaks", dst5);
cv::waitKey(200);
Any valuable suggestion is appreciated to get a better segmentation of the result.
I am trying to detect the biggest/larger rectangular shape and draw bounding box to the detected area.
In my use case, very often (and not always) the object that represent the rectangle shape is in color white and the background is also in color very similar to white.
Before detecting contours, I have preprocessed the image in order to detect perfect edge.
My problem is that I can't detect edges perfectly and i have a lot of noise even after blurring and using 'adaptive threshold' or 'threshold'.
The original image i have used for contours detection
I have tried different way to detect perfect edge in different lighting condition without success.
How can I process image in order to detect perfect edge (edges with no holes) for contour detection ?
Below is the code i am using
public static Mat findRectangleX(Mat original) {
Mat src = original.clone();
Mat gray = new Mat();
Mat binary = new Mat();
MatOfPoint2f approxCurve;
List<MatOfPoint> contours = new ArrayList<MatOfPoint>();
if (original.type() != CvType.CV_8U) {
Imgproc.cvtColor(original, gray, Imgproc.COLOR_BGR2GRAY);
} else {
original.copyTo(gray);
}
Imgproc.GaussianBlur(gray, gray, new Size(5,5),0);
Imgproc.adaptiveThreshold(gray, binary, 255,Imgproc.ADAPTIVE_THRESH_GAUSSIAN_C,Imgproc.THRESH_BINARY_INV,11, 1);
//Imgproc.threshold(gray, binary,0,255,Imgproc.THRESH_BINARY_INV | Imgproc.THRESH_OTSU);
double maxArea = 0;
Imgproc.findContours(binary, contours, new Mat(),Imgproc.RETR_LIST, Imgproc.CHAIN_APPROX_SIMPLE);
for (int i = 0; i<contours.size();i++) {
MatOfPoint contour = contours.get(i);
MatOfPoint2f temp = new MatOfPoint2f(contour.toArray());
double area = Imgproc.contourArea(contour);
approxCurve = new MatOfPoint2f();
Imgproc.approxPolyDP(temp, approxCurve, Imgproc.arcLength(temp, true) * 0.03, true);
if (approxCurve.total() == 4 ) {
Rect rect = Imgproc.boundingRect(contours.get(i));
Imgproc.rectangle(src, rect.tl(), rect.br(), new Scalar(255, 0, 0, .8), 4);
if(maxArea < area)
maxArea = area;
}
}
Log.v(TAG, "Total contours found : " + contours.size());
Log.v(TAG, "Max area :" + maxArea);
return src;
}
I've search similar problems on stackoverflow and try code sample but any of them worked for me. The difficulty i think is the white objet on white background.
How can I process image in order to sharpen the edges for contour detection ?
How can I detect the biggest/larger rectangle shape and draw rectangle line to the detected shape ?
//Updated at : 20/02/2017
i have tried the solution suggested by #Nejc in the post below. The segmentation is better but i still have holes in contour and findcontours fails in detecting the larger contour.
Below is the code provided by #Nejc and translated to java.
public static Mat process(Mat original){
Mat src = original.clone();
Mat hsvMat = new Mat();
Mat saturation = new Mat();
Mat sobx = new Mat();
Mat soby = new Mat();
Mat grad_abs_val_approx = new Mat();
Imgproc.cvtColor(src, hsvMat, Imgproc.COLOR_BGR2HSV);
List<Mat> hsv_channels = new ArrayList<Mat>(3);
Core.split(hsvMat, hsv_channels);
Mat hue = hsv_channels.get( 0 );
Mat sat = hsv_channels.get( 1 );
Mat val = hsv_channels.get( 2 );
Imgproc.GaussianBlur(sat, saturation, new Size(9, 9), 2, 2);
Mat imf = new Mat();
saturation.convertTo(imf, CV_32FC1, 0.5f, 0.5f);
Imgproc.Sobel(imf, sobx, -1, 1, 0);
Imgproc.Sobel(imf, soby, -1, 0, 1);
sobx = sobx.mul(sobx);
soby = soby.mul(soby);
Mat abs_x = new Mat();
Core.convertScaleAbs(sobx,abs_x);
Mat abs_y = new Mat();
Core.convertScaleAbs(soby,abs_y);
Core.addWeighted(abs_x, 1, abs_y, 1, 0, grad_abs_val_approx);
sobx.release();
soby.release();
Mat filtered = new Mat();
Imgproc.GaussianBlur(grad_abs_val_approx, filtered, new Size(9, 9), 2, 2);
final MatOfDouble mean = new MatOfDouble();
final MatOfDouble stdev = new MatOfDouble();
Core.meanStdDev(filtered, mean, stdev);
Mat thresholded = new Mat();
Imgproc.threshold(filtered, thresholded, mean.toArray()[0] + stdev.toArray()[0], 1.0, Imgproc.THRESH_TOZERO);
/*
Mat thresholded_bin = new Mat();
Imgproc.threshold(filtered, thresholded_bin, mean.toArray()[0] + stdev.toArray()[0], 1.0, Imgproc.THRESH_BINARY);
Mat converted = new Mat();
thresholded_bin.convertTo(converted, CV_8UC1);
*/
return thresholded;
}
Here is the image that i have got after running the code above
Image after using #Nejc solution
1) Why my translated code does not output the same image like #Nejc ?
The same code applied to same image should produce the same output ?
2) did i miss something when translating ?
3) For my understanding, why did we multiply the image by itself in this instruction sobx = sobx.mul(sobx); ?
I managed to obtain a pretty nice image of the edge by computing an approximation of the absolute value of gradient of the input image.
EDIT: Before I started working, I resized the input image to 5x smaller size. Click here to see it!. If you use my code on that image, the results will be good. If you want to make my code work well with the image of the original size, then either:
multiply Gaussian kernel sizes and sigmas by 5, or
downsample the image by factor 5, execute the algorithm and then upsample the result by factor 5 (this should work much faster than the first option)
This is the result I got:
My procedure relies on two key features. The first is a conversion to appropriate color space. As Jeru Luke stated in his answer , the saturation channel in HSV color space is the good choice here. The second important thing is the estimation of absolute value of gradient. I used sobel operators and some arithmetics for that purpose. I can provide additional explanations if someone requests them.
This is the code I used to obtain the first image.
using namespace std;
using namespace cv;
Mat img_rgb = imread("letter.jpg");
Mat img_hsv;
cvtColor(img_rgb, img_hsv, CV_BGR2HSV);
vector<Mat> channels_hsv;
split(img_hsv, channels_hsv);
Mat channel_s = channels_hsv[1];
GaussianBlur(channel_s, channel_s, Size(9, 9), 2, 2);
Mat imf;
channel_s.convertTo(imf, CV_32FC1, 0.5f, 0.5f);
Mat sobx, soby;
Sobel(imf, sobx, -1, 1, 0);
Sobel(imf, soby, -1, 0, 1);
sobx = sobx.mul(sobx);
soby = soby.mul(soby);
Mat grad_abs_val_approx;
cv::pow(sobx + soby, 0.5, grad_abs_val_approx);
Mat filtered;
GaussianBlur(grad_abs_val_approx, filtered, Size(9, 9), 2, 2);
Scalar mean, stdev;
meanStdDev(filtered, mean, stdev);
Mat thresholded;
cv::threshold(filtered, thresholded, mean.val[0] + stdev.val[0], 1.0, CV_THRESH_TOZERO);
// I scale the image at this point so that it is displayed properly
imshow("image", thresholded/50);
And this is how I computed the second image:
Mat thresholded_bin;
cv::threshold(filtered, thresholded_bin, mean.val[0] + stdev.val[0], 1.0, CV_THRESH_BINARY);
Mat converted;
thresholded_bin.convertTo(converted, CV_8UC1);
vector<vector<Point>> contours;
findContours(converted, contours, CV_RETR_LIST, CV_CHAIN_APPROX_NONE);
Mat contour_img = Mat::zeros(converted.size(), CV_8UC1);
drawContours(contour_img, contours, -1, 255);
imshow("contours", contour_img);
Thanks for yours comments and suggestion.
The code provided by #NEJC works perfectly and cover 80% of my use case.
Nevertheless, it does not works with similar case like this
case not solved by the current code
and i don't know why.
Perhaps someone have an idea/clue/solution ?
I continue to improve the code and try to find a more generic solution that can cover more case. I will post it if i ever i find.
In any case, below is the working code based on #NEJC solution and notes.
public static Mat process(Mat original){
Mat src = original.clone();
Mat hsvMat = new Mat();
Mat saturation = new Mat();
Mat sobx = new Mat();
Mat soby = new Mat();
Mat grad_abs_val_approx = new Mat();
Imgproc.cvtColor(src, hsvMat, Imgproc.COLOR_BGR2HSV);
List<Mat> hsv_channels = new ArrayList<Mat>(3);
Core.split(hsvMat, hsv_channels);
Mat hue = hsv_channels.get( 0 );
Mat sat = hsv_channels.get( 1 );
Mat val = hsv_channels.get( 2 );
Imgproc.GaussianBlur(sat, saturation, new Size(9, 9), 2, 2);
Mat imf = new Mat();
saturation.convertTo(imf, CV_32FC1, 0.5f, 0.5f);
Imgproc.Sobel(imf, sobx, -1, 1, 0);
Imgproc.Sobel(imf, soby, -1, 0, 1);
sobx = sobx.mul(sobx);
soby = soby.mul(soby);
Mat sumxy = new Mat();
Core.add(sobx,soby, sumxy);
Core.pow(sumxy, 0.5, grad_abs_val_approx);
sobx.release();
soby.release();
sumxy.release();;
Mat filtered = new Mat();
Imgproc.GaussianBlur(grad_abs_val_approx, filtered, new Size(9, 9), 2, 2);
final MatOfDouble mean = new MatOfDouble();
final MatOfDouble stdev = new MatOfDouble();
Core.meanStdDev(filtered, mean, stdev);
Mat thresholded = new Mat();
Imgproc.threshold(filtered, thresholded, mean.toArray()[0] + stdev.toArray()[0], 1.0, Imgproc.THRESH_TOZERO);
/*
Mat thresholded_bin = new Mat();
Imgproc.threshold(filtered, thresholded_bin, mean.toArray()[0] + stdev.toArray()[0], 1.0, Imgproc.THRESH_BINARY_INV);
Mat converted = new Mat();
thresholded_bin.convertTo(converted, CV_8UC1);
*/
Mat converted = new Mat();
thresholded.convertTo(converted, CV_8UC1);
return converted;
}
I am working on an object detection code and I chose the edge orientation histogram as a descriptor for the matching.
I am facing a problem in the back projected histogram as i don't seem to have a good matching , the back projected image is mostly white, which means that i cannot use meanshift or so for detection of the object.
Please help me regarding this matter:
here is what i've done so far:
Take an initial ROI (the target needed to be detected in the video stream).
convert ROI to grayscale
apply sobel operator for both x, y derivatives.
calculate orientation using opencv phase function (from derivative x and derivative y)
make a histogram of the generated orientation. with the following specs:
(range : 0 to 2 PI) , (single channel) , (256 bins)
normalize the histogram
the code for doing these steps is the following:
Mat ROI_grad_x, ROI_grad_y , ROI_grad , ROI_gray;
Mat ROI_abs_grad_x, ROI_abs_grad_y;
cvtColor(ROI, ROI_gray, CV_BGR2GRAY);
/// Gradient X
Sobel( ROI_gray, ROI_grad_x, CV_16S, 1, 0, 3 );
/// Gradient Y
Sobel( ROI_gray, ROI_grad_y, CV_16S, 0, 1, 3 );
convertScaleAbs( ROI_grad_x, ROI_abs_grad_x );
convertScaleAbs( ROI_grad_y, ROI_abs_grad_y );
addWeighted( ROI_abs_grad_x, 0.5, ROI_abs_grad_y, 0.5, 0, ROI_grad );
Mat ROI_orientation = Mat::zeros(ROI_abs_grad_x.rows, ROI_abs_grad_y.cols, CV_32F); //to store the gradients
Mat ROI_orientation_norm ;
ROI_grad_x.convertTo(ROI_grad_x,CV_32F);
ROI_grad_y.convertTo(ROI_grad_y,CV_32F);
phase(ROI_grad_x, ROI_grad_y, ROI_orientation , false);
Mat ROI_orientation_hist ;
float ROI_orientation_range[] = {0 , CV_PI};
const float *ROI_orientation_histRange[] = {ROI_orientation_range};
int ROI_orientation_histSize =256;
//calcHist( &ROI_orientation, 1, 0, Mat(), ROI_orientation_hist, 1, &ROI_orientation_histSize, &ROI_orientation_histRange , true, false);
calcHist( &ROI_orientation, 1, 0, Mat(), ROI_orientation_hist, 1, &ROI_orientation_histSize, ROI_orientation_histRange , true, false);
normalize( ROI_orientation_hist, ROI_orientation_hist, 0, 255, NORM_MINMAX, -1, Mat() );
then , and for each camera frame captured , I do the following steps:
convert to grayscale
apply sobel operator for both x derivative and y derivative.
compute orientation using phase opencv function (using the 2 derivatives mentioned above)
back project the histogram onto the orientation frame matrix to get the matches.
the code used for this part is the following :
Mat grad_x, grad_y , grad;
Mat abs_grad_x, abs_grad_y;
/// Gradient X
Sobel( frame_gray, grad_x, CV_16S, 1, 0, 3 );
/// Gradient Y
Sobel( frame_gray, grad_y, CV_16S, 0, 1, 3 );
convertScaleAbs( grad_x, abs_grad_x );
convertScaleAbs( grad_y, abs_grad_y );
addWeighted( abs_grad_x, 0.5, abs_grad_y, 0.5, 0, grad );
///======================
Mat orientation = Mat::zeros(abs_grad_x.rows, abs_grad_y.cols, CV_32F); //to store the gradients
Mat orientation_norm ;
grad_x.convertTo(grad_x,CV_32F);
grad_y.convertTo(grad_y,CV_32F);
phase(grad_x, grad_y, orientation , false);
Mat EOH_backProj ;
calcBackProject( &orientation, 1, 0, ROI_orientation_hist, EOH_backProj, ROI_orientation_histRange, 1, true );
So , what seems to be the problem in my approach ?!
Thanks alot.
I want to apply the Tenengrad algorithm to a central rectangular region inside the image. Assuming that I have the coordinates of the vertices of the rectangular region or maybe one corner and the dimensions, how can I modify the following code to apply the sharpness measure over the selected region ?
double tenengrad(const cv::Mat& src, int ksize)
{
cv::Mat Gx, Gy;
cv::Sobel(src, Gx, CV_64F, 1, 0, ksize);
cv::Sobel(src, Gy, CV_64F, 0, 1, ksize);
cv::Mat FM = Gx.mul(Gx) + Gy.mul(Gy);
double focusMeasure = cv::mean(FM).val[0];
return focusMeasure;
}
cv::Mat imageRegion;
imageRegion = src(cv::Rect(x, y, width, height));
creates a matrix that points to the region of the original image specified by the rectangle (x, y, width, height). Modifying imageRegion will modify the original image src. So you can use imageRegion instead of src
cv::Mat Gx, Gy;
cv::Sobel(imageRegion, Gx, CV_64F, 1, 0, ksize);
cv::Sobel(imageRegion, Gy, CV_64F, 0, 1, ksize);
cv::Mat FM = Gx.mul(Gx) + Gy.mul(Gy);
double focusMeasure = cv::mean(FM).val[0];
I'm currently trying to rectify stereo cameras to create a disparity map. Unfortunately, I'm having trouble getting past the stereo rectification step because I keep receiving the error
"OpenCV Error: Bad argument in unknown function, file ..\..\..\modules\core\src\matrix.cpp, line 697."
The process is complicated by the fact that I'm not the one one who calibrated the cameras, nor do I have access to the cameras used to record the videos. I was given all of the calibration parameters (intrinsics, distortion coefficients, rotation matrix, and translation vector). As you can see, I've tried to turn these directly into CvMats and use them that way, but I get an error when I try to actually use them.
Thanks in advance.
CvMat li, lm, ri, rm, r, t, Rl, Rr, Pl, Pr;
double init_li[3][3] =
{ {477.984984743, 0, 316.17458671},
{0, 476.861945645, 253.45073026},
{0, 0 ,1} };
double init_lm[5] = {-0.117798518453, 0.147554949385, -0.0549082041898, 0, 0};
double init_ri[3][3] =
{{478.640315323, 0, 299.957994781},
{0, 477.898896505, 251.665771947},
{0, 0, 1}};
double init_rm[5] = {-0.10884732532, 0.12118405303, -0.0322073237741, 0, 0};
double init_r[3][3] =
{{0.999973709051976, 0.00129700728791757, -0.00713435189275776},
{-0.00132096594266573, 0.999993501087837, -0.00335452397041856},
{0.00712995468519435, 0.00336386001267643, 0.99996892361313}};
double init_t[3] = {-0.0830973040641153, -0.00062704210860633, 1.4287643345188e-005};
cvInitMatHeader(&li, 3, 3, CV_64FC1, init_li);
cvInitMatHeader(&lm, 5, 1, CV_64FC1, init_lm);
cvInitMatHeader(&ri, 3, 3, CV_64FC1, init_ri);
cvInitMatHeader(&rm, 5, 1, CV_64FC1, init_rm);
cvInitMatHeader(&r, 3, 3, CV_64FC1, init_r);
cvInitMatHeader(&t, 3, 1, CV_64FC1, init_t);
cvInitMatHeader(&Rl, 3,3, CV_64FC1);
cvInitMatHeader(&Rr, 3,3, CV_64FC1);
cvInitMatHeader(&Pl, 3,4, CV_64FC1);
cvInitMatHeader(&Pr, 3,4, CV_64FC1);
//frame is a cv::MAT holding the first frame of the video.
CvSize imageSize = frame.size();
imageSize.width /= 2;
//IT BREAKS HERE
cvStereoRectify(&li, &ri, &lm, &rm, imageSize, &r, &t, &Rl, &Rr, &Pl, &Pr);
so, you've been bitten by the c-api ? why don't you just turn your back on it ?
use the c++ api whenever possible, don't start learning opencv with the old(1.0), deprecated api, please !
double init_li[9] =
{ 477.984984743, 0, 316.17458671,
0, 476.861945645, 253.45073026,
0, 0 ,1 };
double init_lm[5] = {-0.117798518453, 0.147554949385, -0.0549082041898, 0, 0};
double init_ri[9] =
{ 478.640315323, 0, 299.957994781,
0, 477.898896505, 251.665771947,
0, 0, 1};
double init_rm[5] = {-0.10884732532, 0.12118405303, -0.0322073237741, 0, 0};
double init_r[9] =
{ 0.999973709051976, 0.00129700728791757, -0.00713435189275776,
-0.00132096594266573, 0.999993501087837, -0.00335452397041856,
0.00712995468519435, 0.00336386001267643, 0.99996892361313};
double init_t[3] = {-0.0830973040641153, -0.00062704210860633, 1.4287643345188e-005};
cv::Mat li(3, 3, CV_64FC1, init_li);
cv::Mat lm(5, 1, CV_64FC1, init_lm);
cv::Mat ri(3, 3, CV_64FC1, init_ri);
cv::Mat rm(5, 1, CV_64FC1, init_rm);
cv::Mat r, t, Rl, Rr, Pl, Pr; // note: no initialization needed.
//frame is a cv::MAT holding the first frame of the video.
cv::Size imageSize = frame.size();
imageSize.width /= 2;
//IT won't break HERE
cv::stereoRectify(li, ri, lm, rm, imageSize, r, t, Rl, Rr, Pl, Pr);
// no need ever to release or care about anything
Ok, so I figured out the answer. The problem was that I had only initialized headers for Rl, Rr, Pl, and Pr, but no memory was allocated for the data itself. I was able to fix it as follows:
double init_Rl[3][3];
double init_Rr[3][3];
double init_Pl[3][4];
double init_Pr[3][4];
cvInitMatHeader(&Rl, 3,3, CV_64FC1, init_Rl);
cvInitMatHeader(&Rr, 3,3, CV_64FC1, init_Rr);
cvInitMatHeader(&Pl, 3,4, CV_64FC1, init_Pl);
cvInitMatHeader(&Pr, 3,4, CV_64FC1, init_Pr);
Although, I have a theory that I might have been able to use cv::stereoRectify with cv::Mats as parameters, which would have made life much easier. I don't know if cv::stereoRectify exists, but it seems that versions of many of the other c functions are in the cv namespace. In case it's hard to tell, I'm very new to OpenCV.