I have recently started porting an application to a new platform which runs OpenCV 2.4.5.
Part of my code which uses OpenCV's implementation of FLANN to do hierarchical clustering no longer compiles.
The code is as follows:
cv::Mat mergedFeatures = cvCreateMat(descriptorTotal, descriptorDims, CV_32F);
int counter = 0;
for (uint j = 0; j < ImageFeatures.size(); j++) {
cv::Mat features = ImageFeatures[j];
for (int k = 0; k < features.rows; k++) {
cv::Mat roi = mergedFeatures.row(counter);
features.row(k).copyTo(roi);
counter++;
}
}
cv::Mat centers = cvCreateMat(1000, descriptorDims, CV_32FC1);
cv::flann::KMeansIndexParams k_params = cv::flann::KMeansIndexParams();
cv::flann::AutotunedIndexParams atp = cv::flann::AutotunedIndexParams();
int numClusters = cv::flann::hierarchicalClustering<float, float>(mergedFeatures, centers, k_params);
The error that I am getting (in Eclipse) is that cv::flann::hierarchicalClustering has invalid arguments and that neither of the candidates for this function are met.
Can someone explain how I suddenly seem to be calling this method incorrectly?
Many thanks for any help.
I fixed the problem myself.
Instead of accepting:
cv::flann::KMeansIndexParams k_params
the hierarchicalClustering function actually needs:
cvflann::KMeansIndexParams k_params
It is rather a confusing namespace convention with the FLANN library in OpenCV and I had just overlooked the namespace difference in what the compiler error was telling me.
It is all working now. The KMeansIndexParams type is present in both namespaces and I guess that the hierarchicalClustering method has changed very slightly from OpenCV 2.3 to 2.4.5.
Related
I am trying to compute $\partial{J}{q_i}$ in drake C++ for manipulator and as per my search, the best approach seems to be using autodiff function. I was not able to fully understand autodiff approach from the resources that I found, so I apologize if my approach is not clear enough. I have used my understanding from some already asked questions mentioned on the forum regarding auto diff as well as https://drake.mit.edu/doxygen_cxx/classdrake_1_1multibody_1_1_multibody_plant.html as reference.
As I want to calculate $\partial{J}{q_i}$, the return type will be a tensor i.e. 3 * 7 * 7(or 6 * 7 * 7 depending on the spatial jacobian). I can think of using std::vectorEigen::MatrixXd to allocate the output or alternatively just doing one $q_i$ at a time and computing the respective jacobian for the auto diff. In either case, I was struggling to pass it in the initializing the jacobian function.
I did the following to initialize autodiff
std::unique_ptr<multibody::MultibodyPlant<AutoDiffXd>> mplant_autodiff = systems::System<double>::ToAutoDiffXd(mplant);
std::unique_ptr<systems::Context<AutoDiffXd>> mContext_autodiff = mplant_autodiff->CreateDefaultContext();
mContext_autodiff->SetTimeStateAndParametersFrom(*mContext);
const multibody::Frame<AutoDiffXd>* mFrame_EE_autodiff = &mplant_autodiff->GetBodyByName(mEE_link).body_frame();
const multibody::Frame<AutoDiffXd>* mWorld_Frame_autodiff = &(mplant_autodiff->world_frame());
//Initialize the q as autodiff vector
drake::AutoDiffVecXd q_autodiff = drake::math::InitializeAutoDiff(mq_robot);
MatrixX<AutoDiffXd> mJacobian_autodiff; // Linear Jacobian matrix.
mplant_autodiff->SetPositions(context_autodiff.get(), q_autodiff);
mplant_autodiff->CalcJacobianTranslationalVelocity(*mContext_autodiff,
multibody::JacobianWrtVariable::kQDot,
*mFrame_EE_autodiff,
Eigen::Vector3d::Zero(),
*mWorld_Frame_autodiff,
*mWorld_Frame_autodiff,
&mJacobian_autodiff
);
However, as far as I understand, InitializeAutoDiff initializes to the identity matrix, whereas I want to $\partial{J}{q_i}$, so is there is a better way to do it. In addition, I get error messages when I try to call the jacobian matrix. Is there a way to address this problem both for $\partial{J}{q_i}$ for each q_i and changing q_i in a for loop or directly getting the result in a tensor. My apologies if I am doing something total tangent to the correct approach. I thank you in anticipation.
However, as far as I understand, InitializeAutoDiff initializes to the identity matrix, whereas I want to $\partial{J}{q_i}$, so is there is a better way to do it
That is correct. When you call InitializeAutoDiff and compute mJacobian_autodiff, you get a matrix of AutoDiffXd. Each AutoDiffXd has a value() function that stores the double value, and a derivatives() storing the gradient as an Eigen::VectorXd. We have
mJacobian(i, j).value() = J(i, j)
mJacobian_autodiff(i, j).derivatives()(k) = ∂J(i, j)/∂q(k)
So if you want to create a std::vecot<Eigen::MatrixXd> such that the k'th entry of this vector stores the matrix ∂J/∂q(k), then here is a code
std::vector<Eigen::MatrixXd> dJdq(q_autodiff.rows());
for (int i = 0; i < q_autodiff.rows(); ++i) {
dJdq[i].resize(mJacobian_autodiff.rows(), mJacobian_autodiff.cols());
}
for (int i = 0; i < q_autodiff.rows(); ++i) {
// dJidq stores the gradient of the ∂J.col(i)/∂q, namely dJidq(j, k) = ∂J(j, i)/∂q(k)
auto dJidq = ExtractGradient(mJacobian_autodiff.col(i));
for (int j = 0; j < static_cast<int>(dJdq.size()); ++j) {
dJdq[j].col(i) = dJidq.col(j);
}
}
Compute ∂J/∂q(i) for a single i
If you do not want to compute ∂J/∂q(i) for all i, but only for one specific i, you can change the initialization of q_autodiff from InitializeAutoDiff to this
AutoDiffVecXd q_autodiff(q.rows());
for (int k = 0; k < q_autodiff.rows(); ++k) {
q_autodiff(k).value() = q(k)
q_autodiff(k).derivatives() = Vector1d::Zero();
if (k == i) {
q_autodiff(k).derivatives()(0) = 1;
}
}
namely q_autodiff stores the gradient ∂q/∂q(i), which is 0 for all k != i and 1 when k == i. And then you can compute mJacobian_autodiff using your current code. Now mJacobian_autodiff(m, n).derivatives() store the gradient of ∂J(m, m)/∂q(i) for that specific i. You can extract this gradient as
Eigen::Matrix dJdqi(mJacobian_autodiff.rows(), mJacobian_autodiff.cols());
for (int m = 0; m < dJdqi.rows(); ++m) {
for (int n = 0; n < dJdqi.cols(); ++n) {
dJdqi(m, n) = mJacobian_autodiff(m, n).derivatives()(0);
}
}
Situation: I am trying to get point cloud with pcl::AdaptiveCostSOStereoMatching, which uses two rectified images (pics are ok).
I used these tutorials to learn how to do this:
First tutorial
Second tutorial
Error: programm crashes in runtime when calling "compute" method of AdaptiveCostSOStereoMatching
Question: how to correctly pass images to "compute" method?
I tried:
1) Images converted by png2pcd
(command line: "png2pcd.exe in.png out.pcd")
2) Images converted with function below from cv::Mat
But no luck.
Function which converts cv::Mat to pcl::PointCloud
void MatToPointCloud(Mat& mat, pcl::PointCloud<RGB>::Ptr cloud)
{
int width = mat.cols;
int height = mat.rows;
pcl::RGB val;
val.r = 0; val.g = 0; val.b = 0;
for (int i = 0; i < mat.rows; i++)
for (int j = 0; j < mat.cols; j++)
{
auto point = mat.at<Vec3b>(i, j);
//std::cout << j << " " << i << "\n";
val.b = point[0];
val.g = point[1];
val.r = point[2];
cloud->at(j, i) = val;
}
}
pcl::AdaptiveCostSOStereoMatching (compute)
// Input
Mat leftMat, rightMat;
leftMat = imread("left.png");
rightMat = imread("right.png");
int width = leftMat.cols;
int height = rightMat.rows;
pcl::RGB val;
val.r = 0; val.g = 0; val.b = 0;
pcl::PointCloud<pcl::RGB>::Ptr left_cloud(new pcl::PointCloud<pcl::RGB>(width, height, val));
pcl::PointCloud<pcl::RGB>::Ptr right_cloud(new pcl::PointCloud<pcl::RGB>(width, height, val));
MatToPointCloud(leftMat, left_cloud);
MatToPointCloud(rightMat, right_cloud);
// Calculation
pcl::AdaptiveCostSOStereoMatching stereo;
stereo.setMaxDisparity(60);
//stereo.setXOffest(0); Почему-то не распознается
stereo.setRadius(5);
stereo.setSmoothWeak(20);
stereo.setSmoothStrong(100);
stereo.setGammaC(25);
stereo.setGammaS(10);
stereo.setRatioFilter(20);
stereo.setPeakFilter(0);
stereo.setLeftRightCheck(true);
stereo.setLeftRightCheckThreshold(1);
stereo.setPreProcessing(true);
stereo.compute(*left_cloud, *right_cloud); // <-- CRASHING THERE
stereo.medianFilter(4);
pcl::PointCloud<pcl::PointXYZRGB>::Ptr out_cloud(new pcl::PointCloud<pcl::PointXYZRGB>);
stereo.getPointCloud(318.11220, 224.334900, 368.534700, 0.8387445, out_cloud, left_cloud);
Error information:
Output log: HEAP[App.exe]:
Heap block at 0000006B0F828460 modified at 0000006B0F8284A8 past requested size of 38
App.exe has triggered a breakpoint.
left_cloud (a right cloud looks like left_cloud)
Mini question: if AdaptiveCostSOStereoMatching really allows build point cloud from 2 images, how ACSSM doing this without insintric and excentic parameters?
Problem: I downloaded and installed old version of PCL without stereo.
After that, I downloaded stereo from other PCL pack and add this library to my PCL pack. And it worked incorrectly.
Solution: I compilled PCL 1.8 and my programm is ok now.
OS: Windows
IDE: MSVS 12 2013 x64
If you will try to compile PCL, these links can help you:
Official-tutorial-1
Official-tutorial-2
Good help with FLANN and VTK
Example to verify installation
I'm new to OpenCV and C++ and I'm trying to build a classifier using Gaussian Mixture Model within the OpenCV. I figured out how it works and got it worked ... maybe. I got something like this now:
If I classify the training samples just after the model was trained and saved, I got the result I want. But when I reclassify my training data using the read(), one of the clusters is missing, means I got different cluster result from the same GMM model. I don't get it now because the cluster I want was gone, I can't reproduce the classification again until I retrained the model using the same data. I checked the code in runtime and the result valule in the Vec2d from which predict() returned was never assigned to 1 (I set 3 clusters).
Maybe there's a bug or I did something wrong?
p.s. I'm using 2.4.8 in VS2013
My programs like this:
train part
void GaussianMixtureModel::buildGMM(InputArray _src){
//use source to train GMM and save the model
Mat samples, input = _src.getMat();
createSamples(input, samples);
bool status = em_model.train(samples);
saveModel();
}
save/load the model
FileStorage fs(filename, FileStorage::READ);
if (fs.isOpened()) // if we have file with parameters, read them
{
const FileNode& fn = fs["StatModel.EM"];
em_model.read(fn);
fs.release();
}
FileStorage fs_save(filename, FileStorage::WRITE);
if (fs_save.isOpened()) // if we have file with parameters, read them
{
em_model.write(fs_save);
fs_save.release();
}
predict part
vector<Mat> GaussianMixtureModel::classify(Mat input){
/// samples is a matrix of channels x N elements, each row is a set of feature
Mat samples;
createSamples(input, samples);
for (int k = 0; k < clusterN; k++){
masks[k] = Mat::zeros(input.size(), CV_8UC1);
}
int idx = 0;
for (int i = 0; i < input.rows; i++){
for (int j = 0; j < input.cols; j++){
//process the predicted probability
Mat probs(1, clusterN, CV_64FC1);
Vec2d response = em_model.predict(samples.row(idx++), probs);
int result = cvRound(response[1]);
for (int k = 0; k < clusterN; k++){
if (result == k){
// change to the k-th class's picture
masks[k].at<uchar>(i, j) = 255;
}
...
// something else
}
}
}
}
I suppose my answer will be too late but as I have encountered the same problem the solution I found may be useful for others.
By analysing the source code, I notice that in the case of EM::COV_MAT_DIAGONAL the eigen values of covariances matrix(covsEigenValues in source code) are obtained via SVD after loading the saved data.
However, SVD computes the singular(eigen in our case)values and stores it in ASCENDING order.
To prevent this , I simply extract directly the diagonal element of loaded covariance matrix in covsEigenValues to keep the good order.
As a result of a call to estimateRigidTransform() I get a cv::Mat object named "trans". To retrieve its contained matrix I try to access its elements this way:
for (i=0; i<2; i++) for (j=0; j<3; j++)
{
mtx[j][i]=CV_MAT_ELEM(trans,double,i,j);
}
Unfortunately with VS2010 I get a compiler error
error C2228: left of '.ptr' must have class/struct/union
for the line with CV_MAT_ELEM. When I unwrap this macro I find something like
(mat).data.ptr + (size_t)(mat).step*(row) + (pix_size)*(col))
When I remove the ".ptr" behind (mat).data it compiles. But I can't imagine that's the solution (or can't imagine that's a bug and I'm the only one who noticed it). So what could be wrong here really?
Thanks!
You don't access the mat elements like this. For a traversal see my other answer here with sample code:
color matrix traversal
or see the opencv refman for grayscale Mat:
Mat M; // should be grayscale
int cols = M.cols, rows = M.rows;
for(int i = 0; i < rows; i++)
{
const double* Mi = M.ptr<double>(i);
for(int j = 0; j < cols; j++)
{
Mi[j]; // is the matrix element.
}
}
Just an addendum from my side: meanwhile I found CV_MAT_ELEM expects a structure CvMat (OpenCV-C-interface) but not cv::Mat (the C++-interface). That's why I get this funny error. Conversion from cv::Mat to CvMat can be done simply by casting to CvMat. Funny confusion with the C and C++ interfaces in OpenCV...
After spending a couple of days trying figure out why opencv DFT would give 100% similar results for all three channels I ended up finding out that there might be a bug in the split() function that OpenCV provides for splitting a input image to 3 single channel images.
std::vector<cv::Mat> rgbChannels(3,cv::Mat(inputImage.size(),CV_64FC1));
cv::split(inputImage, rgbChannels);
After saving the image values onto disk and using a file differencing tool, I found out that all values in the split channels were identical.
Have I done something wrong?
My work around was the following function. But that also gave me the exact identical values, giving me a hint that somehow vectors were not being handled correctly by OpenCV.
SplitImage(cv::Mat inputImage)
{
//copy original in BGR order
std::vector<cv::Mat> splittedImage(3,cv::Mat(inputImage.size(),CV_64FC1));
cv::Mat tempImage(inputImage.size(),CV_64FC1);
for (int row = 0; row < inputImage.size().height; row++)
{
for (int col = 0; col < inputImage.size().width; col++)
{
splittedImage[0].at<double>(row, col) = inputImage.at<cv::Vec3d>(row, col)[0];
splittedImage[1].at<double>(row, col) = inputImage.at<cv::Vec3d>(row, col)[1];
splittedImage[2].at<double>(row, col) = inputImage.at<cv::Vec3d>(row, col)[2];
}
}
return splittedImage;
}
And finally wrote the following to solve the problem
SplitImage(cv::Mat inputImage)
{
//copy original in BGR order
std::vector<cv::Mat> splittedImage(3,cv::Mat(inputImage.size(),CV_64FC1));
std::vector<cv::Mat>::iterator it;
it = splittedImage.begin();
for(int channelNo = 0; channelNo < inputImage.channels(); channelNo++)
{
cv::Mat tempImage(inputImage.size(),CV_64FC1);
for (int row = 0; row < inputImage.size().height; row++)
{
for (int col = 0; col < inputImage.size().width; col++)
{
tempImage.at<double>(row, col) = inputImage.at<cv::Vec3d>(row, col)[channelNo];
}
}
it = splittedImage.insert ( it , tempImage );
it++;
}
return splittedImage;
}
Has anyone had a problem with split() function or have I done something wrong?
It is not a bug in OpenCV but there is a problem with your code.
The following line does not create a vector of 3 different Mats:
std::vector<cv::Mat> rgbChannels(3,cv::Mat(inputImage.size(),CV_64FC1));
Instead, this line produces a vector of 3 Mat headers sharing the same memory. It works this way because Mat copy constructor does not make a deep copy - it just increments an internal reference counter.
Just change your code to the following to solve your problem:
std::vector<cv::Mat> rgbChannels(3);
cv::split(inputImage, rgbChannels);