Got different EM::predict() results after EM::read() saved model in OpenCV - opencv

I'm new to OpenCV and C++ and I'm trying to build a classifier using Gaussian Mixture Model within the OpenCV. I figured out how it works and got it worked ... maybe. I got something like this now:
If I classify the training samples just after the model was trained and saved, I got the result I want. But when I reclassify my training data using the read(), one of the clusters is missing, means I got different cluster result from the same GMM model. I don't get it now because the cluster I want was gone, I can't reproduce the classification again until I retrained the model using the same data. I checked the code in runtime and the result valule in the Vec2d from which predict() returned was never assigned to 1 (I set 3 clusters).
Maybe there's a bug or I did something wrong?
p.s. I'm using 2.4.8 in VS2013
My programs like this:
train part
void GaussianMixtureModel::buildGMM(InputArray _src){
//use source to train GMM and save the model
Mat samples, input = _src.getMat();
createSamples(input, samples);
bool status = em_model.train(samples);
saveModel();
}
save/load the model
FileStorage fs(filename, FileStorage::READ);
if (fs.isOpened()) // if we have file with parameters, read them
{
const FileNode& fn = fs["StatModel.EM"];
em_model.read(fn);
fs.release();
}
FileStorage fs_save(filename, FileStorage::WRITE);
if (fs_save.isOpened()) // if we have file with parameters, read them
{
em_model.write(fs_save);
fs_save.release();
}
predict part
vector<Mat> GaussianMixtureModel::classify(Mat input){
/// samples is a matrix of channels x N elements, each row is a set of feature
Mat samples;
createSamples(input, samples);
for (int k = 0; k < clusterN; k++){
masks[k] = Mat::zeros(input.size(), CV_8UC1);
}
int idx = 0;
for (int i = 0; i < input.rows; i++){
for (int j = 0; j < input.cols; j++){
//process the predicted probability
Mat probs(1, clusterN, CV_64FC1);
Vec2d response = em_model.predict(samples.row(idx++), probs);
int result = cvRound(response[1]);
for (int k = 0; k < clusterN; k++){
if (result == k){
// change to the k-th class's picture
masks[k].at<uchar>(i, j) = 255;
}
...
// something else
}
}
}
}

I suppose my answer will be too late but as I have encountered the same problem the solution I found may be useful for others.
By analysing the source code, I notice that in the case of EM::COV_MAT_DIAGONAL the eigen values of covariances matrix(covsEigenValues in source code) are obtained via SVD after loading the saved data.
However, SVD computes the singular(eigen in our case)values and stores it in ASCENDING order.
To prevent this , I simply extract directly the diagonal element of loaded covariance matrix in covsEigenValues to keep the good order.

Related

Autodiff for Jacobian derivative with respect to individual joint angles

I am trying to compute $\partial{J}{q_i}$ in drake C++ for manipulator and as per my search, the best approach seems to be using autodiff function. I was not able to fully understand autodiff approach from the resources that I found, so I apologize if my approach is not clear enough. I have used my understanding from some already asked questions mentioned on the forum regarding auto diff as well as https://drake.mit.edu/doxygen_cxx/classdrake_1_1multibody_1_1_multibody_plant.html as reference.
As I want to calculate $\partial{J}{q_i}$, the return type will be a tensor i.e. 3 * 7 * 7(or 6 * 7 * 7 depending on the spatial jacobian). I can think of using std::vectorEigen::MatrixXd to allocate the output or alternatively just doing one $q_i$ at a time and computing the respective jacobian for the auto diff. In either case, I was struggling to pass it in the initializing the jacobian function.
I did the following to initialize autodiff
std::unique_ptr<multibody::MultibodyPlant<AutoDiffXd>> mplant_autodiff = systems::System<double>::ToAutoDiffXd(mplant);
std::unique_ptr<systems::Context<AutoDiffXd>> mContext_autodiff = mplant_autodiff->CreateDefaultContext();
mContext_autodiff->SetTimeStateAndParametersFrom(*mContext);
const multibody::Frame<AutoDiffXd>* mFrame_EE_autodiff = &mplant_autodiff->GetBodyByName(mEE_link).body_frame();
const multibody::Frame<AutoDiffXd>* mWorld_Frame_autodiff = &(mplant_autodiff->world_frame());
//Initialize the q as autodiff vector
drake::AutoDiffVecXd q_autodiff = drake::math::InitializeAutoDiff(mq_robot);
MatrixX<AutoDiffXd> mJacobian_autodiff; // Linear Jacobian matrix.
mplant_autodiff->SetPositions(context_autodiff.get(), q_autodiff);
mplant_autodiff->CalcJacobianTranslationalVelocity(*mContext_autodiff,
multibody::JacobianWrtVariable::kQDot,
*mFrame_EE_autodiff,
Eigen::Vector3d::Zero(),
*mWorld_Frame_autodiff,
*mWorld_Frame_autodiff,
&mJacobian_autodiff
);
However, as far as I understand, InitializeAutoDiff initializes to the identity matrix, whereas I want to $\partial{J}{q_i}$, so is there is a better way to do it. In addition, I get error messages when I try to call the jacobian matrix. Is there a way to address this problem both for $\partial{J}{q_i}$ for each q_i and changing q_i in a for loop or directly getting the result in a tensor. My apologies if I am doing something total tangent to the correct approach. I thank you in anticipation.
However, as far as I understand, InitializeAutoDiff initializes to the identity matrix, whereas I want to $\partial{J}{q_i}$, so is there is a better way to do it
That is correct. When you call InitializeAutoDiff and compute mJacobian_autodiff, you get a matrix of AutoDiffXd. Each AutoDiffXd has a value() function that stores the double value, and a derivatives() storing the gradient as an Eigen::VectorXd. We have
mJacobian(i, j).value() = J(i, j)
mJacobian_autodiff(i, j).derivatives()(k) = ∂J(i, j)/∂q(k)
So if you want to create a std::vecot<Eigen::MatrixXd> such that the k'th entry of this vector stores the matrix ∂J/∂q(k), then here is a code
std::vector<Eigen::MatrixXd> dJdq(q_autodiff.rows());
for (int i = 0; i < q_autodiff.rows(); ++i) {
dJdq[i].resize(mJacobian_autodiff.rows(), mJacobian_autodiff.cols());
}
for (int i = 0; i < q_autodiff.rows(); ++i) {
// dJidq stores the gradient of the ∂J.col(i)/∂q, namely dJidq(j, k) = ∂J(j, i)/∂q(k)
auto dJidq = ExtractGradient(mJacobian_autodiff.col(i));
for (int j = 0; j < static_cast<int>(dJdq.size()); ++j) {
dJdq[j].col(i) = dJidq.col(j);
}
}
Compute ∂J/∂q(i) for a single i
If you do not want to compute ∂J/∂q(i) for all i, but only for one specific i, you can change the initialization of q_autodiff from InitializeAutoDiff to this
AutoDiffVecXd q_autodiff(q.rows());
for (int k = 0; k < q_autodiff.rows(); ++k) {
q_autodiff(k).value() = q(k)
q_autodiff(k).derivatives() = Vector1d::Zero();
if (k == i) {
q_autodiff(k).derivatives()(0) = 1;
}
}
namely q_autodiff stores the gradient ∂q/∂q(i), which is 0 for all k != i and 1 when k == i. And then you can compute mJacobian_autodiff using your current code. Now mJacobian_autodiff(m, n).derivatives() store the gradient of ∂J(m, m)/∂q(i) for that specific i. You can extract this gradient as
Eigen::Matrix dJdqi(mJacobian_autodiff.rows(), mJacobian_autodiff.cols());
for (int m = 0; m < dJdqi.rows(); ++m) {
for (int n = 0; n < dJdqi.cols(); ++n) {
dJdqi(m, n) = mJacobian_autodiff(m, n).derivatives()(0);
}
}

matchTemplate with openCV in java

i have a code like this:
Mat img = Highgui.imread(inFile);
Mat templ = Highgui.imread(templateFile);
int result_cols = img.cols() - templ.cols() + 1;
int result_rows = img.rows() - templ.rows() + 1;
Mat result = new Mat(result_rows, result_cols, CvType.CV_32FC1);
Imgproc.matchTemplate(img, templ, result, Imgproc.TM_CCOEFF);
/////Core.normalize(result, result, 0, 1, Core.NORM_MINMAX, -1, new Mat());
for (int i = 0; i < result_rows; i++)
for (int j = 0; j < result_cols; j++)
if(result.get(i, j)[0]>?)
//match!
I need to parse the input image to find multiple occurrencies of the template image. I want to have a result like this:
result[0][0]= 15%
result[0][1]= 17%
result[x][y]= 47%
If i use TM_COEFF all results are [-xxxxxxxx.xxx,+xxxxxxxx.xxx]
If i use TM_SQDIFF all results are xxxxxxxx.xxx
If i use TM_CCORR all results are xxxxxxxx.xxx
How can i detect a match or a mismatch? What is the right condition into the if?
If i normalized the matrix the application set a value to 1 and i can't detect if the template isn't stored into the image (all mismatch).
Thanks in advance
You can append "_NORMED" to the method names (For instance: CV_TM_COEFF_NORMED in C++; could be slightly different in Java) to get a sensible value for your purpose.
By 'sensible', I mean that you will get values in the range of 0 to 1 which can be multiplied by 100 for your purpose.
Note: For CV_TM_SQDIFF_NORMED, it will be in the range -1 to 0, and you will have to subtract the value from 1 in order to make sense of it, because the lowest value if used in this method.
Tip: you can use the java equivalent of minMaxLoc() in order to get the minimum and maximum values. It's very useful when used in conjunction with matchtemplate.
I believe 'minMaxLoc' that is located inside the class Core.
Here's a C++ implementation:
matchTemplate( input_mat, template_mat, result_mat, method_NORMED );
double minVal, maxVal;
double percentage;
Point minLoc; Point maxLoc;
minMaxLoc( result, &minVal, &maxVal, &minLoc, &maxLoc, Mat() );
if( method_NORMED == CV_TM_SQDIFF_NORMED )
{
percentage=1-minVal;
}
else
{
percentage=maxVal;
}
Useful C++ docs:
Match template description along with available methods: http://docs.opencv.org/modules/imgproc/doc/object_detection.html
MinMaxLoc documentation:
http://docs.opencv.org/modules/core/doc/operations_on_arrays.html?highlight=minmaxloc#minmaxloc
Another approach will be background differencing. You can observe the distortion.
import org.opencv.core.Core;
import org.opencv.core.Mat;
import org.opencv.highgui.Highgui;
import org.opencv.imgproc.Imgproc;
public class BackgroundDifference {
public static void main(String[] arg){
System.loadLibrary(Core.NATIVE_LIBRARY_NAME);
Mat model = Highgui.imread("e:\\answers\\template.jpg",Highgui.CV_LOAD_IMAGE_GRAYSCALE);
Mat scene = Highgui.imread("e:\\answers\\front7.jpg",Highgui.CV_LOAD_IMAGE_GRAYSCALE);
Mat diff = new Mat();
Core.absdiff(model,scene,diff);
Imgproc.threshold(diff,diff,15,255,Imgproc.THRESH_BINARY);
int distortion = Core.countNonZero(diff);
System.out.println("distortion:"+distortion);
Highgui.imwrite("e:\\answers\\diff.jpg",diff);
}
}

OpenCV 2.4.5: FLANN and hierarchicalClustering

I have recently started porting an application to a new platform which runs OpenCV 2.4.5.
Part of my code which uses OpenCV's implementation of FLANN to do hierarchical clustering no longer compiles.
The code is as follows:
cv::Mat mergedFeatures = cvCreateMat(descriptorTotal, descriptorDims, CV_32F);
int counter = 0;
for (uint j = 0; j < ImageFeatures.size(); j++) {
cv::Mat features = ImageFeatures[j];
for (int k = 0; k < features.rows; k++) {
cv::Mat roi = mergedFeatures.row(counter);
features.row(k).copyTo(roi);
counter++;
}
}
cv::Mat centers = cvCreateMat(1000, descriptorDims, CV_32FC1);
cv::flann::KMeansIndexParams k_params = cv::flann::KMeansIndexParams();
cv::flann::AutotunedIndexParams atp = cv::flann::AutotunedIndexParams();
int numClusters = cv::flann::hierarchicalClustering<float, float>(mergedFeatures, centers, k_params);
The error that I am getting (in Eclipse) is that cv::flann::hierarchicalClustering has invalid arguments and that neither of the candidates for this function are met.
Can someone explain how I suddenly seem to be calling this method incorrectly?
Many thanks for any help.
I fixed the problem myself.
Instead of accepting:
cv::flann::KMeansIndexParams k_params
the hierarchicalClustering function actually needs:
cvflann::KMeansIndexParams k_params
It is rather a confusing namespace convention with the FLANN library in OpenCV and I had just overlooked the namespace difference in what the compiler error was telling me.
It is all working now. The KMeansIndexParams type is present in both namespaces and I guess that the hierarchicalClustering method has changed very slightly from OpenCV 2.3 to 2.4.5.

Compile error in CV_MAT_ELEM

As a result of a call to estimateRigidTransform() I get a cv::Mat object named "trans". To retrieve its contained matrix I try to access its elements this way:
for (i=0; i<2; i++) for (j=0; j<3; j++)
{
mtx[j][i]=CV_MAT_ELEM(trans,double,i,j);
}
Unfortunately with VS2010 I get a compiler error
error C2228: left of '.ptr' must have class/struct/union
for the line with CV_MAT_ELEM. When I unwrap this macro I find something like
(mat).data.ptr + (size_t)(mat).step*(row) + (pix_size)*(col))
When I remove the ".ptr" behind (mat).data it compiles. But I can't imagine that's the solution (or can't imagine that's a bug and I'm the only one who noticed it). So what could be wrong here really?
Thanks!
You don't access the mat elements like this. For a traversal see my other answer here with sample code:
color matrix traversal
or see the opencv refman for grayscale Mat:
Mat M; // should be grayscale
int cols = M.cols, rows = M.rows;
for(int i = 0; i < rows; i++)
{
const double* Mi = M.ptr<double>(i);
for(int j = 0; j < cols; j++)
{
Mi[j]; // is the matrix element.
}
}
Just an addendum from my side: meanwhile I found CV_MAT_ELEM expects a structure CvMat (OpenCV-C-interface) but not cv::Mat (the C++-interface). That's why I get this funny error. Conversion from cv::Mat to CvMat can be done simply by casting to CvMat. Funny confusion with the C and C++ interfaces in OpenCV...

Bug in OpenCV2.3 cv::split() function. Identical values in all 3 channels

After spending a couple of days trying figure out why opencv DFT would give 100% similar results for all three channels I ended up finding out that there might be a bug in the split() function that OpenCV provides for splitting a input image to 3 single channel images.
std::vector<cv::Mat> rgbChannels(3,cv::Mat(inputImage.size(),CV_64FC1));
cv::split(inputImage, rgbChannels);
After saving the image values onto disk and using a file differencing tool, I found out that all values in the split channels were identical.
Have I done something wrong?
My work around was the following function. But that also gave me the exact identical values, giving me a hint that somehow vectors were not being handled correctly by OpenCV.
SplitImage(cv::Mat inputImage)
{
//copy original in BGR order
std::vector<cv::Mat> splittedImage(3,cv::Mat(inputImage.size(),CV_64FC1));
cv::Mat tempImage(inputImage.size(),CV_64FC1);
for (int row = 0; row < inputImage.size().height; row++)
{
for (int col = 0; col < inputImage.size().width; col++)
{
splittedImage[0].at<double>(row, col) = inputImage.at<cv::Vec3d>(row, col)[0];
splittedImage[1].at<double>(row, col) = inputImage.at<cv::Vec3d>(row, col)[1];
splittedImage[2].at<double>(row, col) = inputImage.at<cv::Vec3d>(row, col)[2];
}
}
return splittedImage;
}
And finally wrote the following to solve the problem
SplitImage(cv::Mat inputImage)
{
//copy original in BGR order
std::vector<cv::Mat> splittedImage(3,cv::Mat(inputImage.size(),CV_64FC1));
std::vector<cv::Mat>::iterator it;
it = splittedImage.begin();
for(int channelNo = 0; channelNo < inputImage.channels(); channelNo++)
{
cv::Mat tempImage(inputImage.size(),CV_64FC1);
for (int row = 0; row < inputImage.size().height; row++)
{
for (int col = 0; col < inputImage.size().width; col++)
{
tempImage.at<double>(row, col) = inputImage.at<cv::Vec3d>(row, col)[channelNo];
}
}
it = splittedImage.insert ( it , tempImage );
it++;
}
return splittedImage;
}
Has anyone had a problem with split() function or have I done something wrong?
It is not a bug in OpenCV but there is a problem with your code.
The following line does not create a vector of 3 different Mats:
std::vector<cv::Mat> rgbChannels(3,cv::Mat(inputImage.size(),CV_64FC1));
Instead, this line produces a vector of 3 Mat headers sharing the same memory. It works this way because Mat copy constructor does not make a deep copy - it just increments an internal reference counter.
Just change your code to the following to solve your problem:
std::vector<cv::Mat> rgbChannels(3);
cv::split(inputImage, rgbChannels);

Resources