Compute fundamental matrix with 8 point algorithm - image-processing

I need to write an own implementation of computing the fundamental matrix between two images based on the corresponding image coordinates without using OpenCV.
Is it possible to describe this algorithm in its simplest form in accordance with the following function? a simple and straightforward formula.
FMatrixEightPoint()
Input Arguments:
points1(x,y)−pixel coordinates in the first image ,
corresponding to points2 in the second image
points2(x,y)−pixel coordinates in the second image ,
corresponding to points1 in the first image
Output :
F − the fundamental matrix between the first image and the second image

Yes, it is possible to describe the algorithm in the mentioned form.
If you would use OpenCV, you could just use findFundamentalMat. This also provides the 8-point method for computing the fundamental matrix.
The example (in C++) taken from the OpenCV documentation, but adapted (using the RANSAC algorithm for computing the fundamental matrix):
// Example. Estimation of fundamental matrix using the 8-point algorithm
int point_count = 8; // must be >= 8
vector<Point2f> points1(point_count);
vector<Point2f> points2(point_count);
// initialize the points here ... */
for( int i = 0; i < point_count; i++ )
{
points1[i] = ...;
points2[i] = ...;
}
Mat fundamental_matrix =
findFundamentalMat(points1, points2, CV_FM_8POINT);
If you want to write your own function, it would look like this (no valid code)
Matrix findFundamentalMat(Array points1, Array points2)
{
Matrix fundamentalMatrix;
// compute fundamental matrix based on input points1 and points2 or call OpenCV's findFundamentalMat
return fundamentalMatrix;
}

Related

Dot Product vs Element-Wise multiplication for Backpropogation

I am trying to backpropogate a very primitive / simple ANN.
I've almost got it working. I'm trying to implement the formulas and the article I'm reading does not specify whether to use dot product or element wise multiplication or some other multiplication.
Article: https://ml-cheatsheet.readthedocs.io/en/latest/backpropagation.html
Here's the formula for calculating the error (or delta) of a single Hidden layer:
Or, as I read it in the context of my algorithm,
Delta = prev_delta * prev_weight * zprime
Where delta is the error of this layer, prev_delta is the delta of the previous layer, prev_weight is the weight of the previous layer, and zprime is the derivative of the activation function of the current layer.
Also, for a single Output Layer:
Or, as I read it in the context of my algorithm,
Delta = (output - target) % zprime;
Where output is the final output of the feed-forward and target is the target values.
I've written this code to run this calculation:
void Layer::backward(Matrix & prev_delta, Matrix & prev_weight) {
// all variables are matrices
// except for prev_layer, that's a pointer to a layer object.
// I'm using Armadillo for linear algebra / matrices
// delta, weight, output, and zprime refer to the current layer.
// prev_delta, prev_weight belong the the previous layer.
if (next_layer == nullptr) {
// if next layer is null, this is the output layer.
// in that case, prev_delta is target.
// yHat - y * R'(Zo)
delta = (output - prev_delta) * zprime;
}
else {
// Eo * Wo * R'(Zh)
delta = prev_delta * prev_weight * zprime;
}
// tell the next layer to backpropogate
if (prev_layer != nullptr)
prev_layer -> backward(delta, weight);
}
matrix * matrix indicates a matrix multiplication (dot product)
matrix % matrix indicates element-wise multiplication
The issue I'm having is that these matrices don't seem to multiply properly. I've made sure everything lines up the same way the article has it, but these pieces just don't seem to fit. How should these matrices be multiplied to get the result?
Edit: to clarify, I get errors when I try to take the dot product of these matrices. "invalid size". I've tried using element wise multiplication but then things get weird there too.

How to use convexHull function of openCV in iOS objective C?

I am someone who is new to openCV and has been trying to use convexhull function in the openCV library for an app (objective-C being used), I need to know what is the input format of function arguments, it's pretty confusing. And does this function return the points in a sequence? Like, if I use addLineToPoint to draw a bezierpath of this hull, is it possible?
Some sample code for you:
std::vector<cv::Point> points;
//fill that vector with your points
std::vector<cv::Point> hull;
if (points.size()) {
cv::convexHull(points, hull);
}
cv::Size size = cv::Size(w, h);
//some size for the matrix where you will draw your convex hull
cv::Mat hullMask = Mat::zeros(size, CV_8UC1);
int hull_count = (int)hull.size();
if (hull_count) {
const cv::Point* hull_pts = &hull[0];
cv::fillPoly(hullMask, &hull_pts, &hull_count, 1, cv::Scalar(255));
}
This code will help you to create convex hull and draw it.
Here you can find complete documentation for that function. It will return points in a sequence according to the "clockwise" argument. By default it will be counter-clockwise.

getting segmentation fault with Point2f

I have extracted some feature points of an image using the following code
vector<Point2f> cornersFrame1;
goodFeaturesToTrack( frame1, cornersFrame1, maxCorners, qualityLevel, minDistance, Mat(), blockSize, useHarrisDetector, k );
After that i want to read the values of present at these feature points. So, i am using the following code:
for(int i=0; i<cornersFrame1.size(); i++)
{
float frame1 = calculatedU.at<float>( cornersFrame1[i].x, cornersFrame1[i].y );
}
then i get Segmentation fault.
But if i use the following code in "For loop" then it work.
float frame1 = calculatedU.at<float>( cornersFrame1[i].y, cornersFrame1[i].x );
I am confused because i think that "Point2f" stores pixel information as (row , col). Isn't it?
No, it is not. All types of points in OpenCV are just normal points that you can think about: (x,y). When it comes to coordinate in image this means that 'x' is a column and 'y' is a row. On the other hand at<> requires as input (row, column). This is why you had to provide (y,x) instead of (x,y).
Just to prevent future confusion, one of the ways of using at<> is this one:
float frame1 = calculatedU.at<float>( cornersFrame1[i] );
This way you don't need to think whether you should provide (x,y) or (y,x).

OpenCV - Image histogram value of pixel

What I am doing is trying to implement an Skin Probability Maps algorithm for skin detection in OpenCV.
I've stuck in a place where I should compare SkinHistValue / NonSkinHistValue probability of each pixel with Theta threshold according to http://www.cse.unsw.edu.au/~icml2002/workshops/MLCV02/MLCV02-Morales.pdf and this tutorial http://www.morethantechnical.com/2013/03/05/skin-detection-with-probability-maps-and-elliptical-boundaries-opencv-wcode/
My problems lies in calculating the coords for hist value:
From the tutorial:
calcHist(&nRGB_frame,1,channels,mask,skin_Histogram,2,histSize,ranges,uniform,accumulate);
calcHist(&nRGB_frame,1,channels,~mask,non_skin_Histogram,2,histSize,ranges,uniform,accumulate);
Calculates the histograms. Than i normalize them.
And after that:
for (int i=0; i<nrgb.rows; i++) {
int gbin = cvRound((nrgb(i)[1] - 0)/range_dist[0] * hist_bins[0]);
int rbin = cvRound((nrgb(i)[2] - low_range[1])/range_dist[1] * hist_bins[1]);
float skin_hist_val = skin_Histogram.at<float>(gbin,rbin);
};
Where nrgb is my image, and im trying to get skin_hist_value for that. But the gbin and rbin are probably calculated wrong and it throws an exception (i run outside of array?) when it comes to
skin_Histogram.at<float>(gbin,rbin);
I have totally no idea how to calculate it correctly. Any help?

Manually make pairwise matching in OpenCV from features key points

Here's my problem. I manually extracted key points features with SURF on multiple images. But I also already know which pair of points are going to match. The thing is, I'm trying to create my matching pairs, but I don't understand how. I tried by looking at the code, but it's a mess.
Right now, I know that the size of the features.descriptors, a matrix, is the same as the number of key points (the other dimension is 1). In the code, to detect matching pairs, it's only using the descriptors, so it's comparing rows (or columns, I'm not sure) or two descriptors matrix and determined if there's anything in common.
But in my case, I already know that there's a match between keypoint i from image 1 and keypoint j from image 2. How do I describe that as a MatchesInfo value. Particularly the element matches of type std::vector< cv::DMatch >.
EDIT: So, for this, I don't need to use any matcher or anything like this. I know which pairs are going together!
If I understood you're question correctly, I assume that you want the keypoint matches in std::vector<cv::DMatch> for the purpose of drawing them with the OpenCV cv::drawMatches or usage with some similar OpenCV function. Since I was also doing matching "by hand" recently, here's my code that draws up arbitrary matches contained originally in a std::vector<std::pair <int, int> > aMatches and displays them in a window:
const cv::Mat& pic1 = img_1_var;
const cv::Mat& pic2 = img_2_var;
const std::vector <cv::KeyPoint> &feats1 = img_1_feats;
const std::vector <cv::KeyPoint> &feats2 = img_2_feats;
// you of course can work directly with original objects
// but for drawing you only need const references to
// images & their corresponding extracted feats
std::vector <std::pair <int, int> > aMatches;
// fill aMatches manually - one entry is a pair consisting of
// (index_in_img_1_feats, index_in_img_2_feats)
// the next code draws the matches:
std::vector <cv::DMatch> matches;
matches.reserve((int)aMatches.size());
for (int i=0; i < (int)aMatches.size(); ++i)
matches.push_back(cv::DMatch(aMatches[i].first, aMatches[i].second,
std::numeric_limits<float>::max()));
cv::Mat output;
cv::drawMatches(pic1, feats1, pic2, feats2, matches, output);
cv::namedWindow("Match", 0);
cv::setWindowProperty("Match", CV_WINDOW_FULLSCREEN, 1);
cv::imshow("Match", output);
cv::waitKey();
cv::destroyWindow("Match");
Alternatively, if you need fuller information about the matches for purposes more complicated than drawing then you might also want to set the distance between matches to a proper value. E.g. if you want to calculate distances using L2 distance, you should replace the following line:
for (int i=0; i < (int)aMatches.size(); ++i)
matches.push_back(cv::DMatch(aMatches[i].first, aMatches[i].second,
std::numeric_limits<float>::max()));
with this (note, for this a reference to feature descriptor vectors is also needed):
cv::L2<float> cmp;
const std::vector <std::vector <float> > &desc1 = img_1_feats_descriptors;
const std::vector <std::vector <float> > &desc2 = img_2_feats_descriptors;
for (int i=0; i < (int)aMatches.size(); ++i){
float *firstFeat = &desc1[aMatches[i].first];
float *secondFeat = &desc2[aMatches[i].second];
float distance = cmp(firstFeat, secondFeat, firstFeat->size());
matches.push_back(cv::DMatch(aMatches[i].first, aMatches[i].second,
distance));
}
Note that in the last snippet, descX[i] is a descriptor for featsX[i], each element of the inner vector being one component of the descriptor vector. Also, note that all descriptor vectors should have the same dimensionality.

Resources