In LATCH_match.cpp in opencv_3.1.0 the homography matrix is defined and used as:
Mat homography;
FileStorage fs("../data/H1to3p.xml", FileStorage::READ);
...
fs.getFirstTopLevelNode() >> homography;
...
Mat col = Mat::ones(3, 1, CV_64F);
col.at<double>(0) = matched1[i].pt.x;
col.at<double>(1) = matched1[i].pt.y;
col = homography * col;
...
Why H1to3p.xml is:
<opencv_storage><H13 type_id="opencv-matrix"><rows>3</rows><cols>3</cols><dt>d</dt><data>
7.6285898e-01 -2.9922929e-01 2.2567123e+02
3.3443473e-01 1.0143901e+00 -7.6999973e+01
3.4663091e-04 -1.4364524e-05 1.0000000e+00 </data></H13></opencv_storage>
With which criteria these numbers were chosen? They can be used for any other homography test for filtering keypoints (as in LATCH_match.cpp)?
I assume that your "LATCH_match.cpp in opencv_3.1.0" is
https://github.com/Itseez/opencv/blob/3.1.0/samples/cpp/tutorial_code/xfeatures2D/LATCH_match.cpp
In that file, you find:
// If you find this code useful, please add a reference to the following paper in your work:
// Gil Levi and Tal Hassner, "LATCH: Learned Arrangements of Three Patch Codes", arXiv preprint arXiv:1501.03719, 15 Jan. 2015
And so, looking at http://arxiv.org/pdf/1501.03719v1.pdf you will find
For each set, we compare the first image against each of the remaining
five and check for correspondences. Performance is measured using the
code from [16, 17]1 , which computes recall and 1-precision
using known ground truth homographies between the images.
I think that the image ../data/graf1.png is https://github.com/Itseez/opencv/blob/3.1.0/samples/data/graf1.png that I show here:
According to the comment Homography matrix in Opencv? by Catree the original dataset is at http://www.robots.ox.ac.uk/~vgg/research/affine/det_eval_files/graf.tar.gz where it is said that
Homographies between image pairs included.
So I think that the homography stored in file ../data/H1to3p.xml is the homography between image 1 and image 3.
Related
I am simulating one paper that I have read. They have visualized 3D structure Tensor of knee MRI as you can see on the following image:
Visualization of structure tensor
I surf Internet a lot, but I could not find proper response for my questions. I have computated eigenvalues and eigenvectors of 3D structure Tensor as follows:
function [mu3,mu2,mu1,v3x,v3y,v3z,v2x,v2y,v2z,v1x,v1y,v1z]=EigenVectors3DTensor(G1x.^2, G1x.*G1y, G1x.*G1z, G1y.^2, G1y.*G1z, G1z.^2)
% first derivative of each voxel in three directions (x,y,z) have been saved and vectorized in G1x, G1y and G1z, respectively.
Dxx=G1x.^2
v1x=zeros(size(Dxx),'single');
v1y=zeros(size(Dxx),'single');
v1z=zeros(size(Dxx),'single');
v2x=zeros(size(Dxx),'single');
v2y=zeros(size(Dxx),'single');
v2z=zeros(size(Dxx),'single');
v3x=zeros(size(Dxx),'single');
v3y=zeros(size(Dxx),'single');
v3z=zeros(size(Dxx),'single');
mu1=zeros(size(Dxx),'single');
mu2=zeros(size(Dxx),'single');
mu3=zeros(size(Dxx),'single');
rho=1.1; %sigma out
for i=1:numel(Dxx)
Jxx = imgaussian(Dxx(i),rho,4*rho);
Jxy = imgaussian(Dxy(i),rho,4*rho);
Jxz = imgaussian(Dxz(i),rho,4*rho);
Jyy = imgaussian(Dyy(i),rho,4*rho);
Jyz = imgaussian(Dyz(i),rho,4*rho);
Jzz = imgaussian(Dzz(i),rho,4*rho);
M=[Jxx Jxy Jxz; Jxy Jyy Jyz; Jxz Jyz Jzz];
[v,d]=eig(M);
v=v';
ev1=d(1,1); ev2=d(2,2); ev3=d(3,3);
ev1a=abs(ev1); ev2a=abs(ev2); ev3a=abs(ev3);
if((ev1a>=ev2a)&&(ev1a>ev3a))
d=ev3; dt=ev3a; dat=v(3,:);
ev3=ev1; v(3,:)=v(1,:);
ev1=d; ev1a=dt; v(1,:)=dat;
elseif((ev2a>=ev1a)&&(ev2a>ev3a))
d=ev3; dt=ev3a; dat=v(3,:);
ev3=ev1; v(3,:)=v(2,:);
ev2=d; ev2a=dt; v(2,:)=dat;
end
if(ev1a>ev2a)
d=ev2; dat=v(2,:);
ev2=ev1; v(2,:)=v(1,:);
ev1=d; v(1,:)=dat;
end
mu1(i)=ev1; mu2(i)=ev2; mu3(i)=ev3;
v1x(i)=v(1,1); v1y(i)=v(1,2); v1z(i)=v(1,3);
v2x(i)=v(2,1); v2y(i)=v(2,2); v2z(i)=v(2,3);
v3x(i)=v(3,1); v3y(i)=v(3,2); v3z(i)=v(3,3);
end
I have two major questions:
1) Is the calculation of eigenvalues and eigenvectors of structure tensor correct?
2) How can I visualize the structure tensor on my scans (stack of 2D images)?
I will really appreciate if someone help me.
I am trying to find a simple algorithm to find the correspondence between two sets of 2D points (registration). One set contains the template of an object I'd like to find and the second set mostly contains points that belong to the object of interest, but it can be noisy (missing points as well as additional points that do not belong to the object). Both sets contain roughly 40 points in 2D. The second set is a homography of the first set (translation, rotation and perspective transform).
I am interested in finding an algorithm for registration in order to get the point-correspondence. I will be using this information to find the transform between the two sets (all of this in OpenCV).
Can anyone suggest an algorithm, library or small bit of code that could do the job? As I'm dealing with small sets, it does not have to be super optimized. Currently, my approach is a RANSAC-like algorithm:
Choose 4 random points from set 1 and from set 2.
Compute transform matrix H (using openCV getPerspective())
Warp 1st set of points using H and test how they aligned to the 2nd set of points
Repeat 1-3 N times and choose best transform according to some metric (e.g. sum of squares).
Any ideas? Thanks for your input.
With python you can use Open3D librarry, wich is very easy to install in Anaconda. To your purpose ICP should work fine, so we'll use the classical ICP, wich minimizes point-to-point distances between closest points in every iteration. Here is the code to register 2 clouds:
import numpy as np
import open3d as o3d
# Parameters:
initial_T = np.identity(4) # Initial transformation for ICP
distance = 0.1 # The threshold distance used for searching correspondences
(closest points between clouds). I'm setting it to 10 cm.
# Read your point clouds:
source = o3d.io.read_point_cloud("point_cloud_1.xyz")
target = o3d.io.read_point_cloud("point_cloud_0.xyz")
# Define the type of registration:
type = o3d.pipelines.registration.TransformationEstimationPointToPoint(False)
# "False" means rigid transformation, scale = 1
# Define the number of iterations (I'll use 100):
iterations = o3d.pipelines.registration.ICPConvergenceCriteria(max_iteration = 100)
# Do the registration:
result = o3d.pipelines.registration.registration_icp(source, target, distance, initial_T, type, iterations)
result is a class with 4 things: the transformation T(4x4), 2 metrict (rmse and fitness) and the set of correspondences.
To acess the transformation:
I used it a lot with 3D clouds obteined from Terrestrial Laser Scanners (TLS) and from robots (Velodiny LIDAR).
With MATLAB:
We'll use the point-to-point ICP again, because your data is 2D. Here is a minimum example with two point clouds random generated inside a triangle shape:
% Triangle vértices:
V1 = [-20, 0; -10, 10; 0, 0];
V2 = [-10, 0; 0, 10; 10, 0];
% Create clouds and show pair:
points = 5000
N1 = criar_nuvem_triangulo(V1,points);
N2 = criar_nuvem_triangulo(V2,points);
pcshowpair(N1,N2)
% Registrate pair N1->N2 and show:
[T,N1_tranformed,RMSE]=pcregistericp(N1,N2,'Metric','pointToPoint','MaxIterations',100);
pcshowpair(N1_tranformed,N2)
"criar_nuvem_triangulo" is a function to generate random point clouds inside a triangle:
function [cloud] = criar_nuvem_triangulo(V,N)
% Function wich creates 2D point clouds in triangle format using random
% points
% Parameters: V = Triangle vertices (3x2 Matrix)| N = Number of points
t = sqrt(rand(N, 1));
s = rand(N, 1);
P = (1 - t) * V(1, :) + bsxfun(#times, ((1 - s) * V(2, :) + s * V(3, :)), t);
points = [P,zeros(N,1)];
cloud = pointCloud(points)
end
results:
You may just use cv::findHomography. It is a RANSAC-based approach around cv::getPerspectiveTransform.
auto H = cv::findHomography(srcPoints, dstPoints, CV_RANSAC,3);
Where 3 is the reprojection threshold.
One traditional approach to solve your problem is by using point-set registration method when you don't have matching pair information. Point set registration is similar to method you are talking about.You can find matlab implementation here.
Thanks
i'm trying to remove '5 lines' in section, in music papers, my original image is this : http://en.wikipedia.org/wiki/Requiem_(Mozart)#/media/File:K626_Requiem_Mozart.jpg
First, i apply gaussian filter and binarized with threshold (min:100, max 255).
Then applying dft to this image, erase some appropriate lines, and reconstruct image by inverse dft.
i use sample code in opencv documentation, actually i doubt myself that i understand this code. :(
http://docs.opencv.org/doc/tutorials/core/discrete_fourier_transform/discrete_fourier_transform.html
in this sample code, there's 2 Mats. one is 'complexI' for spectrum, another is 'magI' for actual visualized. the result of cv::dft is complexI, magI is normalized complexI. my question is this. how can i add a black line(to cancel in freq domain) and reconstruct?
OpenCV (now) provides a detailed tutorial on how to deal with periodic noise by spectral filtering: https://docs.opencv.org/trunk/d2/d0b/tutorial_periodic_noise_removing_filter.html
It hinges on using cv::dft(), cv::idft(), cv::mulSpectrums(), and cv::magnitude().
The core function (from the tutorial) to perform the filtering goes like so:
void filter2DFreq(const Mat& inputImg, Mat& outputImg, const Mat& H)
{
Mat planes[2] = { Mat_<float>(inputImg.clone()), Mat::zeros(inputImg.size(), CV_32F) };
Mat complexI;
merge(planes, 2, complexI);
// find FT of image
dft(complexI, complexI, DFT_SCALE);
Mat planesH[2] = { Mat_<float>(H.clone()), Mat::zeros(H.size(), CV_32F) };
Mat complexH;
merge(planesH, 2, complexH);
Mat complexIH;
// apply spectral filter
mulSpectrums(complexI, complexH, complexIH, 0);
// reconstruct the filtered image
idft(complexIH, complexIH);
split(complexIH, planes);
outputImg = planes[0];
}
Refer to the tutorial for more information.
I am trying to extract Rotation matrix and Translation vector from the essential matrix.
<pre><code>
SVD svd(E,SVD::MODIFY_A);
Mat svd_u = svd.u;
Mat svd_vt = svd.vt;
Mat svd_w = svd.w;
Matx33d W(0,-1,0,
1,0,0,
0,0,1);
Mat_<double> R = svd_u * Mat(W).t() * svd_vt; //or svd_u * Mat(W) * svd_vt;
Mat_<double> t = svd_u.col(2); //or -svd_u.col(2)
</code></pre>
However, when I am using R and T (e.g. to obtain rectified images), the result does not seem to be right(black images or some obviously wrong outputs), even so I used different combination of possible R and T.
I suspected to E. According to the text books, my calculation is right if we have:
E = U*diag(1, 1, 0)*Vt
In my case svd.w which is supposed to be diag(1, 1, 0) [at least in term of a scale], is not so. Here is an example of my output:
svd.w = [21.47903827647813; 20.28555196246256; 5.167099204708699e-010]
Also, two of the eigenvalues of E should be equal and the third one should be zero. In the same case the result is:
eigenvalues of E = 0.0000 + 0.0000i, 0.3143 +20.8610i, 0.3143 -20.8610i
As you see, two of them are complex conjugates.
Now, the questions are:
Is the decomposition of E and calculation of R and T done in a right way?
If the calculation is right, why the internal rules of essential matrix are not satisfied by the results?
If everything about E, R, and T is fine, why the rectified images obtained by them are not correct?
I get E from fundamental matrix, which I suppose to be right. I draw epipolar lines on both the left and right images and they all pass through the related points (for all the 16 points used to calculate the fundamental matrix).
Any help would be appreciated.
Thanks!
I see two issues.
First, discounting the negligible value of the third diagonal term, your E is about 6% off the ideal one: err_percent = (21.48 - 20.29) / 20.29 * 100 . Sounds small, but translated in terms of pixel error it may be an altogether larger amount.
So I'd start by replacing E with the ideal one after SVD decomposition: Er = U * diag(1,1,0) * Vt.
Second, the textbook decomposition admits 4 solutions, only one of which is physically plausible (i.e. with 3D points in front of the camera). You may be hitting one of non-physical ones. See http://en.wikipedia.org/wiki/Essential_matrix#Determining_R_and_t_from_E .
I try to compute gradient map using HOGDescriptor.
My code:
HOGDescriptor hog;
hog.compute(faceROI,ders,Size(32,32),Size(0,0),locs);
Mat grad;
Mat sec;
hog.computeGradient(frame_gray, grad, angleofs);
imshow("1", frame_gray);
imshow("2", grad); //here program fails: Unhandled exception at memory location
imshow("3", angleofs); //grad.data = "". Why??
I cant find goot examples of using HOGDescriptor::computeGradient.
Help please!
To visualize OpenCv's HOGDescriptor::Calculate(..), use this, it's amazing.
imshow("2", grad); fails because imshow expects that grad image is a 1, 3 or 4 channel image whereas it is a 2 channel image.
First channel contains the gradient in x direction whereas the second channel contains the gradient in y. You should split the channels in two images to visualize them:
Mat grad_channel[2];
split(grad, grad_channel);
imshow("grad_x", grad_channel[0]);
imshow("grad_y", grad_channel[1]);
Best