I have a set of images of a scene at different angles and the camera intrinsic parameters. I was able to generate the 3D points using point correspondences and triangulation. Is there a built-in method or a way to produce 3D images in MATLAB? From the given set of images, a 3D image as such? By 3D image, I mean a 3D view of the scene based on the colors, depth, etc.?
There was a recent MathWorks blog post on 3D surface rendering, which showed methods using built-in functions and using contributed code.
The built-in method uses isosurface and patch. To display multiple surfaces, the trick is to set the 'FaceAlpha' patch property to get transparency.
The post also highlighted the "vol_3d v2" File Exchange submission, which provides the vol3d function to render the surface. It allows explicit definition of voxel colors and alpha values.
Some file exchange from mathworks:
3D CT/MRI images interactive sliding viewer, 3D imshow, image3, and viewer3D.
If your images matrix I has the dimension of x*y*z, you can try surface as well:
[X,Y] = meshgrid(1:size(I,2), 1:size(I,1));
Z = ones(size(I,1),size(I,2));
for z=1:size(I,3)
k=z*sliceInterval;
surface('XData',X, 'YData',Y, 'ZData',Z*k,'CData',I(:,:,z), 'CDataMapping','direct', 'EdgeColor','none', 'FaceColor','texturemap')
end
The Computer Vision System Toolbox for MATLAB includes a function called estimateUncalibratedRectification, which you can use to rectify a pair of stereo images. Check out this example of how to create a 3-D image, which can be viewed with a pair of red-cyan stereo glasses.
Related
I have an IR and RGB camera set up such that they are 2cm apart from each other (horizontally).
My goal is to fuse both the images (RGB and IR) to obtain a more informative image. Because of the offset between the RGB and the IR data, both the images do not match. Is there a way to align the images so that they are on the same coordinate system.
I would suggest to perform an edge detection (or something similar to reduce the amount of information in both images), and then to make a wrap of one image onto the other.
If you are using OpenCV, I suggest using findTransformECC() method (doc) to find the transformation matrix needed to get the best overlay between your images.
Than you can warp the second image using this transformation matrix with warpAffine() method (doc).
I'm using OpenCV to extract 3D model from multiple views (images).
I got to a stage where the inputs are two images and the results are 3D points that reconstruct the 3D model.
I want to combine more than two images for a full reconstruction from all views.
So I have a 3D points described images 1-2 and a 3D points described images 2-3.
How can I merge the two 3D points arrays to a fully 3D model.
any suggestions?
It is not as simple as transforming the second pc to the frame where the first one is at, since the two frames are independent. There is a scale ambiguity. If you are lucky enough, you might see the merger works fine after some bundle adjustment.
Assuming you are doing the 3D reconstruction using a stereo matching algorithm, the 3D reconstruction between images 1-2 results in 3D points in the coordinate system of image 1. Similarly, the 3D reconstruction between images 2-3 results in 3D points in the coordinate system of image 2.
Hence you simply have to change the 3D coordinate system of your second point cloud, from the one of image 2 to the one of image 1. This makes use of the rotation matrix R and the translation vector T between images 1-2.
EDIT: Note that this way of merging the two point clouds is very basic, and you could improve accuracy by doing a joint 3D reconstruction using images 1-2-3 at once (e.g. bundle adjustment). I don't think that this approach is available in OpenCV though.
I have started on a project to create linear/ strip panorama's of long scenes using video. This meaning that the panorama doesn't revolve around a center but move parallel to a scene eg. vid cam mounted on a vehicle looking perpendicular to the street facade.
The steps I will be following are:
capture frames from video
Feature detection - (SURF)
Feature tracking (Kanade-Lucas-Tomasi)
Homography estimation
Stitching Mosaic.
So far I have been able to save individual frames from video and complete SURF feature detection on only two images. I am not asking for someone to solve my entire project but I am stuck trying complete the SURF detection on the remaing frames captured.
Question: How do I apply SURF detection to successive frames? Do I save it as a YAML or xml?
For my feature detection I used OpenCV's sample find_obj.cpp and just changed the images used.
Has anyone experienced such a project? An example of what I would like to achieve is from Iwane technologies http://www.iwane.com/en/2dpcci.php
While working on a similar project, I created an std::vector of SURF keypoints (both points and descriptors) then used them to compute the pairwise matchings.
The vector was filled while reading frame-by-frame a movie, but it works the same with a sequence of images.
There are not enough points to saturate your memory (and use yml/xml files) unless you have very limited resources or a very very long sequence.
Note that you do not need the feature tracking part, at least in most standard cases: SURF descriptors matching can also provide you an homography estimate (without the need for tracking).
Reading to a vector
Start by declaring a vector of Mat's, for example std::vector<cv::Mat> my_sequence;.
Then, you have two choices:
either you know the number of frames, then you resize the vector to the correct size. Then, for each frame, read the image to some variable and copy it to the correct place in the sequence, using my_sequence.at(i) = frame.clone(); or frame.copyTo(my_sequence.at(i));
or you don't know the size beforehand, and you simply call the push_back() method as usual: my_sequence.push_back(frame);
I have a set of 3-d points and some images with the projections of these points. I also have the focal length of the camera and the principal point of the images with the projections (resulting from previously done camera calibration).
Is there any way to, given these parameters, find the automatic correspondence between the 3-d points and the image projections? I've looked through some OpenCV documentation but I didn't find anything suitable until now. I'm looking for a method that does the automatic labelling of the projections and thus the correspondence between them and the 3-d points.
The question is not very clear, but I think you mean to say that you have the intrinsic calibration of the camera, but not its location and attitude with respect to the scene (the "extrinsic" part of the calibration).
This problem does not have a unique solution for a general 3d point cloud if all you have is one image: just notice that the image does not change if you move the 3d points anywhere along the rays projecting them into the camera.
If have one or more images, you know everything about the 3D cloud of points (e.g. the points belong to an object of known shape and size, and are at known locations upon it), and you have matched them to their images, then it is a standard "camera resectioning" problem: you just solve for the camera extrinsic parameters that make the 3D points project onto their images.
If you have multiple images and you know that the scene is static while the camera is moving, and you can match "enough" 3d points to their images in each camera position, you can solve for the camera poses up to scale. You may want to start from David Nister's and/or Henrik Stewenius's papers on solvers for calibrated cameras, and then look into "bundle adjustment".
If you really want to learn about this (vast) subject, Zisserman and Hartley's book is as good as any. For code, look into libmv, vxl, and the ceres bundle adjuster.
I am new to OpenCV. I have got the Surf Detection sample working. Now I want to place a 3d model on the detected image.
How can I find the 3d Projection matrix?
I guess you are talking about Augment Reality as you say you want to place a 3D model on the detected image (in the camera frame? ). They key of the problem is always to detect at least 4 points that match other 4 "keypoints" in our marker. Then, solving some equations we will get our homography, which will allow us to project any point.
In OpenCV there is a function that performs this task: cvFindHomography
You just need the pairs of matches, select a method (RANSAC, i.e.) and you will get the Homography.
Then you can project the points like explained here: