I am trying to implement an application that projects an image onto a page of a notebook, using OpenCV, a webcam and a projector. To achieve that, I am doing the following steps:
I am using a webcam to detect the four corners points of a page.
A homography is learned between the four corner points of the camera image and their projections on my desk, as seen in the camera. By using the inverse transformation, I will be able to know where I should draw something on my camera image, so that the projection "ends up" at a desired location.
I am applying the inverse transformation to the detected four corners points of the page.
I am warping the desired image to the new, transformed set of points.
So far it works well, if the notebook is on my desk and wide open. Like in this picture:
But if I try to close one side (or both), the following happens:
See the problem? In the first picture the image is perfectly aligned with the edges of the page and remains so if you rotate or translate the notebook, while keeping it on the desk. But that doesn't happen in the second image, where the the top edge of the image is no longer parallel to the top edge of the page (the image becomes more and more skewed).
Can anyone explain why I get this projection problem or at least point me to some resources where I can read about it? I need to mention that the projector and the webcam are placed above and to the left of the notebook, not right above them.
Any tips or suggestions are welcome. Thank you!
You want an effect that is called a key stone correction. The problem you are experiencing is most probably due to the fact that optical axes, positions, and focal lengths of a web camera and a projector are different. I suggest to calibrate your setup so you would know their relative pose and incorporate it in your inverse Homography.
Related
I'm fairly new to Computer Vision and OpenCV. I'm working on a project in which we are building a robot that plays snooker. We installed a camera on top of the table, with the aim of detecting the balls. I originally assumed that getting rid of the barrel distortion would be quite straight forward. However, I'm not very satisfied with the results I'm getting. In the second image I attached, one can clearly see, that after applying the undistortion transformation, the sides of the table are not parallel to each other. Moreover, with respect to the first image, the balls are deformed into a sort of egg shape.
Does anyone have an idea of how I could fix this issues? I tried to take more picture of the chessboard patter is as many different positions as possible, without any visible changes. Also using more parameters to model the distortion didn't seem to yield any improvements.
Distorted Image
Undistorted Image
I have two questions which could be related:
1.) I would like to estimate distances between objects which are positioned in one plane from a photo. Geometrical shape of one object in the photo is rectangular and its dimensions are known, but there is no information on the photo (Camera focal length, photo angle, senor size etc…). For example, say I have the following PCB photo and dimensions of the rectangular chip are known to be 20x10mm, all objects lie in a plane. Is it even possible to estimate the distances (in top view) between other PCB components ?
In this particular case, maximum distance error of 2-3mm would be acceptable.
2.) Say I have similar PCB photo like the above, where I have one feature (object) for which I know it is rectangular shaped. I would like to transform the image perspective so that the object looks rectangular. I have tried imageJ (Fiji) and Interactive Perspective Plugin for this task. First I display rectangular grid over the image and then manually transform the image using the plugin till the object does not appear rectangular. But for some photo angles I find it impossible to manually adjust the control points in order to get rectangular object shape.
Does somebody know alternative approach using imageJ (Fiji) or Octave ? A solution in python would also be ok, although I don’t have much python experience (just recently installed Anaconda with Spyder).
A few years ago, I created a software that seems good for you. It corrects perspective transforming a quadrilateral to a rectangle.
Here is the result:
,
where you can measure distances.
What will be the procedure to correct the following distorted images ? It looks like the images are bulging out from center. These are of the same QR code, and so a combination of such images can be used to arrive at a single correct and straight image.
Please advice.
The distortion you are experiencing is called "barrel distortion". A technical name is "combination of radial distortion and tangential distortions"
The solution for your problem is openCV camera calibration module. Just google it and you will find documentations in openCV wiki. More over, openCV already has built in source code examples of how to calibrate the camera.
Basically, You need to print an image of a chess board, take a few pictures of it, run the calibration module (built in method) and get as output transformation matrix. For each video frame you apply this matrix (I think the method called cvUndistort()) and it will straighten the curved lines in the image.
Note: It will not work if you change the zoom or focal length of the camera.
If camera details are not available and uncontrollable - then your problem is very serious. There is a way to solve the distortion, but I don't know if openCV has built in modules for that. I am afraid that you will need to write a lot of code.
Basically - you need to detect as much as possible long lines. Then from those lines (vertical and horizontal) you build a grid of intersection points. Finally you fit the grid of those points to openCV calibration module.
If you have enough intersection points (say 20 or more) you will be able to calculate the distortion matrix and un-distort the image.
You will not be able to fully calibrate the camera. In other words, you will not be able to run a one time process that calculates the expected distortion. Rather - in each and every video frame, you will calculate the distortion matrix directly - reverse it and un-distort the image.
If you are not familiar with image processing techniques or unable to find a reliable open source code which directly solves your problem - then I am afraid that you will not be able to remove the distortion. sorry
I am a newbie to OpenCV. I would like to work on a small project for tracking the rotation speed of a gear (by using webcam). However, until now, I have no idea how to work on this.
The posted image shows a machine which contains two 'big' gears. What I am interested in the gear only on left hand side (the red line as I highlighted).
Link
My plan is:
Extract the Interested gear region .
Mask all unrelated region. So, the masked image shows the left gear only (ROI).
.....
The problem is, how can I locate/extract/mask the ROI and mask?.I go through some example about cvMatchTemplate(), but it doesn't support rotations and scalings. Due to using webcam, the captured image may scaled or rotated. cvfindcontour() will extract all contours in the image rather then ROI.
If you previously know the gear you can use a picture of it to extract keypoints with SIFT, SURF, FAST or any corner detection algorithm. Then do as follows:
1- Apply FAST on every frame to detect keypoints.
2- Extract SIFT descriptors from those keypoints
3- Match detected points in the scene with your previously extracted points from the image. You can use FLANN matcher for this.
4- Those matches will define a region in the scene containing the gear you are looking for.
This is not trivial so you will need to look for information in OpenCV documentation for using all these functions.
I intend to make a program which will take stereo pair images, taken by a single camera, and then correct and crop them so that when the images are viewed side by side with the parallel or cross eye method, the best 3D effect will be achieved. The left image will be the reference image, the right image will be modified for corrections. I believe OpenCV will be the best software for these purposes. So far I believe the processing will occur something like this:
Correct for rotation between images.
Correct for y axis shift.
Doing so will I imagine result in irregular black borders above and below the right image so:
Crop both images to the same height to remove borders.
Compute stereo-correspondence/disparity
Compute optimal disparity
Correct images for optimal disparity
Okay, so that's my take on what needs doing and the order it occurs in, what I'm asking is, does that seem right, is there anything I've missed, anything in the wrong order etc. Also, which specific functions of OpenCV would I need to use for all the necessary steps to complete this project? Or is OpenCV not the way to go? Much thanks.
OpenCV is great for this.
There is a whole chapter in:
And all the sample code for this in the book ships with the opencv distribution
edit: Roughly the steps are:
Remap each image to remove lens distortions and rotate/translate views to image center.
Crop pixels that don't appear in both views (optional)
Find matching objects in each view (stereoblock matching) create disparity map
Reproject disparity map into 3D model