I have a question for you.
I have performed a single and stereo calibration using 10 different checkerboard poses. I have acquired a image pair and obtained the 3D position of each pixel and saved in a point cloud (PCL).
After that, I have performed another calibration using 60 different checkerboard poses. The obtained calibration parameters are different from those estimated in the previous calibration.
I have used the same image pair to obtain the point cloud and get the 3D reconstruction of the scene and I notice that the corresponding 3D points in the two point clouds have different location in the space.
When the two point clouds are displayed in MeshLab, two separate point clouds in the space are represented.
I think that the origin of the "reconstructed space" is changed somehow according to calibration parameters.
How can I get the transformation between the two different coordinates system so that, known the transformation, I can display the second and the first point clouds overlapping?
The aim is to find this relationship using only stereo-calibration parameters. I know that the transformation could be computed using the correspondences between the same points displayed in the two point clouds, but I need to find out this relationship using only the calibration parameters.
Thanks!
I have four cameras which are at fixed position. So I can measure the distance (even rotation) among them using a ruler (physically). Camera one and two gives me a point cloud and camera three and four gives me another point cloud. I need to merge these point clouds.
As far I have understood that ICP and other such algorithms do a rigid transformation of one point cloud to match the other. My question is how can I use the extra knowledge (distance between the cameras in centimeter) to do this transformation.
I am quite new to such work so, please correct me if I misunderstood something. And thanks in advance.
First, what kind of accuracy are you looking for, and over what volume of space? Achieving 0.1 mm registration accuracy over a 0.5 m tabletop scene is a completely different problem (in terms of mechanical design and constraints) than a few mm over a floor tens of meters wide.
Generally speaking, with a well reconstructed and unambiguous object shape, ICP will always give you a better solution than manual measurements.
If the cameras are static, then what you have is really a calibration problem, and you need to calibrate your 4-camera rig only at setup and when its configuration changes for whatever reason.
I suggest using a calibration object of precisely known size and geometry, e.g. a machined polyhedron. You can generate and ICP-register point clouds for it, then fit the merged cloud to the known geometry, thus obtaining position and orientation of every individual point cloud with respect to the fixed object. From these you can work out the poses of the cameras w.r.t. each other.
I was comparing Kinect V2 with my own ToF sensor and I found a different place.
Below is a point cloud with RGB information produced by Kinect V2, which is placed in front of a object, and the object is placed between two walls. This means the two walls are parallel to Kinect V2's view line.
If I drag the point cloud, you can see the point cloud of two walls in the .ply file are parallel with each other and parallel with the Kinect' view line.
Look from top of the point cloud
However, if I use my own ToF sensor to catch the point cloud under same environment and same view(please ignore the difference of the object in the middle, it has been changed), the point cloud looks like this(color camera hasn't been implemented)
The left wall(red circle area) somehow distorted like a "/"(The right wall cannot visualize due to my sensor's FOV)
I was confused by this phenomenon, I pretty sure Kinect V2 did some processes to fix this issue, but I cannot figure it out.
Can someone give me some clues about the scene I saw?
If there is any further information need to be provided, feel free to ask.
TOF cameras suffer from many systematic errors from both sensor and illumination unit which result in inhomogeneus lighting for example.
I did a detailed study on this topic that can be find at this address:
https://arxiv.org/pdf/1505.05459.pdf
The error gets a lot worth on the border of your frame, as the active illumination has never:
same fov as camera
same power on center and border of the frame
For your case I think the problem is happening at the border where the measured distance gets extremely unlinear for the border pixels. Check figure 5 and 15 in the paper.
Possible solutions:
Try to remain between 1 and 3 meter distance.
Concentrate more on the central 2/3 of the frame.
I have a photocamera mounted vertically under water in a tank, looking downwards.
There is a flat grid on the bottom of the tank (approx 2m away from the camera).
I want to be able to place markers on the bottom, and use computer vision to know their real life exact position.
So, I need to map from pixels to mm.
If I am not mistaken, cv::calibrateCamera(...) does just this, but is dependent on moving a pattern in front of the camera.
I have just static pictures of the scene, and the camera never moves in relation to the grid. Thus, I have only a "single" image to find the parameters.
How can I do this using the grid?
Thank you.
Interesting problem! The "cute" part is the effect on the intrinsic parameters of the refraction at the water-glass interface, namely to increase the focal length (or, conversely, to reduce the field of view) compared to the same lens in air. In theory, you could calibrate in air and then correct for the difference in refraction index, but calibrating directly in water is likely to give you more accurate results.
Do know your accuracy requirements? And have you verified that your lens/sensor combination is adequate to meet them (with an adequate margin)? To answer the question you need to estimate (either by calculation from the lens and sensor specifications, or experimentally using a resolution chart) whether you can resolve in an image the minimal distances required by your application.
From the wording of your question I think that you are interested only in measurements on a single plane. So you only need to (a) remove the nonlinear (barrel or pincushion) lens distortion and (b) estimate the homography between the plane of interest and the image. Once you have the latter, you can directly convert from undistorted image coordinates to world ones by matrix multiplication. Additionally if (as I imagine) the plane of interest is roughly parallel to the image plane, you should not have any problem keeping the entire field-of-view in focus.
Of course, for all of this to work as expected, you should make sure that the tank bottom is really flat, within the measurement tolerances of your application. Otherwise you are really dealing with a 3D problem, and need to modify your procedures accordingly.
The actual procedure depends a lot on the size of the tank, which you don't indicate clearly. If it's small enough that it is practical to manufacture a chessboard-like movable calibration target, by all means go for it. You may want to take a look at this other answer for suggestions. In the following I'll discuss the more interesting case in which your tank is large, e.g. the size of a swimming pool.
I'd proceed by sticking calibration markers in a regular grid at the pool bottom. I'd probably choose checker-like markers like these, maybe printing them myself with a good laser printer on plastic with an adhesive backing (assuming you can leave them in place forever). You should plan on having quite a few of them, say, an 8x8 or 10x10 grid, covering as much as possible of the field of view of the camera in its operating position and pose. To help with lining up the grid nicely you might use a laser line projector of suitable fan angle, or a laser pointer attached to a rotating support. Note carefully that it is not necessary that they be affixed in a precise X-Y grid (which may be complicated, depending on the size of your pool), only that their positions with respect any arbitrarily chosen (but fixed) three of them be known. In other words, you can attach them to the bottom approximately in a grid, then measure the distances of three extreme corners from each other as accurately as you can, thus building a base triangle, then measure the distances of all the other corners from the vertices of the triangle, and finally reconstruct their true positions with a bit of trigonometry. It's basically a surveying problem and, depending on your accuracy requirements and budget, you may want to enroll a local friendly professional surveyor (and their tools) to get it done as precisely as necessary.
Once you have your grid, you can fill the pool, get your camera, focus and f-stop the lens as needed for the application. From now on you may not touch the focus and f-stop ever again, under penalty of miscalibrating - exposure can only be controlled by the exposure time, so make sure to have enough light. Disable any and all auto-focus and auto-iris functions, if any. If the camera has a non-rigid lens mount (e.g. a DLSR), you'll need some kind of mechanical rig to ensure that the lens-body pair stay rigid. F-stop as close as you can, given the available lighting and sensor, so to have a fair bit of depth of field available. Then take several photos (~ 10) of the grid, moving and rotating the camera, and going a bit closer and farther away than your expected operating distance from the plane. You'll want to "see" in some images some significant perspective foreshortening of the grid - this is needed to accurately calibrate the focal length. Avoid JPG and any other lossy compression format when storing the images - use lossless PNG or TIFF.
Once you have the images, you can manually mark and identify the checker markers in the images. For a once-off project like this I would not bother with automatic identification, just do it manually (e.g. in Matlab, or even in Photoshop or Gimp). To help identify the markers, you could, e.g. print a number next to them. Once you have the manual marks, you can refine them automatically to subpixel accuracy, e.g. using cv::findCornerSubpix.
You're almost done. Feed the "reference" measured position of the real corners, and the observed ones in all images, to your favorite camera calibration routine, e.g. cv::calibrateCamera. You use the nominal focal length of the camera (converted to pixels) for an initial estimate, along with null distortion. If all goes well, you will obtain the camera intrinsic parameters, which you will keep, and the camera poses at all images, which you'll throw away.
Now you can mount the camera in your final setup, as needed by your application, and take one further image of the grid. Mark and refine the corner positions as before. Undistort their image positions using the distortion parameters returned by the calibration. Finally compute the homography between the reference positions of the real markers (in meters) and their undistorted positions, and you're done.
HTH
To calibrate the camera you do need multiple images of the checkerboard (or one of the other patterns found here). What you can do, is calibrate the camera outside of the water or do a calibration sequence once.
Once you have that information (focal length, center of lens, distortion, etc). You can use the solvePNP function to estimate the orientation of a single board. This estimation provides you with a distance from the camera to the board.
A completely different alternative could be to find what kind of lens the camera uses and manually fill in the data. I've not tried this, so I'm uncertain how well this would work.
I would like to locate a car (front center point x,y) using a high resolution single camera. The camera setup is fixed at 1-2m high, and tilted around 25 degrees. The camera can provide images in where the front side of the car is visible. The intrinsic and extrinsic parameters are known.
So far, I tried to detect the headlights and number plates. Issues... Headlights are not detected as blobs all the time. The shape of the headlights are changing depending on the distance. Also, the number plate is not visible in the dark.
Is there a robust algorithm to detect a car? or to detect headlights? or detect number plate?How could I proceed?
Thanks in advance,
Are you detecting the same car everytime? If yes, then presumably the appearance remains consistent. Rather than detect and recognise blobs and shapes, you may be better off using scale and rotation invariant features combined with a machine learning algorithm. Look into the SIFT and SURF feature descriptors. For easy experimentation, use OpenCV's implementation of feature description and matching. Take a look at this example.
This is not an easy problem because of the change in the scale and point of view. Ideally, you would need a collection of training images with the car seen from different points of view to match later some of them to your input image. Then, you need local features (SIFT, SURF) or some classifier to decide on the match.
On the other hand, if you are tracking the same car all the time, check out the MeanShift algorithm. The problem is you need an initial position to carry on with the tracking.