I captured raw magnetometer data on the iPhone 5 by accident, and I actually require the calibrated data. The problem is that I can't go and recapture the data I originally got. Does anyone know what the iPhone's hard bias (device bias) calibration values are and how I can apply them to my data to get a similar output to what the iPhone would have given me?
Alternatively what is the best approach to calibrate for the device bias? I don't care about soft bias in my measurements.
Thanks
The iPhone needs calibration data not to calibrate for the internal sensor (that is always accounted for, even in "raw" data, which actually isn't as raw as you might think). It is actually to calibrate for external factors that might disrupt or interfere with the Earth's natural magnetic field, like high voltage power lines, or steel beams overhead. The iPhone creates a 3D distortion map of the field (which is why the compass app asks you to make a figure eight) to offset these external influences.
Finally, even if you could recreate the exact distortions, Apple provides no way to peek into their black-box filtering, let alone apply your own distortion map to their data. So no, you cannot recompute the calibrated data after the fact.
Related
I need to get distance from the camera to points in the camera image with AVDepthData. I understand there are two kinds of accuracy associated to AVDepthData: relative and absolute, the latter being the one which corresponds to real life distance.
I cannot seem to generate an AVDepthData with absolute accuracy. Is it possible at all?
AVDepthData is a generic model object for representing depth maps from a variety of possible sources, including parallax-based disparity inference, time-of-flight-based depth inference, data recorded by third-party cameras, or data synthesized by a 3D rendering engine. Thus, it can represent and describe more types of data than the device you're currently using can capture.
(It's like having an image format that supports 10-bit-per-component color: just because UIImage or some other API can tell you it's holding a wide-color image doesn't mean you have a camera that captures such images.)
More specifically... you didn't say whether you're using the front or back camera on iPhone X, but that matters quite a bit to what kind of depth maps you can capture.
builtInDualCamera, which iPhone X has for the back-facing camera (as do iPhone 7/8 Plus), infers disparity — which is not quite the same as depth, but related — by analyzing the parallax offsets between two camera images. This technique doesn't produce absolute measurements of depth, but because disparity is inversely proportional to depth you can know which points are deeper than others. (And using the cameraCalibrationData you can do some math and maybe get some decent estimates of absolute depth.)
builtInTrueDepthCamera, which iPhone X (and so far only iPhone X) has for its front-facing camera, can measure disparity or depth with time-of-flight analysis. (And sharks with fricking laser beams!) This technique produces absolute measurements pretty well, as long as you can safely assume the speed of light.
Which technique is used determines what kind of measurement you can get, and which technique is used depends on the capture device you select. (And by the way, there's a wealth of information on how these techniques work in the WWDC17 talk on capturing depth.)
If you're looking for back-camera depth measurements in an absolute frame of reference, you might do better to look at ARKit — that's not going to get you accurate depth values for every pixel, because it depends on coarse scene reconstruction, but the distance values you can get are absolute.
We have XSENS MTi IMU-Device and use the ROS-Framework (Ubuntu / Fuerte).
We subscribe to the IMU-Data and all data looks good except orientation.
In Euler-Outputmode like in Quaternion-Outputmode the values are constantly changing. Not randomly, they increase or decrease slowly at a more or less constant rate, and sometimes I observed the change to flatten out and then change it's direction.
When the Value at Second X may be:
x: 7.79210457616
y: -6.58661204898
z: 41.2841955308
the Z value changes in a range of about 10 within a few seconds (10-20 seconds I think).
What can cause this behaviour? Do we misinterpret the data or is there something wrong with the driver? The strange thing is, this also happend with 2 other drivers, and one other IMU device (we have 2). Same results in each combination.
Feel free to ask for more precise data or whatever you'd like to know that may be able to help us out. We are participating at the Spacebot-Cup in November, so it would be quite a relief to get the IMU done. :)
Perfectly normal if you have no magnetometer to give a corrected heading.
Gyroscope alone measures rate of turn only, and has no idea of orientation at any given time on any axis. Integrating the rate of turn gives the heading if you know the initial heading and the gyro is 100% accurate. It drifts anyway, even if it's perfectly calibrated, as you are sampling at discrete intervals rather than continuously.
Adding an accelerometer will at least fix the downward direction (because it measures gravity, which is towards the Earth's centre). This will keep the Z axis solution aligned with vertical, but it won't fix the horizontal direction (the heading or yaw). That will continue to drift, as you are seeing.
Adding a magnetometer will fix the heading relative to the Earth's magnetic field. This will give you a heading relative to magnetic North. You will need to apply a shift for local magnetic declination to get True North. These are generally available on line and reasonably constant over tens of km. Google ITREF.
Some integrated sensors don't have a magnetometer. That's why the heading drifts. Units like the MPU6050 have firmware built in, and can access a magnetometer, but the usual firmware doesn't use it, so you have to implement Madgwick, etc., on your micro controller or a connected PC anyway. Bosch have a new single module with a processing unit built in. Hopefully, it uses 9 DOF rather than the 6 you get with the DMP on the MPU6050.
Magnetic sensors are accurate to about 2 degrees. Local magnetic declination corrections also have an error. You may be able to perform additional calibrations by using a GPS on a long base line to get better results. It's also worth noting that heading and course made good are often different, due to crosswind / cross currents.
The Madgwick algorithm is fairly stable and easy to implement, and uses fewer resources than a Kalman filter, which needs to perform matrix inversion. It still gives minor jitter, but minor smoothing of results shouldn't induce too much lag.
If you have the IMU version, I assume that no signal processing has been done on the device. (but I don't know the product). So the data you get for the orientation should be only the integral of the gyroscope data.
The drift you can see is normal and can come from the integration of the noise, a bad zero rate calibration, or the bias of the gyroscope.
To correct this drift, we usually use an AHRS or a VRU algorithm (depending the need of a corrected yaw). It's a fusion sensor algorithm which take the gravity from the accelerometer and the magnetometer data (for AHRS) to correct this drift.
The algorithms often used are the Kalman filter and the complementary filter (Madgwick/Mahony).
It's not an easy job and request a bit of reading and experimenting on matlab/python to configure these filters :)
I'm currently developing an iPhone App (on iPhone 5, iOS 7, Xcode 5) which requires a very accurate determination of the current attitude. The "attitude" of CMDeviceMotion does not fulfil these requirements because Apple's sensor fusion algorithm seems to rely too much on the gyroscope which drifts away rather fast (in my experience). That's why I decided to read out the bare sensor data and later I want to combine it within a sensor fusion algorithm by myself.
When asking for magnetometer data one has two possibilities:
via CMMagnetometerData in CMMotionManager
via CMCalibratedMagneticField in CMDeviceMotion about which Apple says
The CMCalibratedMagneticField returned by this property gives you the total magnetic field in the device’s vicinity without device bias. Unlike the magneticField property of the CMMagnetometer class, these values reflect the earth’s magnetic field plus surrounding fields, minus device bias.
In principle (2.) is exactly what I want.
There is a very simple test if magnetometer data is calibrated properly. For simplicity one can restrict oneself to two dimensions. When the device lies on it's back, the combination B_x^2 + B_y^2 must be constant, independent of the direction the device is pointing to. It must just equal the horizontal component of the Earth's magnetic field (assuming no other fields in the vicinity of the device). Thus, when performing a 360 degrees turn of the device which lies on it's back, the measured data B_y over B_x should display a circle. See here for details.
Now the point: the data of CMCalibratedMagneticField does NOT result in a circle!
Does anyone have an explanation for that? Or does anyone know, how the CMCalibratedMagneticField comes about? Is the magnetometer calibrated in the sense of the link from above when performing the "eight-shaped" movement of the device or what is the movement good for?
Btw. why the "eight-shaped" movement and not flipping the device around it's three axis, which would allow a calibration as described in the link from above?
I would be very glad for any clarification with this issue... Thanks!
There is a problem with the magnetometer in iOS 7, it has an error of +-7º. Try using the 7.1 beta version.
EDIT
The magnetometer has zero-drift over time, but is pretty inaccurate for sudden changes in position. The accelerometer and gyroscope on the other hand adjust quickly for sudden changes but, being inertial sensors, they lose accuracy over a period of time.
So when CMCalibratedMagneticField tries compensate for your rotational motion it uses data from the gyroscope and accelerometer. This is when the accelerometer and gyroscope's +-7º error creeps in and throws your circle off track. Check this answer and this wikipedia article for more info.
As regards to the figure of eight:
Both do the same thing, they orient the "North" of your device in each direction in hope of cancelling out magnetic interference. Flipping your device along all three axes will work better but it is harder to perform and not as easily understood by the user.
Hope this helps.
I want to find the cardinal direction accelerated by an iphone. I thought I could just use the accelerometer to do this, however, as you can see from the picture below the accelerometers axes are defined by the device orientation.
I figured that if i used the gyroscope to correct for yaw, spin, rotation then I could get a more accurate reading and not have to hold the phone in the same orientation during movement.
But this still does not tell me what cardinal direction the iphone is moving in. For that I would also have to use the the magnetometer.
Can anybody tell me how to use a three sensor readings to find the cardinal direction accelerated in? I dont even know where to start. I dont even know if the phone takes these measurements at the same rates of time either.
Taking the cross product of the magnetometer vector with the "down" vector will give you a horizontal magnetic east/west vector; from that, a second cross product gets the magnetic north/south vector. That's the easy part.
The harder problem is tracking the "down" vector effectively. If you integrate the accelerometers over time, you can filter out the motion of a hand-held mobile device, to get the persistent direction of gravity. Or, you could, if your device weren't rotating at the same time...
That's where the rate gyros come in: the gyros can let you compensate for the dynamic rotation of the hand-held device, so you can track your gravity in real-time. The classic way to do this is called a Kalman filter, which can integrate (both literally and figuratively) multiple data sources in order to evaluate the most likely state of your system.
A Kalman filter requires a mathematical model both of your physical system, and of the sensors that observe it; each of these models must be both accurate and "sufficiently linear" for the Kalman filter to work properly. As it happens, the iphone/accelerometer/gyro system is in fact sufficiently linear.
The Kalman filter uses both calculus and linear algebra, so if you're rolling your own, you will need a certain amount of math.
Also, as a practical matter, you should understand that physical sensors typically have offsets that need to be compensated for -- in particular, you need to pay attention to the rate gyro offsets in this kind of inertial navigation system, or your tracker will never stabilize. This means you will need to add your rate gyro offsets to your Kalman state vector and system model.
I am totally new to camera calibration techniques... I am using OpenCV chessboard technique... I am using a webcam from Quantum...
Here are my observations and steps..
I have kept each chess square side = 3.5 cm. It is a 7 x 5 chessboard with 6 x 4 internal corners. I am taking total of 10 images in different views/poses at a distance of 1 to 1.5 m from the webcam.
I am following the C code in Learning OpenCV by Bradski for the calibration.
my code for calibration is
cvCalibrateCamera2(object_points,image_points,point_counts,cvSize(640,480),intrinsic_matrix,distortion_coeffs,NULL,NULL,CV_CALIB_FIX_ASPECT_RATIO);
Before calling this function I am making the first and 2nd element along the diagonal of the intrinsic matrix as one to keep the ratio of focal lengths constant and using CV_CALIB_FIX_ASPECT_RATIO
With the change in distance of the chess board the fx and fy are changing with fx:fy almost equal to 1. there are cx and cy values in order of 200 to 400. the fx and fy are in the order of 300 - 700 when I change the distance.
Presently I have put all the distortion coefficients to zero because I did not get good result including distortion coefficients. My original image looked handsome than the undistorted one!!
Am I doing the calibration correctly?. Should I use any other option than CV_CALIB_FIX_ASPECT_RATIO?. If yes, which one?
Hmm, are you looking for "handsome" or "accurate"?
Camera calibration is one of the very few subjects in computer vision where accuracy can be directly quantified in physical terms, and verified by a physical experiment. And the usual lesson is that (a) your numbers are just as good as the effort (and money) you put into them, and (b) real accuracy (as opposed to imagined) is expensive, so you should figure out in advance what your application really requires in the way of precision.
If you look up the geometrical specs of even very cheap lens/sensor combinations (in the megapixel range and above), it becomes readily apparent that sub-sub-mm calibration accuracy is theoretically achievable within a table-top volume of space. Just work out (from the spec sheet of your camera's sensor) the solid angle spanned by one pixel - you'll be dazzled by the spatial resolution you have within reach of your wallet. However, actually achieving REPEATABLY something near that theoretical accuracy takes work.
Here are some recommendations (from personal experience) for getting a good calibration experience with home-grown equipment.
If your method uses a flat target ("checkerboard" or similar), manufacture a good one. Choose a very flat backing (for the size you mention window glass 5 mm thick or more is excellent, though obviously fragile). Verify its flatness against another edge (or, better, a laser beam). Print the pattern on thick-stock paper that won't stretch too easily. Lay it after printing on the backing before gluing and verify that the square sides are indeed very nearly orthogonal. Cheap ink-jet or laser printers are not designed for rigorous geometrical accuracy, do not trust them blindly. Best practice is to use a professional print shop (even a Kinko's will do a much better job than most home printers). Then attach the pattern very carefully to the backing, using spray-on glue and slowly wiping with soft cloth to avoid bubbles and stretching. Wait for a day or longer for the glue to cure and the glue-paper stress to reach its long-term steady state. Finally measure the corner positions with a good caliper and a magnifier. You may get away with one single number for the "average" square size, but it must be an average of actual measurements, not of hopes-n-prayers. Best practice is to actually use a table of measured positions.
Watch your temperature and humidity changes: paper adsorbs water from the air, the backing dilates and contracts. It is amazing how many articles you can find that report sub-millimeter calibration accuracies without quoting the environment conditions (or the target response to them). Needless to say, they are mostly crap. The lower temperature dilation coefficient of glass compared to common sheet metal is another reason for preferring the former as a backing.
Needless to say, you must disable the auto-focus feature of your camera, if it has one: focusing physically moves one or more pieces of glass inside your lens, thus changing (slightly) the field of view and (usually by a lot) the lens distortion and the principal point.
Place the camera on a stable mount that won't vibrate easily. Focus (and f-stop the lens, if it has an iris) as is needed for the application (not the calibration - the calibration procedure and target must be designed for the app's needs, not the other way around). Do not even think of touching camera or lens afterwards. If at all possible, avoid "complex" lenses - e.g. zoom lenses or very wide angle ones. For example, anamorphic lenses require models much more complex than stock OpenCV makes available.
Take lots of measurements and pictures. You want hundreds of measurements (corners) per image, and tens of images. Where data is concerned, the more the merrier. A 10x10 checkerboard is the absolute minimum I would consider. I normally worked at 20x20.
Span the calibration volume when taking pictures. Ideally you want your measurements to be uniformly distributed in the volume of space you will be working with. Most importantly, make sure to angle the target significantly with respect to the focal axis in some of the pictures - to calibrate the focal length you need to "see" some real perspective foreshortening. For best results use a repeatable mechanical jig to move the target. A good one is a one-axis turntable, which will give you an excellent prior model for the motion of the target.
Minimize vibrations and associated motion blur when taking photos.
Use good lighting. Really. It's amazing how often I see people realize late in the game that you need a generous supply of photons to calibrate a camera :-) Use diffuse ambient lighting, and bounce it off white cards on both sides of the field of view.
Watch what your corner extraction code is doing. Draw the detected corner positions on top of the images (in Matlab or Octave, for example), and judge their quality. Removing outliers early using tight thresholds is better than trusting the robustifier in your bundle adjustment code.
Constrain your model if you can. For example, don't try to estimate the principal point if you don't have a good reason to believe that your lens is significantly off-center w.r.t the image, just fix it at the image center on your first attempt. The principal point location is usually poorly observed, because it is inherently confused with the center of the nonlinear distortion and by the component parallel to the image plane of the target-to-camera's translation. Getting it right requires a carefully designed procedure that yields three or more independent vanishing points of the scene and a very good bracketing of the nonlinear distortion. Similarly, unless you have reason to suspect that the lens focal axis is really tilted w.r.t. the sensor plane, fix at zero the (1,2) component of the camera matrix. Generally speaking, use the simplest model that satisfies your measurements and your application needs (that's Ockam's razor for you).
When you have a calibration solution from your optimizer with low enough RMS error (a few tenths of a pixel, typically, see also Josh's answer below), plot the XY pattern of the residual errors (predicted_xy - measured_xy for each corner in all images) and see if it's a round-ish cloud centered at (0, 0). "Clumps" of outliers or non-roundness of the cloud of residuals are screaming alarm bells that something is very wrong - likely outliers due to bad corner detection or matching, or an inappropriate lens distortion model.
Take extra images to verify the accuracy of the solution - use them to verify that the lens distortion is actually removed, and that the planar homography predicted by the calibrated model actually matches the one recovered from the measured corners.
This is a rather late answer, but for people coming to this from Google:
The correct way to check calibration accuracy is to use the reprojection error provided by OpenCV. I'm not sure why this wasn't mentioned anywhere in the answer or comments, you don't need to calculate this by hand - it's the return value of calibrateCamera. In Python it's the first return value (followed by the camera matrix, etc).
The reprojection error is the RMS error between where the points would be projected using the intrinsic coefficients and where they are in the real image. Typically you should expect an RMS error of less than 0.5px - I can routinely get around 0.1px with machine vision cameras. The reprojection error is used in many computer vision papers, there isn't a significantly easier or more accurate way to determine how good your calibration is.
Unless you have a stereo system, you can only work out where something is in 3D space up to a ray, rather than a point. However, as one can work out the pose of each planar calibration image, it's possible to work out where each chessboard corner should fall on the image sensor. The calibration process (more or less) attempts to work out where these rays fall and minimises the error over all the different calibration images. In Zhang's original paper, and subsequent evaluations, around 10-15 images seems to be sufficient; at this point the error doesn't decrease significantly with the addition of more images.
Other software packages like Matlab will give you error estimates for each individual intrinsic, e.g. focal length, centre of projection. I've been unable to make OpenCV spit out that information, but maybe it's in there somewhere. Camera calibration is now native in Matlab 2014a, but you can still get hold of the camera calibration toolbox which is extremely popular with computer vision users.
http://www.vision.caltech.edu/bouguetj/calib_doc/
Visual inspection is necessary, but not sufficient when dealing with your results. The simplest thing to look for is that straight lines in the world become straight in your undistorted images. Beyond that, it's impossible to really be sure if your cameras are calibrated well just by looking at the output images.
The routine provided by Francesco is good, follow that. I use a shelf board as my plane, with the pattern printed on poster paper. Make sure the images are well exposed - avoid specular reflection! I use a standard 8x6 pattern, I've tried denser patterns but I haven't seen such an improvement in accuracy that it makes a difference.
I think this answer should be sufficient for most people wanting to calibrate a camera - realistically unless you're trying to calibrate something exotic like a Fisheye or you're doing it for educational reasons, OpenCV/Matlab is all you need. Zhang's method is considered good enough that virtually everyone in computer vision research uses it, and most of them either use Bouguet's toolbox or OpenCV.