In F#. How to efficiently compare floats for equality that are almost equal? It should work for very large and very small values too. I am thinking of first comparing the Exponent and then the Significand (Mantissa) while ignoring the last 4 bits of the its 52 bits. Is that a good approach? How can I get the Exponent and Significand of a float?
An F# float is just a shorthand for System.Double. That being the case, you can use the BitConverter.DoubleToInt64Bits method to efficiently (and safely!) "cast" an F# float value to int64; this is useful because it avoids allocating a byte[], as John mentioned in his comment. You can get the exponent and significand from that int64 using some simple bitwise operations.
As John said though, you're probably better off with a simple check for relative accuracy. It's likely to be the fastest solution and "close enough" for many use cases (e.g., checking to see if an iterative solver has converged on a solution). If you need a specific amount of accuracy, take a look at NUnit's code -- it has some nice APIs for asserting that values are within a certain percentage or number of ulps of an expected value.
When you ask how to compare floating-point values that are almost equal, you are asking:
I have two values, x and y, that have been computed with floating-point arithmetic, so they contain rounding errors and are approximations of ideal mathematical values x and y. How can I use the floating-point x and y to compare the mathematical x and y for equality?
There are two problems here:
We do not know how much error there may be in x or y. Some combinations of arithmetic magnify errors, while others shrink them. It is possible for the errors in x and y to range from zero to infinity, and you have not given us any information about this.
It is often assumed that the goal is to produce a result of “equal” when x and y are unequal but close to each other. This converts false negatives (inequality would be reported even though the mathematical x and y would be equal) into positives. However, it creates false positives (equality is reported even though the mathematical x and y would be unequal).
There is no general solution for these problems.
It is impossible to know in general whether an application can tolerate being told that values are equal when they should be unequal or vice-versa without knowing specific details about that application.
It is impossible to know in general how much error there may be in x and y.
Therefore, there is no correct general test for equality in values that have been computed appoximately.
Note that this problem is not really about testing for equality. Generally, it is impossible to compute any function of incorrect data (except for trivial functions such as constant functions). Since x and y contain errors, it is impossible to use x to compute log(x) without errors, or to compute arcosine(y) or sqrt(x) without errors. In fact, if the errors have made y slightly greater than 1 while y is not or made x slightly less than zero while x is not, then computing acos(y) or sqrt(x) will produce exceptions and NaNs even though the ideal mathematical values would work without problem.
What this all means is that you cannot simply convert exact mathematical arithmetic to approximate floating-point arithmetic and expect to get a good result (whether you are testing for equality or not). You must consider the effects of converting exact arithmetic to approximate arithmetic and evaluate how they affect your program and your data. The use of floating-point arithmetic, including comparisons for equality, must be tailored to individual situations.
Related
I would like to understand whether using fixed point Q31 is better than floating-point (single precision) for DSP applications where accuracy is important.
More details, I am currently working with ARM Cortex-M7 microcontroller and I need to perform FFT with high accuracy using CMSIS library. I understand that the SP has 24 bits for the mantissa while the Q31 has 31 bits, therefore, the precision of the Q31 should be better, but I read everywhere that for algorithms that require multiplication and so on, the floating-point representation should be used, which I do not understand why.
Thanks in advance.
Getting maximum value out of fixed point (that extra 6 or 7 bits of mantissa accuracy), as well as avoiding a ton of possible underflow and overflow problems, requires knowing precisely the bounds (min and max) of every arithmetic operation in your CMSIS algorithms for every valid set of input data.
In practice, both a complete error analysis turns out to be difficult, and the added operations needed to rescale all intermediate values to optimal ranges reduces performance so much, that only a narrower set of cases seems worth the effort, over using either IEEE signal or double, which the M7 supports in hardware, and where the floating point exponent range hides an enormous amount (but not all !!) of intermediate result numerical scaling issues.
But for some more simple DSP algorithms, sometimes analyzing and fixing the scaling isn't a problem. Hard to tell which without disassembling the numeric range of every arithmetic operation in your needed algorithm. Sometimes the work required to use integer arithmetic needs to be done because the processors available don't support floating point arithmetic well or at all.
what is the usual precision for Real variables in Z3? Is exact arithmetic used?
Is there a way to set the accuracy level manually?
If Real means that exact arithmetic must be used, is there any other data type for floating point values which has limited precision?
Finally: from this point of view, is z3 different with respect to the other popular SMT solvers, or is this standardised in the SMT-LIB definition?
See this answer: z3 existential theory of the reals
Regarding printing precision, see this one: algebraic reals: does z3 do rounding when pretty printing?
In short, yes they are precisely represented as roots of polynomials. Not every real number can be represented by the Real type (transcendentals, e, pi, etc.); but all polynomial roots are representable.
This paper discusses how to also deal with transcendentals.
I'd like to write a function to "compute" the length of a list. http://rise4fun.com/Z3/Irsl1, based on list concat in z3 and Proving inductive facts in Z3.
I can't seem to be able to make it work, it fails with a timeout. Would such a function be expressible in Z3?
The larger context is that I'm trying to model and solve questions like "how many even positive integers are less than 9?", or "there are 5 even positive integers smaller than than x, what is x?".
The larger context is that I'm trying to model and solve questions like "how many even positive integers are less than 9?", or "there are 5 even positive integers smaller than than x, what is x?".
If this is what you are really trying to solve, then I would suggest not creating lists or comprehensions directly, but to encode problems using just arithmetic.
For example, you can obtain even integers by multiplying by two, and express
properties of intervals of integers using quantifiers.
For operations on sequences, there are some emerging options.
For example, sequences and lengths are partially handled: http://rise4fun.com/z3/tutorial/sequences. Rather simple properties
of sequences and lengths are discharged using the built-in procedure.
If you start encoding properties like you hint at above, it is unlikely to
do much good as the main support is around ground (quantifier-free) properties.
I am wondering whether large integer values have impact on the performance of SMT. Sometimes I need to work with large values. Mostly I do arithmetic operations (mainly addition and multiplication) on them (i.e., different integer terms) and need to compare the resultant value with constraints (i.e., some other integer term).
Large integers and/or rationals in the input problem is not a definitive indicator of hardness.
Z3 may generate large numbers internally even when the input contains only small numbers.
I have observed many examples where Z3 spends a lot of time processing large rational numbers.
A lot of time is spent computing the GCD of the numerator and denominator.
Each GCD computation takes a relatively small amount of time, but on hard problems Z3 will perform millions of them.
Note that, Z3 uses rational numbers for solving pure integer problems, because it uses a Simplex-based algorithm for solving linear arithmetic.
If you post your example, I can give you a more precise answer.
I am using Lua 5.1
print(10.08 - 10.07)
Rather than printing 0.01, above prints 0.0099999999999998.
Any idea how to get 0.01 form this subtraction?
You got 0.01 from the subtraction. It is just in the form of a repeating decimal with a tiny amount of lost precision.
Lua uses the C type double to represent numbers. This is, on nearly every platform you will use, a 64-bit binary floating point value with approximately 23 decimal digits of precision. However, no amount of precision is sufficient to represent 0.01 exactly in binary. The situation is similar to attempting to write 1/3 in decimal.
Furthermore, you are subtracting two values that are very similar in magnitude. That all by itself causes an additional loss of precision.
The solution depends on what your context is. If you are doing accounting, then I would strongly recommend that you not use floating point values to represent account values because these small errors will accumulate and eventually whole cents (or worse) will appear or disappear. It is much better to store accounts in integer cents, and divide by 100 for display.
In general, the answer is to be aware of the issues that are inherent to floating point, and one of them is this sort of small loss of precision. It is easily handled by rounding answers to an appropriate number of digits for display, and never comparing results of calculations for equality.
Some resources for background:
The semi-official explanation at the Lua Users Wiki
This great page of IEEE floating point calculators where you can enter values in decimal and see how they are represented, and vice-versa.
Wiki on IEEE floating point.
Wiki on floating point numbers in general.
What Every Computer Scientist Should Know About Floating-Point Arithmetic is the most complete discussion of the fine details.
Edit: Added the WECSSKAFPA document after all. It really is worth the study, although it will likely seem a bit overwhelming on the first pass. In the same vein, Knuth Volume 2 has extensive coverage of arithmetic in general and a large section on floating point. And, since lhf reminded me of its existence, I inserted the Lua wiki explanation of why floating point is ok to use as the sole numeric type as the first item on the list.
Use Lua's string.format function:
print(string.format('%.02f', 10.08 - 10.07))
Use an arbitrary precision library.
Use a Decimal Number Library for Lua.