Can someone explain why in lua running:
return 256.65 * 1000000 + .000000005 - 256 * 1000000 gives 649999.99999997
whereas
return 255.65 * 1000000 + .000000005 - 255 * 1000000 and
return 268.65 * 1000000 + .000000005 - 268 * 1000000 give 650000.0 ?
From what i can see it seems to be an issue strictly for decimal 65 (and it seems also 15) and for whole numbers within the range 256 - 267. I know this is related to doing these calculations with floating points, but I'm still curious as to what is special about these values in particular
What is special about these values is that 0.65 is not a binary fraction (even though it is a decimal fraction), and so cannot be represented exactly in floating point.
For the record, this is not specific to Lua. The same thing will happen in C.
For the same reason that 10/3 is a repeating fraction in base 10. In base 3, dividing by 3 would result in whole numbers. In base 2 -- which is used to represent numbers in a computer -- the numbers you're producing similarly result in fractions that can be exactly represented.
Further reading.
Related
I am currently converting some python statistics library that needs to produce a number with high decimal precision. For example, I did this:
i = 1
n = 151
sum = (i - 3/8) / (n + 1/4)
it will result to 0.
My question is how to always show decimal precision automatically when I do this kind of computation?
My desired output is:
0.004132231404958678
In ruby all the arithmetic operations result in the value of the same type as operands (the one having better precision.)
That said, 3/4 is an integer division, resulting in 0.
To make your example working, you are to ensure you are not losing precision anywhere:
i = 1.0
n = 151.0
sum = (i - 3.0/8) / (n + 1/4.0)
Please note, that as in most (if not all) languages, Float is tainted:
0.1 + 0.2 #⇒ 0.30000000000000004
If you need an exact value, you might use BigDecimal or Rational.
Here is my Swift Code
print("\(Int64.max)")
print("\(Double(Int64.max))")
It produce following output
9223372036854775807
9.223372036854776e+18
Why the both value is entire different
9.223372036854776e+18 - 9223372036854775807 = 193 FYI
The value of Double you are seeing in the output is only an approximation to some number of significant figures. We can see more significant figures by String(format:)
print(String(format: "%.1f", Double(Int64.max)))
This prints:
9223372036854775808.0
So actually, the difference is not as big as what you claimed it was (193). It's just a difference of 1.
Why is there a difference?
Double stores value using a floating point representation. It can represent a wide range of numbers, but not every number in that range. Double uses 53 bits of mantissa, 1 sign bit and 11 bits to store the exponent. The mantissa represents the significant digits of the number, and the exponent tells you where to put the decimal point. Everything on one side of the decimal point represents positive powers of 2 and everything on the other side represent negative powers of 2. For example:
0.1010000 0010
mantissa exponent
The exponent says to move the decimal point to the right 3 times, so the mantissa becomes 010.10000. The 1 on the left represents 2, and the 1 on the right represents a half (2^-1), so this floating point number represents the number 2.5
To represent Int64.max (2^63-1), you need 63 bits of mantissa to all be 1s and the value in the exponent to be 63. But Double does not have that many bits of mantissa! So it can only approximate. Int64.max + 1 is actually representable by a Double, because it is equal to 2^63. You just need one 1 followed by 52 0s in the mantissa and the exponent can store 64. And that's what Double did.
I'm using float value in my project. when I try to access in Project, it's expanding to 1/billions decimal but when it comes to playground it works perfectly.
In xcodeproj:
let sampleFloat: Float = 0.025
print(sampleFloat) // It prints 0.0250000004
In Playground:
let sampleFloat: Float = 0.025
print(sampleFloat) // It prints 0.025
Any clue what's happening here? how can I avoid expansion in xcodeproj?
Lots of comments, but nobody's posted all the info as an answer yet.
The answer is that internally, floating point numbers are represented with binary powers of 2.
In base 10, the tenths digit represents how many 1/10ths are in the value. The hundredths digit represents how many 1/100ths are in the value, the thousandths digit represents how many 1/1000ths are in the value, and so on. In base 10, you can't represent 1/3 exactly. That is 0.33333333333333333...
In binary floating point, the first fractional binary digit represents how many 1/2s are in the value. The second digit represents how many 1/4ths are in in th value, the next digit represents how many 1/8ths are in the value, and so on. There are some (lots of) decimal values that can't be represented exactly in binary floating point. The value 0.1 (1/10) is one such value. That will be approximated by something like 1/16 + 1/32 + 1/256 + 1/512 + 1/4096 + 1/8192.
The value 0.025 is another value that can't be represented exactly in binary floating point.
There is an alternate number format, NSDecimalNumber (Decimal in Swift 3) that uses decimal digits to represent numbers, so it CAN express any decimal value exactly. (Note that it still can't express a fraction like 1/3 exactly.)
How to do % to negative number in VF?
MOD(10,-3) = -2
MOD(-10,3) = 2
MODE(-10,-3) = -1
Why?
It is a regular modulo:
The mod function is defined as the amount by which a number exceeds
the largest integer multiple of the divisor that is not greater than
that number.
You can think of it like this:
10 % -3:
The largest multiple of 10 that is less than -3 is -2.
So 10 % -3 is -2.
-10 % 3:
Now, why -10 % 3 is 2?
The easiest way to think about it is to add to the negative number a multiple of 2 so that the number becomes positive.
-10 + (4*3) = 2 so -10 % 3 = (-10 + 12) % 3 = 2 % 3 = 3
Here's what we said about this in The Hacker's Guide to Visual FoxPro:
MOD() and % are pretty straightforward when dealing with positive numbers, but they get interesting when one or both of the numbers is negative. The key to understanding the results is the following equation:
MOD(x,y) = x - (y * FLOOR(x/y))
Since the mathematical modulo operation isn't defined for negative numbers, it's a pleasure to see that the FoxPro definitions are mathematically consistent. However, they may be different from what you'd initially expect, so you may want to check for negative divisors or dividends.
A little testing (and the manuals) tells us that a positive divisor gives a positive result while a negative divisor gives a negative result.
I'm not sure if this is a bug or not, so I thought that maybe you folks might want to take a look.
The problem lies with this code:
for i=0,1,.05 do
print(i)
end
The output should be:
0
.05
.1
--snip--
.95
1
Instead, the output is:
0
.05
.1
--snip--
.95
This same problem happened with a while loop:
w = 0
while w <= 1 do
print(w)
w = w + .05
end
--output:
0
.05
.1
--snip--
.95
The value of w is 1, which can be verified by a print statement after the loop.
I have verified as much as possible that any step that is less than or equal .05 will produce this error. Any step above .05 should be fine. I verified that 1/19 (0.052631579) does print a 1. (Obviously, a decimal denominator like 19.9 or 10.5 will not produce output from [0,1] inclusive.) Is there a possibility that this is not an error of the language? Both the interpreter and a regular Lua file produce this error.
This is a rounding problem. The issue is that 0.05 is represented as a floating point binary number, and it does not have an exact representation in binary. In base 2 (binary), it is a repeating decimal similar to numbers like 1/3 in base 10. When added repeatedly, the rounding results in a number which is slightly more than 1. It is only very, very slightly more than 1, so if you print it out, it shows 1 as the output, but it is not exactly 1.
> x=0.05+0.05+0.05+0.05+0.05+0.05+0.05+0.05+0.05+0.05+0.05+0.05+0.05+0.05+0.05+0.05+0.05+0.05+0.05+0.05
> print(x)
1
> print(1==x)
false
> print(x-1)
2.2204460492503e-16
So, as you can see, although really close to 1, it is actually slightly more.
A similar situation can come up in decimal when we have repeating fractions. If we were to add together 1/3 + 1/3 + 1/3, but we had to round to six digits to work with, we would add 0.333333 + 0.333333 + 0.333333 and get 0.999999 which is not actually 1. This is an analogous case for binary math. 1/20 cannot be precisely represented in binary.
Note that the rounding is slightly different for multiplication so
> print(0.05*20-1)
0
> print(0.05*20==1)
true
As a result, you could rewrite your code to say
for i=0,20,1 do
print(i*0.05)
end
And it would work correctly. In general, it's advisable not to use floating point numbers (that is, numbers with decimal points) for controlling loops when it can be avoided.
This is a result of floating-point inaccuracy. A binary64 floating point number is unable to store 0.05 and so the result will be rounded to a number which is very slightly more than 0.05. This rounding error remains in the repeated sum, and eventually the final value will be slightly more than 1.0, and so will not be displayed.
This is a floating point thing. Computers don't represent floating point numbers exactly. Tiny rounding errors make it so that 20 additions of +0.05 does not result in precisely 1.0.
Check out this article: "What every programmer should know about floating-point arithmetic."
To get your desired behavior, you could loop i over 1..20, and set f=i*0.05
This is not a bug in Lua. The same thing happens in the C program below. Like others have explained, it's due to floating-point inaccuracy, more precisely, to the fact that 0.05 is not a binary fraction (that is, does not have a finite binary representation).
#include <stdio.h>
int main(void)
{
double i;
for (i=0; i<=1; i+=0.05) printf("%g\n",i);
return 0;
}