I'm fairly new to lua, but I read that it does not have integers, so all numbers are floats.
I noticed in my log that some numbers get a slight inaccuracy added to it. For instance 0.14 is written as 0.14000000059605.
In that function it doesn't really matter if the number is a little diferent, as it is a comparison with a random number. But I do a lot of equals comparisons with numbes like NumReg() == 2 where it would give a wrong result if the 2 gets unrounded.
Do do I have to account for this by rounding them down, or are non-tiny numbers not affected?
You can format a float for your needs.
Example to play with in an interactive Lua console...
> _VERSION
Lua 5.4
tonumber(string.format('%.3f', 0.14000000059605))
0.14
> print(tonumber(string.format('%.3f', 0.14000000059605)))
0.14
> type(tonumber(string.format('%.3f', 0.14000000059605)))
number
> math.type(tonumber(string.format('%.3f', 0.14000000059605)))
float
> tonumber(string.format('%.3f', 0.14000000059605)) == 0.140
true
Related
This question already has an answer here:
Dealing with big numbers in Lua
(1 answer)
Closed 1 year ago.
When using lua to handle floating point numbers I found that lua can handle very limited precision, for example:
print(3.14159265358979)
output:
3.1415926535898
The result will be missing a few decimal places, which will lead to calculation bias. How can I deal with such a lack of precision
By default, Lua only displays 14 digits of a number. A float can require 15 to 17 digits to be represented exactly as a base-10 string. We can use a loop to find the right number of digits. Note that %g will drop the trailing zeros, so we can start our search at 15 digits, not 1. This is the function I use:
local function floatToString(x)
for precision = 15, 17 do
-- Use a 2-layer format to try different precisions with %g.
local s <const> = ('%%.%dg'):format(precision):format(x)
-- See if s is an exact representation of x.
if tonumber(s) == x then
return s
end
end
end
print(floatToString(3.14159265358979))
Output: 3.14159265358979
I have the following test case:
Lua 5.3.2 Copyright (C) 1994-2015 Lua.org, PUC-Rio
> foo = 1000000000000000000
> bar = foo + 1
> bar
1000000000000000001
> string.format("%.0f", foo)
1000000000000000000
> string.format("%.0f", bar)
1000000000000000000
That last line should be 1000000000000000001, since that's the value of bar, but for some reason it's not. This doesn't only apply to 1000000000000000000, I've yet to find another number over that one which gives the correct value. Can anyone give an explanation for why this happens?
You're formatting the number as floating-point, not integer. That's what %.0f is doing. At some point, floats lose precision. double, for example, will lose precision after about 16 decimal digits.
If you want to format an integer as an integer, then you need to format it as an integer, using standard printf rules:
string.format("%i", bar)
log2(1000000000000000000) is between 59 and 60, which means that the binary representation of that number needs 60 bits. double-precision floating point numbers have only 53 bits of precision, plus a power-of-two exponent with 11 bits of range. So to store that large of a number as floating point (which is what you requested with the %f format specifier), six to seven bits of precision are chopped off the end of the number, and the whole thing is multiplied by a power of two to get it back in range (259 in this case, I think). Chopping off those final bits removes the precision that allows 1000000000000000000 and 1000000000000000001 to be distinct from each other.
(This is not a particularly precise description of floating point, apologies if my numbers or descriptions are not exact.)
The result of math.sqrt(2) seems to be irrational so this occurs:
> return math.sqrt(2)
1.4142135623731
> return math.sqrt(2) == 1.4142135623731
false
How do I make this "irrational" variable same as if I got the variable different way (like in the example above)?
The variable is not irrational, it is floating-point, so it isn't even real. (the square-root of 2 is irrational though, and thus cannot be accurately represented by it)
Just use more digits for your literal, and the round-trip conversion will work. An IEEE double-precision floating-point value needs 17 significant decimal digits to safely represent it, not 14.
Let's see what happens when we take the number 1 and uptick it in the least significant bit. (The '0x' means the numeral is hexadecimal. That makes it easier for me to control the bits for this example.):
x = 0x1.0000000000001
> print(x == 1)
false
> print(('%.16g'):format(x))
1
> print(('%.17g'):format(x))
1.0000000000000002
Even though Lua does not differentiate between floating point numbers and integers, there are some cases when you want to use integers. What is the best way to covert a number to an integer if you cannot do a C-like cast or without something like Python's int?
For example when calculating an index for an array in
idx = position / width
how can you ensure idx is a valid array index? I have come up with a solution that uses string.find, but maybe there is a method that uses arithmetic that would obviously be much faster. My solution:
function toint(n)
local s = tostring(n)
local i, j = s:find('%.')
if i then
return tonumber(s:sub(1, i-1))
else
return n
end
end
You could use math.floor(x)
From the Lua Reference Manual:
Returns the largest integer smaller than or equal to x.
Lua 5.3 introduced a new operator, called floor division and denoted by //
Example below:
Lua 5.3.1 Copyright (C) 1994-2015 Lua.org, PUC-Rio
>12//5
2
More info can be found in the lua manual
#Hofstad is correct with the math.floor(Number x) suggestion to eliminate the bits right of the decimal, you might want to round instead. There is no math.round, but it is as simple as math.floor(x + 0.5). The reason you want to round is because floats are usually approximate. For example, 1 could be 0.999999996
12.4 + 0.5 = 12.9, floored 12
12.5 + 0.5 = 13, floored 13
12.6 + 0.5 = 13.1, floored 13
local round = function(a, prec)
return math.floor(a + 0.5*prec) -- where prec is 10^n, starting at 0
end
why not just use math.floor()? it would make the indices valid so long as the numerator and denominator are non-negative and in valid ranges.
I am doing the following in actionscript in Coldfusion Flash Forms:
90 / 3.7
Gives me:
24.3243243243243
Whereas the calculator gives me:
24.32432432432432
Note the extra 2 at the end.
So my problem occurs when I am trying to get the original value of 90 by taking the 24.3243243243243 * 3.7 and then I get 89.9999999999 which is wrong.
Why is Actionscript truncating the value and how do I avoid this so I get the proper amount that the calculator gets?
Thanks so much.
Round your number using a routine like this
var toFixed:Function = function(number, factor) {
return (Math.round(number * factor)/factor);
}
Where the factor is 10, 100, 1000 etc, a simple way to think about it is the number of 0's in the factor is the number of decimal places
so
toFixed(1.23341230123, 100) = 1.23
Good explanation of numeric in ActionScript can be found at http://docstore.mik.ua/orelly/web2/action/ch04_03.htm. See section 4.3.2.1. Floating-point precision
A relavant quote:
"In order to accommodate for the minute discrepancy, you should round your numbers manually if the difference will adversely affect the behavior of your code. "