This question already has answers here:
Is floating point math broken?
(31 answers)
Closed 4 years ago.
Trying to round off double values to two decimal places using function FormatFloat (Format string '0.##').
Below are the input and output values
231.545 -> 231.54 (but expected output is 231.55)
2.315 -> 2.31 (but expected output is 2.32)
23.045 -> 23.05 (gives expected output 23.05)
23.145 -> 23.14 (but expected output 23.15)
23.245 -> 23.25 (gives expected output 23.25)
23.345 -> 23.34 (but expected output 23.35)
23.445 -> 23.45 (gives expected output 23.45)
23.545 -> 23.55 (gives expected output 23.55)
23.645 -> 23.65 (but expected output 23.64)
23.745 -> 23.75 (gives expected output 23.75)
23.845 -> 23.84 (but expected output 23.84)
23.945 -> 23.95 (gives expected output 23.95)
why this strange behaviour happening? am using Delphi 7.
Binary floating point values can not represent every value exactly. This is what you are seeing.
For example, the value 2.315 is represented in double precision by:
2.31499 99999 99999 94670 92948 17992 48605 96656 79931 64062 5
This will be rounded to 2.31
If you can use a decimal data type, like currency, you can get the wanted output (if currency is within limits of your working range):
var
c : Currency;
begin
c := 2.315;
WriteLn(FormatFloat('0.##',c)); // Outputs 2.32
end.
An alternative is to use a decimals library, like BigDecimals, but that would require a modern Delphi version with support of records with methods.
Related
This question already has an answer here:
Dealing with big numbers in Lua
(1 answer)
Closed 1 year ago.
When using lua to handle floating point numbers I found that lua can handle very limited precision, for example:
print(3.14159265358979)
output:
3.1415926535898
The result will be missing a few decimal places, which will lead to calculation bias. How can I deal with such a lack of precision
By default, Lua only displays 14 digits of a number. A float can require 15 to 17 digits to be represented exactly as a base-10 string. We can use a loop to find the right number of digits. Note that %g will drop the trailing zeros, so we can start our search at 15 digits, not 1. This is the function I use:
local function floatToString(x)
for precision = 15, 17 do
-- Use a 2-layer format to try different precisions with %g.
local s <const> = ('%%.%dg'):format(precision):format(x)
-- See if s is an exact representation of x.
if tonumber(s) == x then
return s
end
end
end
print(floatToString(3.14159265358979))
Output: 3.14159265358979
While converting an old iOS app to Sift 3.0 I hit the following issue:
The code is:
cutRange = numberString.index(numberString.startIndex, offsetBy:2)...numberString.index(numberString.startIndex, offsetBy:5)
The error message I get is:
No '...' candidates produce the expected contextual result type 'Range<String.Index>' (aka 'Range<String.CharacterView.Index>')
I have seen a few post related to the subject, but was not very satisfied.
So what is the simplest way to solve this problem?
In Swift 3, two range operators generate different results:
closed range operator ... -> ClosedRange (by default)
(half open) range operator ..< -> Range (by default)
So, assuming your cutRange is declared as Range<String.Index>, you need to use half open range operator ..<:
cutRange = numberString.index(numberString.startIndex, offsetBy:2)..<numberString.index(numberString.startIndex, offsetBy:6)
(Please do not miss the last offset is changed to 6.)
print(2^62)
print(2^63)
print(2^64)
In Lua 5.2, all numbers are doubles. The output of the above code is:
4.6116860184274e+18
9.2233720368548e+18
1.844674407371e+19
Lua 5.3 has support for integers and does automatic conversion between integer and float representation. The same code outputs:
4611686018427387904
-9223372036854775808
0
I want to get the float result. 2.0^64 works, but what if it's not a literal:
local n = io.read("*n") --user input 2
print(n^64)
One possible solution is to divide the number by 1: (n/1)^64 because in / division , the operands are always converted to float, but I'm looking for a more elegant solution.
Tested on Lua 5.3.0 (work2).
io.read("*n") always returns a float. So no surprises there.
If you need to convert an integer to a float, add 0.0 to it.
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
How to format a number with padding in Erlang
I developed this function :
check() ->
case get_val(Text, H) of
{ok, {Montant}} -> io:format("~s", [Montant]);
{error, pas_echeance} -> io:format("erreur")
end.
this function displayed Montant, this is an example of Montant :45
My goal is to convert this value "45" (which denotes Tunisian dinars) in the "45000" form (so I would have Tunisian millime -- each dinar consist of 1000 millimes).
Why dont you convert the value to integer (using list_to_integer("45") or list_to_integer(get_val(Text,H)) and multiply it by 1000.
can someone please help me understand what's going on here
lists:dropwhile(fun(X) -> X < 8 end, lists:seq(1,10)).
"\b\t\n" % ??? what is this ? why not [8,9,10]
lists:dropwhile(fun(X) -> X < 7 end, lists:seq(1,10)).
[7,8,9,10] % this is correct
Your results are actually correct in both cases. The unexpected string in the first case
is due to the fact that in Erlang strings are just lists of integers. Therefore, Erlang chooses to interpret your first list as a string, since it contains only printable ASCII codes. In the second case the list contains the code 7, which is not printable, so Erlang is forced to interpret it as an integer list.
You can always print the actual integer list by using
MyList = lists:dropwhile(fun(X) -> X < 8 end, lists:seq(1,10)),
io:format("~w", [MyList]).