Is there a way in x++ 2012. that i can force two decimal places to .00 even if it is a whole number. so 400 would be 400.00
I was using this conversion.
num2str0(amount, 0, 2, 1,0)
It works fine if there is a decimal but if not, just outputs 400 for example.
Related
Can anyone explain why the result is 252.99999999999997 and not 253? What should be used instead to get 253?
double x = 2.11;
double y = 0.42;
print(((x + y) * 100)); // print 252.99999999999997
I am basically trying to convert a currency value with 2 decimal (ie £2.11) into pence/cent (ie 211p)
Thanks
In short: Because many fractional double values are not precise, and adding imprecise values can give even more imprecise results. That's an inherent property of IEEE-754 floating point numbers, which is what Dart (and most other languages and the CPUs running them) are working with.
Neither of the rational numbers 2.11 and 0.42 are precisely representable as a double value. When you write 2.11 as source code, the meaning of that is the actual double values that is closest to the mathematical number 2.11.
The value of 2.11 is precisely 2.109999999999999875655021241982467472553253173828125.
The value of 0.42 is precisely 0.419999999999999984456877655247808434069156646728515625.
As you can see, both are slightly smaller than the value you intended.
Then you add those two values, which gives the precise double result 2.529999999999999804600747665972448885440826416015625. This loses a few of the last digits of the 0.42 to rounding, and since both were already smaller than 2.11 and 0.42, the result is now even more smaller than 2.53.
Finally you multiply that by 100, which gives the precise result 252.999999999999971578290569595992565155029296875.
This is different from the double value 253.0.
The double.toString method doesn't return a string of the exact value, but it does return different strings for different values, and since the value is different from 253.0, it must return a different string. It then returns a string of the shortest number which is still closer to the result than to the next adjacent double value, and that is the string you see.
The Erlang float_to_binary function truncates decimals strangely. For instance, I would expect it to convert 0.45 with no decimal places to "0". Instead we get (example in Elixir):
iex> :erlang.float_to_binary(0.45, [decimals: 0])
"1"
iex> :erlang.float_to_binary(0.445, [decimals: 0])
"1"
> :erlang.float_to_binary(0.444, [decimals: 0])
"0"
Thus, it seems like rounding is being applied iteratively from right to left until the desired number of decimals is reached.
Is this expected behavior? Why doesn't it either round correctly or just truncate? Both of those options seem much more predictable to me.
This was a bug in Erlang which was fixed on Jan 15 2018 and first included in Erlang 20.3. If you upgrade to Erlang 20.3 or later, you should get "0" for 0.445.
Ok...I think I'm missing something very obvious here but haven't been able to google myself through this solution. I have two simple rails methods that calculate the number of up votes and down votes. They will always return a fraction because i'm trying to show a percentage (up_vote_count / votal_vote_count). I open the rails console and run the following:
y = #somespecificrecord
then...
y.up_vote_count
This returns 1 as is expected
y.down_vote_count
This returns 1 as is expected
y.total_vote_count
This returns 2 as is expected.
However, when I run in the console...
y.up_vote_count / y.total_vote_count
This returns 0 when it should return .50. I've been reading about floats/integers/decimals, etc and I do see this in the schema on the model i'm working from:
t.float "value", default: 0.0
Is this my problem?...and if so what do I have to do to allow myself to do a simple formula like the one above in rails console that will return the correct decimal rounded to 2 digits (i.e, .50 in this case above). I don't know if I want to run any migrations to change data types because this is a gem (& as a beginner I tend to stay away from customizing code from any gems I'm using). Is there another way? something small i'm missing hopefully?
UPDATE:
I'm learning decimals are slower than floats also, so is there any way to accomplish this with continuing to use t.float "value", default: 0.0
thanks for any help.
1 / 2 = 0.5
With integers this will round down to 0
You can get around this by casting the divisor to a float, forcing it to do division with floating point accuracy.
y.up_vote_count / y.total_vote_count.to_f
Float
Float objects represent inexact real numbers using the native
architecture's double-precision floating point representation.
Floating point has a different arithmetic and is an inexact number.
its important to know if you divide 2 hole numbers you are going to get a hole number.
if you are looking a decimal you should first convert your numbers to a decimal or a "float" like this
up_votes = y.up_vote_count.to_f
total_vote = y.down_vote_count.to_f
(up_votes / total_vote) * 100.0
I hope that this helps
I am doing the following in actionscript in Coldfusion Flash Forms:
90 / 3.7
Gives me:
24.3243243243243
Whereas the calculator gives me:
24.32432432432432
Note the extra 2 at the end.
So my problem occurs when I am trying to get the original value of 90 by taking the 24.3243243243243 * 3.7 and then I get 89.9999999999 which is wrong.
Why is Actionscript truncating the value and how do I avoid this so I get the proper amount that the calculator gets?
Thanks so much.
Round your number using a routine like this
var toFixed:Function = function(number, factor) {
return (Math.round(number * factor)/factor);
}
Where the factor is 10, 100, 1000 etc, a simple way to think about it is the number of 0's in the factor is the number of decimal places
so
toFixed(1.23341230123, 100) = 1.23
Good explanation of numeric in ActionScript can be found at http://docstore.mik.ua/orelly/web2/action/ch04_03.htm. See section 4.3.2.1. Floating-point precision
A relavant quote:
"In order to accommodate for the minute discrepancy, you should round your numbers manually if the difference will adversely affect the behavior of your code. "
I created a calculator class that does basic +,-, %, * and sin, cos, tan, sqrt and other math functions.
I have all the variables of type double, everything is working fine for big numbers, so I can calculate numbers like 1.35E122, but the problem is with extremely small numbers. For example if I do calculation 1/98556321 I get 0 where I would like to get something 1.01464E-8.
Should I rewrite my code so that I only manipulate NSDecimalNumber's and if so, what do I do with sin and cos math functions that accept only double and long double values.
1/98556321
This division gives you 0 because integer division is performed here - the result is an integer part of division. The following line should give you floating point result:
1/(double)98556321
integer/integer is always an integer
So either you convert the upper or the lower number to decimal
(double)1/98556321
or
1/(double)98556321
Which explicitely convert the number to double.
Happy coding....