I am doing the following in actionscript in Coldfusion Flash Forms:
90 / 3.7
Gives me:
24.3243243243243
Whereas the calculator gives me:
24.32432432432432
Note the extra 2 at the end.
So my problem occurs when I am trying to get the original value of 90 by taking the 24.3243243243243 * 3.7 and then I get 89.9999999999 which is wrong.
Why is Actionscript truncating the value and how do I avoid this so I get the proper amount that the calculator gets?
Thanks so much.
Round your number using a routine like this
var toFixed:Function = function(number, factor) {
return (Math.round(number * factor)/factor);
}
Where the factor is 10, 100, 1000 etc, a simple way to think about it is the number of 0's in the factor is the number of decimal places
so
toFixed(1.23341230123, 100) = 1.23
Good explanation of numeric in ActionScript can be found at http://docstore.mik.ua/orelly/web2/action/ch04_03.htm. See section 4.3.2.1. Floating-point precision
A relavant quote:
"In order to accommodate for the minute discrepancy, you should round your numbers manually if the difference will adversely affect the behavior of your code. "
Related
Ok...I think I'm missing something very obvious here but haven't been able to google myself through this solution. I have two simple rails methods that calculate the number of up votes and down votes. They will always return a fraction because i'm trying to show a percentage (up_vote_count / votal_vote_count). I open the rails console and run the following:
y = #somespecificrecord
then...
y.up_vote_count
This returns 1 as is expected
y.down_vote_count
This returns 1 as is expected
y.total_vote_count
This returns 2 as is expected.
However, when I run in the console...
y.up_vote_count / y.total_vote_count
This returns 0 when it should return .50. I've been reading about floats/integers/decimals, etc and I do see this in the schema on the model i'm working from:
t.float "value", default: 0.0
Is this my problem?...and if so what do I have to do to allow myself to do a simple formula like the one above in rails console that will return the correct decimal rounded to 2 digits (i.e, .50 in this case above). I don't know if I want to run any migrations to change data types because this is a gem (& as a beginner I tend to stay away from customizing code from any gems I'm using). Is there another way? something small i'm missing hopefully?
UPDATE:
I'm learning decimals are slower than floats also, so is there any way to accomplish this with continuing to use t.float "value", default: 0.0
thanks for any help.
1 / 2 = 0.5
With integers this will round down to 0
You can get around this by casting the divisor to a float, forcing it to do division with floating point accuracy.
y.up_vote_count / y.total_vote_count.to_f
Float
Float objects represent inexact real numbers using the native
architecture's double-precision floating point representation.
Floating point has a different arithmetic and is an inexact number.
its important to know if you divide 2 hole numbers you are going to get a hole number.
if you are looking a decimal you should first convert your numbers to a decimal or a "float" like this
up_votes = y.up_vote_count.to_f
total_vote = y.down_vote_count.to_f
(up_votes / total_vote) * 100.0
I hope that this helps
Can somebody explain why multiplying by 100 here gives a less accurate result but multiplying by 10 twice gives a more accurate result?
± % sc
Loading development environment (Rails 3.0.1)
>> 129.95 * 100
12994.999999999998
>> 129.95*10
1299.5
>> 129.95*10*10
12995.0
If you do the calculations by hand in double-precision binary, which is limited to 53 significant bits, you'll see what's going on:
129.95 = 1.0000001111100110011001100110011001100110011001100110 x 2^7
129.95*100 = 1.1001011000010111111111111111111111111111111111111111011 x 2^13
This is 56 significant bits long, so rounded to 53 bits it's
1.1001011000010111111111111111111111111111111111111111 x 2^13, which equals
12994.999999999998181010596454143524169921875
Now 129.95*10 = 1.01000100110111111111111111111111111111111111111111111 x 2^10
This is 54 significant bits long, so rounded to 53 bits it's 1.01000100111 x 2^10 = 1299.5
Now 1299.5 * 10 = 1.1001011000011 x 2^13 = 12995.
First off: you are looking at the string representation of the result, not the actual result itself. If you really want to compare the two results, you should format both results explicitly, using String#% and you should format both results the same way.
Secondly, that's just how binary floating point numbers work. They are inexact, they are finite and they are binary. All three mean that you get rounding errors, which generally look totally random, unless you happen to have memorized the entirety of IEEE754 and can recite it backwards in your sleep.
There is no floating point number exactly equal to 129.95. So your language uses a value which is close to it instead. When that value is multiplied by 100, the result is close to 12995, but it just so happens to not equal 12995. (It is also not exactly equal to 100 times the original value it used in place of 129.95.) So your interpreter prints a decimal number which is close to (but not equal to) the value of 129.95 * 100 and which shows you that it is not exactly 12995. It also just so happens that the result 129.95 * 10 is exactly equal to 1299.5. This is mostly luck.
Bottom line is, never expect equality out of any floating point arithmetic, only "closeness".
I am trying to convert an erlang time format tuple, {megasec,sec,microsec}, into a floating point number and back again.
I can do this one way, e.g.:
{Megasec,Sec,Usec} = erlang:now().
Total = Megasec*1000000+Sec+Usec/1000000.
1352802601.427
But I am struggling to convert this number back to the time format. I have a general idea to divide by 1000000 and round but I get rounding errors. e.g.
Mega = erlang:round(Total/1000000).
1353
If I could get this accurately I could apply similar steps to get Seconds and Microseconds.
Any ideas?
You can use erlang:trunc instead of erlang:round.
Following #Falco Hirschenberger's suggestion here's how I did it:
Mega = erlang:trunc(Total/1000000).
1352
Sec = erlang:trunc(Total - Mega*1000000).
802601
Usec = erlang:round((Total - Mega*1000000 - Sec)*1000000).
427000
Note. I had to use erlang:round to get Usec (else the answer would have been 427000.0457763672 - I think this is due to a rounding error introduced when I divided by 1000000)
I'm implementing line charts of primefaces(3.0) , I'm trying to change the value of X-scale
The values which I'm using are minX="0" maxX="38" , since primefaces linecharts is using jqplot , I added this script
<script>
$(function(){
widget_category.plot.axes.xaxis._tickInterval = 1;
widget_category.plot.axes.xaxis.numberTicks = 38;
});
</script>
But still the coordinates is coming in decimals.
I would like to mention that for Y scale, the values I used are minY="40" maxY="110" with style="height:1005px;" , As i figured out for a scale value , which can be 10 if height is defined as 1005px i.e. 5 * 14 = 70 which means Y scale is of 5 intervals , with 14 values and the line height is 1005 as 5*14*14 = 980 + 25 (which is top-margin added) 1005.
Though the same is not working out for X-Scale.
Any help would be helpful.
The arithmetic in your Y values are all multiplication operations on whole numbers, which will always result in a whole number. These whole numbers correlate perfectly to pixels.
Your X range however involves a multiplication of 1.0 and 38, one being an integer value and the other being determined as a float or double number. When performing a multiplication operation where one number is a float then the result value will always be a float, and standard floating point artithmetic rules will apply. This is why the coordinates are coming in decimals which don't equate perfectly to pixels.
When using Javascript you need to be careful of these kinds of pitfalls because it is not a strongly typed language like Java and it will not point things like this out to you.
I am very curious why Ruby render decimal with precision 2 inconsistently.
For example:
helper.number_to_currency 9.995
=> "$9.99"
whilst
helper.number_to_currency 10.995
=> "$11.00"...should it be "$10.99"?
It is a floating point precision error. I will probably submit a patch to fix this because I'm using number_to_currency on my website too.
Here's what's happening in more detail:
number_to_currency ends up just calling number_with_precision to get the number correctly formatted. number_with_precision immediately converts the number to a Float. Essentially, it comes down to this line in the Rails code:
# File actionpack/lib/action_view/helpers/number_helper.rb, line 280
rounded_number = BigDecimal.new((number * (10 ** precision)).to_s).round.to_f / 10 ** precision
The number you provide is multiplied by 100 before it is converted to BigDecimal. Look at this simple irb session:
irb(main):001:0> 9.995 * 100
=> 999.4999999999999
That number would obviously round down to 999, then it will be divided by 100 to give you 9.99.
The only workaround for this that I can think of for the moment is to do your own rounding before you pass in the number.
Looks like it is using "Round half to odd" rule.
So "1.5" gets rounded down to the nearest odd number (1) and 2.5 gets rounded up to the nearest odd number (3).
For a randomly distributed set of numbers to be rounded this form of rounding will consitently produce the smallest difference between the sum of unrounded numbers and the sum of rounded numbers.