I have a string which I want to convert to a decimal.
eg.
tax_rate = "0.07"
tax_rate.to_d
In most cases it converts to 0.07 but sometimes it converts it to 0.07000000000000001. Why?
If it helps, when I look at the log of when I insert this value into the DB, this is the relevant part:
["tax_rate", #<BigDecimal:7f9b6221d7c0,'0.7000000000 000001E-1',18(45)>]
Hopefully that makes sense to someone.
Disclaimer:
I have a feeling someone is going to ask why I'm doing this.
I've created a simple settings model, where a user can update some global settings.
The Setting model has name, value and var_type columns.
The value column is a string. I use a helper method to retrieve the value in the appropriate format depending on the value of the var_type column.
I cannot explain why but there is a chance I can tell you how to avoid having this kind of trouble when dealing with numbers: use rationals.
Here is the documentation: http://ruby-doc.org/core-1.9.3/Rational.html
As stated, a rational will always give you the exact number you want and thus will avoid rounding errors.
From the doc:
10.times.inject(0){|t,| t + 0.1} #=> 0.9999999999999999
10.times.inject(0){|t,| t + Rational('0.1')} #=> (1/1)
Let me know if this solves your problem. : )
Related
I have problem with comparison of two variables of "Real" type. One is a result of mathematical operation, stored in a dataset, second one is a value of an edit field in a form, converted by StrToFloat and stored to "Real" variable. The problem is this:
As you can see, the program is trying to tell me, that 121,97 is not equal to 121,97... I have read
this topic, and I am not copletely sure, that it is the same problem. If it was, wouldn't be both the numbers stored in the variables as an exactly same closest representable number, which for 121.97 is 121.96999 99999 99998 86313 16227 83839 70260 62011 71875 ?
Now let's say that they are not stored as the same closest representable number. How do I find how exactly are they stored? When I look in the "CPU" debugging window, I am completely lost. I see the adresses, where those values should be, but nothing even similar to some binary, hexadecimal or whatever representation of the actual number... I admit, that advanced debugging is unknown universe to me...
Edit:
those two values really are slightly different.
OK, I don't need to understand everything. Although I am not dealing with money, there will be maximum 3 decimal places, so "currency" is the way out
BTW: The calculation is:
DATA[i].Meta.UnUsedAmount := DATA[i].AMOUNT - ObjQuery.FieldByName('USED').AsFloat;
In this case it is 3695 - 3573.03
For reasons unknown, you cannot view a float value (single/double or real48) as hexadecimal in the watch list.
However, you can still view the hexadecimal representation by viewing it as a memory dump.
Here's how:
Add the variable to the watch list.
Right click on the watch -> Edit Watch...
View it as memory dump
Now you can compare the two values in the debugger.
Never use floats for monetary amounts
You do know of course that you should not use floats to count money.
You'll get into all sorts of trouble with rounding and comparisons will not work the way you want them too.
If you want to work with money use the currency type instead. It does not have these problems, supports 4 decimal places and can be compared using the = operator with no rounding issues.
In your database you use the money or currency datatype.
I have a model that has an attribute named value this is saved in the db as an integer.
I input the number in the view using a text_field, I am also performing a method on it :before_save that takes the value (something like 21.37) and using the money gem convert it to just cents.
However, it seems that before I can even perform the method that converts it from a float to an integer it is being converted to some kind of integer and the decimal is being lost.
I have tested this by outputting value in the method that runs before_save: and it round it to 21
Does anyone know why this might be happening, Im not performing any other changes to it.
I'm guessing you're doing something like Coupon.new(params) in your controller, and since Rails knows Coupon's value should be an integer, it helpfully calls to_i on params[:value] for you under the covers, which turns 21.37 into 21. Here's a rather inelegant but effective fix:
params[:value] ~= /.*\.(\d+)/
params[:value] = params[:value].to_f * (10**$1.length) if $1
Do that before you do Coupon.new(params).
Another option is to simply assume you don't care about anything after the 2nd decimal place (you're dealing with cents after all). Then you can simply do params[:value] = params[:value].to_f.round(2) * 100.
Also you should consider using number_field instead of text_field in your view so you can at least be sure you're getting a number. Something like number_field(:coupon, :value, step: 0.01) should ensure your users can enter decimals.
I agree with #kitkat seems like something that would happen in the model layer. You might be able to implement your conversion to cents logic in 'before_validation' or perhaps a custom setter for 'coupon'.
I have a function that returns a float value like this:
1.31584870815277
I need a function that returns TRUE comparing the value and the two numbers after the dot.
Example:
if 1.31584870815277 = 1.31 then ShowMessage('same');
Sorry for my english.
Can someone help me? Thanks
Your problem specification is a little vague. For instance, you state that you want to compare the values after the decimal point. In which case that would imply that you wish 1.31 to be considered equal to 2.31.
On top of this, you will need to specify how many decimal places to consider. A number like 1.31 is not representable exactly in binary floating point. Depending on the type you use, the closest representable value could be less than or greater than 1.31.
My guess is that what you wish to do is to use round to nearest, to a specific number of decimal places. You can use the SameValue function from the Math unit for this purpose. In your case you would write:
SameValue(x, y, 0.01)
to test for equality up to a tolerance of 0.01.
This may not be precisely what you are looking for, but then it's clear from your question that you don't yet know exactly what you are looking for. If your needs are specifically related to decimal representation of the values then consider using a decimal type rather than a binary type. In Delphi that would be Currency.
If speed isn't the highest priority, you can use string conversion:
if Copy(1.31584870815277.ToString, 1, 4) = '1.31' then ShowMessage('same');
I have a field -
:Revenue
and it should accept values like 10,000.00, but if I input such value it stores 10 into database instead of 10000.00
What should I do to strip of commas before I save?
I've tried to find a few solutions online but wasn't able to implement them as I found them incomplete. If someone could help me I would really appreciate it.
**The problem now I am facing is that as soon as I enter the value rails converts string in to float value before it can run the gsub function, like if I enter 50,000.00 its converting into float 50.0 before calling the gsub, is there any way to over the to_f method which rails is calling on the string.
Removing commas is pretty simple:
value.gsub(/,/, '').to_f
Keep in mind that European formatting often uses comma as the decimal value separator so your results would be off by a factor of 100 if processing those sorts of numbers.
You can take a String#delete.
"10,000,000.00".delete(',').to_f
# => 10000000.0
I found the solution after looking at few places and combining few solutions, since I had to use gsub before the linking to model has to be done. so I created the method in my controller and called it before create and update action. and wrote the following code in the method
params[:record][:Revenue] = params[:record][:Revenue].gsub(/,/,"")
I have stringgrid on delphi form and i am trying to divide values of one cell with value of another cell in another column.
But the problem is, stringgrid cells are populated with different types of numbers, so I am getting ConvertErrors.
For example the numbers in cells can look like
0.37 or 34 or 0.0013 or 0.00 or 0.35 or 30.65 or 45.9108 or 0.0307 or 6854.93.
In another words I never know is it going to be real, float, integer or any other kind of type in those cells.
I have looked everywhere on internet but no luck. Anyone any ideas. By the way I am not exactly Delphi expert. Thanks.
For each string, convert it first to a float value using StrToFloat function in SysUtils.pas . This should allow for any numerical type to be dealt with (unless you have something unusual like complex numbers). As you have some zero values in your list above you should also ensure that you check for divide by zero conditions as this will also potentially throw an exception.
SysUtils has many functions such as TryStrToFloat, TryStrToInt, TryStrToInt64 etc for this purpose. These functions accept a reference parameter (var parameter) for returning the converted value and function itself returns true if the conversion is successful.
If you are sure that the string has a valid number then you can check the input string to see if it has a decimal point before deciding which function to use.
Treat all the numbers as float. Use StrToFloat, divide the numbers, and then convert the result back to string with FloatToStr. If the result is an integer, no decimal point would be produced.