Bug #13210
closedWrong float substraction result
Description
0.1 - 0.0001 => 0.0999 0.1 - 0.001 => 0.099 0.1 - 0.01 => 0.09000000000000001 0.01 - 0.0001 => 0.00999 0.01 - 0.001 => 0.0099 0.01 - 0.01 => 0.009000000000000001 There is something strange going on.
As you can see, the subtraction of A-B with B being ten times smaller as A returns a wrong result. But this is somehow not always the case:
2.4.0 :001 > for i in 1..21 do p "#{0.1*i} - #{0.01*i} = #{0.1*i - 0.01*i}" end "0.1 - 0.01 = 0.09000000000000001" "0.2 - 0.02 = 0.18000000000000002" "0.30000000000000004 - 0.03 = 0.27" "0.4 - 0.04 = 0.36000000000000004" "0.5 - 0.05 = 0.45" "0.6000000000000001 - 0.06 = 0.54" "0.7000000000000001 - 0.07 = 0.6300000000000001" "0.8 - 0.08 = 0.7200000000000001" "0.9 - 0.09 = 0.81" "1.0 - 0.1 = 0.9" "1.1 - 0.11 = 0.9900000000000001" "1.2000000000000002 - 0.12 = 1.08" "1.3 - 0.13 = 1.17" "1.4000000000000001 - 0.14 = 1.2600000000000002" "1.5 - 0.15 = 1.35" "1.6 - 0.16 = 1.4400000000000002" "1.7000000000000002 - 0.17 = 1.5300000000000002" "1.8 - 0.18 = 1.62" "1.9000000000000001 - 0.19 = 1.7100000000000002" "2.0 - 0.2 = 1.8" "2.1 - 0.21 = 1.8900000000000001" => 1..21 Even 0.1*x seems sometimes to fail:
2.4.0 :002 > for i in 1..21 do p "0.1*#{i} = #{0.1*i}" end "0.1*1 = 0.1" "0.1*2 = 0.2" "0.1*3 = 0.30000000000000004" "0.1*4 = 0.4" "0.1*5 = 0.5" "0.1*6 = 0.6000000000000001" "0.1*7 = 0.7000000000000001" "0.1*8 = 0.8" "0.1*9 = 0.9" "0.1*10 = 1.0" "0.1*11 = 1.1" "0.1*12 = 1.2000000000000002" "0.1*13 = 1.3" "0.1*14 = 1.4000000000000001" "0.1*15 = 1.5" "0.1*16 = 1.6" "0.1*17 = 1.7000000000000002" "0.1*18 = 1.8" "0.1*19 = 1.9000000000000001" "0.1*20 = 2.0" "0.1*21 = 2.1" => 1..21 And to be sure that this can lead to problems:
2.4.0 :003 > 0.1 * 3 == 0.3 => false I successfully reproduced the wrong results on another device. Both devices are running on Ubuntu. I also reproduced it with ruby 2.3.1.
The wrong results somehow have a pattern, but I'm not sure about their origin.
I suggest ranking this bug as critical, as ruby should be able to calculate correctly.
BR Alex
Edit: typo
Updated by dexter3 (Alex V) over 8 years ago
- Description updated (diff)
Updated by duerst (Martin Dürst) over 8 years ago
- Status changed from Open to Rejected
Floating point calculations have limited precision!
Please read http://docs.oracle.com/cd/E19957-01/806-3568/ncg_goldberg.html.
It's linked from the top page of this site (https://bugs.ruby-lang.org/).