floating point problem

[gunn]

Hi everyone,

Dim a as Double = 292452.13
Dim b as Double = 36556.52
Dim c as Double = a - b

The value for variable c is 255895.61000000002. It is suppose to be 255895.61. How can this simple substraction can be wrong? Can anyone help me? Is that because of my computer setting?

Please help me... URGENT

Thank you very much

Andy

[Kimmy4]

Have you tried formatting your variable so that it only has 2 digits after the decimal point?

Hope this helps

[gunn]

Thank you for the reply

Yes, it works if i format it. But I am looking for a solution which isn't need for formatting... Last time it was fine, but then when i tried to play around with REGIONAL SETTING, then the result become like this... I have tried to check the decimal point on regional setting, it is 2... Do you have any idea on this?

Thank you very much

[Shaggy Hiker]

Your problem is the inability to show certain floating point numbers to infinite precision. There was a thread on this many years ago, but I forget it now. As long as you are using doubles, this will always happen. You might change your data type to decimal, but otherwise you will need to understand that addition, subtraction, multiplication, or division will produce an answer that will be the right one plus or minus a reeeeeeaaaaallly small amount.

[gunn]

Is this the bug of VB.NET? because when i put the same code on VB6, it was fine... I become confuse about this...

[Shaggy Hiker]

No, this is a fundamental issue with floating point operations. If it never happened in VB6, it was only because you got lucky. Floating point operations are accurate only to a certain number of decimal places. However, they are longer than they are accurate, so there will often be a little noise. Use Decimal in place of double, and you will generally be happier.

Basically, in all languages, VB6, .NET, C/C++, etc., you can never assume that a double will be exactly equal to any particular number. Therefore, you should never have a line like this:

If d = n Then

Where d is a double, and n is anything at all. There are circumstances where this will work, but there are many situations where this will fail because d will be just slightly different.

[gunn]

Thank you Shaggy Hiker...
That is the problem that I face... I always try to compare by using IF and it always wrong.... I know now, I have to format it before I compare it...

Thank you

[Shaggy Hiker]

Again, try decimal. It may prove to be just what you were hoping for.

One way I have seen to get around this is to have a function like this:

Public Function IsEqual(d1 as double, d2 as double) as boolean
 dim someTinyNo as Double = 0.00001
 
 if d2< (d1+someTinyNo) AndAlso (d2 > d1-someTinyNo) Then
  return true
 else
  return False
end function

OF course, someTinyNo in this example is actually larger than it should be for all possible comparisons, but would be good enough for most. Also, this could be overloaded to take different types other than just doubles. It isn't a really fast solution, but it is ok for most cases.