I'm fairly new to VB.Net, but am fairly fluent in VB6. I have some code like this:
Where dblTwo = 17 and dblOne = 16.94 and dblTotal ends up being equal to "0.0599999999999987"Code:dblTotal = dblTwo - dblOne
huh? :confused:
Printable View
I'm fairly new to VB.Net, but am fairly fluent in VB6. I have some code like this:
Where dblTwo = 17 and dblOne = 16.94 and dblTotal ends up being equal to "0.0599999999999987"Code:dblTotal = dblTwo - dblOne
huh? :confused:
Its a seems to be a big problem in VB6 with double precision, too.
Try casting to a Decimal...
VB Code:
Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click Dim d1, d2, d3 As Double d1 = 16.94 d2 = 17 d3 = CType(d2, Decimal) - CType(d1, Decimal) MessageBox.Show(d3) End Sub
That's a well known bug ("feature" in MS lingo) of floating point numbers in binary. It's the reason you don't check floating point numbers for exact equality, just for an arbitrarily small difference. Like crpptcblade says, you'll get the same problem in VB6 as well, not to mention C, C++, Java, Pascal, etc.
I switched to using the Decimal datatype and it works fine.