0.768 + 0.032 = 0.800000000000001 ??? what??

I am incrementing a double value by a small decimal less than 1. Can anyone explain how the app can return something like 0.8000000000001. for the first few cycles, it works OK but then it starts giving weird numbers, and then a few cycles later it settles down again. WHATS GOIN ON?

Is it something to do with double data types? how do I get around this?

I tried your equation and it worked. Incase you are interested, here is the code I used.

Private Sub Command1_Click()

    Dim a, b, c As Double
    a = 0.768
    b = 0.032
    c = a + b
    MsgBox (c) ' Display my number

End Sub

[parksie]

you can't get around the problem, because it happens when you get to the limit of the binary system, such as numbers like 0.1 which are easy in decimal, but a recurring number in base 2. also, small numbers get rounded a lot, and then it arses up. if you premultiply by some arbitrary number, then divide, it should be fine.

Dim a, b, c As Double

that declares a & b as variants, only c is the double, you need to do this.

Dim a As Double, b As Double, c As Double

[kedaman]

Yeah, dennis you're right. But i think you can use currency instead of double, it's a integer with 4 decimals so in this case it's the best way.

[kmchong]

It is a type conversion problem, I think.

I hit the same problem almost 10 years ago in using Turbo Pascal. 2-1 will equal to 1.0000.......0001 (can't you believe, there are total of 30 zero)

After that, I very seldom use figures declared as scientific representation. All those float, double etc may raise these kind of problem.

My solution is:

1. Try to use integer/currency as much as possible
2. If not, then add the rounding code for every calculation afterward. (This was the way I used in Turbo Pascal, but it is painful to spend 2 weeks just to round all the figures)

[parksie]

it's a problem with the way floating-point numbers are stored. see this for a description: http://www.cerfnet.com/~mpcline/c++-...tml#[34.8]

[Paul282]

wossname,

I've never had these problems, can you post some test code that failed, I'm not sure what's failing. Are you incrementing in a loop?

I just chucked 0.768 + 0.032 into the immediate window and got 0.8 ...?

Please explain to the curious

[kedaman]

Paul, that's because immediate window doesn't show it at all, same thing with msgbox, you need to substract to get a exponent value

[Paul282]

Tried it in an app to...

The example says "+" and "increment"

Anyone got any code to display

[kedaman]

Try something like this

?value-int(value/10^x)*x

[HarryW]

It's just as Parksie said - it's because of the way floating point numbers are stored in binary.

If I remember correctly a floating point value is stored in binary by storing it in something like standard form, but with a base of 2 instead of 10. I'm not sure if it's exactly like that but it's stored in two parts, like a mantissa and exponent or something.

Some numbers just don't fit nicely into this, even though they may seem simple in decimal.

This anomaly is the reason why you shouldn't really check floating point variables for equality, you should just check that they are very close. Although the maths involved might suggest they should be equal, the rounding when they are stored in binary means that the results can be a little off what you expect. In wossname's case 0.0000000000001 off what was expected

if you read my original post, you will find that it is not just that sum, i am doing several increments of the value, if you add 0.032 to 0.8 then you get 0.8000000.....001.

I am not incrementing in a loop, but that woulndt make any difference anyway.

This clearly seems to be a massive discrepancy in the way computers work!

I can't be bothered with this anymore, it seems so stupid because a miss is as good as a mile as far as maths is concerned.

And we trust computers to give out things like values for medicine doses and voltages for cardiac machines!!!!!

[kedaman]

Also if you need more than 4 decimals, that currency provide, use the decimal datatype, and you will have 28, but it's a variant so you need to use cdec to convert it...

[Paul282]

wossname,
I believe it's the job of the programmer to make the technology work! You take a look at how a CPU works and you'll be impressed at how far they've come in just a few years, life is easy for us compared with the recent past, if the only cost of that is having to make a few checks... better that than VB being even higher level! go forbid if we couldn't even work with the bits!.

If you can think up a better way of accurately doing math with huge varying floating point numbers in just ones and zeros, I'd be VERY impressed.

Use classes then, build you're own floating point types for better precision!

[Joacim Andersson]

If you just add 0.768 + 0.032 then you add two single presition values.
To use double presition values change your code to something simular to this:

Dim a As Double

a = 0.768# + 0.032#

Good luck!

[kedaman]

Paul, I think that would slow down the calculations enormously, so i would rather just make some functions that operates values in strings, but that's still slow, i think decimal datatype is enough for most calculations

[HarryW]

Well wossname, this is a problem that has been around for longer than you or I (assuming you're aged less than about 50 or something) and it's something that is quite easily dealt with. It just takes a little thought. And I wouldn't call a difference of 0.0000000000001 in 0.8 a massive discrepancy.

Like I said, you can just round the figures to a reasonable accuracy if you're outputting them, or if you want to use the figure in a boolean expression then just make sure you check if the two values you are comparing are very close to each other, not strictly equal.

This was something that I was told at my first programming lecture on real numbers, it's not hard to handle.

[kmchong]

wossname,

You trust the computer and it 'betrays' you that make the life very difficult.

That's why I hate M$, as a OS, it should be and need to be trusted. But often it come out bugs, sometimes even more than the AP. When face with end users, we have to bear the blame and M$ bear the glory, FAIR?

[Paul282]

Originally posted by kedaman
Paul, I think that would slow down the calculations enormously, so i would rather just make some functions that operates values in strings, but that's still slow, i think decimal datatype is enough for most calculations

I don't doubt that for a minute. I didn't say I'd do it, nor that it was a good idea.

Just my way of saying, "Deal with it, it goes with the territory"

[kedaman]

Anyway if they could work out a processor to work with decimal floating points, it would be nice.
kmchong, i think that's not microsofts fault.

[Paul282]

Fundamental Bit math.

It's in differing degrees on different chips (poor in the first pentium chip with the floating point bug) but it's not a VB issue.

If you do a bit or research around the web I'm sure you'll get enough info to beable to avoid problems...

I've never needed to. I don't do much floating point math.

I might take a bit of a look though. If I find any good sites I'll post back here

[Paul282]

The Crowd goes wild

(Complete) Tutorial to Understand IEEE Floating-Point Errors



http://support.microsoft.com/support...s/q42/9/80.asp