Printable View
Hi,
When I multiply a variable of integer data type and a variable of single data type, the result isn't always the right one.
For example - 0.02 * 5 isn't 0.1, but 0.099999994.
Why does this happen? I've tried using double as well, but it's the same problem.
you will want to format it for display. I just do this:
Format(cDbl(variable * variable), "##0.0000")
Floating point values are not supposed to be accurate.
If you are looking for accuracy do not ever use them.
We do financial and payroll systems here - we never ever use float.
Try using decimal rather than single.
In general, the problem is that a single/double is the number out to a certain precision. Since the number is binary, not all decimal fractions can be accurately represented in the number of bytes allocated to the number. 0.1 is one tenth, which is a concept hard to capture in a binary system. The way around this is to approximate it. The result you got is 0.1 when you round it off, back to several decimal places, so it is correct....unless you use the entire precision of the number.
There was a lengthy explanation of how fractional numbers are represented in a binary system in the Classic VB forum a few years back. It was thorough, detailed, and probably more than anyone wanted to know, but it has disappeared into the mist of time now.
The decimal type takes care of this better, but you should keep in mind that as long as you are using a floating point number, things like this can happen. Formatting is a good way to display things correctly.
Thanks :)