Numbers funny business - please help

Could anyone explain the following for me because I am completely baffled:

int(2.01*100) returns 200
int(2.05*100) returns 204

but as expected

int(2.02*100) returns 202
int(2.06*100) returns 206
int(2.09*100) returns 209

and

int(201) returns 201
int(205) returns 205

Before you ask why do I need to use Int it is because the first number, which in my code is a variable, may have more than two decimal places.

And another baffler which probably is behind the results above

Dim Number As Double = 2.01 * 100 'returns 200.99999999999997
Dim Number As Double = 2.05 * 100 'returns 204.99999999999997


Please help me I am going crazy here.

Welcome to the wonderful world of floating-point numbers. This phenomenon is well documented. You should do some reading on floating-point numbers to get a full understanding.

Maybe you should use Math.Round instead of Int.

I can't use Math.Round because I want to round down always. Is there no solution to this phenomenon

Try using Decimal instead of Double, which suffers far less from such errors. That's why Decimal is used for financial calculations. Decimal is bigger and slower than Double though, hence the reason that Double is used wherever it can without causing issues.

Thanks for that. Decimal works fine.

This is a mighty big error. Why hasn't it been addressed?

Because floating points are imprecise... it's always been that way. Mostly has to do with the fact that decimals don't translate into binary very well sometimes.

And it has been addressed. That's why we have decimal type.

-tg

Quote:

---

Originally Posted by natrap

This is a mighty big error. Why hasn't it been addressed?

---

You haven't done any reading on floating-point numbers, have you? If you do, then you'll understand.

Quote:

---

Originally Posted by natrap

I can't use Math.Round because I want to round down always. Is there no solution to this phenomenon

---

Use Math.Floor then (but I realize this won't solve your main issue).

Floating point manipulation can be frustrating - there are entire university course dedicated to numerical analysis which cover this and more.
It can seem unintuitive to realize that a $5 calculator (or Windows 'Calculator') can perform rings around the output that we get for simple floating point calculations, but there it is. I've often thought that programming languages might benefit from having a 'calculator' mode - that is, perform as well as a calculator without having to have a math degree to make it happen.

That's because calculators don't use floating point processors... they only have to deal with the first 8-10 significant digits, and therefore use what would be the equivelent of a decimal type.

-tg

Calculator mode = Decimal.
But Decimal = relatively slow
Therefore Calculator = relatively slow (but your fingers are far slower, so you never notice).