[RESOLVED] Odd Round behaviour
Something odd I've noticed in VBA.
If you evaluate Round(16 * 1000/100,1) you will get 160.
If you do Round(16 * 10000/1000,1) you will generate an overflow exception.
If you put a tiny decimal in, though, you will be fine.
Round(16.0001 * 10000/1000,1) = 160
There must be some form of coercion to an integer form with a specific number of bytes if you don't use a decimal, which then trips the overflow.
Re: [RESOLVED] Odd Round behaviour
I guess I'm a little surprised too that more people in this thread haven't run into this before. I've dealt with a lot of financial-based systems over the years so this was something I ran to pretty early in my career. I guess it's something more common when $250,000 is chump change.
-tg
Re: [RESOLVED] Odd Round behaviour
Always used currency datatypes myself - so that's a longword with a 4-digit implied decimal point.
[edit] back when I did vb6 [/edit]