divide bug?

[DNA7433]

How come CInt(750 / 100) = CInt(850 / 100)?

They both equal 8...how is that possible? and how would i emulate this weird formula evaluation in C++?

I'm trying to convert some VB code (that works perfectly) to C++ and this is a bug I tracked down. C++ evaluates static_cast<long>(750 / 100) = 7, static_cast<long>(850 / 100) = 8

[RobDog888]

You need to use a double converter.

Option Explicit
 
Private Sub Command1_Click()
    MsgBox CDbl(750 / 100) '= 7.5
    MsgBox CDbl(850 / 100) '=8.5
    MsgBox CDbl(750 / 100) = CDbl(850 / 100) '=False
End Sub

[DNA7433]

Can't. The original code is some complex compression algorithm written by someone else. Unless you know of some good free C++ JPEG compression code that takes a byte stream and spits another one out this is my only option. The value is ultimately stored in a VB Long variable (well, its an array of data), but it seems this algorithm just works with this bug(?) and I don't know how to convert that to C++.

[chemicalNova]

I swear I saw a few examples of JPEG compression at Planet Source Code. They use DIBBits and the like and Byte arrays.

Phreak

[BuggyProgrammer]

I have seen the Jpeg thing in pure VB code as well on PSC.

Perhaps it is doing rounding where if the most sigfig is odd and the most sigfig after the decimal is 5, it will round up. IE:

1.5->2
2.5->2
3.5->4
4.5->4

(I did say maybe, but I'm pretty sure.)

It's called banker's rounding by the way: http://blogs.msdn.com/ericlippert/ar.../26/53107.aspx

[MartinLiss]

Originally posted by DNA7433
How come CInt(750 / 100) = CInt(850 / 100)?

They both equal 8...how is that possible? and how would i emulate this weird formula evaluation in C++?

I'm trying to convert some VB code (that works perfectly) to C++ and this is a bug I tracked down. C++ evaluates static_cast<long>(750 / 100) = 7, static_cast<long>(850 / 100) = 8

What you are seeing is actually a result of the VB's use of what is called Banker's rounding. The results of those divisions are of course 7.5 and 8.5 respectivly, but since you are asking VB to convert them to Integers they have to be rounded. Here is VB's explanation of Banker's Rounding

When you add rounded values together, always rounding .5 in the same direction results in a bias that grows with the more numbers you add together. One way to minimize the bias is with banker's rounding.

Banker's rounding rounds .5 up sometimes and down sometimes. The convention is to round to the nearest even number, so that both 1.5 and 2.5 round to 2, and 3.5 and 4.5 both round to 4. Banker's rounding is symmetric.

In Visual Basic for Applications, the following numeric functions perform banker's rounding: CByte(), CInt(), CLng(), CCur(), and Round().

[DNA7433]

Thanks everybody. I wrote some code to mimic the Bankers rounding and it doesn't crash anymore. The JPEG comes out as a big gray square (not the image i was compressing). I looked at the files in binary and it seems half the data isn't being written. More debugging!

[BuggyProgrammer]

Why don't you just use the intel jpeg library? Although you don't directly set the bytes and return a byte array, you can set the bytes to a dib (which the jpeg dll uses) and get the bytes again after.

[DNA7433]

I was using that, and if I remember the only thing you could do was save the compressed bytes to a file. I don't want to save them to a file, I want them back.

[szlamany]

Look at this simple example:

Private Sub Form_Load()
 
Dim x1 As Long
Dim x2 As Long
Dim x3 As Long
Dim x4 As Long
 
Dim r1 As Long
Dim r2 As Long
 
x1 = 750
x2 = 100
x3 = 850
x4 = 100
 
Debug.Print x1 / x2
Debug.Print x3 / x4
 
r1 = x1 / x2
r2 = x3 / x4
 
Debug.Print "Now for r1 and r2"
Debug.Print r1, r2
 
 
End Sub

It returns:

7.5
8.5
Now for r1 and r2
8 8

This illustrates the evils of coercion (I just did a post on this two days ago). Debug.Print is treating the "expression" x1/x2 as a variant - and furthermore as a float. Even though both operands are interger type (long).

But if I stick the results in a LONG, they both end up being 8 - makes debugging in a language so loose in coercion a pain-in-the...

BTW - if you are working with colors, never, ever use DOUBLE - these are INTEGERS - LONG and need to be treated as such throughout your code. As soon as you loose control of the datatype, it's over

Don't know if this will help, but try PUBLIC CONST... for all your "literal" values - maybe that will help VB focus on a particular datatype