[RESOLVED] Rounding issues ???? or Not !!

[JohnSavage]

Now last night when I was trying to sort this out (with plenty of help )

http://www.vbforums.com/showthread.p...87#post3048587

An issue arose highlighted by LaVolpe: at post 29

Rounding issues when dividing.
Calculate minutes differently: Minutes = Int(TimeVal) Mod 60

Now given many of my app’s use Double’s and Division could I have this problem all over the place in my calculations

If so what do I need to search for in my code

Is it only with Division of Doubles

[LaVolpe]

Rounding is probably depending on the pc's processor and the exact value of .5 is the culprit when converting decimal to non-decimal data types. A decision has to be made: is .5 rounded down or rounded up. Different processors, different rules? But the rule that appears to be used is whether or not the whole number is even or odd - crappy rule, huh? Honestly I thought VB did the tie-breaker but between your system & my system, we got different values (are we both running VB SP6?). Rounding all other decimal values are determined by being > or < .5

Things to look for. Anything where you convert a decimal value to a non-decimal variable type. Anything? Huh?, Yep, the following will return the same value. Horrible

? cint(31.5);cint(32.5)

Obviously, when working with decimal types, it is best to not convert back and forth. Building your own rounding rule may help too, basically deciding beforehand if .5 will be rounded down or up if the decimal portion is exactly .5

[JohnSavage]

First, yes I am running VB6 SP6

I use a lot of Doubles in my app’s but when I store them in my database I convert them into text and then when I load my app I convert them back into doubles

Using CDbl() & CStr()

Is this the type of thing that may cause me problems

Or is it only when using whole numbers

[LaVolpe]

Can't see how. CStr() does no conversion to integer/long. CDbl() & Val() return double values when used on a string's value.

Edited: When db is used, why not store them in a data field appropriate for doubles. Double uses 8 bytes, whereas a string uses 1 to 2 bytes per character (ansi vs unicode). Do all dbs store strings in 2 byte format?

[JohnSavage]

So the problem is only when dividing whole numbers and getting a decimal answer ???

[JohnSavage]

Let me explain myself better

Dim Number as Integer
Number = 1/5

This above maybe a problem

But this below will always be OK

Dim Number as Double
Number = 1/5

[LaVolpe]

So the problem is only when dividing whole numbers and getting a decimal answer ???

No, the problem is when converting decimal numbers to byte, integer, long only when the decimal portion is exactly .5
VB's Round() function suffers from say problem: ? Round(31.5); Round(32.5) along with integer division (\) and any function that expects a non-decimal value

[JohnSavage]

You ask:

When db is used, why not store them in a data field appropriate for doubles

Because my encryption routine can only work with text values

Why do I use an encryption routine ? Because I use Access and it’s not built in

[JohnSavage]

the problem is when converting decimal numbers to byte, integer, long only when the decimal portion is exactly .5

OK now I understand - thank for your time AGAIN !!

[LaVolpe]

John, take a look at this link from developerfusion -- a nice site.

[JohnSavage]

Thanks for that I have just read it and will go back later but it says in that article that "John is 25 years old" now given that this statement is way out by a matter of some 30 years I can now see how this issue causes real problems with simple calculations

[randem]

Hey John,

Binary is binary, even if you encrypted 1001111010011101010100011100
if you stored it in a double it would still be a double. No need to convert to a string... Am I missing something?

[JohnSavage]

Randem

My encryption routine makes an encrypted text string therefore if I tried to save it to a database field of type Double then I would get an error.

Now as a result of this, all of my field types are text and therefore when I get a record from my database I have to use:

IJR = CDbl(Encrypt(RS1.Fields("32").Value, Key))

Which first de-encyrpts it and then changes it back to a double

or similar

But I could be wrong again

If there is a strong encryption rountine that works regardless of data type then that would make my life at the keyboard a whole lot less complicated

[randem]

Good Encryption routines should not care about data type. As I stated all data is binary. So encrypting a 8 byte double field would still yield 8 bytes or less and can be stored in the same data type.

[JohnSavage]

Randem

You say:

“Good Encryption routines should not care about data type. As I stated all data is binary. So encrypting an 8 byte double field would still yield 8 bytes or less and can be stored in the same data type”

I am sure you are correct but how does this statement help me?

You have a link in your signature “String Encryption” is that not good then?

All I can do is repeat:

If there is a strong encryption routine that works regardless of data type then please tell me were to find it as it would make things a lot easier for me.

[randem]

The link in my signature doesn't care about text, it's a binary string which can contain any binary signature.

Ex.

Take a long say, &FFFF then AND it with &1111 (Basically Encrypting it). It still stays a Long and can be stored as such. You don't need to turn it into a string first then save it as a string or text field.

[JohnSavage]

Many thanks - I was under the impression that you did have to give the function a string and then convert it back !

But that just about sums up my VB knowledge!