Doubles - Overflow

**Mc Brain** · Jul 28th, 2003, 07:37 PM

Does anybody know how not to get "Overflow" in the last line of this code??

VB Code:

Dim dLevel As Double
    Dim dBitWiseAnd As Double
 
    'Just in case, the value of that field is rs!DAEGR_Level=2147483648 
    dBitWiseAnd = CDbl(rs!DAEGR_Level) And dLevel

**phinds** · Jul 28th, 2003, 07:54 PM

probably bit-wise AND acts as an integer operation but you're using it on a floating point number.

**VictorB212** · Jul 28th, 2003, 07:54 PM

I believe the And operator only works on integer types.

**VictorB212** · Jul 28th, 2003, 08:02 PM

What exactly are you doing? Bitwise operators aren't defined for floating-point numbers because the operation is meaningless. If you really want to use them on floating point numbers, use the CopyMemory API to convert them to longs, do the operation, and then convert mack to doubles. I don't understand why all the code is needed that DiGiTaIErRoR just provided (if any code is NOT self-documenting, that's a good example).

**DiGiTaIErRoR** · Jul 28th, 2003, 08:08 PM

Originally posted by VictorB212
What exactly are you doing? Bitwise operators aren't defined for floating-point numbers because the operation is meaningless. If you really want to use them on floating point numbers, use the CopyMemory API to convert them to longs, do the operation, and then convert mack to doubles. I don't understand why all the code is needed that DiGiTaIErRoR just provided (if any code is NOT self-documenting, that's a good example).

The bitwise operators are limited to the max signed value of the Long type.

You wouldn't need to CopyMemory. You would just have to convert values over 2147483648 to it's respective negative signed Long value.

So yeah... I'll remove my post.

**Mc Brain** · Jul 28th, 2003, 08:09 PM

Originally posted by DiGiTaIErRoR
The And operator only works upto the max signed Long value.

VB Code:

Private Enum OpE
opAnd = 1
opOr = 2
opXor = 3
End Enum

Private Function dOps(ByVal Val1 As Double, ByVal Val2 As Double, ByVal OpCode As OpE) As Double
Dim sVal1 As String, sVal2 As String
sVal1 = B10To256(Val1, 6)
sVal2 = B10To256(Val2, 6)
For x = 1 To 6
Select Case OpCode
Case 1
rs = rs & Chr$((Asc(Mid$(sVal1, x, 1)) And Asc(Mid$(sVal2, x, 1))))
Case 2
rs = rs & Chr$((Asc(Mid$(sVal1, x, 1)) Or Asc(Mid$(sVal2, x, 1))))
Case 3
rs = rs & Chr$((Asc(Mid$(sVal1, x, 1)) Xor Asc(Mid$(sVal2, x, 1))))
End Select
Next
dOps = B256To10(rs)
End Function

Private Function B10To256(ByVal initval As Double, ByVal Character_Return As Long) As String
Dim rt As String
Dim x As Long
oi = initval
If initval < 0 Then
initval = 4294967296# + initval
End If
goal = initval
DoEvents
For x = Character_Return - 1 To 1 Step -1
If goal >= (256 ^ x) Then
rt = rt & Chr$(Int(goal / (256 ^ x)))
goal = goal - (Int(goal / (256 ^ x)) * (256 ^ x))
Else
rt = rt & Chr$(0)
End If
Next
rt = rt & Chr$(goal)
B10To256 = rt
End Function

Private Function B256To10(ByVal Base256Chrs As String) As Double
Dim rt As Double
Dim x As Long
If Len(Base256Chrs) > 1 Then
For x = Len(Base256Chrs) - 1 To 1 Step -1
i = i + 1
rt = rt + (Asc(Mid$(Base256Chrs, x, 1)) * (256 ^ i))
Next
End If
B256To10 = rt + Asc(Right$(Base256Chrs, 1))
End Function

It's not fast enough for say, computing a CRC. But it should do just fine if not intensively used.

VictorB212, that's exactly what I thought.

Anyway, thanks so much for the code, DiGiTaIErRoR. I was thinking that I should do something like you did. Fortunately... you saved me a couple of coding-hours. Thanks a lot!!

**DiGiTaIErRoR** · Jul 28th, 2003, 08:10 PM

Originally posted by Mc Brain
VictorB212, that's exactly what I thought.

Anyway, thanks so much for the code, DiGiTaIErRoR. I was thinking that I should do something like you did. Fortunately... you saved me a couple of coding-hours. Thanks a lot!!

Read my above post.

And it'll save even more time.

**Mc Brain** · Jul 28th, 2003, 08:16 PM

Anyway... I'm having problems with SQL now!!

Code:

UPDATE DAEUsers
SET DAEUser_Level = DAEUser_Level - 4294967296 + 536870912
WHERE (DAEUser_Level & 4294967296) > 0

raises:
Servidor: mensaje 403, nivel 16, estado 1, línea 1
Invalid operator for data type. Operator equals boolean AND, type equals numeric.

**DiGiTaIErRoR** · Jul 28th, 2003, 08:19 PM

Like this:

If val > 2147483647 then res = val - 4294967296

**Mc Brain** · Jul 28th, 2003, 08:27 PM

I don't get what you mean.

**DiGiTaIErRoR** · Jul 28th, 2003, 08:34 PM

Originally posted by Mc Brain
I don't get what you mean.

Just another method of using bitwise operators on 4 byte quanitities.

if the number is > 2147483647 and < 4294967296 then subtract the number from 4294967296 and do the calculation, then add the 4294967296 back after.

FFFFFFFF = -1 with a signed Long, unsigned it's 4294967295.

Thus, 4294967295 - 4294967296 = -1.

Unless VB supports unsigned longs....

**Mc Brain** · Jul 28th, 2003, 08:43 PM

Ok, and how would you do a query like:

UPDATE DAEUsers
SET DAEUser_Level = DAEUser_Level - 2147483648
WHERE (DAEUser_Level & 2147483648) > 0

**DiGiTaIErRoR** · Jul 28th, 2003, 08:55 PM

Originally posted by Mc Brain
Ok, and how would you do a query like:

UPDATE DAEUsers
SET DAEUser_Level = DAEUser_Level - 2147483648
WHERE (DAEUser_Level & 2147483648) > 0

What's Daeuser_Level initially?

2147483648 would be -2147483648 which is 80000000 and it's also 2147483648, but the sign messes it up.

It's weird that the negative works, while the positive doesn't.

Go figure.

**DiGiTaIErRoR** · Jul 28th, 2003, 08:56 PM

This works:

MsgBox -2147483648# And -2147483648#

This overflows:

MsgBox 2147483648# And 2147483648#

almost makes you

**Mc Brain** · Jul 28th, 2003, 09:02 PM

Originally posted by DiGiTaIErRoR
What's Daeuser_Level initially?

2147483648 would be -2147483648 which is 80000000 and it's also 2147483648, but the sign messes it up.

It's weird that the negative works, while the positive doesn't.

Go figure.

That's a problem! I don't know tha initial value of DAEUser_Level. I'm deleting a grant from the table, and since I'm freeing the User = 2 ^ x, I have to change all the UserLevel for all the users who had access to that action. This is why I don't know the initial value, because I want to change all the users' level (which had access to this level). Am I clear?

**DiGiTaIErRoR** · Jul 29th, 2003, 01:05 AM

So do:

VB Code:

if DAEUser_Level > 2147483647 and DAEUser_Level < 4294967296 Then DAEUser_Level = 4294967296 - DAEUser_Level
 
dBitWiseAnd = -2147483648 And DAEUser_Level

**DiGiTaIErRoR** · Jul 29th, 2003, 10:33 AM

This resolved?

**Mc Brain** · Jul 29th, 2003, 10:39 AM

I don't know. I'll get at work in 2:30 hours time. Anyway, I'll change the way I'm doing things. Instead of doubles, I will have a table with all the permissions, a table with all the users, and a table for each the user_id and grant_id. If the user has a record in the latter that matches with the grant_id needed, he has access... if not, he hasn't.

Thanks anyway for your help. But I'll guess this way should be much easier.

Thread: Doubles - Overflow

Thread Tools

Display

Doubles - Overflow

Posting Permissions