|
-
Jan 13th, 2015, 12:52 PM
#1
Thread Starter
Member
Preserve decimal precision in Bitwise Operation
I have a number that is a decimal , fractional number, less than 1000 in an array, EG: 518.25
Adding the value 1024 "Flags" this number as unique among the rest. To get to the original number I "And" it with 1023.
However, I lose the ".25" and get 518.
Code is as follows:
Dim N1 As Long
N1 = 518.25
MsgBox 1023& And (N1 + 1024&)
Produces "518"
Is there a way to retrieve the precision and get 518.25?
Thanks
-
Jan 13th, 2015, 01:13 PM
#2
Re: Preserve decimal precision in Bitwise Operation
Dim N1 As Long
N1 is an integer type, so can't hold fractions, so...
N1 = 518.25
N1 is now equal to 518, it rounded down, in this case.
You could multiply the number by 100 when you assign it so you know it has 2 decimal points (essentially Fixed Point representation).
You'll need to use a larger mask and flag bit so you don't interfere with your number, I would say (128 * 1024 - 1) so your mask would be 131071, and your flag would be 131072. (these things are usually easier in Hex, so your mask would be &H1FFFF and your flag &H20000).
Divide by 100 after masking to recover your two decimal digits.
Binary operations only work on integer type numbers.
Last edited by passel; Jan 13th, 2015 at 01:17 PM.
-
Jan 13th, 2015, 02:47 PM
#3
Thread Starter
Member
Re: Preserve decimal precision in Bitwise Operation
Thanks passel,
The following now works:
Dim N1 as single 'Long wont work
N1=518.25
N1=N1 * 100
MsgBox (&H1FFFF And (N1 + &H20000)) / 100
Produces 518.25
Thanks for the quick reply
-
Jan 13th, 2015, 03:09 PM
#4
Re: Preserve decimal precision in Bitwise Operation
If multiplying your number by 100 will make it exceed max Long value, another option is to remove & then add the decimal portion
Code:
X = (N1 - Int(N1)) + (Int(N1) And 1023)
Tags for this Thread
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
|
Click Here to Expand Forum to Full Width
|