Results 1 to 20 of 20

Thread: [RESOLVED] Why Is This Bit Mask Not Working?

  1. #1

    Thread Starter
    Master Of Orion ForumAccount's Avatar
    Join Date
    Jan 2009
    Location
    Canada
    Posts
    2,802

    Resolved [RESOLVED] Why Is This Bit Mask Not Working?

    Here is the code:
    Code:
    Option Strict On
    
    Dim mask As ULong = &HFFFFFFFFUL
    Dim result1 As ULong = 20L And mask '//disallows implicit conversions
    Dim result2 As ULong = 20L And &HFFFFFFFFUL
    If you put that into the IDE with Option Strict On, it will not let you compile because of 'result1'. I'm not sure I understand why. In the 'result2' assignment, I am explicitly saying to use an unsigned long, however it's not giving me any grief. Is the IDE doing that conversion to long without telling me?

  2. #2
    Powered By Medtronic dbasnett's Avatar
    Join Date
    Dec 2007
    Location
    Jefferson City, MO
    Posts
    9,897

    Re: Why Is This Bit Mask Not Working?

    Very interesting.
    My First Computer -- Documentation Link (RT?M) -- Using the Debugger -- Prime Number Sieve
    Counting Bits -- Subnet Calculator -- UI Guidelines -- >> SerialPort Answer <<

    "Those who use Application.DoEvents have no idea what it does and those who know what it does never use it." John Wein

  3. #3

    Thread Starter
    Master Of Orion ForumAccount's Avatar
    Join Date
    Jan 2009
    Location
    Canada
    Posts
    2,802

    Re: Why Is This Bit Mask Not Working?

    Quote Originally Posted by dbasnett View Post
    Very interesting.
    I think so too. Does the IDE treat the literal and the variable differently? I.e. does the conversion with the literal, and demands a conversion with the variable.

  4. #4

    Thread Starter
    Master Of Orion ForumAccount's Avatar
    Join Date
    Jan 2009
    Location
    Canada
    Posts
    2,802

    Re: Why Is This Bit Mask Not Working?

    It actually got more interesting:
    Code:
    Option Strict On
    
    Dim mask As ULong = &HFFFFFFFFUL
    Const mask2 As ULong = &HFFFFFFFFUL
    Dim result1 As ULong = 20L And mask '//disallows implicit conversions
    Dim result2 As ULong = 20L And &HFFFFFFFFUL '//valid
    Dim result3 As ULong = 20L And mask2 '//valid
    The variables 'result2' and 'result3' are both valid, however, 'result1' is still an implicit conversion.

  5. #5

    Re: Why Is This Bit Mask Not Working?

    Code:
    Dim mask As ULong = &HFFFFFFFFUL
    Dim result1 As ULong = CULng(20) And mask
    It now compiles on Option Strict...this is kind of weird.

  6. #6
    Powered By Medtronic dbasnett's Avatar
    Join Date
    Dec 2007
    Location
    Jefferson City, MO
    Posts
    9,897

    Re: Why Is This Bit Mask Not Working?

    Quote Originally Posted by formlesstree4 View Post
    Code:
    Dim mask As ULong = &HFFFFFFFFUL
    Dim result1 As ULong = CULng(20) And mask
    It now compiles on Option Strict...this is kind of weird.
    Why would this be weird?
    My First Computer -- Documentation Link (RT?M) -- Using the Debugger -- Prime Number Sieve
    Counting Bits -- Subnet Calculator -- UI Guidelines -- >> SerialPort Answer <<

    "Those who use Application.DoEvents have no idea what it does and those who know what it does never use it." John Wein

  7. #7

    Re: Why Is This Bit Mask Not Working?

    Quote Originally Posted by dbasnett View Post
    Why would this be weird?
    Because it was saying that "mask" needed to be converted to Long, then the entire thing be converted to ULong afterwards:

    Code:
    Dim mask As ULong = &HFFFFFFFFUL
    Dim result1 As ULong = CULng(20 And CLng(mask))
    That's how VS wanted result1 to look, which seems to be a bit overkill and doesn't change the fact that mask was already declared as ULong while VS was still "yelling" at us about it. It just seems to be going around one's elbow to reach their hand.

  8. #8
    PowerPoster
    Join Date
    Apr 2007
    Location
    The Netherlands
    Posts
    5,070

    Re: Why Is This Bit Mask Not Working?

    I'm not sure why but I am seeing different results. For me none of your bitwise operations work, they all complain about being unable to convert from ULong to Long.

    It makes sense, doesn't it? Your mask is ULong while '20L' is not (it's type Long). The And operator is apparently not defined for 'Long And ULong', so VS complains.

    I can fix it by changing 20L to 20UL.



    (The green line underneath 'mask' is ReSharper suggesting to make it into a Const. It doesn't do that because that might fix the problem, it always suggests that when you assign a value once and never change it).

  9. #9
    PowerPoster stanav's Avatar
    Join Date
    Jul 2006
    Location
    Providence, RI - USA
    Posts
    9,290

    Re: Why Is This Bit Mask Not Working?

    Quote Originally Posted by NickThissen View Post
    I'm not sure why but I am seeing different results. For me none of your bitwise operations work, they all complain about being unable to convert from ULong to Long.

    It makes sense, doesn't it? Your mask is ULong while '20L' is not (it's type Long). The And operator is apparently not defined for 'Long And ULong', so VS complains.

    I can fix it by changing 20L to 20UL.



    (The green line underneath 'mask' is ReSharper suggesting to make it into a Const. It doesn't do that because that might fix the problem, it always suggests that when you assign a value once and never change it).
    What version of VS are you using? I'm on VS2010, and this is what I got:
    Attached Images Attached Images  
    Let us have faith that right makes might, and in that faith, let us, to the end, dare to do our duty as we understand it.
    - Abraham Lincoln -

  10. #10

    Thread Starter
    Master Of Orion ForumAccount's Avatar
    Join Date
    Jan 2009
    Location
    Canada
    Posts
    2,802

    Re: Why Is This Bit Mask Not Working?

    Nick, I understand that all three shouldn't work, but the thing is that the IDE doesn't complain on 2 of 3.

  11. #11
    PowerPoster
    Join Date
    Apr 2007
    Location
    The Netherlands
    Posts
    5,070

    Re: Why Is This Bit Mask Not Working?

    I'm on VS2010 as well. I understand that you understand that they shouldn't work, I was just pointing out that they don't for me...

    It might be ReSharper complaining, but the error doesn't look like a ReSharper error but more like a VS error... Strange.

    On second thought it probably is ReSharper complaining, since stanav doens't get the red lines but a blue line instead. It's still strange though...

  12. #12
    Powered By Medtronic dbasnett's Avatar
    Join Date
    Dec 2007
    Location
    Jefferson City, MO
    Posts
    9,897

    Re: Why Is This Bit Mask Not Working?

    Quote Originally Posted by stanav View Post
    What version of VS are you using? I'm on VS2010, and this is what I got:
    Me too.
    My First Computer -- Documentation Link (RT?M) -- Using the Debugger -- Prime Number Sieve
    Counting Bits -- Subnet Calculator -- UI Guidelines -- >> SerialPort Answer <<

    "Those who use Application.DoEvents have no idea what it does and those who know what it does never use it." John Wein

  13. #13

    Thread Starter
    Master Of Orion ForumAccount's Avatar
    Join Date
    Jan 2009
    Location
    Canada
    Posts
    2,802

    Re: Why Is This Bit Mask Not Working?

    VS bug...?

  14. #14
    PowerPoster
    Join Date
    Apr 2007
    Location
    The Netherlands
    Posts
    5,070

    Re: Why Is This Bit Mask Not Working?

    In my case it was flagged as an error by ReSharper, not VS. When I turn off ReSharper VS does not complain, the same as with you guys. Whether its a bug in VS... I dunno.

    It also seems ReSharper doesn't know what to do with it, even though it strikes it as an error. When I hover over the offending line:
    Code:
    Dim result1 = 20L And &HFFFFFFUL
    ReSharper offers to 'Cast second operand to Long'. When I accept, it changes it to:
    Code:
    Dim result1 = 20L And DirectCast (&HFFFFFFUL, Long)
    and that does not compile ("ULong cannot be converted to Long")... That's the first time I've seen ReSharper suggest something that won't compile!

  15. #15
    You don't want to know.
    Join Date
    Aug 2010
    Posts
    4,578

    Re: Why Is This Bit Mask Not Working?

    At first I was going to make a snarky statement about reading the documentation, but this required extra credit. I was also going to joke about real men reading IL, but it turns out the conversions happen before the IL is generated!

    My first thought was to check the documentation to see what the operator definitions looked like. Turns out the operators for primitive types don't follow operator overloading. That means the language probably defines the rules. So I had a peek at the Visual Basic Language Reference.

    Here's what And operator has to say:
    For a bitwise comparison, the result data type is a numeric type appropriate for the data types of expression1 and expression2. See the "Relational and Bitwise Comparisons" table in Data Types of Operator Results (Visual Basic).
    Okey dokey. Bitwise And says for arguments of type Long and ULong, the result is always Long. For the record, I'm getting the same results as ForumAccount in #1: result2 As ULong = 20L And &HFFFFFFFFUL is allowed. Interesting. So I asked myself, "Can Long be converted to ULong?" Turns out the answer should be no. So what's happening? I asked ILDASM (I annotated with comments
    Code:
      IL_0000:  nop
      IL_0001:  ldc.i4.m1        ' Push -1 onto the stack as an Int32.
      IL_0002:  conv.u8          ' Convert to a UInt64
      IL_0003:  stloc.0          ' mask = -1UL
      IL_0004:  ldc.i4.s   20    ' Load 20 onto the stack as an Int32.
      IL_0006:  conv.i8          ' Convert the Int32 to an Int64.
      IL_0007:  stloc.1          ' result2 = 20 ???
      IL_0008:  nop
      IL_0009:  ret
    Huh. Looks like the compiler's being clever. It knows &HFFFFFFFFUL And any numer is that number. So no And is performed. Since &HFFFFFFFFUL is already a ULong, you have no problems. So long as the mask has the same bits as 20 set, there's no error. So that IL's uninteresting.

    It turns out that as long as you're working with literals and constants the compiler does the masking itself. When I tried 20L And &H4UL the IL was just storing 4 in result2. This explains your results in #4: since mask2 is a constant, the compiler knows its value and performs the bitmask itself. &HFFFFFFFFUL is a literal and thus constant. mask isn't constant, so the compiler doesn't allow it.

    If you don't trust me, all you have to do is set up a scenario where the compiler can't guarantee a constant is used:
    Code:
    Sub Main()
        Dim mask As ULong = &H4UL
        Dim result2 As ULong = DoAnd(20, mask)
    End Sub
    
    Function DoAnd(ByVal left As Long, ByVal right As ULong) As ULong
        Return left And right
    End Function
    The compiler complains with Option Strict on because there's no guarantees about the value of the right parameter. The compiler doesn't go to the trouble of checking all calls.

    The Visual Basic Language Specification has the nitty gritty details of how this all works if you're *real* curious. Here's snippets from section 11.2 that describes constant expressions:
    A constant expression is an expression whose value can be fully evaluated at compile time. The type of a constant expression can be Byte, SByte, UShort, Short, UInteger, Integer, ULong, Long, Char, Single, Double, Decimal, Date, Boolean, String, Object, or any enumeration type. The following constructs are permitted in constant expressions:

    * Literals (including Nothing).
    [...]
    * The +, –, *, ^, Mod, /, \, <<, >>, &, And, Or, Xor, AndAlso, OrElse, =, <, >, <>, <=, and => binary operators, provided each operand and result is of a type listed above.
    [...]
    Aaaaaand here's the final piece of the puzzle:
    Constant expressions of an integral type (ULong, Long, UInteger, Integer, UShort, Short, SByte, or Byte) can be implicitly converted to a narrower integral type, and constant expressions of type Double can be implicitly converted to Single, provided the value of the constant expression is within the range of the destination type. These narrowing conversions are allowed regardless of whether permissive or strict semantics are being used.
    So the compiler figures out it's dealing with a constant expression because two literals are involved with an And operator. ULong <-> Long conversion is narrowing, but the compiler reserves the right to make them if possible so long as they're in a constant expression. My guess is inside the VB compiler the types are ignored and raw bits have And applied, and the result is initially interpreted as Long. If it's not done that way, I bet &HFFFFFFFFUL has its bits interpreted as a Long; then you have Long And Long which returns a Long. Since a ULong is needed and it's being assigned the result of a constant expression, it's implicitly converted. Whew. I bet if you carefully crafted an expression whose result was a negative Long you might not get such a happy ending, but again if "conversion" means "interpret the bits" it'll work.

    ReSharper probably handles it incorrectly because they assumed VB uses similar rules to C#, where the bitwise operators behave a little differently (I believe C# uses bits rather than type conversion.) It should be reported as an R# bug.
    Last edited by Sitten Spynne; Jul 14th, 2011 at 04:02 PM.

  16. #16

    Re: Why Is This Bit Mask Not Working?

    That was a pretty good explanation Sitten. I would +rep you if I could but it appears I need to spread some more around. It looks like you went through a bit of work to find all these answers.

  17. #17

  18. #18
    PowerPoster i00's Avatar
    Join Date
    Mar 2002
    Location
    1/2 way accross the galaxy.. and then some
    Posts
    2,390

    Re: Why Is This Bit Mask Not Working?

    Working fine for me in VS2008!

    Kris

  19. #19
    Powered By Medtronic dbasnett's Avatar
    Join Date
    Dec 2007
    Location
    Jefferson City, MO
    Posts
    9,897

    Re: Why Is This Bit Mask Not Working?

    Quote Originally Posted by formlesstree4 View Post
    That was a pretty good explanation Sitten. I would +rep you if I could but it appears I need to spread some more around. It looks like you went through a bit of work to find all these answers.
    Agreed, and I can, and did.
    My First Computer -- Documentation Link (RT?M) -- Using the Debugger -- Prime Number Sieve
    Counting Bits -- Subnet Calculator -- UI Guidelines -- >> SerialPort Answer <<

    "Those who use Application.DoEvents have no idea what it does and those who know what it does never use it." John Wein

  20. #20
    PowerPoster SJWhiteley's Avatar
    Join Date
    Feb 2009
    Location
    South of the Mason-Dixon Line
    Posts
    2,256

    Re: Why Is This Bit Mask Not Working?

    If in doubt, Reflect it.
    "Ok, my response to that is pending a Google search" - Bucky Katt.
    "There are two types of people in the world: Those who can extrapolate from incomplete data sets." - Unk.
    "Before you can 'think outside the box' you need to understand where the box is."

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Click Here to Expand Forum to Full Width