dcsimg
Results 1 to 9 of 9

Thread: [RESOLVED] Int64 test if can fit into Int32 w/o truncation

  1. #1

    Thread Starter
    Frenzied Member
    Join Date
    Jun 2012
    Posts
    1,281

    Resolved [RESOLVED] Int64 test if can fit into Int32 w/o truncation

    I have a given Int64 as Currency (master value).

    How can I test it? I would like to decide if I can expose it as Long or as Decimal.
    Currently I extract it to LoLong and HiLong and do a test according to my observation.

    Code:
    Private Type LongLong
    LoLong As Long
    HiLong As Long
    End Type
    Private Declare Function GetMem8 Lib "msvbvm60.dll" Alias  (ByRef Value As Currency, ByRef i64Out As LongLong) As Long
    Private Declare Function VarDecFromI8 Lib "oleaut32" (ByVal LoLong As Long, ByVal HiLong As Long, ByRef pDecOut As Variant) As Long
    
    Dim MasterValue As Currency ' <- True Int64 value inside
    Dim Int64 As LongLong
    GetMem8 MasterValue, Int64
    With Int64
    If (.LoLong >= 0 And .HiLong = 0) Or (.LoLong < 0 And .HiLong = &HFFFFFFFF) Then
        RetVal = .LoLong ' Expose As Long
    Else
        RetVal = CDec(0)
        VarDecFromI8 .LoLong, .HiLong, RetVal ' Expose As Decimal
    End If
    End With
    Is this the correct approach? The code marked in Red is due to observation. I don't know if it's really correct.

    Also is it efficient? (Fast)
    Maybe there is a better way to directly test the Currency master value instead of the workaround with LoLong and HiLong.
    Last edited by Krool; Sep 9th, 2019 at 01:53 PM.

  2. #2
    PowerPoster Elroy's Avatar
    Join Date
    Jun 2014
    Location
    Near Nashville TN
    Posts
    5,663

    Re: Int64 test if can fit into Int32 w/o truncation

    Personally, regardless of whether it's in a LongLong or a Currency, here's how I'd think of it. We're dealing with 64 bits (0 thru 63). For this Int64, the high (63rd) bit is the sign bit. All others are the integer's mantissa (in 2s compliment). So, if we want it to fit into a signed Long (which only has 32 bits, counting the sign bit), we've got to have 32 bits of our Int64 we're not using.

    And specifically, they have to be bits 31 thru 62. So, without thinking of low DWORD and high DWORD, we could conceptualize a mask of &h800000007FFFFFFF. From there, we could do something like:

    Code:
    If (Int64 Or &h800000007FFFFFFF) = &h800000007FFFFFFF Then MsgBox "The Int64 will fit into a Long"
    That's just the way I'd think about it. But that does ignore that it takes a Variant to get a LongLong into VB6. It also ignores the complexities of getting that long of a hex into something we can use. This also assumes that your Int64 is signed.

    Good Luck,
    Elroy

    EDIT1: Just as a further FYI, an Int64 will always fit into a Decimal, as a Decimal has a full 96 bits of mantissa, with the sign bit stored separately. However, the reverse (Decimal fitting into Int64) certainly isn't necessarily true. Testing whether or not a Decimal would fit into an Int64 would be a bit tricky, other than just doing straight-up bounds checking. The mixed-byte-endianness of Decimals makes them difficult to deal with.
    Last edited by Elroy; Sep 9th, 2019 at 01:54 PM.
    Any software I post in these forums written by me is provided “AS IS” without warranty of any kind, expressed or implied, and permission is hereby granted, free of charge and without restriction, to any person obtaining a copy. Please understand that I’ve been programming since the mid-1970s and still have some of that code. My contemporary VB6 project is approaching 1,000 modules. In addition, I have a “VB6 random code folder” that is overflowing. I’ve been at this long enough to truly not know with absolute certainty from whence every single line of my code has come, with much of it coming from programmers under my employ who signed intellectual property transfers. I have not deliberately attempted to remove any licenses and/or attributions from any software. If someone finds that I have inadvertently done so, I sincerely apologize, and, upon notice and reasonable proof, will re-attach those licenses and/or attributions. To all, peace and happiness.

  3. #3
    PowerPoster
    Join Date
    Feb 2006
    Posts
    20,420

    Re: Int64 test if can fit into Int32 w/o truncation

    Often you can simply leave it Currency.

    If you need to compare it to a literal use a Currency literal: 0.0001@ would be 1 as an Int64.

  4. #4

    Thread Starter
    Frenzied Member
    Join Date
    Jun 2012
    Posts
    1,281

    Re: Int64 test if can fit into Int32 w/o truncation

    Thanks Elory. Thanks dilettante.

    The background is for SQLite3. Storing values of a result set in an Variant array. (value matrix)
    Every value in SQLite3 has one of five fundamental datatypes:
    * 64-bit signed integer -> Long/Decimal
    * 64-bit IEEE floating point number -> Double
    * string -> String
    * BLOB -> Byte Array
    * NULL -> Null

    SQLite3 offers two functions for the 64-bit signed integer.
    - 'sqlite3_column_int' As Long
    - 'sqlite3_column_int64' As Currency
    However, 'sqlite3_column_int' is a "trick" as it just extracts the LoDWord of the actual Int64.

    Due to this background I can't just store the original 'Currency' in the Variant due to the decimal offset at display and for comparison.

    Of course I could opt to always convert the 'Currency' to a Decimal Variant? However, a Long Variant would be faster and applicable in most situtations, when possible.

    That's why I look for a way to quickly decide if the' Currency' shall be converted to Long Variant or Decimal Variant and do the best conversion option in either case.
    Last edited by Krool; Sep 9th, 2019 at 02:43 PM.

  5. #5
    PowerPoster Elroy's Avatar
    Join Date
    Jun 2014
    Location
    Near Nashville TN
    Posts
    5,663

    Re: Int64 test if can fit into Int32 w/o truncation

    And, with care, VB6 will work with I8:

    Code:
    
    Option Explicit
    '
    Private Declare Sub CopyMemory Lib "kernel32" Alias "RtlMoveMemory" (ByRef Dest As Any, ByRef Source As Any, ByVal Bytes As Long)
    Private Declare Function VariantChangeTypeEx Lib "oleaut32" (ByRef pvargDest As Variant, ByRef pvarSrc As Variant, ByVal LCID As Long, ByVal wFlags As Integer, ByVal vt As Integer) As Long
    Private Const VT_I8 As Integer = &H14   ' 20
    Private Const LOCALE_INVARIANT As Long = &H7F&
    '
    
    
    Public Function LongLongZero() As Variant
        ' This returns a Variant of type LongLong set to ZERO.
        VariantChangeTypeEx LongLongZero, LongLongZero, LOCALE_INVARIANT, 0, VT_I8
    End Function
    
    Public Function LngLngFromCurrBits(curr As Currency) As Variant
        LngLngFromCurrBits = LongLongZero
        CopyMemory ByVal VarPtr(LngLngFromCurrBits) + 8&, curr, 8&      ' Offset to data area of variant. Must change to PtrAdd if 4Gig will ever be used.
    End Function
    
    Public Function CurrFromLngLngBits(v As Variant) As Currency
        If VarType(v) <> VT_I8 Then Exit Function                       ' Just get out if wrong type.
        CopyMemory CurrFromLngLngBits, ByVal VarPtr(v) + 8&, 8&         ' Offset to data area of variant. Must change to PtrAdd if 4Gig will ever be used.
    End Function
    
    
    
    Private Sub Form_Load()
        Dim v1 As Variant
        Dim v2 As Variant
        Dim v3 As Variant
        Dim c As Currency
    
    
        c = 1234.5678
        v1 = LngLngFromCurrBits(c)
        Debug.Print v1              '<--- output:   12345678
        Debug.Print VarType(v1)     '<--- output = VT_I8
    
    
        v1 = v1 * 100&
        Debug.Print VarType(v1)     '<--- output = VT_I8
    
    
        v1 = v1 \ 200&
        Debug.Print VarType(v1)     '<--- output = VT_I8
    
    
    
        v2 = LongLongZero()
        v2 = v2 Or &H400& ' 1024
        Debug.Print v2              '<--- output:   1024
        Debug.Print VarType(v2)     '<--- output = VT_I8
    
    
    
        v3 = v1 Or v2
        Debug.Print v3              '<--- output:   6173863
        Debug.Print VarType(v3)     '<--- output = VT_I8
    
    
    
    End Sub
    
    Any software I post in these forums written by me is provided “AS IS” without warranty of any kind, expressed or implied, and permission is hereby granted, free of charge and without restriction, to any person obtaining a copy. Please understand that I’ve been programming since the mid-1970s and still have some of that code. My contemporary VB6 project is approaching 1,000 modules. In addition, I have a “VB6 random code folder” that is overflowing. I’ve been at this long enough to truly not know with absolute certainty from whence every single line of my code has come, with much of it coming from programmers under my employ who signed intellectual property transfers. I have not deliberately attempted to remove any licenses and/or attributions from any software. If someone finds that I have inadvertently done so, I sincerely apologize, and, upon notice and reasonable proof, will re-attach those licenses and/or attributions. To all, peace and happiness.

  6. #6

    Thread Starter
    Frenzied Member
    Join Date
    Jun 2012
    Posts
    1,281

    Re: Int64 test if can fit into Int32 w/o truncation

    Quote Originally Posted by Elroy View Post
    And, with care, VB6 will work with I8:
    I thought that too. So you mean never store a Long Variant and just go full ?
    I will check if I just take VarDecFromI8 or use VariantChangeTypeEx with VT_I8.

    According to this discussion http://www.vbforums.com/showthread.p...y-calculations
    a LongLong Variant is slower than a Decimal Variant ?

    EDIT:
    VarDecFromI8 is somewhat faster than VariantChangeTypeEx with VT_I8. Also a Decimal Variant is safe with TypeName().
    However, it would be nice to store Long (Int32) Variant, if applicable of course.. and otherwise then a Decimal Variant.
    Last edited by Krool; Sep 9th, 2019 at 04:22 PM.

  7. #7
    PowerPoster Elroy's Avatar
    Join Date
    Jun 2014
    Location
    Near Nashville TN
    Posts
    5,663

    Re: Int64 test if can fit into Int32 w/o truncation

    Quote Originally Posted by Krool View Post
    LongLong Variant is slower than a Decimal Variant ?
    I don't have time right now to test it, but I can't imagine that's true. A LongLong will fit into CPU registers whereas the weird mixed-byte-endianness of Decimals has to make using them very slow. Also, 96 bits isn't going to fit into registers easily so doing math with them has to be doing all kinds of gyrations. But, I suppose a test is in order. Maybe tomorrow.

    I could believe that LongLong math is slower than Currency math, as VB6 probably does the Currency math rather directly, whereas it's probably passing the LongLong values into the oleaut32.dll to get it done. But, slower than Decimal? I'll have to see that one to believe it.

    Take Care,
    Elroy

    EDIT1: Actually, I read that thread more closely, and that does seem to be the case. I guess passing the LongLong Variant values into oleaut32 is slowing it down. I might still test myself just to see.
    Any software I post in these forums written by me is provided “AS IS” without warranty of any kind, expressed or implied, and permission is hereby granted, free of charge and without restriction, to any person obtaining a copy. Please understand that I’ve been programming since the mid-1970s and still have some of that code. My contemporary VB6 project is approaching 1,000 modules. In addition, I have a “VB6 random code folder” that is overflowing. I’ve been at this long enough to truly not know with absolute certainty from whence every single line of my code has come, with much of it coming from programmers under my employ who signed intellectual property transfers. I have not deliberately attempted to remove any licenses and/or attributions from any software. If someone finds that I have inadvertently done so, I sincerely apologize, and, upon notice and reasonable proof, will re-attach those licenses and/or attributions. To all, peace and happiness.

  8. #8

    Thread Starter
    Frenzied Member
    Join Date
    Jun 2012
    Posts
    1,281

    Re: Int64 test if can fit into Int32 w/o truncation

    My initial thought was maybe too complicated...

    I now found the fastest approach.. which is actually the most easiest..

    I just do the math of the Int64 with native Currency to see if it fits into a Long (red marked)

    Code:
    Select Case stub_sqlite3_column_type(hStmt, iCol)
        Case SQLITE_INTEGER
            Int64 = stub_sqlite3_column_int64(hStmt, iCol)
            If Int64 >= -214748.3648@ And Int64 <= 214748.3647@ Then
                MyVar = stub_sqlite3_column_int(hStmt, iCol) ' Long
            Else
                MyVar = CDec(0)
                VarDecFromI8 Int64, MyVar ' Decimal
            End If
    The only thing I need to decide if I should take Decimal or VT_I8 (LongLong) for ranges beyond Int32.

    Though I do feel more comfortable with the Decimal instead of LongLong, bacause:
    1. No issue with TypeName()
    2. Seems to be slightly faster

    Conclusion: I take Decimal

    So this topic is resolved.
    Last edited by Krool; Sep 15th, 2019 at 03:51 AM.

  9. #9
    PowerPoster
    Join Date
    Feb 2006
    Posts
    20,420

    Re: [RESOLVED] Int64 test if can fit into Int32 w/o truncation

    Seems like a reasonable conclusion.

    We may as well give up on ultimate performance anyway. Soon we'll all be working in JavaScript or Python and we'll face far greater performance problems just adding 2 + 2.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Featured


Click Here to Expand Forum to Full Width