[RESOLVED] Hacking Variant

I'm trying to understand the data structure of the VB variant data type and am getting a puzzling result:

VB Code:

Sub VariantTest()
    Dim vA(1 To 2) As Variant
    Dim A(1 To 2) As Double, d1 As Double, d2 As Double
    Dim vB As Variant
    Dim itype As Integer
    
    A(1) = 0.35
    A(2) = 0.07
    vA(1) = A(1)
    vA(2) = A(2)
    
    vB = vA
    CopyMemory itype, vB(1), 2                  'get the variant descriptor type
    CopyMemory d1, ByVal VarPtr(vB(1)) + 8, 8   'get 1st value
    CopyMemory d2, ByVal VarPtr(vB(1)) + 24, 8  'get 2nd value
    Debug.Print TypeName(vB), itype, d1, d2     'Variant() 5 0.35 0.07
    
    vB = A
    CopyMemory itype, vB(1), 2                  'get the variant descriptor type
    CopyMemory d1, ByVal VarPtr(vB(1)) + 8, 8   'get 1st value
    CopyMemory d2, ByVal VarPtr(vB(1)) + 24, 8  'get 2nd value
    Debug.Print TypeName(vB), itype, d1, d2     'Double() 16389 3.75776923216955E-316 0
End Sub

If vB is assigned to the variant array (of doubles), the resulting data structure makes perfect sense. But, if vB is assigned to the array of doubles, the variant data structure makes no sense at all. The descriptor type is 16389 and I can't find the data values anywhere.

Can someone explain what's going on?

vB(1) is most likely refering to memory location of an array item, and since Double is only 8 bytes long in the first place, you're going out of the array space.

So, see what you get when you do:

Code:

---

' itype not applicaple for a Double array, since you're accessing Double array data, not Variant information
CopyMemory d1, ByVal VarPtr(vB(1)), 8
CopyMemory d2, ByVal VarPtr(vB(2)), 8
Debug.Print TypeName(vB), d1, d2

---

So in both cases you are accessing an array item data, not the Variant thatis holding the pointer information to the arrays. The array items remain of the type they are declared to and the fact you have assigned the arrays to a Variant doesn't change the actual array data to a Variant.

If you wish to doodle more, take a look to VarPtrArray.

First of all, I don't really know what I'm talking about, so don't expect me to be able to explain anything I've presented here in much detail.

Second, to get the descriptor, you need to pass the variant variable itself, not one of its array elements:

Code:

---

CopyMemory itype, vB, 2

---

In these cases, itype will be 8204 for 'Array of Variant' or 8197 for 'Array of Double'. The data in the variant is a pointer to a SafeArray structure.

Finally, I've been experimenting with this for a little while. Here is what I've found:

When vB contains an array of Variant, VarPtr(vB(1)) returns a pointer to the actual array item data as expected. When vB contains an array of Double, VarPtr(vB(1)) returns a pointer to a variant which holds a pointer to the array item data. This is curious, to say the least.

Code:

---

Sub VariantTest()
    Dim vA(1 To 2) As Variant
    Dim A(1 To 2) As Double, d1 As Double, d2 As Double
    Dim vB As Variant
    Dim itype As Integer
    Dim ptr1 As Long, ptr2 As Long
    Dim bt(15) As Byte, i As Long
   
    A(1) = 0.35
    A(2) = 0.07
    vA(1) = A(1)
    vA(2) = A(2)
   
    vB = vA
    CopyMemory itype, vB, 2                  'get the variant descriptor type: vbArray + vbVariant = 8192 + 12
    CopyMemory d1, ByVal VarPtr(vB(1)) + 8, 8 'get 1st value
    CopyMemory d2, ByVal VarPtr(vB(2)) + 8, 8 'get 2nd value
    Debug.Print TypeName(vB), itype, d1, d2  'Variant() 8204 0.35 0.07
    CopyMemory ptr1, ByVal VarPtr(vB) + 8, 4  ' get pointer to array structure
    CopyMemory ptr2, ByVal ptr1 + 12, 4      ' get pointer to array data
    Debug.Print ptr2, VarPtr(vB(1)), VarPtr(vB(2))
    CopyMemory d1, ByVal ptr2 + 8, 8  'get 1st value
    CopyMemory d2, ByVal ptr2 + 24, 8 'get 2nd value
    Debug.Print d1, d2
    CopyMemory ByVal VarPtr(bt(0)), vB, 16
    DisplayBytes bt, "vB"
    CopyMemory ByVal VarPtr(bt(0)), vB(1), 16
    DisplayBytes bt, "vB(1)"
    CopyMemory ByVal VarPtr(bt(0)), vB(2), 16
    DisplayBytes bt, "vB(2)"
   
    vB = A
    CopyMemory itype, vB, 2                  'get the variant descriptor type: vbArray + vbDouble = 8192 + 5
    CopyMemory d1, ByVal VarPtr(vB(1)), 8    'get 1st value
    CopyMemory d2, ByVal VarPtr(vB(2)), 8    'get 2nd value
    Debug.Print TypeName(vB), itype, d1, d2  'Double() 8197 8.09724186969219E-320 8.09724186969219E-320
    CopyMemory ptr1, ByVal VarPtr(vB) + 8, 4 ' get pointer to array structure
    CopyMemory ptr2, ByVal ptr1 + 12, 4      ' get pointer to array data
    Debug.Print ptr2, VarPtr(vB(1)), VarPtr(vB(2))
    CopyMemory d1, ByVal ptr2, 8    'get 1st value
    CopyMemory d2, ByVal ptr2 + 8, 8 'get 2nd value
    Debug.Print d1, d2
    CopyMemory ByVal VarPtr(bt(0)), vB, 16
    DisplayBytes bt, "vB"
    CopyMemory ByVal VarPtr(bt(0)), vB(1), 16
    DisplayBytes bt, "vB(1)"
    CopyMemory ByVal VarPtr(bt(0)), vB(2), 16
    DisplayBytes bt, "vB(2)"
    CopyMemory ptr1, ByVal VarPtr(vB(1)) + 8, 4
    CopyMemory d1, ByVal ptr1, 8
    CopyMemory ptr1, ByVal VarPtr(vB(2)) + 8, 4
    CopyMemory d2, ByVal ptr1, 8
    Debug.Print d1, d2
    Debug.Print
End Sub

Private Sub DisplayBytes(btArr() As Byte, strText As String)
  Dim i As Long
  For i = LBound(btArr) To UBound(btArr)
    If btArr(i) < 16 Then Debug.Print "0";
    Debug.Print Hex(btArr(i)); " ";
  Next i
  Debug.Print "<< "; strText
End Sub

---

Quote:

---

Originally Posted by Logophobic

When vB contains an array of Variant, VarPtr(vB(1)) returns a pointer to the actual array item data as expected. When vB contains an array of Double, VarPtr(vB(1)) returns a pointer to a variant which holds a pointer to the array item data. This is curious, to say the least

---

Indeed it is strange! Here's a related puzzle. I tried using varPtrArray with A() and vA(). I thought this was suppose to return the address of the base of the SAFEARRAY meaning + 12 bytes will be the address of the 1st element. The 1st two bytes weren't the array dimension and I couldn't find the address of the 1st element anywhere. I must be doing something wrong...

SAFEARRAYHEADER locates in an entirely different location and has a pointer to the actual location of the data. The header's length may vary depending on how many dimensions there are, the more dimensions the longer the header.

So you have these in different parts of memory:

• Variant variable (with a pointer to array header)
• Array header (with a pointer to array data)
• Array data

You need algorithm first that determines type of variant before performing copy memory operations.

Sorry, I didn't follow either of the last 2 posts. Let's say we have:

Code:

---

Dim vA(1 To 2) As Variant
vA(1) = CDbl(1.2)
vB(2) = CDbl(0.7)

---

Thanks to Logophobic, I completely understand what varPtr(vA), varPtr(vA(1)), and varPtr(vA(2)) return.

My question: what does varPtrArray(vA()) return? I thought it was a pointer to the SAFEARRAY structure, but it seems not.

You need to know what the variant contains (native data type, or array, etc) so you know whether to perform VarPtr() or VarPtrArray() necessary for CopyMemory. There is no one size fits all solution; its data dependent.

I'm confused, too. This isn't how VarPtrArray is expected to work. The return value is the address of a pointer to the SAFEARRAY structure?

Code:

---

  Dim A(5) As Long
  Dim B(3, 3) As Long
  Dim pA As Long
  Dim pB As Long
  Dim pSA As Long
  Dim cdims As Integer
  Dim pdata As Long
 
  pA = VarPtrArray(A())
  CopyMemory pSA, ByVal pA, 4
  CopyMemory cdims, ByVal pSA, 2
  CopyMemory pdata, ByVal pSA + 12, 4
  Debug.Print pSA, cdims, pdata, VarPtr(A(0))
 
  pB = VarPtrArray(B())
  CopyMemory pSA, ByVal pB, 4
  CopyMemory cdims, ByVal pSA, 2
  CopyMemory pdata, ByVal pSA + 12, 4
  Debug.Print pSA, cdims, pdata, VarPtr(B(0, 0))
  Debug.Print pA, pB

---

The error result because of assumption that VarPtr() and VarPtrArray() works as expected. It isn't so. You really have to get the pointer values yourself and jump to the indicated memory location. I managed to output 0.35.

Code:

---

Private Sub Form_Load()
Dim vA(1 To 2) As Variant
Dim A(1 To 2) As Double
Dim vB As Variant
Dim itype As Integer

Dim ptrSA    As Long
Dim cbElements As Long
Dim pvData    As Long

Dim tempV As Variant
Dim tempD As Double

  A(1) = 0.35
  A(2) = 0.07
  vA(1) = A(1)
  vA(2) = A(2)
         
  vB = vA
  CopyMemory ptrSA, ByVal VarPtr(vB) + 8, 4
  Debug.Print "ptrSA " & ptrSA
  CopyMemory cbElements, ByVal ptrSA + 4, 4
  Debug.Print "cbElements " & cbElements
  CopyMemory pvData, ByVal ptrSA + 12, 4
  Debug.Print "pvData " & pvData & " --- VarPtr(vB(1)) " & VarPtr(vB(1))
 
  CopyMemory tempV, ByVal pvData, cbElements '<-- copying variant to variant
  Debug.Print "vB(1) " & vB(1) & " --- tempV " & tempV
 
  Debug.Print vbCrLf
 
  vB = A
  CopyMemory ptrSA, ByVal VarPtr(vB) + 8, 4
  Debug.Print "ptrSA " & ptrSA
  CopyMemory cbElements, ByVal ptrSA + 4, 4
  Debug.Print "cbElements " & cbElements
  CopyMemory pvData, ByVal ptrSA + 12, 4
  Debug.Print "pvData " & pvData & " --- VarPtr(vB(1)) " & VarPtr(vB(1)) 'note discrepancy, VarPtr(vB(1)) fails
  CopyMemory tempD, ByVal pvData, cbElements '<-- copying double to double
  Debug.Print "vB(1) " & vB(1) & " --- tempD " & tempD
End Sub

---

I tried same extraction method starting with ptrSA = VarPtrArray(A()) to see if 0.35 can be extracted from A(1)... VB crashed when I tried to use pvData as source in copymemory.

Logophobic, guess what!

VarPtrArray() returns address of safe array pointer and not the address of the base as we expected. I managed to get 0.35 via VarPtrArray() with following code

Code:

---

ptrSA = VarPtrArray(A())
  CopyMemory ptrSA, ByVal ptrSA, 4  '<--- get memory location of base from memory location of safe array pointer
  Debug.Print ptrSA
  Debug.Print "ptrSA " & ptrSA
  CopyMemory cbElements, ByVal ptrSA + 4, 4
  Debug.Print "cbElements " & cbElements
  CopyMemory pvData, ByVal ptrSA + 12, 4
  Debug.Print "pvData " & pvData & " --- VarPtr(A(1)) " & VarPtr(A(1))
  CopyMemory tempD, ByVal pvData, cbElements '<-- copying double to double
  Debug.Print "vB(1) " & vB(1) & " --- tempD " & tempD

---

Quote:

---

Originally Posted by leinad31

VarPtrArray() returns address of safe array pointer and not the address of the base as we expected.

---

Yep, I arrived at the same conclusion studying the code from Logophobic's last post and expanding:

Code:

---

Sub ArrayPointerTest()
    Dim A(5) As Long
    Dim B(3, 3) As Long
    Dim C(2) As Double
    Dim D(4) As Variant
    Dim pA As Long, pB As Long, pC As Long, pD As Long
    Dim pSA As Long
    Dim cdims As Integer
    Dim pdata As Long
    Dim dValue As Double, dValue1 As Double
    Dim lValue As Long, lValue1 As Long
   
    '1D array of long
    A(0) = 99
    A(1) = 58
    pA = VarPtrArray(A())
    CopyMemory pSA, ByVal pA, 4
    CopyMemory cdims, ByVal pSA, 2
    CopyMemory pdata, ByVal pSA + 12, 4
    CopyMemory lValue, ByVal pdata, 4
    CopyMemory lValue1, ByVal pdata + 4, 4
    Debug.Print pA, pSA, cdims, pdata, VarPtr(A(0)), lValue, lValue1
   
    '2D array of long
    B(0, 0) = 315
    B(0, 1) = 123
    pB = VarPtrArray(B())
    CopyMemory pSA, ByVal pB, 4
    CopyMemory cdims, ByVal pSA, 2
    CopyMemory pdata, ByVal pSA + 12, 4
    CopyMemory lValue, ByVal pdata, 4
    CopyMemory lValue1, ByVal pdata + 16, 4
    Debug.Print pB, pSA, cdims, pdata, VarPtr(B(0, 0)), lValue, lValue1
   
    '1D array of double
    C(0) = CDbl(3.5)
    C(1) = CDbl(0.17)
    pC = VarPtrArray(C())
    CopyMemory pSA, ByVal pC, 4
    CopyMemory cdims, ByVal pSA, 2
    CopyMemory pdata, ByVal pSA + 12, 4
    CopyMemory dValue, ByVal pdata, 8
    CopyMemory dValue1, ByVal pdata + 8, 8
    Debug.Print pC, pSA, cdims, pdata, VarPtr(C(0)), dValue, dValue1
 
    '1D array of variant
    D(0) = CDbl(0.8)
    D(1) = CLng(835)
    pD = VarPtrArray(D())
    CopyMemory pSA, ByVal pD, 4
    CopyMemory cdims, ByVal pSA, 2
    CopyMemory pdata, ByVal pSA + 12, 4
    CopyMemory dValue, ByVal pdata + 8, 8
    CopyMemory lValue, ByVal pdata + 24, 4
    Debug.Print pC, pSA, cdims, pdata, VarPtr(D(0)), dValue, lValue
End Sub

---

Furthermore, varPtrArray() is a fixed address, whose value changes every time varPtrArray() is called. Curious indeed!

Thanks everybody, I'm marking this resolved.:thumb:

Other mystery left is discrepancy between pvData and VarPtr(vB(1)) when vB = A

Yeah, I agree. I did some more investigating and found that VarPtr(vB(0)) has a different meaning depending on the type of array that is assigned to vB!

If an array of Variant is assigned to vB, VarPtr(vB(0)) is the address of the 1st array element (variant data type). But, if an array of Long or Double is assigned to vB, VarPtr(vB(0)) is the address of what appears to be a variant data structure, but the value of the type field makes no sense. The address of the 1st array element is offset 8 bytes. Hmmm, just thought of something to be further investigated: what does VarPtr(vB(1)) represent?

Another curious tid bit: if an array is assigned to a vB, vB can be indexed like an array, typename indicates that vB is an array, but VarPtrArray(vB) produces an error because vB isn't an array.

There is another very odd thing I observed. If VarPtr(vB(0)) is assigned to a variable, say p1, and 16 bytes of memory values are extracted, the result is different if p1 is used vs VarPtr(vB(0)) even though p1 and VarPtr(vB(0)) evaluate to the same number! This has got to be the oddest thing I've ever seen! :sick:

Sure could use some Bruce McKinney expertise here. Anybody have a connection???? :D

Code:

---

Sub ArrayPointerTest2()
    'Using ideas from Logophobic's code
    Dim p1&, p2&, lValue1&, lValue2&
    Dim dValue1#, dValue2#
    Dim bt(15) As Byte
    Dim itype1%, itype2%
    Dim A(1) As Long                            '1D array of Long
    Dim B(1) As Variant                        '1D array of Variant
    Dim vB As Variant                          'Variant

    A(0) = 837&                                '1st array value
    A(1) = 58&                                  '2nd array value
    B(0) = CDbl(2.37)                          '1st array value is Double
    B(1) = CLng(12)                            '2nd array value is Long
   
    vB = B                                      'assign array of Variant to vB
    p1 = VarPtr(vB(0))                          'address 1st array element (variant)
    CopyMemory bt(0), ByVal VarPtr(vB(0)), 16  'copy bytes from VarPtr(vB(0))
    DisplayBytes bt, "Variant data structure"  'variant data structure
    CopyMemory bt(0), ByVal p1, 16              'use p1 instead of VarPtr(vB(0))
    DisplayBytes bt, "no difference"            'no difference
    CopyMemory itype1, ByVal p1, 2              '5 = double
    CopyMemory dValue1, ByVal p1 + 8, 8        '1st array value = double
    p2 = p1 + 16                                'address of 2nd array element (variant)
    CopyMemory bt(0), ByVal p2, 16              'copy bytes
    DisplayBytes bt, "Variant data structure"  'variant data structure
    CopyMemory itype2, ByVal p2, 2              '3 = long
    CopyMemory lValue1, ByVal p2 + 8, 4        '2nd array value = long
    Debug.Print p1, p2, itype1, itype2, dValue1, lValue1
    'VarPtr(vB(0)) is the address of the 1st array element
       
    vB = A                                      'assign array of Long to vB
    p1 = VarPtr(vB(0))                          'address of what?
    CopyMemory bt(0), ByVal VarPtr(vB(0)), 16  'copy bytes from VarPtr(vB(0))
    DisplayBytes bt, "What is this?"            'looks like variant data structure
    CopyMemory itype1, ByVal VarPtr(vB(0)), 2  '16387 valid data type???
    CopyMemory bt(0), ByVal p1, 16              'use p1 instead of VarPtr(vB(0))
    DisplayBytes bt, "1st 2 bytes missing!"    'missing bytes, very strange!
    CopyMemory p2, ByVal p1 + 8, 4              'address of 1st array element
    CopyMemory lValue1, ByVal p2, 4            '1st array value
    CopyMemory lValue2, ByVal p2 + 4, 4        '2nd array value
    Debug.Print p1, p2, itype1, " ", lValue1, lValue2
    'VarPtr(vB(0))is an address that is offset 8 bytes from the address of the 1st array element

    '05 00 00 00 00 00 00 00 F6 28 5C 8F C2 F5 02 40 << Variant data structure
    '05 00 00 00 00 00 00 00 F6 28 5C 8F C2 F5 02 40 << no difference
    '03 00 00 00 00 00 00 00 0C 00 00 00 C2 F5 02 40 << Variant data structure
    ' 77036264      77036280      5            3            2.37          12
    '03 40 00 00 00 00 00 00 20 55 97 04 00 00 00 00 << What is this?
    '00 00 00 00 00 00 00 00 20 55 97 04 00 00 00 00 << 1st 2 bytes missing!
    ' 1308148      77026592      16387                      837          58
End Sub

Private Sub DisplayBytes(btArr() As Byte, strText As String)
  Dim i As Long
  For i = LBound(btArr) To UBound(btArr)
    If btArr(i) < 16 Then Debug.Print "0";
    Debug.Print Hex(btArr(i)); " ";
  Next i
  Debug.Print "<< "; strText
End Sub

---

I've tidied this up a touch, I'm pretty confident it can handle all the different kinds of Variant

Code:

---

Option Explicit

Public Declare Sub RtlMoveMemory Lib "kernel32" (Destination As Any, Source As Any, ByVal Length As Long)

Sub Main()
Dim Lng As Long
Dim LngArr() As Long
Dim Dbl As Double
Dim Byt As Byte
Dim Txt As String
Dim Txt3 As String * 3
Dim DblArr(2) As Double
Dim Obj As Object
Dim Var As Variant
Dim Dat As Date

  Debug.Print "================================================="
 
  DblArr(0) = 123
  Dbl = 123
  Var = DblArr
  Set Obj = New StdPicture
 
  DebugVariant Lng
  DebugVariant LngArr
  DebugVariant (LngArr)
  DebugVariant Dbl
  DebugVariant Var
  DebugVariant Byt
  DebugVariant Txt
  DebugVariant Txt3
  DebugVariant Obj
  DebugVariant (Obj)
  DebugVariant Dat
  DebugVariant (Dat)
  DebugVariant DblArr
  DebugVariant (DblArr)
  DebugVariant Var(0)
  DebugVariant CDec("123456789012345678901234567")
 
End Sub

Public Sub DebugVariant(Var As Variant)
Dim VarByts(15) As Byte, Ptr As Long
Dim i As Long
 
  RtlMoveMemory VarByts(0), ByVal VarPtr(Var), 16
  Debug.Print "----------------------------"
  Debug.Print "TypeName returns """ & TypeName(Var) & """"
  Debug.Print "VarType() returns " & VarType(Var) & ", the High type byte is " & VarByts(1)
 
  Select Case VarByts(1)
      Case 0 'No high byte flag
        Ptr = VarPtr(Var) + 8
        Debug.Print "The variant contains the passed variable"
        Debug.Print "The data resides at " & Ptr
      Case 32 'Flag &H2000 (8192) IsArray
        RtlMoveMemory Ptr, VarByts(8), 4
        If Ptr Then
            Debug.Print "The variant contains a pointer to a safearray structure"
            Debug.Print "The Structure resides at " & Ptr
            RtlMoveMemory Ptr, ByVal Ptr + 12, 4
            Debug.Print "The first element resides at " & Ptr
        Else
            Debug.Print "The variant contains nothing the array is unintialised"
        End If
      Case 64 'Flag &H4000 (16384) IsPointer
        Debug.Print "The variant contains a pointer to the sub variable"
          RtlMoveMemory Ptr, VarByts(8), 4
        Debug.Print "The data resides at " & Ptr
      Case 96 'Flags &H6000 (24576) IsArray Or IsPointer
        RtlMoveMemory Ptr, VarByts(8), 4
        RtlMoveMemory Ptr, ByVal Ptr, 4
        If Ptr Then
            Debug.Print "The variant contains a pointer to a pointer to a safearray structure"
            Debug.Print "The Structure resides at " & Ptr
            RtlMoveMemory Ptr, ByVal Ptr + 12, 4
            Debug.Print "The first element resides at " & Ptr
        Else
            Debug.Print "The variant contains a pointer to an unintialised array"
        End If
      Case Else
        Debug.Print ":( Don't know what the variant contains. " & VarByts(1) * 256 & "?" 'let me know if you see this
  End Select
 
  ''Return addresses confirmed by returning ptr as a function
  ''DebugVariant = Ptr 'return the address of the variable (first element if array)
 
  'Extra bit to display the Variants bytes
  For i = 15 To 0 Step -1
      If i And 1 Then
        Select Case i
            Case 1: Debug.Print vbNewLine & "  Type ";
            Case 7: Debug.Print vbNewLine & "Reseved ";
            Case 15: Debug.Print vbNewLine & "  Data ";
            Case Else: Debug.Print "|"; 'for hex output
        End Select
      End If
      Debug.Print Right$("0" & Hex(VarByts(i)), 2); ' & " "; 'for hex output
      'Debug.Print Format(VarByts(i), "000 "); 'for dec output'
  Next i
  Debug.Print
 
End Sub

---

If a decimal is passed the reserved bytes are used to make up the missing data (if required).

I've tested the above by checking the addresses for the right data, it seems to work fine.

It does not explain (it just avoids) the weird stuff with VarPtr.