[RESOLVED] Pointers, 2D Arrays and Debugging a crash

[sancarn]

I'm trying to make an efficient method of storing a reference to an array for later use, without storing the array itself. This is because many arrays I use are huge.

To accomplish this I made the following class (VBA):

VERSION 1.0 CLASS
BEGIN
  MultiUse = -1  'True
END
Attribute VB_Name = "stdRefArray"
Attribute VB_GlobalNameSpace = False
Attribute VB_Creatable = False
Attribute VB_PredeclaredId = True
Attribute VB_Exposed = False
'Status WIP
'High level wrapper around flood risk.
'Ultimate goal is to make a class which calculates affordability, total project benefit, historic and overridden likelihood,
'and ultimately port code out of HFRR core into a class wrapper.

#Const DEBUG_PERF = False

'Variables for pData
Private Declare PtrSafe Sub CopyMemory Lib "kernel32" Alias "RtlMoveMemory" (lpDest As Any, ByVal lpSource As LongPtr, ByVal cbCopy As Long)
Private pArrayPtr As LongPtr
Private Const VARIANT_SIZE As Long = 16
Private iRowOffset As LongPtr
Private iColOffset As LongPtr

Public Function Create(ByRef Data As Variant) As stdRefArray
    Set Create = New stdRefArray
    Call Create.Init(Data)
End Function
Public Sub Init(ByRef Data As Variant)
    'Store array pointer
    pArrayPtr = VarPtr(Data(1, 1))
    
    'Store array offsets (we will use these later to quickly obtain data from array without storing it. See property pData for more info
    iRowOffset = VARIANT_SIZE
    iColOffset = UBound(Data, 1) * VARIANT_SIZE
End Sub

'Method Data as Variant2D()
'  Originally the class stored copies of pData however, for large pData this ground processes to a halt.
'  The following property private property is a work around for this issue. Instead we store a pointer to the original ByRef array.
'  This speeds operations which used to take 3s to 3ms (i.e. 10^3 speed boost)
'
'  When dealing with arrays in memory the landscape is a little different to how they look from VBA...
'  The memory landscape of 2D arrays is as follows (e.g. a 3 row by 2 column array [[1,2,3],[4,5,6]]:
'    | Offset | iRow | iCol |
'    |    1   |  1   |  1   |
'    |    2   |  2   |  1   |
'    |    3   |  3   |  1   |
'    |    4   |  1   |  2   |
'    |    5   |  2   |  2   |
'    |    6   |  3   |  2   |
'  Therefore we need to use iRowOffset (size of variable, in this case SizeOf(VARIANT) [16], and iColOffset (number of rows * size of variable) [UBound * 16]
'  Using these in combination with CopyMemory allows us to copy portions of memory from the array into the return value.
Function Data(iRow As Long, iCol As Long) As Variant
Attribute Data.VB_UserMemId = 0
    'Get the location of a value in 2d array
    Dim pValueLocation As LongPtr: pValueLocation = pArrayPtr + iRowOffset * (iRow - 1) + iColOffset * (iCol - 1)
    
    On Error GoTo ErrorOccurred:
    
    'Return value
    Dim v
    Call CopyMemory(v, pValueLocation, VARIANT_SIZE)
    
    Data = v
    Exit Function
ErrorOccurred:
    Debug.Assert False
End Function

Private Sub Class_Terminate()
    pArrayPtr = 0
    iRowOffset = 0
    iColOffset = 0
End Sub

Currently there is a huge problem however, namely "The project crashes at de-initialize time".

Sub aaTestStuff()
  Const ub1 As Long = 100
  Const ub2 As Long = 100
  
  Dim x(1 To ub1, 1 To ub2)
  For i = 1 To ub1
    For j = 1 To ub2
      x(i, j) = "yo"
    Next
  Next
  
  'Create reference to x
  Dim y As stdRefArray
  Set y = stdRefArray.Create(x)
  
  For i = 1 To ub1
    For j = 1 To ub2
      Call y(i, j) ' we don't care about the return value (for now), so use Call
    Next
  Next
  MsgBox "alert"
End Sub      'The application (Excel) crashes here?!

Note the crash ONLY occurs when the 2d-array contains strings!

Questions:

1. If the code is ported to VB6, does the code work as is?
2. What am I doing wrong? Can you really not store the reference to an array and use it at a later date?
3. Is there an easier way to do what I'm trying to do?

[sancarn]

I believe this is the VB6 port (removed LongPtr for Long, and PtrSafe:

VERSION 1.0 CLASS
BEGIN
  MultiUse = -1  'True
END
Attribute VB_Name = "stdRefArray"
Attribute VB_GlobalNameSpace = False
Attribute VB_Creatable = False
Attribute VB_PredeclaredId = True
Attribute VB_Exposed = False
'Status WIP
'High level wrapper around flood risk.
'Ultimate goal is to make a class which calculates affordability, total project benefit, historic and overridden likelihood,
'and ultimately port code out of HFRR core into a class wrapper.

#Const DEBUG_PERF = False

'Variables for pData
Private Declare Sub CopyMemory Lib "kernel32" Alias "RtlMoveMemory" (lpDest As Any, ByVal lpSource As Long, ByVal cbCopy As Long)
Private pArrayPtr As Long
Private Const VARIANT_SIZE As Long = 16
Private iRowOffset As Long
Private iColOffset As Long

Public Function Create(ByRef Data As Variant) As stdRefArray
    Set Create = New stdRefArray
    Call Create.Init(Data)
End Function
Public Sub Init(ByRef Data As Variant)
    'Store array pointer
    pArrayPtr = VarPtr(Data(1, 1))
    
    'Store array offsets (we will use these later to quickly obtain data from array without storing it. See property pData for more info
    iRowOffset = VARIANT_SIZE
    iColOffset = UBound(Data, 1) * VARIANT_SIZE
End Sub

'Method Data as Variant2D()
'  Originally the class stored copies of pData however, for large pData this ground processes to a halt.
'  The following property private property is a work around for this issue. Instead we store a pointer to the original ByRef array.
'  This speeds operations which used to take 3s to 3ms (i.e. 10^3 speed boost)
'
'  When dealing with arrays in memory the landscape is a little different to how they look from VBA...
'  The memory landscape of 2D arrays is as follows (e.g. a 3 row by 2 column array [[1,2,3],[4,5,6]]:
'    | Offset | iRow | iCol |
'    |    1   |  1   |  1   |
'    |    2   |  2   |  1   |
'    |    3   |  3   |  1   |
'    |    4   |  1   |  2   |
'    |    5   |  2   |  2   |
'    |    6   |  3   |  2   |
'  Therefore we need to use iRowOffset (size of variable, in this case SizeOf(VARIANT) [16], and iColOffset (number of rows * size of variable) [UBound * 16]
'  Using these in combination with CopyMemory allows us to copy portions of memory from the array into the return value.
Function Data(iRow As Long, iCol As Long) As Variant
Attribute Data.VB_UserMemId = 0
    'Get the location of a value in 2d array
    Dim pValueLocation As Long: pValueLocation = pArrayPtr + iRowOffset * (iRow - 1) + iColOffset * (iCol - 1)
    
    On Error GoTo ErrorOccurred:
    
    'Return value
    Dim v
    Call CopyMemory(v, pValueLocation, VARIANT_SIZE)
    
    Data = v
    Exit Function
ErrorOccurred:
    Debug.Assert False
End Function

Private Sub Class_Terminate()
    pArrayPtr = 0
    iRowOffset = 0
    iColOffset = 0
End Sub

[Arnoutdv]

A string array is not stored as a single block of data.
So to work with string arrays you have to create a specific class and not a generic one

[sancarn]

I solved the issue after stumbling across this stack overflow post.

VERSION 1.0 CLASS
BEGIN
  MultiUse = -1  'True
END
Attribute VB_Name = "stdRefArray"
Attribute VB_GlobalNameSpace = False
Attribute VB_Creatable = False
Attribute VB_PredeclaredId = True
Attribute VB_Exposed = False
'Status WIP
'High level wrapper around flood risk.
'Ultimate goal is to make a class which calculates affordability, total project benefit, historic and overridden likelihood,
'and ultimately port code out of HFRR core into a class wrapper.

#Const DEBUG_PERF = False

'Variables for pData
Private Declare PtrSafe Sub FillMemory Lib "kernel32" Alias "RtlFillMemory" (Destination As Any, ByVal Length As Long, ByVal Fill As Byte)
Private Declare PtrSafe Sub CopyMemory Lib "kernel32" Alias "RtlMoveMemory" (lpDest As Any, lpSource As Any, ByVal cbCopy As Long)


Public Data As Variant

Private Const VARIANT_SIZE As Long = 16

Public Function Create(ByRef Data As Variant) As stdRefArray
    Set Create = New stdRefArray
    Call Create.Init(Data)
End Function
Public Sub Init(ByRef DataIn As Variant)
    'Create direct reference to array:
    CopyMemory Data, DataIn, VARIANT_SIZE
End Sub

Private Sub Class_Terminate()
   'Clean up array reference
   FillMemory Data, VARIANT_SIZE, 0
End Sub

Public Function GetData(ByVal iRow as long, ByVal iCol as long) as Variant
  Attribute GetData.VB_UserMemID=0
  GetData = GetData(iRow,iCol)
End Function

Note: It'll only work with variants. In theory you should be able to copy the SafeArray structure so another variable simply points to the old variable, but in the end I wasn't able to figure that out. This is a lot more hacky but also a fair amount more direct. So that's nice!

[dilettante]

Yep, a String array is an array of pointers to BSTR structs.