dcsimg
Results 1 to 5 of 5

Thread: [RESOLVED] Pointers, 2D Arrays and Debugging a crash

  1. #1

    Thread Starter
    Junior Member
    Join Date
    Apr 2019
    Posts
    21

    Resolved [RESOLVED] Pointers, 2D Arrays and Debugging a crash

    I'm trying to make an efficient method of storing a reference to an array for later use, without storing the array itself. This is because many arrays I use are huge.

    To accomplish this I made the following class (VBA):

    Code:
    VERSION 1.0 CLASS
    BEGIN
      MultiUse = -1  'True
    END
    Attribute VB_Name = "stdRefArray"
    Attribute VB_GlobalNameSpace = False
    Attribute VB_Creatable = False
    Attribute VB_PredeclaredId = True
    Attribute VB_Exposed = False
    'Status WIP
    'High level wrapper around flood risk.
    'Ultimate goal is to make a class which calculates affordability, total project benefit, historic and overridden likelihood,
    'and ultimately port code out of HFRR core into a class wrapper.
    
    #Const DEBUG_PERF = False
    
    'Variables for pData
    Private Declare PtrSafe Sub CopyMemory Lib "kernel32" Alias "RtlMoveMemory" (lpDest As Any, ByVal lpSource As LongPtr, ByVal cbCopy As Long)
    Private pArrayPtr As LongPtr
    Private Const VARIANT_SIZE As Long = 16
    Private iRowOffset As LongPtr
    Private iColOffset As LongPtr
    
    Public Function Create(ByRef Data As Variant) As stdRefArray
        Set Create = New stdRefArray
        Call Create.Init(Data)
    End Function
    Public Sub Init(ByRef Data As Variant)
        'Store array pointer
        pArrayPtr = VarPtr(Data(1, 1))
        
        'Store array offsets (we will use these later to quickly obtain data from array without storing it. See property pData for more info
        iRowOffset = VARIANT_SIZE
        iColOffset = UBound(Data, 1) * VARIANT_SIZE
    End Sub
    
    'Method Data as Variant2D()
    '  Originally the class stored copies of pData however, for large pData this ground processes to a halt.
    '  The following property private property is a work around for this issue. Instead we store a pointer to the original ByRef array.
    '  This speeds operations which used to take 3s to 3ms (i.e. 10^3 speed boost)
    '
    '  When dealing with arrays in memory the landscape is a little different to how they look from VBA...
    '  The memory landscape of 2D arrays is as follows (e.g. a 3 row by 2 column array [[1,2,3],[4,5,6]]:
    '    | Offset | iRow | iCol |
    '    |    1   |  1   |  1   |
    '    |    2   |  2   |  1   |
    '    |    3   |  3   |  1   |
    '    |    4   |  1   |  2   |
    '    |    5   |  2   |  2   |
    '    |    6   |  3   |  2   |
    '  Therefore we need to use iRowOffset (size of variable, in this case SizeOf(VARIANT) [16], and iColOffset (number of rows * size of variable) [UBound * 16]
    '  Using these in combination with CopyMemory allows us to copy portions of memory from the array into the return value.
    Function Data(iRow As Long, iCol As Long) As Variant
    Attribute Data.VB_UserMemId = 0
        'Get the location of a value in 2d array
        Dim pValueLocation As LongPtr: pValueLocation = pArrayPtr + iRowOffset * (iRow - 1) + iColOffset * (iCol - 1)
        
        On Error GoTo ErrorOccurred:
        
        'Return value
        Dim v
        Call CopyMemory(v, pValueLocation, VARIANT_SIZE)
        
        Data = v
        Exit Function
    ErrorOccurred:
        Debug.Assert False
    End Function
    
    Private Sub Class_Terminate()
        pArrayPtr = 0
        iRowOffset = 0
        iColOffset = 0
    End Sub
    Currently there is a huge problem however, namely "The project crashes at de-initialize time".

    Code:
    Sub aaTestStuff()
      Const ub1 As Long = 100
      Const ub2 As Long = 100
      
      Dim x(1 To ub1, 1 To ub2)
      For i = 1 To ub1
        For j = 1 To ub2
          x(i, j) = "yo"
        Next
      Next
      
      'Create reference to x
      Dim y As stdRefArray
      Set y = stdRefArray.Create(x)
      
      For i = 1 To ub1
        For j = 1 To ub2
          Call y(i, j) ' we don't care about the return value (for now), so use Call
        Next
      Next
      MsgBox "alert"
    End Sub      'The application (Excel) crashes here?!
    Note the crash ONLY occurs when the 2d-array contains strings!

    Questions:
    1. If the code is ported to VB6, does the code work as is?
    2. What am I doing wrong? Can you really not store the reference to an array and use it at a later date?
    3. Is there an easier way to do what I'm trying to do?
    Last edited by sancarn; Dec 17th, 2019 at 09:57 AM.

  2. #2

    Thread Starter
    Junior Member
    Join Date
    Apr 2019
    Posts
    21

    Re: Pointers, 2D Arrays and Debugging a crash

    I believe this is the VB6 port (removed LongPtr for Long, and PtrSafe:

    Code:
    VERSION 1.0 CLASS
    BEGIN
      MultiUse = -1  'True
    END
    Attribute VB_Name = "stdRefArray"
    Attribute VB_GlobalNameSpace = False
    Attribute VB_Creatable = False
    Attribute VB_PredeclaredId = True
    Attribute VB_Exposed = False
    'Status WIP
    'High level wrapper around flood risk.
    'Ultimate goal is to make a class which calculates affordability, total project benefit, historic and overridden likelihood,
    'and ultimately port code out of HFRR core into a class wrapper.
    
    #Const DEBUG_PERF = False
    
    'Variables for pData
    Private Declare Sub CopyMemory Lib "kernel32" Alias "RtlMoveMemory" (lpDest As Any, ByVal lpSource As Long, ByVal cbCopy As Long)
    Private pArrayPtr As Long
    Private Const VARIANT_SIZE As Long = 16
    Private iRowOffset As Long
    Private iColOffset As Long
    
    Public Function Create(ByRef Data As Variant) As stdRefArray
        Set Create = New stdRefArray
        Call Create.Init(Data)
    End Function
    Public Sub Init(ByRef Data As Variant)
        'Store array pointer
        pArrayPtr = VarPtr(Data(1, 1))
        
        'Store array offsets (we will use these later to quickly obtain data from array without storing it. See property pData for more info
        iRowOffset = VARIANT_SIZE
        iColOffset = UBound(Data, 1) * VARIANT_SIZE
    End Sub
    
    'Method Data as Variant2D()
    '  Originally the class stored copies of pData however, for large pData this ground processes to a halt.
    '  The following property private property is a work around for this issue. Instead we store a pointer to the original ByRef array.
    '  This speeds operations which used to take 3s to 3ms (i.e. 10^3 speed boost)
    '
    '  When dealing with arrays in memory the landscape is a little different to how they look from VBA...
    '  The memory landscape of 2D arrays is as follows (e.g. a 3 row by 2 column array [[1,2,3],[4,5,6]]:
    '    | Offset | iRow | iCol |
    '    |    1   |  1   |  1   |
    '    |    2   |  2   |  1   |
    '    |    3   |  3   |  1   |
    '    |    4   |  1   |  2   |
    '    |    5   |  2   |  2   |
    '    |    6   |  3   |  2   |
    '  Therefore we need to use iRowOffset (size of variable, in this case SizeOf(VARIANT) [16], and iColOffset (number of rows * size of variable) [UBound * 16]
    '  Using these in combination with CopyMemory allows us to copy portions of memory from the array into the return value.
    Function Data(iRow As Long, iCol As Long) As Variant
    Attribute Data.VB_UserMemId = 0
        'Get the location of a value in 2d array
        Dim pValueLocation As Long: pValueLocation = pArrayPtr + iRowOffset * (iRow - 1) + iColOffset * (iCol - 1)
        
        On Error GoTo ErrorOccurred:
        
        'Return value
        Dim v
        Call CopyMemory(v, pValueLocation, VARIANT_SIZE)
        
        Data = v
        Exit Function
    ErrorOccurred:
        Debug.Assert False
    End Function
    
    Private Sub Class_Terminate()
        pArrayPtr = 0
        iRowOffset = 0
        iColOffset = 0
    End Sub

  3. #3
    PowerPoster Arnoutdv's Avatar
    Join Date
    Oct 2013
    Posts
    3,826

    Re: Pointers, 2D Arrays and Debugging a crash

    A string array is not stored as a single block of data.
    So to work with string arrays you have to create a specific class and not a generic one

  4. #4

    Thread Starter
    Junior Member
    Join Date
    Apr 2019
    Posts
    21

    Re: Pointers, 2D Arrays and Debugging a crash

    I solved the issue after stumbling across this stack overflow post.

    Code:
    VERSION 1.0 CLASS
    BEGIN
      MultiUse = -1  'True
    END
    Attribute VB_Name = "stdRefArray"
    Attribute VB_GlobalNameSpace = False
    Attribute VB_Creatable = False
    Attribute VB_PredeclaredId = True
    Attribute VB_Exposed = False
    'Status WIP
    'High level wrapper around flood risk.
    'Ultimate goal is to make a class which calculates affordability, total project benefit, historic and overridden likelihood,
    'and ultimately port code out of HFRR core into a class wrapper.
    
    #Const DEBUG_PERF = False
    
    'Variables for pData
    Private Declare PtrSafe Sub FillMemory Lib "kernel32" Alias "RtlFillMemory" (Destination As Any, ByVal Length As Long, ByVal Fill As Byte)
    Private Declare PtrSafe Sub CopyMemory Lib "kernel32" Alias "RtlMoveMemory" (lpDest As Any, lpSource As Any, ByVal cbCopy As Long)
    
    
    Public Data As Variant
    
    Private Const VARIANT_SIZE As Long = 16
    
    Public Function Create(ByRef Data As Variant) As stdRefArray
        Set Create = New stdRefArray
        Call Create.Init(Data)
    End Function
    Public Sub Init(ByRef DataIn As Variant)
        'Create direct reference to array:
        CopyMemory Data, DataIn, VARIANT_SIZE
    End Sub
    
    Private Sub Class_Terminate()
       'Clean up array reference
       FillMemory Data, VARIANT_SIZE, 0
    End Sub
    
    Public Function GetData(ByVal iRow as long, ByVal iCol as long) as Variant
      Attribute GetData.VB_UserMemID=0
      GetData = GetData(iRow,iCol)
    End Function
    Note: It'll only work with variants. In theory you should be able to copy the SafeArray structure so another variable simply points to the old variable, but in the end I wasn't able to figure that out. This is a lot more hacky but also a fair amount more direct. So that's nice!

  5. #5
    PowerPoster
    Join Date
    Feb 2006
    Posts
    20,972

    Re: Pointers, 2D Arrays and Debugging a crash

    Yep, a String array is an array of pointers to BSTR structs.

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Featured


Click Here to Expand Forum to Full Width