Results 1 to 5 of 5

Thread: Executing assembly language functions in VB6 using DispCallFunc.

  1. #1

    Thread Starter
    Angel of Code Niya's Avatar
    Join Date
    Nov 2011
    Posts
    6,505

    Executing assembly language functions in VB6 using DispCallFunc.

    Yes, I know this has been done to death on these forums but one more couldn't hurt.

    This is yet another sample showing how to call function pointers in VB6. In this sample I wrote 3 functions in assembly language, assembled them to x86 machine code and directly execute them from normal byte arrays. This method uses the DispCallFunc API to call them through function pointers.

    The 3 assembly demo functions are as follows: One that adds two Longs and returns the result. One that takes an array of Longs, sums them and returns the result. The last one is a sub string function that works like Mid$. These functions were written by me purely for demonstration purposes. I'm far from being an assembly language expert so don't expect them to be top notch, super performant examples of good assembly code.

    All the assembly functions observe the standard calling convention.

    Additional Credits

    The author of this article for showing me how to use DispCallFunc correctly.

    The trick

    For suggesting the DispCallFunc API which I didn't know about.

    Olaf Schmidt

    For providing information about a bug I encountered with Variants and the VT_BYREF flag.

    Additional Links

    The thread where this began. The OP in this thread also demonstrates a method of calling function pointers using CallWindowProc.

    A more detailed and robust version of the DispCallFunc method by Olaf Schmidt. His version can also make cdecl calls.

    Some additional information on this topic by Olaf Schmidt.
    Attached Files Attached Files
    Treeview with NodeAdded/NodesRemoved events | BlinkLabel control | Calculate Permutations | Object Enums | ComboBox with centered items | .Net Internals article(not mine) | Wizard Control | Understanding Multi-Threading | Simple file compression | Demon Arena

    Copy/move files using Windows Shell

    C++ programmers will dismiss you as a cretinous simpleton for your inability to keep track of pointers chained 6 levels deep and Java programmers will pillory you for buying into the evils of Microsoft. Meanwhile C# programmers will get paid just a little bit more than you for writing exactly the same code and VB6 programmers will continue to whitter on about "footprints". - FunkyDexter

    There's just no reason to use garbage like InputBox. - jmcilhinney

    The threads I start are Niya and Olaf free zones. No arguing about the benefits of VB6 over .NET here please. Happiness must reign. - yereverluvinuncleber

  2. #2

    Thread Starter
    Angel of Code Niya's Avatar
    Join Date
    Nov 2011
    Posts
    6,505

    Re: Executing assembly language functions in VB6 using DispCallFunc.

    Here are some more functions I wrote just for the fun of it:-

    Add these to modAsmFunctions:-
    Code:
    Public Function GetToUpperProc() As Byte()
        '******************************************************************************************************
        'This function was generated by a tool. The tool also assembled the x86 native code
        'from assembly language source code and converted it to a base 64 String.
        'Created in VB.Net by Niya(Sept 20, 2021)
        '******************************************************************************************************
    
        '******************************************************************************************************
        'Assembly source code
        '******************************************************************************************************
        '; Native prototype:
        ';===================================================================================================================
        '; void ToUpper(wchar_t* str);
        ';===================================================================================================================
        ';
        '
        'org 0                                  ; Base this code at address 0
        'use32                                  ; Tell FASM.Net that this is 32 bit code
        '
        '                                       ; Set up stack frame
        '; -----------------------------------------------------------------------------------------
        'push ebp                               ; Save the calling function's base pointer
        'mov ebp, esp                           ; Set the base pointer to represent this function's stack frame
        '
        'push ebx
        'push ecx
        '; -----------------------------------------------------------------------------------------
        '
        'xor eax, eax                           ; EAX is our index. Set it to 0
        '
        'mov ebx, [ebp + 8]                     ; Copy string pointer from parameter into EBX
        '
        'strLoop:
        '
        '   mov cx, word [ebx + eax * 2]        ; Get current character
        '
        '   cmp cx, 0                           ; If char is null.....
        '   je endLoop                          ; then end the loop
        '
        '   cmp cx, 97                          ; If current char code is less that 97.....
        '   jl skipChar                         ; then it's not a lower case letter so skip it
        '
        '   cmp cx,122                          ; If current char code is greater than 122.....
        '   jg skipChar                         ; then it's not a lower case letter so skip it
        '
        '       sub cx, 32                      ; Subtract 32 to make it upper case letter
        '       mov [ebx + eax * 2], cx         ; Put the capital letter back into the String
        '
        '   skipChar:
        '
        '   inc eax
        '
        'jmp strLoop
        '
        'endLoop:
        '
        '
        '
        '                                       ; Tear down stack frame
        '; -----------------------------------------------------------------------------------------
        'pop ecx
        'pop ebx
        '
        'mov esp, ebp
        'pop ebp
        '
        'ret 4
        
        '******************************************************************************************************
    
        Dim proc As String
    
        proc = "VYnlU1ExwItdCGaLDENmg/kAdBdmg/lhfA5mg/l6fwhmg+kgZokMQ0Dr31lbiexdwgQA"
    
        GetToUpperProc = FromBase64(proc)
    
    End Function
    
    Public Function GetStrLenProc() As Byte()
        '******************************************************************************************************
        'This function was generated by a tool. The tool also assembled the x86 native code
        'from assembly language source code and converted it to a base 64 String.
        'Created in VB.Net by Niya(Sept 20, 2021)
        '******************************************************************************************************
    
        '******************************************************************************************************
        'Assembly source code
        '******************************************************************************************************
        '; Native prototype:
        ';===================================================================================================================
        '; int StrLen(wchar_t* str);
        ';===================================================================================================================
        ';
        '
        'org 0                                  ; Base this code at address 0
        'use32                                  ; Tell FASM.Net that this is 32 bit code
        '
        '                                       ; Set up stack frame
        '; -----------------------------------------------------------------------------------------
        'push ebp                               ; Save the calling function's base pointer
        'mov ebp, esp                           ; Set the base pointer to represent this function's stack frame
        '
        'push ebx
        '
        'mov eax, -1                            ; Set EAX to -1 so we can start the loop with an index increment
        '
        'mov ebx, [ebp + 8]                     ; Get pointer to string
        '
        'strLoop:
        '   inc eax                             ; Increment index
        '
        '   cmp word [ebx + eax * 2], 0         ; Is current character null?
        'jne strLoop                            ; Goto strLoop if current character is not equal to null
        '
        '
        'pop ebx
        '
        'mov esp, ebp
        'pop ebp
        '
        'ret 4
        
        '******************************************************************************************************
    
        Dim proc As String
    
        proc = "VYnlU7j/////i10IQGaDPEMAdfhbiexdwgQA"
    
        GetStrLenProc = FromBase64(proc)
    
    End Function
    
    
    Public Function GetExecCBProc() As Byte()
        '******************************************************************************************************
        'This function was generated by a tool. The tool also assembled the x86 native code
        'from assembly language source code and converted it to a base 64 String.
        'Created in VB.Net by Niya(Sept 20, 2021)
        '******************************************************************************************************
    
        '******************************************************************************************************
        'Assembly source code
        '******************************************************************************************************
        ';Native prototype:
        ';===================================================================================================================
        '; int ExecuteCallBack(void* pCallbackAddr, int data1, int data2);
        ';===================================================================================================================
        ';Callback signature:
        ';===================================================================================================================
        '; int CallBack(int data1, int data2);
        ';===================================================================================================================
        ';
        ';Note: The callback is called using standard calling convention
        '
        'org 0                                  ; Base this code at address 0
        'use32                                  ; Tell FASM.Net that this is 32 bit code
        '
        '                                       ; Set up stack frame
        '; -----------------------------------------------------------------------------------------
        'push ebp                               ; Save the calling function's base pointer
        'mov ebp, esp                           ; Set the base pointer to represent this function's stack frame
        '
        'push dword [ebp + 16]                  ; Push 3rd parameter onto the stack
        'push dword [ebp + 12]                  ; Push 2nd parameter onto the stack
        '
        'call dword [ebp + 8]                   ; Call the callback function. It's return value will be in EAX.
        '                                       ; The standard calling convention is assumed so no need to pop
        '                                       ; the arguments passed off the stack
        'mov esp, ebp
        'pop ebp
        '
        'ret 12                                 ; Clear the 3 arguments of the stack as per the standard calling convention
        
        '******************************************************************************************************
    
        Dim proc As String
    
        proc = "VYnl/3UQ/3UM/1UIiexdwgwA"
    
        GetExecCBProc = FromBase64(proc)
    
    End Function
    Add these to ModMain:-
    Code:
    Public Sub TestExecuteCallBack()
    
        Dim data1 As Long
        Dim data2 As Long
        Dim callbackResult As Long
        Dim func() As Byte
        
        data1 = 50
        data2 = 200
    
        ';Native prototype:
        ';===================================================================================================================
        '; int ExecuteCallBack(void* pCallbackAddr, int data1, int data2);
        ';===================================================================================================================
        ';Callback signature:
        ';===================================================================================================================
        '; int CallBack(int data1, int data2);
        ';===================================================================================================================
        func = GetExecCBProc() 'Get x86 machine code for ExecuteCallback function
        
        'Make sure the memory can be used to execute code
        PrepProcMem func
    
        callbackResult = CallFunctionPointer(VarPtr(func(0)), vbLong, AddressOf Callback, data1, data2)
           
        Dim msg As String
        
        msg = msg & "Data passed to callback:-" & vbCrLf
        msg = msg & "data1 = " & CStr(data1) & vbCrLf
        msg = msg & "data2 = " & CStr(data2) & vbCrLf
        msg = msg & vbCrLf
        msg = msg & "Callback result = " & CStr(callbackResult)
           
           
        MsgBox msg
        
    End Sub
    
    Public Function Callback(ByVal data1 As Long, ByVal data2 As Long) As Long
                
        'This callback will add the 2 numbers and return the result
        Callback = data1 + data2
            
    End Function
    
    
    Public Sub TestStrLen()
            
        Dim myStr As String
        Dim strLen As Long
        Dim func() As Byte
        
        myStr = "ABC DEF"
            
        '; Native prototype:
        ';===================================================================================================================
        '; int StrLen(wchar_t* str);
        ';===================================================================================================================
        func = GetStrLenProc() 'Get x86 machine code for StrLen function
        
        'Make sure the memory can be used to execute code
        PrepProcMem func
    
        strLen = CallFunctionPointer(VarPtr(func(0)), vbLong, StrPtr(myStr))
        
        MsgBox "String = " & myStr & vbCrLf & "Length = " & CStr(strLen)
    
    End Sub
    
    
    Public Sub TestToUpper()
        
        
        Dim myStr As String
        Dim upperStr As String
        Dim func() As Byte
        
        myStr = "00**(( just an orginary string ))**00"
        upperStr = myStr
        
            '; Native prototype:
        ';===================================================================================================================
        '; void ToUpper(wchar_t* str);
        ';===================================================================================================================
        func = GetToUpperProc() 'Get x86 machine code for ToUpper function
        
        'Make sure the memory can be used to execute code
        PrepProcMem func
    
        CallFunctionPointer VarPtr(func(0)), vbLong, StrPtr(upperStr)
        
        MsgBox "Lower case:-" & vbCrLf & myStr & vbCrLf & vbCrLf & "Upper case:-" & vbCrLf & upperStr
        
    
    End Sub
    And update Sub Main in ModMain:-
    Code:
    Public Sub Main()
        TestAdd
        TestStrLen
        TestToUpper
        TestSubString
        TestSumArray
        TestExecuteCallBack
    End Sub
    Those 3 functions are my own assembly versions of Strlen, UCASE and a function that can execute a callback that takes arguments and returns a value in VB6. The string functions are designed for UTF-16 Unicode Strings.

    Let me stress again, do not assume them to be examples good assembly code. This is just me having a bit of fun and hopefully in the process someone can learn a thing or two.

    I didn't bother to update the ZIP in the OP with these new functions because that project because it is sufficient as it is to demonstrate calling these kinds of functions using function pointers. This is just a bit of extra stuff that someone may or may not find interesting.
    Treeview with NodeAdded/NodesRemoved events | BlinkLabel control | Calculate Permutations | Object Enums | ComboBox with centered items | .Net Internals article(not mine) | Wizard Control | Understanding Multi-Threading | Simple file compression | Demon Arena

    Copy/move files using Windows Shell

    C++ programmers will dismiss you as a cretinous simpleton for your inability to keep track of pointers chained 6 levels deep and Java programmers will pillory you for buying into the evils of Microsoft. Meanwhile C# programmers will get paid just a little bit more than you for writing exactly the same code and VB6 programmers will continue to whitter on about "footprints". - FunkyDexter

    There's just no reason to use garbage like InputBox. - jmcilhinney

    The threads I start are Niya and Olaf free zones. No arguing about the benefits of VB6 over .NET here please. Happiness must reign. - yereverluvinuncleber

  3. #3
    Fanatic Member
    Join Date
    May 2014
    Location
    Kallithea Attikis, Greece
    Posts
    1,016

    Re: Executing assembly language functions in VB6 using DispCallFunc.

    @Niya
    GetExecCBProc not found in the code above
    Also examples TestAdd, TestSumArray not exist.

  4. #4
    PowerPoster Arnoutdv's Avatar
    Join Date
    Oct 2013
    Posts
    4,702

    Re: Executing assembly language functions in VB6 using DispCallFunc.

    The code is in post #2

  5. #5

    Thread Starter
    Angel of Code Niya's Avatar
    Join Date
    Nov 2011
    Posts
    6,505

    Re: Executing assembly language functions in VB6 using DispCallFunc.

    Quote Originally Posted by georgekar View Post
    @Niya
    GetExecCBProc not found in the code above
    Also examples TestAdd, TestSumArray not exist.
    The code in post #2 is meant to be added via copy paste to the project I attached in the OP.

    Like I said, the code in post #2 doesn't really add anything to the point of the project. It was just something extra I did for my own fun that I decided to share.
    Last edited by Niya; Sep 24th, 2021 at 05:20 AM. Reason: Just to add a comma ;)
    Treeview with NodeAdded/NodesRemoved events | BlinkLabel control | Calculate Permutations | Object Enums | ComboBox with centered items | .Net Internals article(not mine) | Wizard Control | Understanding Multi-Threading | Simple file compression | Demon Arena

    Copy/move files using Windows Shell

    C++ programmers will dismiss you as a cretinous simpleton for your inability to keep track of pointers chained 6 levels deep and Java programmers will pillory you for buying into the evils of Microsoft. Meanwhile C# programmers will get paid just a little bit more than you for writing exactly the same code and VB6 programmers will continue to whitter on about "footprints". - FunkyDexter

    There's just no reason to use garbage like InputBox. - jmcilhinney

    The threads I start are Niya and Olaf free zones. No arguing about the benefits of VB6 over .NET here please. Happiness must reign. - yereverluvinuncleber

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Click Here to Expand Forum to Full Width