[RESOLVED] WideCharToMultiByte problem

I can't manage to upload binary files via WinHTTP (i.e. send post data in byte type), because it appears that there is a problem with function that converts string to bytes - WideCharToMultiByte (I have tried to put CP_UNICODE in CodePage parameter but again it won't convert properly). Can someone help me in solving this?
(...)

EDIT: Finally found a working function here: ToCPString(String, CP_ACP)

You get yourself in trouble when you do things wrong, and trying to use String expressions blindly with binary data is about as wrong as things get.

You don't want a String body payload. That just means numerous gyrations pouring data from glass to glass to glass doing multiple potentially unsafe character encoding conversions.

I fail to see why so many people try to do this and then have to attempt exotic manipulations on top of it, hoping to compensate for the mess. Instead consider using a sort of "binary string builder" as implemented in the attachment.

Code:

---

Option Explicit

Private Declare Sub CopyMemory Lib "kernel32" Alias "RtlMoveMemory" ( _
    ByRef Destination As Any, _
    ByRef Source As Any, _
    ByVal Length As Long)

Private Const BINARY_CHUNK As Long = 256

Private Binary() As Byte
Private BinaryNext As Long

Private Sub ClearBinary()
    Binary = "" 'Creates an initialized but empty Byte array.
    ResetBinary
End Sub

Private Sub ResetBinary()
    BinaryNext = 0 'Retain allocated space for reuse.
End Sub

Private Sub CatBinary(ByRef Bytes() As Byte)
    Dim BytesLen As Long

    BytesLen = UBound(Bytes) - LBound(Bytes) + 1
    If BinaryNext + BytesLen - 1 > UBound(Binary) Then
        If BytesLen > BINARY_CHUNK Then
            ReDim Preserve Binary(BinaryNext + BytesLen - 1)
        Else
            ReDim Preserve Binary(BinaryNext + BINARY_CHUNK - 1)
        End If
    End If
    CopyMemory Binary(BinaryNext), Bytes(LBound(Bytes)), BytesLen
    BinaryNext = BinaryNext + BytesLen
End Sub

Private Sub CatBinaryString(ByRef Text As String)
    Dim Bytes() As Byte

    Bytes = StrConv(Text, vbFromUnicode) 'Safe if Text only has valid 7-bit US-ASCII chars.
    CatBinary Bytes
End Sub

Private Sub TrimBinary()
    If BinaryNext > 0 Then
        ReDim Preserve Binary(BinaryNext - 1)
    Else
        ClearBinary
    End If
End Sub

---

The attachment is large because it includes a sample PDF document to be wrapped as a payload. The sample program writes the result to disk so you can view it.

Thanks for this, it will help me alot!

I'm facing a problem right now. Why this code below causes crash after executing CopyMemory?

VB Code:

Private Declare Function WideCharToMultiByte Lib "Kernel32.dll" ( _
    ByVal CodePage As Long, _
    ByVal dwFlags As Long, _
    ByVal lpWideCharStr As Long, _
    ByVal cchWideChar As Long, _
    ByVal lpMultiByteStr As Long, _
    ByVal cbMultiByte As Long, _
    ByVal lpDefaultChar As Long, _
    ByVal lpUsedDefaultChar As Long _
) As Long
 
Private Const CP_ACP        As Long = 0
 
Private Declare Function CopyMemory Lib "kernel32" Alias "RtlMoveMemory" (ByVal _
        dest As Long, ByVal src As Long, ByVal Length As Long) As Long
 
Private Const BINARY_CHUNK As Long = 256
 
Private Function ToCPString(ByRef the_sValue As String) As Byte()
 
    Dim abytOutput()                As Byte
    Dim nValueLen                   As Long
    Dim nOutputByteLen              As Long
 
    ' Cache the input length.
    nValueLen = Len(the_sValue)
 
    ' See how big the output buffer will be.
    nOutputByteLen = WideCharToMultiByte(CP_ACP, 0&, StrPtr(the_sValue), nValueLen, 0&, 0&, 0&, 0&)
 
    If nOutputByteLen > 0 Then
        ' Resize output byte array to the size of the UTF-8 string.
        ReDim abytOutput(1 To nOutputByteLen)
 
        ' Make this API call again, this time giving a pointer to the output byte array.
        WideCharToMultiByte CP_ACP, 0&, StrPtr(the_sValue), nValueLen, VarPtr(abytOutput(1)), nOutputByteLen, 0&, 0&
    End If
 
    ' Return the array.
    ToCPString = abytOutput()
 
End Function
 
Private Sub CatBinary(bytData() As Byte, Bytes() As Byte)
    Dim BytesLen As Long, BinaryNext As Long
    
    BinaryNext = UBound(bytData) + 1
    BytesLen = UBound(Bytes) - LBound(Bytes) + 1
    If BinaryNext + BytesLen - 1 > BinaryNext Then
        If BytesLen > BINARY_CHUNK Then
            ReDim Preserve bytData(BinaryNext + BytesLen - 1)
        Else
            ReDim Preserve bytData(BinaryNext + BINARY_CHUNK - 1)
        End If
    End If
    CopyMemory bytData(BinaryNext), Bytes(LBound(Bytes)), BytesLen
End Sub
 
Private Sub CatBinaryString(bytData() As Byte, Text As String)
    Dim Bytes() As Byte
    
    Bytes = ToCPString(Text)
    CatBinary bytData, Bytes
End Sub
 
Private Sub TrimBinary(bytData() As Byte)
    If UBound(bytData) > 0 Then
        ReDim Preserve bytData(UBound(bytData) - 1)
    Else
        bytData = ""
    End If
End Sub
 
Sub Main()
Dim bytData() As Byte: bytData = ""
CatBinaryString bytData, "test"
MsgBox StrConv(bytData, vbUnicode)
End Sub

When copymemory crashes, likely one or both of these are the reason:

1. BinaryNext + BytesLen - 1 > UBound(bytData)
2. LBound(Bytes) + BytesLen - 1 > UBound(Bytes)

You should verify this & if either are true, your calculations are off

Can you test it, I think that it's all right with what you said? Just put the code above into a module and run it.

Quote:

---

Originally Posted by MikiSoft

I'm facing a problem right now. Why this code below causes crash after executing CopyMemory?

---

Code:

---

Private Declare Function CopyMemory Lib "kernel32" Alias "RtlMoveMemory" (ByVal _
        dest As Long, ByVal src As Long, ByVal Length As Long) As Long

---

With dest and src declared as ByVal parameters, you must pass VarPtrs:

Code:

---

CopyMemory VarPtr(bytData(BinaryNext)), VarPtr(Bytes(LBound(Bytes))), _
BytesLen

---

HTH, Wolfgang

It's the API declaration and API call that are the problem, ByVal vs ByRef

Either

1. Pass your parameters ByVal vs ByRef in your CopyMemory call, i.e.,
CopyMemory VarPtr(bytData(BinaryNext)), VarPtr(Bytes(LBound(Bytes))), BytesLen

2. Or redefine your CopyMemory API definition and remove the ByVal for the 1st two parameters

Thanks from both of you, it works! I've put VarPtr as you said.
But, I have now another problem with this:

VB Code:

Sub Main()
Dim bytData() As Byte: bytData = ""
CatBinaryString bytData, "aa"
CatBinaryString bytData, "bb"
CatBinaryString bytData, "cc"
TrimBinary bytData
Debug.Print UBound(bytData)
MsgBox StrConv(bytData, vbUnicode)
End Sub

It only displays me "aa" in message box? :confused:

UPDATE: Solved it now (just had to remove BINARY_CHUNK), here's the final code:

VB Code:

Private Declare Function WideCharToMultiByte Lib "Kernel32.dll" ( _
    ByVal CodePage As Long, _
    ByVal dwFlags As Long, _
    ByVal lpWideCharStr As Long, _
    ByVal cchWideChar As Long, _
    ByVal lpMultiByteStr As Long, _
    ByVal cbMultiByte As Long, _
    ByVal lpDefaultChar As Long, _
    ByVal lpUsedDefaultChar As Long _
) As Long
 
Private Const CP_ACP        As Long = 0
 
Private Declare Function CopyMemory Lib "kernel32" Alias "RtlMoveMemory" (ByVal _
        dest As Long, ByVal src As Long, ByVal Length As Long) As Long
 
Private Function ToCPString(ByRef the_sValue As String) As Byte()
 
    Dim abytOutput()                As Byte
    Dim nValueLen                   As Long
    Dim nOutputByteLen              As Long
 
    ' Cache the input length.
    nValueLen = Len(the_sValue)
 
    ' See how big the output buffer will be.
    nOutputByteLen = WideCharToMultiByte(CP_ACP, 0&, StrPtr(the_sValue), nValueLen, 0&, 0&, 0&, 0&)
 
    If nOutputByteLen > 0 Then
        ' Resize output byte array to the size of the UTF-8 string.
        ReDim abytOutput(1 To nOutputByteLen)
 
        ' Make this API call again, this time giving a pointer to the output byte array.
        WideCharToMultiByte CP_ACP, 0&, StrPtr(the_sValue), nValueLen, VarPtr(abytOutput(1)), nOutputByteLen, 0&, 0&
    End If
 
    ' Return the array.
    ToCPString = abytOutput()
 
End Function
 
Private Sub CatBinary(bytData() As Byte, Bytes() As Byte)
    Dim BytesLen As Long, BinaryNext As Long
    
    BinaryNext = UBound(bytData) + 1
    BytesLen = UBound(Bytes) - LBound(Bytes) + 1
    If BinaryNext + BytesLen > BinaryNext Then ReDim Preserve bytData(BinaryNext + BytesLen - 1)
    CopyMemory VarPtr(bytData(BinaryNext)), VarPtr(Bytes(LBound(Bytes))), BytesLen
End Sub
 
Private Sub CatBinaryString(bytData() As Byte, Text As String)
    Dim Bytes() As Byte
    
    Bytes = ToCPString(Text)
    CatBinary bytData, Bytes
End Sub
 
Sub Main()
Dim bytData() As Byte: bytData = ""
CatBinaryString bytData, "aa"
CatBinaryString bytData, "bb"
CatBinaryString bytData, "cc"
MsgBox StrConv(bytData, vbUnicode)
End Sub

I think that it's good now (if someone thinks opposite then correct me).