Results 1 to 6 of 6

Thread: XML JSON UTF-8 problem

  1. #1

    Thread Starter
    Addicted Member Davor Geci's Avatar
    Join Date
    Sep 2009
    Posts
    222

    XML JSON UTF-8 problem

    Hello,
    I'm sending a JSON with an XML to server and sometimes I get an error:
    {"":{"Value":"","Messages":["Invalid character in the given encoding. Line 21, position 47."]}}

    It looks like the message that I'm sending is not in UTF-8 format.

    Here is the code (and a strip down XML) that gives me this error:

    Code:
    Public Sub sendtoserver()
    Dim objHTTP As New MSXML2.ServerXMLHTTP60
    Dim strJSON As String
    Dim Url As String
    Dim strUserName As String
    Dim strPassword As String
    Dim strCompanyId As String
    Dim strSoftwareId As String
    Dim strXML As String
    Dim result As String
    
    
    strXML = ""
    strXML = strXML & "<?xml version='1.0' encoding='utf-8'?>"
    strXML = strXML & "<OutgoingInvoicesData xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance' xmlns:xsd='http://www.w3.org/2001/XMLSchema' xmlns='http://fina.hr/eracun/erp/OutgoingInvoicesData/v3.2'>"
    strXML = strXML & "  <Header>"
    strXML = strXML & "    <SupplierID>435455335</SupplierID>"
    strXML = strXML & "    <InvoiceType>1</InvoiceType>"
    strXML = strXML & "  </Header>"
    strXML = strXML & "  <OutgoingInvoice>"
    strXML = strXML & "    <SupplierInvoiceID>1212-1-1</SupplierInvoiceID>"
    strXML = strXML & "    <BuyerID>455465456</BuyerID>"
    strXML = strXML & "    <InvoiceEnvelope>"
    strXML = strXML & "      <Invoice xmlns:cbc='urn:oasis:names:specification:ubl:schema:xsd:CommonBasicComponents-2' xmlns:sac='urn:oasis:names:specification:ubl:schema:xsd:SignatureAggregateComponents-2' xmlns:ext='urn:oasis:names:specification:ubl:schema:xsd:CommonExtensionComponents-2' xmlns:cac='urn:oasis:names:specification:ubl:schema:xsd:CommonAggregateComponents-2' xmlns:sig='urn:oasis:names:specification:ubl:schema:xsd:CommonSignatureComponents-2' xsi:schemaLocation='urn:oasis:names:specification:ubl:schema:xsd:Invoice-2 HRInvoice.xsd' xmlns='urn:oasis:names:specification:ubl:schema:xsd:Invoice-2'>"
    strXML = strXML & "        <ext:UBLExtensions>"
    strXML = strXML & "          <ext:UBLExtension>"
    strXML = strXML & "            <cbc:ID>HRINVOICE1</cbc:ID>"
    strXML = strXML & "            <cbc:Name>InvoiceIssuePlaceData</cbc:Name>"
    strXML = strXML & "            <ext:ExtensionAgencyID>MINGORP</ext:ExtensionAgencyID>"
    strXML = strXML & "            <ext:ExtensionAgencyName>MINGORP</ext:ExtensionAgencyName>"
    strXML = strXML & "            <ext:ExtensionAgencyURI>MINGORP</ext:ExtensionAgencyURI>"
    strXML = strXML & "            <ext:ExtensionURI>urn:invoice:hr:issueplace</ext:ExtensionURI>"
    strXML = strXML & "            <ext:ExtensionReasonCode>MandatoryField</ext:ExtensionReasonCode>"
    strXML = strXML & "            <ext:ExtensionReason>Mjesto izdavanja raèuna prema Pravilniku o PDV-u</ext:ExtensionReason>"
    strXML = strXML & "            <ext:ExtensionContent>"
    strXML = strXML & "              <ext:InvoiceIssuePlace>MyTown</ext:InvoiceIssuePlace>"
    strXML = strXML & "            </ext:ExtensionContent>"
    strXML = strXML & "          </ext:UBLExtension>"
    strXML = strXML & "        </ext:UBLExtensions>"
    strXML = strXML & "     </Invoice>"
    strXML = strXML & "   </InvoiceEnvelope>"
    strXML = strXML & "  </OutgoingInvoice>"
    strXML = strXML & "</OutgoingInvoicesData>"
    
    
    
        Url = "http://demo.webadress.com/apis/v2/send/"
        strUserName = "ThisIsTheUserName"
        strPassword = "PaSSworD"
        strCompanyId = "33122611302"
        strSoftwareId = "Test-001"
    
    
        strJSON = ""
        strJSON = strJSON & "{" & Chr(34) & "Username" & Chr(34) & ": " & strUserName & ","
        strJSON = strJSON & Chr(34) & "Password" & Chr(34) & ": " & Chr(34) & strPassword & Chr(34) & ","
        strJSON = strJSON & Chr(34) & "CompanyId" & Chr(34) & ": " & Chr(34) & strCompanyId & Chr(34) & ","
        strJSON = strJSON & Chr(34) & "SoftwareId" & Chr(34) & ": " & Chr(34) & strSoftwareId & Chr(34) & ","
        strJSON = strJSON & Chr(34) & "File" & Chr(34) & ": " & Chr(34) & strXML & Chr(34) & "}"
    
    
    
        objHTTP.Open "POST", Url, False
    
       objHTTP.setRequestHeader "Content-type", "application/json"
       objHTTP.send (strJSON)
       result = objHTTP.responseText
    
    
    ExitSub:
        Set objHTTP = Nothing
    
    
    End Sub
    This line in XML triggers this error:
    Code:
    <ext:ExtensionReason>Mjesto izdavanja raèuna prema Pravilniku o PDV-u</ext:ExtensionReason>"
    It contains a special character è

    How could I convert the strJSON to UTF-8?
    My projects:
    Virtual Forms
    VBA Telemetry

  2. #2
    PowerPoster Arnoutdv's Avatar
    Join Date
    Oct 2013
    Posts
    5,872

    Re: XML JSON UTF-8 problem

    A string variable can not contain UTF8

    Code:
    '---------------------------------------------------------------------------------------
    ' Module    : modUTF8.bas
    ' DateTime  : 20-9-2011
    ' Author    : ActiveVB http://www.activevb.de/rubriken/faq/faq0155.html
    ' Purpose   :
    '---------------------------------------------------------------------------------------
    Option Explicit
    
    Private Declare Function WideCharToMultiByte Lib "kernel32.dll" ( _
                             ByVal CodePage As Long, _
                             ByVal dwFlags As Long, _
                             ByVal lpWideCharStr As Long, _
                             ByVal cchWideChar As Long, _
                             ByVal lpMultiByteStr As Long, _
                             ByVal cbMultiByte As Long, _
                             ByVal lpDefaultChar As Long, _
                             ByVal lpUsedDefaultChar As Long) As Long
                             
    Private Declare Function MultiByteToWideChar Lib "kernel32.dll" ( _
                             ByVal CodePage As Long, _
                             ByVal dwFlags As Long, _
                             ByVal lpMultiByteStr As Long, _
                             ByVal cbMultiByte As Long, _
                             ByVal lpWideCharStr As Long, _
                             ByVal cchWideChar As Long) As Long
                             
    Private Const CP_UTF8 As Long = 65001
    
    Public Function ConvertToUTF8(ByRef Source As String) As Byte()
      Dim Length As Long
      Dim Pointer As Long
      Dim Size As Long
      Dim Buffer() As Byte
      
      If Len(Source) > 0 Then
        Length = Len(Source)
        Pointer = StrPtr(Source)
        Size = WideCharToMultiByte(CP_UTF8, 0, Pointer, Length, 0, 0, 0, 0)
        If Size > 0 Then
          ReDim Buffer(0 To Size - 1)
          
          WideCharToMultiByte CP_UTF8, 0, Pointer, Length, VarPtr(Buffer(0)), Size, 0, 0
          ConvertToUTF8 = Buffer
        End If
      End If
      
    End Function
    
    Public Function ConvertFromUTF8(ByRef Source() As Byte) As String
      Dim Size As Long
      Dim Pointer As Long
      Dim Length As Long
      Dim Buffer As String
      
      Size = UBound(Source) - LBound(Source) + 1
      Pointer = VarPtr(Source(LBound(Source)))
        
      If pUTF8header(Source) Then
        ' Ignore BOM header
        ' http://en.wikipedia.org/wiki/Byte_order_mark
        Size = Size - 3
        Pointer = Pointer + 3
      End If
        
      Length = MultiByteToWideChar(CP_UTF8, 0, Pointer, Size, 0, 0)
      Buffer = Space$(Length)
      MultiByteToWideChar CP_UTF8, 0, Pointer, Size, StrPtr(Buffer), Length
      ConvertFromUTF8 = Buffer
    End Function
    
    '---------------------------------------------------------------------------------------
    ' Procedure : pUTF8header
    ' DateTime  : 14-6-2013
    ' Reference : http://en.wikipedia.org/wiki/Byte_order_mark
    '---------------------------------------------------------------------------------------
    Private Function pUTF8header(Source() As Byte) As Boolean
    
      If UBound(Source) >= 2 Then
        If Source(0) = &HEF Then
          If Source(1) = &HBB Then
            If Source(2) = &HBF Then
              pUTF8header = True
              Exit Function
            End If
          End If
        End If
      End If
    
    End Function

  3. #3

    Thread Starter
    Addicted Member Davor Geci's Avatar
    Join Date
    Sep 2009
    Posts
    222

    Re: XML JSON UTF-8 problem

    How to implement this in my code from 1. post?
    Last edited by Davor Geci; Jul 31st, 2019 at 12:07 PM.
    My projects:
    Virtual Forms
    VBA Telemetry

  4. #4
    PowerPoster
    Join Date
    Feb 2006
    Posts
    24,482

    Re: XML JSON UTF-8 problem

    As it says in the Fine Manual:

    send Method (ServerXMLHTTP/IServerXMLHTTPRequest)

    If the input type is a BSTR, the response is always encoded as UTF-8. The caller must set a Content-Type header with the appropriate content type and include a charset parameter.
    Both XML and JSON can handle Unicode in string literals as long as you have escaped their respective reserved characters.

    Then it becomes a question of how standards compliant both ends are. If the server is being as hackish as your client code, fiddling with the raw text directly, all bets are pretty much off.

    You might at least try sending the Content-Type as application/json; charset=utf-8 though, and see whether it helps. In many HTTP clients and servers the default encoding is first 7-bit ASCII, then if that fails the current ANSI codepage is tried. The exact heuristics followed depends on each specific implementation.

    But I can't see how trying to replicate what ServerXMLHTTP.send already does when fed a String is going to make any difference here.

  5. #5
    PowerPoster wqweto's Avatar
    Join Date
    May 2011
    Location
    Sofia, Bulgaria
    Posts
    5,120

    Re: XML JSON UTF-8 problem

    Quote Originally Posted by Davor Geci View Post
    How to implement this in my code from 1. post?
    Try something like this

    Code:
       objHTTP.setRequestHeader "Content-type", "application/json; charset=utf-8" '--- dilettante's idea
       objHTTP.send ConvertToUTF8(strJSON) '--- Arnoutdv's ides
    cheers,
    </wqw>

  6. #6
    New Member
    Join Date
    Nov 2021
    Posts
    1

    Re: XML JSON UTF-8 problem

    This worked like a charm.
    Thanks for good solution.

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Click Here to Expand Forum to Full Width