Results 1 to 7 of 7

Thread: Base64 decoded string is different with different regional settings!

  1. #1

    Thread Starter
    Addicted Member
    Join Date
    Sep 2015
    Posts
    225

    Base64 decoded string is different with different regional settings!

    The following Base64 encoded string gives different decoded string when changing regional settings Format from (Arabic Egypt) to (English United States)
    Code:
    AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAPABUAIwAkADEANwBAAEMAQwA7ADsALgAtACMAIQAaABkAGQAbABYAFgAaAB0AHgAmACQALgA4AEMASQBUAFkAXwBrAHsAewCBAIgAnwCpAL4A1ADgAOMA6gD7AP8A9wDwAOsA2wDKAMgAzwC2AK8ApwCcAI0AiAB9AHcAYwBbAE8ASQBMAEoARQBFAEgASgBIAEwATABXAFYAVQBUAFEAVABWAFYAXABWAFcAVgBaAFcAWwBgAF8AWgBVAFQAXABkAGMAXABcAGMAaABoAG0AaQBlAGYAYQBeAFwAXABeAFwAYABfAFoAXgBbAFcAWQBUAEoASgBJAEkASQBJAEYAPQA+AEMAOwAyADEALgAqACYAJAAfABoAGAAZABYAEQARABEADwAOAA4ACwALAAoACgAKAAsACQAHAAcABwAFAAUABQACAAEAAQABAAAAAAAAAAAAAAAAAAEAAQABAAEAAQABAAEAAQABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
    I used many Base64 Encode/Decode modules, classes and even Chilkat libraries and all gives different results when changing regional settings!
    Any ideas?

  2. #2
    PowerPoster
    Join Date
    Jun 2013
    Posts
    7,219

    Re: Base64 decoded string is different with different regional settings!

    Quote Originally Posted by labmany View Post
    The following Base64 encoded string gives different decoded string when changing regional settings Format from (Arabic Egypt) to (English United States)
    Code:
    AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAPABUAIwAkADEANwBAAEMAQwA7ADsALgAtACMAIQAaABkAGQAbABYAFgAaAB0AHgAmACQALgA4AEMASQBUAFkAXwBrAHsAewCBAIgAnwCpAL4A1ADgAOMA6gD7AP8A9wDwAOsA2wDKAMgAzwC2AK8ApwCcAI0AiAB9AHcAYwBbAE8ASQBMAEoARQBFAEgASgBIAEwATABXAFYAVQBUAFEAVABWAFYAXABWAFcAVgBaAFcAWwBgAF8AWgBVAFQAXABkAGMAXABcAGMAaABoAG0AaQBlAGYAYQBeAFwAXABeAFwAYABfAFoAXgBbAFcAWQBUAEoASgBJAEkASQBJAEYAPQA+AEMAOwAyADEALgAqACYAJAAfABoAGAAZABYAEQARABEADwAOAA4ACwALAAoACgAKAAsACQAHAAcABwAFAAUABQACAAEAAQABAAAAAAAAAAAAAAAAAAEAAQABAAEAAQABAAEAAQABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
    I used many Base64 Encode/Decode modules, classes and even Chilkat libraries and all gives different results when changing regional settings!
    Any ideas?
    Since a Base64-String "came to live" from "raw-bytes" (as e.g. the raw FileContent-bytearrays of a JPG -
    or even a TextFile), your default decoding-output of such a B64-string should be a ByteArray (in a first stage) as well...

    Any (valid) B64-String-Input will give always the same ByteArray-Content on any machine on any system.

    So, now you're sitting there with a decoded ByteArray and don't know how to convert it further...

    Solution: Ask the provider of the B64-String, what his ByteArray-content was originally based on.

    If it came from an "ANSI-TextFile" - ask him for the LCID of the system this file came from
    (and use this LCID in the Optional 3rd-Param of StrConv(...), when decoding the ByteArray into a String in a second stage)

    If it came from an "UTF8-TextFile" - then use an "UTF8BytesToVBString"-converter-routine,
    to decode your bytes.

    If your Bytes came from a JPG-File, then use a "ByteArrayToImage-converter", etc...

    So, Base64 is to encode "raw ByteArrays" (and the decoding will produce exactly that again).
    If you think about B64 in that way ("it's just another form to express a ByteArray"),
    then the question about the "format of these bytes" prior to encoding them, becomes automatic.

    If you are the originator of the encoding,
    then make sure to transform your text-strings into such a ByteArray with very explicit "transform instructions" -
    e.g. transforming Strings to UTF8-ByteArrays (prior to a B64-encoding) is common these days.

    HTH

    Olaf
    Last edited by Schmidt; Nov 23rd, 2022 at 10:42 PM.

  3. #3

    Thread Starter
    Addicted Member
    Join Date
    Sep 2015
    Posts
    225

    Re: Base64 decoded string is different with different regional settings!

    Quote Originally Posted by Schmidt View Post
    Since a Base64-String "came to live" from "raw-bytes" (as e.g. the raw FileContent-bytearrays of a JPG -
    or even a TextFile), your default decoding-output of such a B64-string should be a ByteArray (in a first stage) as well...

    Any (valid) B64-String-Input will give always the same ByteArray-Content on any machine on any system.

    So, now you're sitting there with a decoded ByteArray and don't know how to convert it further...

    Solution: Ask the provider of the B64-String, what his ByteArray-content was originally based on.

    If it came from an "ANSI-TextFile" - ask him for the LCID of the system this file came from
    (and use this LCID in the Optional 3rd-Param of StrConv(...), when decoding the ByteArray into a String in a second stage)

    If it came from an "UTF8-TextFile" - then use an "UTF8BytesToVBString"-converter-routine,
    to decode your bytes.

    If your Bytes came from a JPG-File, then use a "ByteArrayToImage-converter", etc...

    So, Base64 is to encode "raw ByteArrays" (and the decoding will produce exactly that again).
    If you think about B64 in that way ("it's just another form to express a ByteArray"),
    then the question about the "format of these bytes" prior to encoding them, becomes automatic.

    If you are the originator of the encoding,
    then make sure to transform your text-strings into such a ByteArray with very explicit "transform instructions" -
    e.g. transforming Strings to UTF8-ByteArrays (prior to a B64-encoding) is common these days.

    HTH

    Olaf
    Thanks Olaf, this is helpful.

  4. #4
    PowerPoster wqweto's Avatar
    Join Date
    May 2011
    Location
    Sofia, Bulgaria
    Posts
    5,120

    Re: Base64 decoded string is different with different regional settings!

    Quote Originally Posted by Schmidt View Post
    Any (valid) B64-String-Input will give always the same ByteArray-Content on any machine on any system.
    Funny fact: it's possible two different B64-String-Input to produce the same ByteArray-Content.

    That's why Base64 input cannot be treated as canonical encoding esp. if produced by malicious party.

    cheers,
    </wqw>

  5. #5
    Frenzied Member
    Join Date
    Dec 2012
    Posts
    1,470

    Re: Base64 decoded string is different with different regional settings!

    I used the Base64 string you provided to create a binary file called Test.b64 and verified its contents. Then I ran the following command line:
    certutil -decode test.b64 test.bin

    Then I changed the country code to Egypt and ran this command:
    certutil -decode test.b64 test.egy

    Then I ran a file compare:
    fc test.bin test.egy
    Comparing files test.bin and TEST.EGY
    FC: no differences encountered

    What this tells me is that the problem exists in your conversion routine.

    J.A. Coutts

  6. #6
    PowerPoster
    Join Date
    Feb 2006
    Posts
    24,482

    Re: Base64 decoded string is different with different regional settings!

    Seems to work fine here.

    Code:
    Option Explicit
    
    'Text1: Font = Consolas 9, MultiLine = True, ScrollBars = Both
    
    Private Const CRYPT_STRING_HEXASCIIADDR = 11
    
    Private Declare Function CryptBinaryToString Lib "Crypt32" _
        Alias "CryptBinaryToStringW" ( _
        ByRef Binary As Byte, _
        ByVal cbBinary As Long, _
        ByVal dwFlags As Long, _
        ByVal pszString As Long, _
        ByRef cchString As Long) As Long
    
    Private Const CRYPT_STRING_BASE64 = 1
    
    Private Declare Function CryptStringToBinary Lib "Crypt32" _
        Alias "CryptStringToBinaryW" ( _
        ByVal pszString As Long, _
        ByVal cchString As Long, _
        ByVal dwFlags As Long, _
        ByRef bBinary As Byte, _
        ByRef cbBinary As Long, _
        ByRef dwSkip As Long, _
        ByRef dwFlagsActual As Long) As Long
    
    Private Sub Form_Load()
        Const BASE64_TEXT As String = _
            "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAPABUAIwAkADEANwBAAEMA" _
          & "QwA7ADsALgAtACMAIQAaABkAGQAbABYAFgAaAB0AHgAmACQALgA4AEMASQBUAFkAXwBrAHsA" _
          & "ewCBAIgAnwCpAL4A1ADgAOMA6gD7AP8A9wDwAOsA2wDKAMgAzwC2AK8ApwCcAI0AiAB9AHcA" _
          & "YwBbAE8ASQBMAEoARQBFAEgASgBIAEwATABXAFYAVQBUAFEAVABWAFYAXABWAFcAVgBaAFcA" _
          & "WwBgAF8AWgBVAFQAXABkAGMAXABcAGMAaABoAG0AaQBlAGYAYQBeAFwAXABeAFwAYABfAFoA" _
          & "XgBbAFcAWQBUAEoASgBJAEkASQBJAEYAPQA+AEMAOwAyADEALgAqACYAJAAfABoAGAAZABYA" _
          & "EQARABEADwAOAA4ACwALAAoACgAKAAsACQAHAAcABwAFAAUABQACAAEAAQABAAAAAAAAAAAA" _
          & "AAAAAAEAAQABAAEAAQABAAEAAQABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA" _
          & "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA" _
          & "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA="
        Dim DecodedCount As Long
        Dim dwFlagsActual As Long
        Dim Decoded() As Byte
        Dim DumpedCount As Long
        Dim Dumped As String
    
        'Convert Base64 to binary.
        CryptStringToBinary StrPtr(BASE64_TEXT), _
                            Len(BASE64_TEXT), _
                            CRYPT_STRING_BASE64, _
                            ByVal 0&, _
                            DecodedCount, _
                            0, _
                            dwFlagsActual
        ReDim Decoded(DecodedCount - 1)
        If CryptStringToBinary(StrPtr(BASE64_TEXT), _
                               Len(BASE64_TEXT), _
                               CRYPT_STRING_BASE64, _
                               Decoded(0), _
                               DecodedCount, _
                               0, _
                               dwFlagsActual) <> 0 Then
            
            'Dump the binary output.
            CryptBinaryToString Decoded(0), _
                                DecodedCount, _
                                CRYPT_STRING_HEXASCIIADDR, _
                                0&, _
                                DumpedCount
            Dumped = String$(DumpedCount - 1, 0)
            If CryptBinaryToString(Decoded(0), _
                                   DecodedCount, _
                                   CRYPT_STRING_HEXASCIIADDR, _
                                   StrPtr(Dumped), _
                                   DumpedCount) <> 0 Then
                Text1.Text = Replace$(Dumped, vbTab, ": ")
            Else
                MsgBox "CryptBinaryToString Error " & CStr(Err.LastDllError)
            End If
        Else
            MsgBox "CryptStringToBinary Error " & CStr(Err.LastDllError)
        End If
    End Sub
    
    Private Sub Form_Resize()
        If WindowState <> vbMinimized Then
            Text1.Move 0, 0, ScaleWidth, ScaleHeight
        End If
    End Sub
    However the contents look very sketchy. The encoded data appears to be garbage Unicode text for some reason. That's pretty bizarre and hints of serious logical flaws and bad assumptions in the rest of the program.

  7. #7

    Thread Starter
    Addicted Member
    Join Date
    Sep 2015
    Posts
    225

    Re: Base64 decoded string is different with different regional settings!

    Quote Originally Posted by couttsj View Post
    I used the Base64 string you provided to create a binary file called Test.b64 and verified its contents. Then I ran the following command line:
    certutil -decode test.b64 test.bin

    Then I changed the country code to Egypt and ran this command:
    certutil -decode test.b64 test.egy

    Then I ran a file compare:
    fc test.bin test.egy
    Comparing files test.bin and TEST.EGY
    FC: no differences encountered

    What this tells me is that the problem exists in your conversion routine.

    J.A. Coutts
    Well, I used a byte array instead of parsing the returned decoded string one character at a time and it worked ok whatever the regional settings is.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Click Here to Expand Forum to Full Width