-
Nov 23rd, 2022, 09:19 PM
#1
Thread Starter
Addicted Member
Base64 decoded string is different with different regional settings!
The following Base64 encoded string gives different decoded string when changing regional settings Format from (Arabic Egypt) to (English United States)
Code:
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAPABUAIwAkADEANwBAAEMAQwA7ADsALgAtACMAIQAaABkAGQAbABYAFgAaAB0AHgAmACQALgA4AEMASQBUAFkAXwBrAHsAewCBAIgAnwCpAL4A1ADgAOMA6gD7AP8A9wDwAOsA2wDKAMgAzwC2AK8ApwCcAI0AiAB9AHcAYwBbAE8ASQBMAEoARQBFAEgASgBIAEwATABXAFYAVQBUAFEAVABWAFYAXABWAFcAVgBaAFcAWwBgAF8AWgBVAFQAXABkAGMAXABcAGMAaABoAG0AaQBlAGYAYQBeAFwAXABeAFwAYABfAFoAXgBbAFcAWQBUAEoASgBJAEkASQBJAEYAPQA+AEMAOwAyADEALgAqACYAJAAfABoAGAAZABYAEQARABEADwAOAA4ACwALAAoACgAKAAsACQAHAAcABwAFAAUABQACAAEAAQABAAAAAAAAAAAAAAAAAAEAAQABAAEAAQABAAEAAQABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
I used many Base64 Encode/Decode modules, classes and even Chilkat libraries and all gives different results when changing regional settings!
Any ideas?
-
Nov 23rd, 2022, 10:38 PM
#2
Re: Base64 decoded string is different with different regional settings!
Originally Posted by labmany
The following Base64 encoded string gives different decoded string when changing regional settings Format from (Arabic Egypt) to (English United States)
Code:
AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAPABUAIwAkADEANwBAAEMAQwA7ADsALgAtACMAIQAaABkAGQAbABYAFgAaAB0AHgAmACQALgA4AEMASQBUAFkAXwBrAHsAewCBAIgAnwCpAL4A1ADgAOMA6gD7AP8A9wDwAOsA2wDKAMgAzwC2AK8ApwCcAI0AiAB9AHcAYwBbAE8ASQBMAEoARQBFAEgASgBIAEwATABXAFYAVQBUAFEAVABWAFYAXABWAFcAVgBaAFcAWwBgAF8AWgBVAFQAXABkAGMAXABcAGMAaABoAG0AaQBlAGYAYQBeAFwAXABeAFwAYABfAFoAXgBbAFcAWQBUAEoASgBJAEkASQBJAEYAPQA+AEMAOwAyADEALgAqACYAJAAfABoAGAAZABYAEQARABEADwAOAA4ACwALAAoACgAKAAsACQAHAAcABwAFAAUABQACAAEAAQABAAAAAAAAAAAAAAAAAAEAAQABAAEAAQABAAEAAQABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
I used many Base64 Encode/Decode modules, classes and even Chilkat libraries and all gives different results when changing regional settings!
Any ideas?
Since a Base64-String "came to live" from "raw-bytes" (as e.g. the raw FileContent-bytearrays of a JPG -
or even a TextFile), your default decoding-output of such a B64-string should be a ByteArray (in a first stage) as well...
Any (valid) B64-String-Input will give always the same ByteArray-Content on any machine on any system.
So, now you're sitting there with a decoded ByteArray and don't know how to convert it further...
Solution: Ask the provider of the B64-String, what his ByteArray-content was originally based on.
If it came from an "ANSI-TextFile" - ask him for the LCID of the system this file came from
(and use this LCID in the Optional 3rd-Param of StrConv(...), when decoding the ByteArray into a String in a second stage)
If it came from an "UTF8-TextFile" - then use an "UTF8BytesToVBString"-converter-routine,
to decode your bytes.
If your Bytes came from a JPG-File, then use a "ByteArrayToImage-converter", etc...
So, Base64 is to encode "raw ByteArrays" (and the decoding will produce exactly that again).
If you think about B64 in that way ("it's just another form to express a ByteArray"),
then the question about the "format of these bytes" prior to encoding them, becomes automatic.
If you are the originator of the encoding,
then make sure to transform your text-strings into such a ByteArray with very explicit "transform instructions" -
e.g. transforming Strings to UTF8-ByteArrays (prior to a B64-encoding) is common these days.
HTH
Olaf
Last edited by Schmidt; Nov 23rd, 2022 at 10:42 PM.
-
Nov 23rd, 2022, 11:14 PM
#3
Thread Starter
Addicted Member
Re: Base64 decoded string is different with different regional settings!
Originally Posted by Schmidt
Since a Base64-String "came to live" from "raw-bytes" (as e.g. the raw FileContent-bytearrays of a JPG -
or even a TextFile), your default decoding-output of such a B64-string should be a ByteArray (in a first stage) as well...
Any (valid) B64-String-Input will give always the same ByteArray-Content on any machine on any system.
So, now you're sitting there with a decoded ByteArray and don't know how to convert it further...
Solution: Ask the provider of the B64-String, what his ByteArray-content was originally based on.
If it came from an "ANSI-TextFile" - ask him for the LCID of the system this file came from
(and use this LCID in the Optional 3rd-Param of StrConv(...), when decoding the ByteArray into a String in a second stage)
If it came from an "UTF8-TextFile" - then use an "UTF8BytesToVBString"-converter-routine,
to decode your bytes.
If your Bytes came from a JPG-File, then use a "ByteArrayToImage-converter", etc...
So, Base64 is to encode "raw ByteArrays" (and the decoding will produce exactly that again).
If you think about B64 in that way ("it's just another form to express a ByteArray"),
then the question about the "format of these bytes" prior to encoding them, becomes automatic.
If you are the originator of the encoding,
then make sure to transform your text-strings into such a ByteArray with very explicit "transform instructions" -
e.g. transforming Strings to UTF8-ByteArrays (prior to a B64- encoding) is common these days.
HTH
Olaf
Thanks Olaf, this is helpful.
-
Nov 24th, 2022, 12:59 AM
#4
Re: Base64 decoded string is different with different regional settings!
Originally Posted by Schmidt
Any (valid) B64-String-Input will give always the same ByteArray-Content on any machine on any system.
Funny fact: it's possible two different B64-String-Input to produce the same ByteArray-Content.
That's why Base64 input cannot be treated as canonical encoding esp. if produced by malicious party.
cheers,
</wqw>
-
Nov 24th, 2022, 11:22 AM
#5
Re: Base64 decoded string is different with different regional settings!
I used the Base64 string you provided to create a binary file called Test.b64 and verified its contents. Then I ran the following command line:
certutil -decode test.b64 test.bin
Then I changed the country code to Egypt and ran this command:
certutil -decode test.b64 test.egy
Then I ran a file compare:
fc test.bin test.egy
Comparing files test.bin and TEST.EGY
FC: no differences encountered
What this tells me is that the problem exists in your conversion routine.
J.A. Coutts
-
Nov 24th, 2022, 03:27 PM
#6
Re: Base64 decoded string is different with different regional settings!
Seems to work fine here.
Code:
Option Explicit
'Text1: Font = Consolas 9, MultiLine = True, ScrollBars = Both
Private Const CRYPT_STRING_HEXASCIIADDR = 11
Private Declare Function CryptBinaryToString Lib "Crypt32" _
Alias "CryptBinaryToStringW" ( _
ByRef Binary As Byte, _
ByVal cbBinary As Long, _
ByVal dwFlags As Long, _
ByVal pszString As Long, _
ByRef cchString As Long) As Long
Private Const CRYPT_STRING_BASE64 = 1
Private Declare Function CryptStringToBinary Lib "Crypt32" _
Alias "CryptStringToBinaryW" ( _
ByVal pszString As Long, _
ByVal cchString As Long, _
ByVal dwFlags As Long, _
ByRef bBinary As Byte, _
ByRef cbBinary As Long, _
ByRef dwSkip As Long, _
ByRef dwFlagsActual As Long) As Long
Private Sub Form_Load()
Const BASE64_TEXT As String = _
"AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABAAPABUAIwAkADEANwBAAEMA" _
& "QwA7ADsALgAtACMAIQAaABkAGQAbABYAFgAaAB0AHgAmACQALgA4AEMASQBUAFkAXwBrAHsA" _
& "ewCBAIgAnwCpAL4A1ADgAOMA6gD7AP8A9wDwAOsA2wDKAMgAzwC2AK8ApwCcAI0AiAB9AHcA" _
& "YwBbAE8ASQBMAEoARQBFAEgASgBIAEwATABXAFYAVQBUAFEAVABWAFYAXABWAFcAVgBaAFcA" _
& "WwBgAF8AWgBVAFQAXABkAGMAXABcAGMAaABoAG0AaQBlAGYAYQBeAFwAXABeAFwAYABfAFoA" _
& "XgBbAFcAWQBUAEoASgBJAEkASQBJAEYAPQA+AEMAOwAyADEALgAqACYAJAAfABoAGAAZABYA" _
& "EQARABEADwAOAA4ACwALAAoACgAKAAsACQAHAAcABwAFAAUABQACAAEAAQABAAAAAAAAAAAA" _
& "AAAAAAEAAQABAAEAAQABAAEAAQABAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA" _
& "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA" _
& "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA="
Dim DecodedCount As Long
Dim dwFlagsActual As Long
Dim Decoded() As Byte
Dim DumpedCount As Long
Dim Dumped As String
'Convert Base64 to binary.
CryptStringToBinary StrPtr(BASE64_TEXT), _
Len(BASE64_TEXT), _
CRYPT_STRING_BASE64, _
ByVal 0&, _
DecodedCount, _
0, _
dwFlagsActual
ReDim Decoded(DecodedCount - 1)
If CryptStringToBinary(StrPtr(BASE64_TEXT), _
Len(BASE64_TEXT), _
CRYPT_STRING_BASE64, _
Decoded(0), _
DecodedCount, _
0, _
dwFlagsActual) <> 0 Then
'Dump the binary output.
CryptBinaryToString Decoded(0), _
DecodedCount, _
CRYPT_STRING_HEXASCIIADDR, _
0&, _
DumpedCount
Dumped = String$(DumpedCount - 1, 0)
If CryptBinaryToString(Decoded(0), _
DecodedCount, _
CRYPT_STRING_HEXASCIIADDR, _
StrPtr(Dumped), _
DumpedCount) <> 0 Then
Text1.Text = Replace$(Dumped, vbTab, ": ")
Else
MsgBox "CryptBinaryToString Error " & CStr(Err.LastDllError)
End If
Else
MsgBox "CryptStringToBinary Error " & CStr(Err.LastDllError)
End If
End Sub
Private Sub Form_Resize()
If WindowState <> vbMinimized Then
Text1.Move 0, 0, ScaleWidth, ScaleHeight
End If
End Sub
However the contents look very sketchy. The encoded data appears to be garbage Unicode text for some reason. That's pretty bizarre and hints of serious logical flaws and bad assumptions in the rest of the program.
-
Nov 25th, 2022, 07:36 AM
#7
Thread Starter
Addicted Member
Re: Base64 decoded string is different with different regional settings!
Originally Posted by couttsj
I used the Base64 string you provided to create a binary file called Test.b64 and verified its contents. Then I ran the following command line:
certutil -decode test.b64 test.bin
Then I changed the country code to Egypt and ran this command:
certutil -decode test.b64 test.egy
Then I ran a file compare:
fc test.bin test.egy
Comparing files test.bin and TEST.EGY
FC: no differences encountered
What this tells me is that the problem exists in your conversion routine.
J.A. Coutts
Well, I used a byte array instead of parsing the returned decoded string one character at a time and it worked ok whatever the regional settings is.
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
|
Click Here to Expand Forum to Full Width
|