[RESOLVED] Code with Asc and Chr$ functions works on all but one PC where needs AscW and CHRW

[vbrad]

The encryption code in this stackoverflow answer given by wqw has worked fine for me for a long time.
( https://stackoverflow.com/questions/...using-password ).
It is used on the exact same data on hundreds of computers and has only failed on one.
The computers in question are using everything from winxp to windows 11, 32 bit and 64 bit, home, professional and server editions of windows.

It worked on this single problem pc only after I changed the encryption code to use the Ascw function instead of Asc and and Chrw instead of Chr$. This change was recommended by someone who had users in Asia. This PC is set to English and UK region.

I'm trying to figure out what's different about this pc (it's win 10 pro 64 bit which is about the most common set up I see). I can't find anything in settings that makes it unique.

I'd be grateful for any suggestions

[wqweto]

I'm trying to figure out what's different about this pc (it's win 10 pro 64 bit which is about the most common set up I see). I can't find anything in settings that makes it unique.

This PC is probably using a far-eastern locale where some "ANSI" symbols are encoded in 2 or more bytes. In Windows you have Unicode strings which are UTF-16 encoded i.e. 2 bytes per symbol and MultiByte strings which can have symbols from 1 to 2 bytes (not sure about up to 4).

In European languages the MultiByte symbols are exactly 1 byte so Asc/Chr operate on bytes but if you notice the signature of these functions it's Integer that they use because in far-eastern locales Asc can return 2 byte MultiByte symbols and in the case Asc returns a value above 256 the RC4 cipher implementation (more like abomination) you linked above fails badly.

The simplest solution would be to convert *every* string parameter to a utf-8 encoded byte-array internally so that no matter the current locale and no matter the content of the input text and no matter the content of the password it will work without a flaw. Unfortunately this will make some of the already encoded ciphertexts impossible to decrypt i.e. everything outside ASCII will suffer the transition to utf-8.

cheers,
</wqw>

[vbrad]

This PC is probably using a far-eastern locale where some "ANSI" symbols are encoded in 2 or more bytes. In Windows you have Unicode strings which are UTF-16 encoded i.e. 2 bytes per symbol and MultiByte strings which can have symbols from 1 to 2 bytes (not sure about up to 4).

In European languages the MultiByte symbols are exactly 1 byte so Asc/Chr operate on bytes but if you notice the signature of these functions it's Integer that they use because in far-eastern locales Asc can return 2 byte MultiByte symbols and in the case Asc returns a value above 256 the RC4 cipher implementation (more like abomination) you linked above fails badly.

The simplest solution would be to convert *every* string parameter to a utf-8 encoded byte-array internally so that no matter the current locale and no matter the content of the input text and no matter the content of the password it will work without a flaw. Unfortunately this will make some of the already encoded ciphertexts impossible to decrypt i.e. everything outside ASCII will suffer the transition to utf-8.

cheers,
</wqw>

Thanks very much wqweto!

I was fairly certain it wasn't a locale issue but I checked again:

'Control Panel ->Region-> click "Administrative tab"-> click "change system locale" button '

A window called 'Region Settings' opens
At the bottom of this, there was a ticked, checkbox captioned
"Beta: Use Unicode UTF-8 for worldwide language support".

I'm 99% certain that was the cause especially after a test on a vm here which then reproduced the error and also looking at some of the other machines that I have access to, none of which had this box ticked.

I wonder if this problem I had is a bug in this beta feature or a normal behaviour that will be rolled out to everyone at some point?

[wqweto]

Now that was unexpected turn of events. I'll have to fiddle with this checkbox just to fathom how it affects some of the runtime functions.

The BETA ends with the feature becoming officially optional I guess i.e. yes, some of your users might try to turn it on but most will not.

cheers,
</wqw>

[vbrad]

The BETA ends with the feature becoming officially optional I guess i.e. yes, some of your users might try to turn it on but most will not.
</wqw>

Makes sense, at least I'll have a heads up now!

[dilettante]

Might look at: https://learn.microsoft.com/en-us/wi...utf8-code-page

[Eduardo-]

Might look at: https://learn.microsoft.com/en-us/wi...utf8-code-page

I guess this should need a manifest, it seems that they missed that.

[wqweto]

I guess this should need a manifest, it seems that they missed that.

My understanding is that you need custom manifest to force UTF-8 codepage for you app no matter what global checkbox state is. At the same time with global checkbox ON you activate UTF-8 codepage for all non-manifested apps.

Last time I tried to set <ActiveCodePage>cp1251</ActiveCodePage> (Cyrillic) with custom manifest for my app it was not working at all so probably only UTF-8 is valid value here.

cheers,
</wqw>

[Eduardo-]

At the same time with global checkbox ON you activate UTF-8 codepage for all non-manifested apps.

But that should not be an option, the final user shouldn't be able to set that, because it is totally wrong.
The one that knows if the App can work with UTF-8 or not is the developer, not the final user. Or in any case, each App should be set separately (in the compatibility settings window), but not a setting for all programs.

[wqweto]

But that should not be an option, the final user shouldn't be able to set that, because it is totally wrong.
The one that knows if the App can work with UTF-8 or not is the developer, not the final user. Or in any case, each App should be set separately (in the compatibility settings window), but not a setting for all programs.

The promise is that coming out of BETA the feature will be tested enough and Microsoft will make it work even for non-intended applications. You get UTF-8 support for free in your legacy apps which lived in the ANSI world up until now. Magic!

Btw, there is no point to "implement" UTF-8 support in your Unicode applications -- you gain nothing by such shenanigans. This is clearly aimed at ANSI (VB6) apps only.

cheers,
</wqw>

[VanGoghGaming]

Last time I tried to set <ActiveCodePage>cp1251</ActiveCodePage> (Cyrillic) with custom manifest for my app it was not working at all so probably only UTF-8 is valid value here.

According to this page (https://devblogs.microsoft.com/oldne...1-00/?p=106697) the only accepted settings for "ActiveCodePage" are: "UTF-8" (case sensitive), "Legacy" (case sensitive) or a specific locale name like "en-US" (not case sensitive).

[wqweto]

According to this page (https://devblogs.microsoft.com/oldne...1-00/?p=106697) the only accepted settings for "ActiveCodePage" are: "UTF-8" (case sensitive), "Legacy" (case sensitive) or a specific locale name like "en-US" (not case sensitive).

I tried cp1251, Windows-1251, bg_BG, bg-BG and every other combination I could fathom and it was *not* working as expected. I wanted my Cyrillic labels and buttons in a VB6 application to appear correct on OSes with English locale for non-unicode applications -- nothing special, just emulate that I had bg_BG in "locale for non-unicode applications" set for my application only but got nothing. Not even an event log entry something's wrong with my manifest.

cheers,
</wqw>

[VanGoghGaming]

Yeah just tested this "activeCodePage" setting with locale names and it doesn't seem to make any difference at all. The only setting that seems to make a difference is "UTF-8" and even that is flawed (guess that's why it's still in BETA):

Program running in IDE (textbox, command button and label at the bottom all showing question marks):



Compiled program with manifest (textbox and command button work fine, label at the bottom not so much but at least it tried to display something other than question marks):

[wqweto]

This looks promising actually.

Is it that setting Label's caption at run-time not working on second right picture?

cheers,
</wqw>

[VanGoghGaming]

Yeah the caption is being set dynamically to whatever is selected in the RichEdit control:

Private Sub richTest1_SelectionChange(SelText As String)
    cmdTest.Caption = SelText
    lblTest = SelText
End Sub

It's worth mentioning that even the command button is not working correctly all the time. Only the Cyrillic text is displayed correctly. The Chinese text is missing some of the characters (not many but still). The only control that is "spot-on" after implementing the manifest is the textbox.

[wqweto]

The Label is using Unicode API (i.e. DrawTextW) to display the text which is UTF-8 encoded ANSI and fails badly. If it were using DrawTextA it would actually succeed.

cheers,
</wqw>

[dilettante]

I played with this when it was new, but for VB6 programs it is pretty erratic because of the sort of things already described.

[VanGoghGaming]

The Label is using Unicode API (i.e. DrawTextW) to display the text which is UTF-8 encoded ANSI and fails badly. If it were using DrawTextA it would actually succeed.

It turns out it was "TextOutA" instead! I've put together a small "cCapW" class that can hook this obsolete "TextOutA" and replace it with a custom function that ends up calling "TextOutW" and so the label caption is rendered correctly:

Unicode Captions for all intrinsic controls