Xor Cipher - Revisited

[dilettante]

Beginners are often attracted to the simple XOR cipher. Besides having serious limitations as ciphers go, they usually try to approach it ignoring many important topics.

The first one is that most ciphers don't really apply to text at all. Sure, there are some that do but these are mostly obsolete and of little but historical interest. So when using most ciphers today, including simple Xor, you want to combine the plaintext and the key as arrays of bytes:

Private Function XorCipher(ByRef Bytes() As Byte, ByRef PassBytes() As Byte) As Byte()
    Dim Temp() As Byte
    Dim Length As Long
    Dim I As Long

    ReDim Temp(UBound(Bytes))
    'Assumes LBound() = 0 for both arrays passed to us:
    Length = UBound(PassBytes) + 1
    For I = 0 To UBound(Bytes)
        Temp(I) = Bytes(I) Xor PassBytes(I Mod Length)
    Next I
    XorCipher = Temp
End Function

But how to get those bytes?

Typically people start off with String values for both "inputs" and to get them as Byte arrays they'll either just slop them in and get lots of zero bytes or much worse they'll convert the String to ANSI.

Converting to ANSI is ok as long as you enforce the use of the 7-bit ASCII subset. But if you don't do that you can run into serious portability problems. ANSI is not ANSI, but a family of ANSI encodings that differ for different language alphabets. And you can also run into Double Byte Character Set issues in many Asian alphabets.

See the classic:

The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!)

Or perhaps the newer take on the subject:

What every programmer absolutely, positively needs to know about encodings and character sets to work with text


Text in VB6

First you need some grasp of text in VB6. String values in VB6 are normally 16-bit Windows Unicode UTF-16LE characters. Yes, a String variable can hold other things such as ASCII, ANSI, DBCS, even UTF-8 and UTF-7. However this is rarely done and you'd want to a good handle on the subject before you attempt it.

So one can almost say VB6 Strings are always Unicode (UTF-16LE).

This can get confusing because to help bridge the gap from MS-DOS and earlier versions of VB, VB6 "helps" you. Most text I/O operations and many controls do implicit conversion to and from ANSI using the current session codepage. This means that in many ways VB6 is "Unicode on the inside and ANSI on the outside."

People also get confused because 7-bit US-ASCII is a proper subset of every ANSI encoding as well. Then the somewhat misleading notion of "extended 8-bit ASCII" came along in the MS-DOS era.

Once again see the links above for some conceptual help.


Demo

So here is a demo that attempts to address some of these things.

It uses the Text Object Model (TOM) interface of the Win32 RichEdit control which VB6's RichTextBox control wraps to get us some Unicode capabilities. This should work on Windows 2000 or later.

A small helper class RtbTom.cls wraps an API call to grab ITextDocument references for our RichTextBox controls.

It converts Unicode text to UTF-8 text in Byte arrays to operate upon them. UTF-8 is a multibyte character set, though be careful. The 7-bit ASCII subset uses one byte per character so people often get mislead into thinking things are that simple. Other characters get encoded as 2 to 4 bytes instead of just one. Still, at least with UTF-8 you can compress out most of the zero bytes you'd have using "Unicode" (UTF-16LE).

After applying the XorCipher() function above, it encodes the ciphertext bytes in Base64. Base64 is relatively compact compared to hex, etc. and it uses only "safe" characters - safe for ASCII, ANSI, almost anything.

We don't have much character encoding support in VB6, so the helper class TextCodec.cls wraps a few API calls to accomplish this.

Then the cipher is reversed by converting the Base64 back to bytes, applying XorCipher() once more using the same passphrase, and finally converts the resulting UTF-8 back to "Unicode."

When the program starts it reads the data from PlainTextIn.txt into rtbPlainText1 (a RichTextBox). The file is Unicode, so we bring in lots of characters from various alphabets. You can type over this or leave it as-is.

When you type a passphrase into rtbPass it enables cmdEncrypt allowing you to click on it.

Clicking that does the encryption and displays the results in rtbCipherText and also saves it to disk as a 7-bit ASCII file Encrypted.txt.

Then it enables cmdDecrypt.

Clicking on that reverses the encryption process. Results are written into rtbPlainText2 and also saves it to disk as a Unicode file PlainTextOut.txt.


Requirements

VB6, Windows 2000 or later.


Summary

Xor Cipher is simple though limited, but using it effectively can be trickier than new programmers expect.

In order to avoid corrupting your data in the process you need to be aware of character encodings and make some choices about which ones to use and how to use them.

We could have skipped encoding as UTF-8 but we'd produce a far larger "encrypted" ciphertext in most cases, with a lot of "dead air" zero bytes. Don't rely on ANSI conversion though, it can be a trap.

The ciphertext itself is not "text" as such, but binary data. If you need to represent this as actual text you should encode it using Base64, hex, or something similar. Web and email do tons of this very thing under the covers and most people aren't aware of it.

There is a lot here that can be intimidating and confusing, but these are good concepts to understand.

[Elroy]

Hi Dilettante,

Since you're being rather precise here, it's worth pointing out that the VB6 internal (memory) handling of strings (treated as text) is actually the UCS-2 subset of the UTF-16 character-set (with LE (little endian) or BE (big endian) being yet another separate issue). The full UTF-16 (LE or BE) is capable of handling characters that are sometimes longer than two bytes.

There is much mis-information on the interwebbies about this, such as this page which is wrong. However, Wikipedia has it correct. As stated by Wikipedia, the full UTF-16 characterset (which is not natively supported by VB6) can support 1,112,064 possible characters, which is far more than a maximum allowed by two bytes.

Best Regards,
Elroy

EDIT1: Just as some more FYI, the mono-languages support the full UTF-16 specification. Also, Microsoft, to some degree has attempted to co-op the word "Unicode" and use it in this UCS-2 subset of UTF-16. However, originally, "Unicode" referred to all the attempts to encode extended character-sets beyond the original one-byte ASCII/ANSI standard.

[dilettante]

Microsoft hasn't co-opted anything. Unicode is a separate concept from any specific representation in an encoding. I'm pretty sure the author of the second linked article above attempts to make that point.

The "supplementary planes" that drive UTF-16 encodings to 32 bits are seldom used but I agree it is useful to be aware of them. However you will have to dig deep to find anything in the Windows SDK that refers to UCS-2 rather than using the term UTF-16 instead. So if you want to split hairs you are correct, but it won't get you too far in accomplishing much.

This does mean it can be clumsy to work with characters like that in VB6 because ChrW$() and friends are limited to 16-bit values. But it is a rare program that needs to. Clumping around pretending ANSI encodings are ASCII is far more hazardous. So-called "extended ASCII" is really just the old OEM-US (codepage 437) from the MS-DOS era and it doesn't even use the same character set as Windows Western ANSI (codepage 1252).


However all I was trying to illustrate is how people often go wrong trying to write simple "Xor encryption" programs. They get in just as much trouble trying to double-convert hoping to squeeze through implicit encoding conversions in VB6.

[Elroy]

Hi Dilettante,

Yep, I think it's excellent that you pointed out some of the potential pitfalls of XOR encoding with VB6 string data. And, as pointed out above, full use of UTF-16 isn't entirely dead. There are still many implementations of mono being used (which does implement the full UTF-16). One rather active and publicly used example is the little C-based Linden Scripting Language used in their online Second-Life platform.

And the reason I suggest that Microsoft is possibly co-opting the word Unicode is because of statements like the following:

It converts Unicode text to UTF-8

I believe those are your words, but its not difficult to find similar statements on Microsoft pages. That sentence is saying that we converted "some color to red". What if it was already red to begin with? IMHO, it's just Microsoft confusing things, as they often do.

All The Best,
Elroy

[DEXWERX]

More food for thought. https://msdn.microsoft.com/en-us/lib...or=-2147217396