[2005] WriteAllText and the Byte Order Mark

I ran across this little quirk recently and thought I'd let everyone in on it so you won't get burned by it.

The Byte Order Mark is a set of characters at the beginning of a Unicode text file that denote how the file is encoded. For UTF-8, the common default in .NET, the characters are the byte sequence EF BB BF, which appears as the characters "".

Here's the quirk. There are three common ways to write out a small text file in VB.NET 2005: My.Computer.FileSystem.WriteAllText, System.IO.File.WriteAllText, and StreamWriter. All three use UTF-8 encoding by default, however only My.Computer.FileSystem.WriteAllText includes the Byte Order Mark, the others don't.

So, if you use My.Computer.FileSystem.WriteAllText and don't specify encoding you will write out a text file that includes the Byte Order Mark. This can confuse other programs that read your file since the byte sequence is probably unexpected. To make matters worse, if you load a file created this way into Notepad or Wordpad it will look perfectly fine to you because they read, but don't display, the header bytes. If you don't know what's going on this can lead to some frustration. :eek2:

So, my recommendation is to always specify the right encoding method no matter which method you use.

I hope this helps anyone who might encounter this situation.