Results 1 to 23 of 23

Thread: how to encode output to text file

  1. #1

    Thread Starter
    Member
    Join Date
    Jun 2012
    Posts
    43

    how to encode output to text file

    I have a string that contains accented characters (read from a database varchar field). I want to write this to a text file but when I do this, the accented characters come out as Unicode 2-byte characters. I don't want this. How do I get it to write the characterss the same way they are stored in the database (single-byte)? using encode.ascii does not work.

  2. #2
    Angel of Code Niya's Avatar
    Join Date
    Nov 2011
    Posts
    8,598

    Re: how to encode output to text file

    Try UTF8 instead:-
    vbnet Code:
    1. '
    2.         Dim myTest As String = "Bad Dog"
    3.  
    4.         System.IO.File.WriteAllText("c:\MyText.txt", myTest, System.Text.Encoding.UTF8)
    Treeview with NodeAdded/NodesRemoved events | BlinkLabel control | Calculate Permutations | Object Enums | ComboBox with centered items | .Net Internals article(not mine) | Wizard Control | Understanding Multi-Threading | Simple file compression | Demon Arena

    Copy/move files using Windows Shell | I'm not wanted

    C++ programmers will dismiss you as a cretinous simpleton for your inability to keep track of pointers chained 6 levels deep and Java programmers will pillory you for buying into the evils of Microsoft. Meanwhile C# programmers will get paid just a little bit more than you for writing exactly the same code and VB6 programmers will continue to whitter on about "footprints". - FunkyDexter

    There's just no reason to use garbage like InputBox. - jmcilhinney

    The threads I start are Niya and Olaf free zones. No arguing about the benefits of VB6 over .NET here please. Happiness must reign. - yereverluvinuncleber

  3. #3

    Thread Starter
    Member
    Join Date
    Jun 2012
    Posts
    43

    Re: how to encode output to text file

    UTF8 doesn't work either. The string "les votes de l'élection" comes out as "les votes de l'‚lection"

  4. #4
    Angel of Code Niya's Avatar
    Join Date
    Nov 2011
    Posts
    8,598

    Re: how to encode output to text file

    Does it get retrieved from the database properly ? I.e. If you did a Debug.Writeline would it display properly ?
    Treeview with NodeAdded/NodesRemoved events | BlinkLabel control | Calculate Permutations | Object Enums | ComboBox with centered items | .Net Internals article(not mine) | Wizard Control | Understanding Multi-Threading | Simple file compression | Demon Arena

    Copy/move files using Windows Shell | I'm not wanted

    C++ programmers will dismiss you as a cretinous simpleton for your inability to keep track of pointers chained 6 levels deep and Java programmers will pillory you for buying into the evils of Microsoft. Meanwhile C# programmers will get paid just a little bit more than you for writing exactly the same code and VB6 programmers will continue to whitter on about "footprints". - FunkyDexter

    There's just no reason to use garbage like InputBox. - jmcilhinney

    The threads I start are Niya and Olaf free zones. No arguing about the benefits of VB6 over .NET here please. Happiness must reign. - yereverluvinuncleber

  5. #5

    Thread Starter
    Member
    Join Date
    Jun 2012
    Posts
    43

    Re: how to encode output to text file

    Yes, it shows up correctly in a messagebox or datagridview.

  6. #6
    Angel of Code Niya's Avatar
    Join Date
    Nov 2011
    Posts
    8,598

    Re: how to encode output to text file

    Well it works fine for me:-
    vbnet Code:
    1. '
    2.         Dim ss As String = "les votes de l'élection"
    3.  
    4.         File.WriteAllText("C:\TestText.txt", ss, Encoding.UTF8)
    5.  
    6.         MsgBox(File.ReadAllText("C:\TestText.txt", Encoding.UTF8))

    What code are you using to read and write this text file ?
    Treeview with NodeAdded/NodesRemoved events | BlinkLabel control | Calculate Permutations | Object Enums | ComboBox with centered items | .Net Internals article(not mine) | Wizard Control | Understanding Multi-Threading | Simple file compression | Demon Arena

    Copy/move files using Windows Shell | I'm not wanted

    C++ programmers will dismiss you as a cretinous simpleton for your inability to keep track of pointers chained 6 levels deep and Java programmers will pillory you for buying into the evils of Microsoft. Meanwhile C# programmers will get paid just a little bit more than you for writing exactly the same code and VB6 programmers will continue to whitter on about "footprints". - FunkyDexter

    There's just no reason to use garbage like InputBox. - jmcilhinney

    The threads I start are Niya and Olaf free zones. No arguing about the benefits of VB6 over .NET here please. Happiness must reign. - yereverluvinuncleber

  7. #7

    Thread Starter
    Member
    Join Date
    Jun 2012
    Posts
    43

    Re: how to encode output to text file

    Well, that doesn't really help me, as writealltext writes one line and closes the file. How about if I want to write lots of lines?

    My code is as follows:
    [code]
    Dim Est As Single
    Dim str As String
    saveFileDialog1.Filter = "txt files (*.txt)|*.txt|All files (*.*)|*.*"
    saveFileDialog1.FilterIndex = 2
    saveFileDialog1.RestoreDirectory = True
    Dim ndel As Integer = dgvLots.SelectedRows.Count
    If saveFileDialog1.ShowDialog() = DialogResult.OK Then
    Dim fs As New FileStream(saveFileDialog1.FileName, FileMode.Create)
    sw = New StreamWriter(fs, Encoding.UTF8)
    Dim Arow As DataGridViewRow
    Dim nrow As Integer = 0
    For i = 0 To ndel - 1
    Arow = dgvLots.SelectedRows(i)
    nrow += 1
    sw.Write("[8pt] Ref: ")
    sw.Write(Arow.Cells("Whatisit").Value)
    sw.Write("/")
    sw.Write(Arow.Cells("Seqno").Value.ToString)
    sw.Write("/")
    sw.Write(Arow.Cells("Notes").Value)
    sw.Write("/")
    sw.Write(Arow.Cells("Location").Value)
    sw.WriteLine("[10pt]")
    sw.Write("[b]" & Arow.Cells("Country").Value.ToString & "[b] ")
    sw.Write("(" & Arow.Cells("Oyear").Value.ToString & ") ")
    str = Arow.Cells("Descrip").Value.ToString
    sw.Write(str)
    Est = Convert.ToSingle(Arow.Cells("Estimate").Value)
    sw.WriteLine(" " & Est.ToString("C2", CultureInfo.CurrentCulture))
    Next
    sw.Close()
    MessageBox.Show(nrow.ToString & " written.")
    End If

  8. #8
    Angel of Code Niya's Avatar
    Join Date
    Nov 2011
    Posts
    8,598

    Re: how to encode output to text file

    Attach a sample of the badly written text file. As far as I can tell your code is write. I want to see how my system interprets the text file.
    Treeview with NodeAdded/NodesRemoved events | BlinkLabel control | Calculate Permutations | Object Enums | ComboBox with centered items | .Net Internals article(not mine) | Wizard Control | Understanding Multi-Threading | Simple file compression | Demon Arena

    Copy/move files using Windows Shell | I'm not wanted

    C++ programmers will dismiss you as a cretinous simpleton for your inability to keep track of pointers chained 6 levels deep and Java programmers will pillory you for buying into the evils of Microsoft. Meanwhile C# programmers will get paid just a little bit more than you for writing exactly the same code and VB6 programmers will continue to whitter on about "footprints". - FunkyDexter

    There's just no reason to use garbage like InputBox. - jmcilhinney

    The threads I start are Niya and Olaf free zones. No arguing about the benefits of VB6 over .NET here please. Happiness must reign. - yereverluvinuncleber

  9. #9

    Thread Starter
    Member
    Join Date
    Jun 2012
    Posts
    43

    Re: how to encode output to text file

    I'm sorry, but attachments don't seem to be working for me. Here is a link to the file:

    http://www.stampsbythemes.com/kn6/cat.txt

  10. #10
    Angel of Code Niya's Avatar
    Join Date
    Nov 2011
    Posts
    8,598

    Re: how to encode output to text file

    I'm getting a "Not Found" error with that link.
    Treeview with NodeAdded/NodesRemoved events | BlinkLabel control | Calculate Permutations | Object Enums | ComboBox with centered items | .Net Internals article(not mine) | Wizard Control | Understanding Multi-Threading | Simple file compression | Demon Arena

    Copy/move files using Windows Shell | I'm not wanted

    C++ programmers will dismiss you as a cretinous simpleton for your inability to keep track of pointers chained 6 levels deep and Java programmers will pillory you for buying into the evils of Microsoft. Meanwhile C# programmers will get paid just a little bit more than you for writing exactly the same code and VB6 programmers will continue to whitter on about "footprints". - FunkyDexter

    There's just no reason to use garbage like InputBox. - jmcilhinney

    The threads I start are Niya and Olaf free zones. No arguing about the benefits of VB6 over .NET here please. Happiness must reign. - yereverluvinuncleber

  11. #11

    Thread Starter
    Member
    Join Date
    Jun 2012
    Posts
    43

    Re: how to encode output to text file

    OK,. maybe this time the attachment worked?
    Attached Files Attached Files

  12. #12
    Angel of Code Niya's Avatar
    Join Date
    Nov 2011
    Posts
    8,598

    Re: how to encode output to text file

    Hmmm......
    Code:
    [8pt] Ref: VAR/18156/GM CT.RL á-X XL/[10pt]
    [b]ETATS UNIS[b] (1977) Urne* pour les votes de l'‚lection g‚n‚rale.  Yvert No 1182.  Scott No 1584.  Le vote libre.  Jolie vari‚t‚ de piquage en bloc de 6.  **. $150.00
    It looks exactly like that when I open it in NotePad. I assume its not supposed to look like that ?
    Treeview with NodeAdded/NodesRemoved events | BlinkLabel control | Calculate Permutations | Object Enums | ComboBox with centered items | .Net Internals article(not mine) | Wizard Control | Understanding Multi-Threading | Simple file compression | Demon Arena

    Copy/move files using Windows Shell | I'm not wanted

    C++ programmers will dismiss you as a cretinous simpleton for your inability to keep track of pointers chained 6 levels deep and Java programmers will pillory you for buying into the evils of Microsoft. Meanwhile C# programmers will get paid just a little bit more than you for writing exactly the same code and VB6 programmers will continue to whitter on about "footprints". - FunkyDexter

    There's just no reason to use garbage like InputBox. - jmcilhinney

    The threads I start are Niya and Olaf free zones. No arguing about the benefits of VB6 over .NET here please. Happiness must reign. - yereverluvinuncleber

  13. #13

    Thread Starter
    Member
    Join Date
    Jun 2012
    Posts
    43

    Re: how to encode output to text file

    No, the string "l'‚lection g‚n‚rale" should be "l'élection génénerale". The acute accent e's are coming out as commas in the text file.

  14. #14

    Thread Starter
    Member
    Join Date
    Jun 2012
    Posts
    43

    Re: how to encode output to text file

    Actually, notepad is reformatting the text. If you open it in a real editor like VEDIT, you will see the text in the file is actually:

    [8pt] Ref: VAR/18156/GM CT.RL á-X XL/[10pt]
    [b]ETATS UNIS[b] (1977) Urne* pour les votes de l'‚lection g‚n‚rale. Yvert No 1182. Scott No 1584. Le vote libre. Jolie vari‚t‚ de piquage en bloc de 6. **. $150.00

  15. #15

    Thread Starter
    Member
    Join Date
    Jun 2012
    Posts
    43

    Re: how to encode output to text file

    Quote Originally Posted by Niya View Post
    Well it works fine for me:-
    vbnet Code:
    1. '
    2.         Dim ss As String = "les votes de l'élection"
    3.  
    4.         File.WriteAllText("C:\TestText.txt", ss, Encoding.UTF8)
    5.  
    6.         MsgBox(File.ReadAllText("C:\TestText.txt", Encoding.UTF8))

    What code are you using to read and write this text file ?
    I ran this code and the file actually contains the following (despite what messagebox says):

    les votes de l'élection

  16. #16
    Angel of Code Niya's Avatar
    Join Date
    Nov 2011
    Posts
    8,598

    Re: how to encode output to text file

    I'm gonna venture a guess here and say that the file is being written to correctly and its this program VEDIT that's not decoding it properly. Its the only explanation that makes sense to me. NotePad is fully capable of reading various encodings which is why it looks more accurate in NotePad. Even word can read it. Word offers several decoders and it appears close to correct when using UTF-8. Only the accents are missing. Also, you might want to message Evil_Giraffe as he knows a whole lot more about unicode than me.
    Treeview with NodeAdded/NodesRemoved events | BlinkLabel control | Calculate Permutations | Object Enums | ComboBox with centered items | .Net Internals article(not mine) | Wizard Control | Understanding Multi-Threading | Simple file compression | Demon Arena

    Copy/move files using Windows Shell | I'm not wanted

    C++ programmers will dismiss you as a cretinous simpleton for your inability to keep track of pointers chained 6 levels deep and Java programmers will pillory you for buying into the evils of Microsoft. Meanwhile C# programmers will get paid just a little bit more than you for writing exactly the same code and VB6 programmers will continue to whitter on about "footprints". - FunkyDexter

    There's just no reason to use garbage like InputBox. - jmcilhinney

    The threads I start are Niya and Olaf free zones. No arguing about the benefits of VB6 over .NET here please. Happiness must reign. - yereverluvinuncleber

  17. #17

    Thread Starter
    Member
    Join Date
    Jun 2012
    Posts
    43

    Re: how to encode output to text file

    Nope, it's definitely NOT vedit that's the problem. I know that for a fact. If any editor can show a file as it really is then it's Vedit. Just to confirm, I tried to import the file into Microsoft Word and got the same problem -- the file looks just like it does in Vedit.

    How do I send a message to Evil_Giraffe?

  18. #18
    Angel of Code Niya's Avatar
    Join Date
    Nov 2011
    Posts
    8,598

    Re: how to encode output to text file

    The upper left corner of the page should have a "Private Messages" link where you can go. You can direct him to this thread so the discussion can take place in public where others can learn.
    Treeview with NodeAdded/NodesRemoved events | BlinkLabel control | Calculate Permutations | Object Enums | ComboBox with centered items | .Net Internals article(not mine) | Wizard Control | Understanding Multi-Threading | Simple file compression | Demon Arena

    Copy/move files using Windows Shell | I'm not wanted

    C++ programmers will dismiss you as a cretinous simpleton for your inability to keep track of pointers chained 6 levels deep and Java programmers will pillory you for buying into the evils of Microsoft. Meanwhile C# programmers will get paid just a little bit more than you for writing exactly the same code and VB6 programmers will continue to whitter on about "footprints". - FunkyDexter

    There's just no reason to use garbage like InputBox. - jmcilhinney

    The threads I start are Niya and Olaf free zones. No arguing about the benefits of VB6 over .NET here please. Happiness must reign. - yereverluvinuncleber

  19. #19
    PowerPoster Evil_Giraffe's Avatar
    Join Date
    Aug 2002
    Location
    Suffolk, UK
    Posts
    2,555

    Re: how to encode output to text file

    Okay, your issue basically boils down to this fact:

    There is no character é in ASCII.

    ASCII only defines the values 0-127. Those in the range 128-255 are not part of ASCII. These are colloquially known as Extended ASCII, but I don't know how official that terminology is.

    Asking for the ASCII encoding to write your text in .NET gives you ASCII encoding (as you might hope). You need to use "extended ASCII" - but here's the problem:

    There is no single extended ASCII. You need to know what code page you are going to use for the upper 128 characters. Until you figure that out (your underlying database can probably tell you what code page it is using for single-byte text), you can't encode properly. Once you have figured it out, I would assume that you can use the System.Text.Encoding.GetEncoding() method to get the correct encoder, but I wouldn't know for sure because I see no reason not to use Unicode and have never needed to get a text encoder for a specific code page.

    Presumably you have a reason, but would you mind sharing it? It may be founded on an incorrect assumption, and if you can work with Unicode you'll probably find a lot of things a lot easier.

  20. #20
    PowerPoster Evil_Giraffe's Avatar
    Join Date
    Aug 2002
    Location
    Suffolk, UK
    Posts
    2,555

    Re: how to encode output to text file

    Okay, your issue basically boils down to this fact:

    There is no character é in ASCII.

    ASCII only defines the values 0-127. Those in the range 128-255 are not part of ASCII. These are colloquially known as Extended ASCII, but I don't know how official that terminology is.

    Asking for the ASCII encoding to write your text in .NET gives you ASCII encoding (as you might hope). You need to use "extended ASCII" - but here's the problem:

    There is no single extended ASCII. You need to know what code page you are going to use for the upper 128 characters. Until you figure that out (your underlying database can probably tell you what code page it is using for single-byte text), you can't encode properly. Once you have figured it out, I would assume that you can use the System.Text.Encoding.GetEncoding() method to get the correct encoder, but I wouldn't know for sure because I see no reason not to use Unicode and have never needed to get a text encoder for a specific code page.

    Presumably you have a reason, but would you mind sharing it? It may be founded on an incorrect assumption, and if you can work with Unicode you'll probably find a lot of things a lot easier.

  21. #21

    Thread Starter
    Member
    Join Date
    Jun 2012
    Posts
    43

    Re: how to encode output to text file

    The output from this program has to go into a file loaded by WordPerfect 11, which does not seem to import Unicode text properly. Also the Wordperfect 11 file will be processed by several macros which do not recognize Unicode characters. Legacy code, legacy problems. I should probably just rewrite everything to be consistent with Unicode, but there are time and effort constraints on the project.

  22. #22
    PowerPoster Evil_Giraffe's Avatar
    Join Date
    Aug 2002
    Location
    Suffolk, UK
    Posts
    2,555

    Re: how to encode output to text file

    Quote Originally Posted by franceint View Post
    The output from this program has to go into a file loaded by WordPerfect 11, which does not seem to import Unicode text properly.
    Seems like a valid reason to me. In which case you should check which code page the version of WordPerfect you've installed is expecting. You might get away with using System.Text.Encoding.Default, but I'm really not too familiar with the code page related parts of the framework, sorry.

    Edit: And if that doesn't work, I'd next try interrogating the system for its default code page like this:
    http://stackoverflow.com/questions/9.../921981#921981
    Then use the overload of System.Text.Encoding.GetEncoding() that takes an integer argument.
    Last edited by Evil_Giraffe; Nov 19th, 2012 at 11:59 AM.

  23. #23

    Thread Starter
    Member
    Join Date
    Jun 2012
    Posts
    43

    Re: how to encode output to text file

    I seem to be able to get away with using Encoding.Default so far. Thanks for your help.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Click Here to Expand Forum to Full Width