[RESOLVED] Reading Delimited Text

[dotty]

After some thinking, I think I figured out how to read delimited text. I was just curious as to whether it'd be better to use StreamReader.ReadLine or File.ReadAllLine or if it matters at all? I'm assuming the ReadLine method would be better if I'm working with a larger file since ReadAllLine would store the entire thing in memory?

Here's my code - I'm reading from a text file and displaying into a listview. I'm still new to vb so let me know if there's a more efficient way of doing this or if there's a problem with how I'm doing it now.

File.ReadAllLines

        Dim lines() As String
        Dim parts() As String
   
        lines = File.ReadAllLines("C:\blah.txt")

        For Each line As String In lines
            parts = line.Split("|")

            Dim li As New ListViewItem

            li.Text = parts(0)
            li.SubItems.Add(parts(1))
            li.SubItems.Add(parts(2))
            li.SubItems.Add(parts(3))
            li.SubItems.Add(parts(4))

            ListView1.Items.Add(li)

        Next

StreamReader

        Dim lines As String
        Dim parts() As String
        Dim fs As New FileStream("C:\blah.txt", FileMode.Open)
        Dim sr As New StreamReader(fs)

        Do Until sr.EndOfStream

            lines = sr.ReadLine
            parts = lines.Split("|")

            Dim li As New ListViewItem

            li.Text = parts(0)
            li.SubItems.Add(parts(1))
            li.SubItems.Add(parts(2))
            li.SubItems.Add(parts(3))
            li.SubItems.Add(parts(4))

            ListView1.Items.Add(li)

        Loop

[jmcilhinney]

For reading delimited text files I would recommend a TextFieldParser. The class documentation has code examples.

With regards to your question specifically, as you're throwing away the data you read after processing it I would say that using a StreamReader and calling ReadLine is better. You don't need all the data at the same time so there's point getting all the data at the same time.

Also, you really should create a List(Of ListViewItem) and add your items to that, then call AddRange at the end to add them all in batch. The way you're doing it is inefficient because the control will redraw after adding each item. That can slow things down a lot if you have a lot of items.

[dotty]

When you say the control have to be redraw, you're referring to the listview control, right?

Is this how I would do it if I were to add it all at once? Have a question regarding this though. When I tried this, if there was a blank line at the end of the file, nothing gets added to the listview because the the sub exit after "li.SubItems.Add(parts(1))" instead of finishing the loop and going to ListView1.Items.AddRange. Why is that? It doesn't happen if I add the item one at a time.

I'll look at the documentation for the textfieldparser, but if I code it like this, say it's a comma delimited file and the field is "Some Company, LLC"..how would I deal with this?

Dim lines As String
Dim parts() As String
Dim fs As New FileStream("C:\blah.txt", FileMode.Open)
Dim sr As New StreamReader(fs)
Dim litlist As New List(Of ListViewItem)

Do

lines = sr.ReadLine
parts = lines.Split("|")

Dim li As New ListViewItem

li.Text = parts(0)
li.SubItems.Add(parts(1))
li.SubItems.Add(parts(2))
li.SubItems.Add(parts(3))
li.SubItems.Add(parts(4))

litlist.Add(li)

Loop Until sr.EndOfStream

ListView1.Items.AddRange(litlist.ToArray)

Thanks!

[jmcilhinney]

That looks good. One change I'd make is putting the EndOfStream test at the start of the loop, just in case there's no data in the file. Also, there's a ListViewItem constructor that takes a String array representing the subitems as an argument. That means your code can be simplied from this:

lines = sr.ReadLine
parts = lines.Split("|")

Dim li As New ListViewItem

li.Text = parts(0)
li.SubItems.Add(parts(1))
li.SubItems.Add(parts(2))
li.SubItems.Add(parts(3))
li.SubItems.Add(parts(4))

litlist.Add(li)

to this:

lines = sr.ReadLine
parts = lines.Split("|"c)

Dim li As New ListViewItem(parts)

litlist.Add(li)

or even just this:

litlist.Add(New ListViewItem(sr.ReadLine().Split("|"c)))

Note that I'm passing a Char literal to Split, which is required with Option Strict On. Your code relies on your String literal being implicitly converted to a Char. Relying on implicit conversions makes your code slower and more brittle. The sooner you turn Option Strict On and make all your conversions explicit the better.

[dotty]

Thanks for the tip. Checked out the TextFieldParser and I love it. It takes care of the problem with fields containing delimiter.