[RESOLVED] Transferring ListView Checked Items to Text box taking too long

[victorb17]

Hey guys, I have a list view that takes values from an analog sensor hooked up to an arduino board. The sensor reads values for a while then I need to transfer them to a text box, or string, or stream reader, etc. Just something that I can format so that one listview item ends up on one line in a text file with no line feed after the last value in the file and no delimiter.

I have it working but the problem is, on occasion there can be as many as 50,000 items in the list view which can take up to 15-20mins to transfer. This is unacceptable. I have tried several methods to quicken up the process. Here is the current code that produces the fastest result:

Dim cItems As ListView.CheckedListViewItemCollection = Main.listview.CheckedItems
        Dim ccount As Integer = cItems.Count

        ProgressBar1.Maximum = ccount
        ProgressBar1.Value = 1

        Dim num As New Integer
        num = 0
        datastring = Main.listview.CheckedItems(num).Text

        num = num + 1

        If ccount > 1 Then

            Do While num < Main.listview.CheckedItems.Count

                datastring = datastring + Environment.NewLine

                datastring = datastring + Main.listview.CheckedItems(num).Text
                previewtextbox.Text = datastring
                ProgressBar1.Value = ProgressBar1.Value + 1
                num = num + 1
            Loop

        Else

            previewtextbox.Text = datastring
            ProgressBar1.Value = ProgressBar1.Maximum

        End If

here is code that works, but gets slower and slower, and basically comes to a crawl, make take over an hour to do the transfer.

        Dim i As Integer

        ProgressBar1.Maximum = Main.listview.CheckedItems.Count
        ProgressBar1.Value = 1

        For i = 0 To Main.listview.CheckedItems.Count - 1

            textbox.AppendText(Main.listview.CheckedItems(i).Text + vbCrLf)
            ProgressBar1.PerformStep()
        Next

Is the answer that it is just a large amount of data and I need to suck it up? I am just thinking that the computer should be able to append text at the speed of light and it should only take a few seconds. When I watch the task manager the software isn't eating up tons of memory. In fact it stays somewhere around 25000k. Maybe it doesn't show up there? I'm not the world's genius when it comes to memory management, so maybe that is the problem. Thanks for any pointers.

[dunfiddlin]

My first question would be why you're using a listview at all, especially if there are no subitems as it would appear. Surely you're not ploughing through 10s of 1000s of values manually checking those you want transferred?

[jmcilhinney]

The issue is that you are adding text to the TextBox for the items one at a time. You should construct the text and then update the TextBox only once. You should be using a For each loop and doing one of three things:

1. Use a StringBuilder and call AppendLine in the loop, then call ToString at the end to get a single String to update the TextBox once.
2. Use a List(Of String) and call String.Join to create a single String to update the TextBox.
3. Use a List(Of String) and call ToArray to create a String array to assign to the Lines property of the TextBox.

[victorb17]

My first question would be why you're using a listview at all, especially if there are no subitems as it would appear. Surely you're not ploughing through 10s of 1000s of values manually checking those you want transferred?

The arduino tells the listview which ones to check. I have a button to invert the selection. I save the checked items, delete them, then select the remaining and add them to a file with a different name.

[victorb17]

I tried the stringbuilder. It is consistantly the same speed, but still is just as slow as the code I already have working. From what I read the stringbuilder is probably the fastest way to go. Here is the code now:

Dim sb As New System.Text.StringBuilder()

        Dim cItems As ListView.CheckedListViewItemCollection = Main.listview.CheckedItems
        ProgressBar1.Maximum = cItems.Count
        ProgressBar1.Value = 1

        Dim num As New Integer
        num = 0
        sb.Append(Main.listview.CheckedItems(num).Text)
        num = num + 1

        If cItems.Count > 1 Then

            Do While num < Main.listview.CheckedItems.Count

                sb.Append(vbCrLf + Main.listview.CheckedItems(num).Text)

                num = num + 1
                ProgressBar1.Value = num
            Loop

        Else

            previewtextbox.Text = sb.ToString
            ProgressBar1.Value = ProgressBar1.Maximum

        End If

        previewtextbox.Text = sb.ToString

[victorb17]

One thought that I had that maybe some of the VB guru's here could help me with is this: Perhaps the shear amount of data trying to be processed at once is the problem? Would it make a difference if I appended 100 checked items at a time to the string builder, then appended the SB to the textbox. Cleared the SB, then appended the next 100 items? etc.etc.etc till all was processed? That way it would be working on smaller blocks of data at a time.

I am not a home to try that. I will when I get back unless someone gives a reason not to. At this point I'm stuck.

[dunfiddlin]

If the arduino gives the information relevant to checking can you not simply build two strings or textboxes as the values come in? It seems odd to create a listview item and not take the opportunity to retain the information in the form you ultimately want to keep either at the same time or instead.

[victorb17]

If the arduino gives the information relevant to checking can you not simply build two strings or textboxes as the values come in? It seems odd to create a listview item and not take the opportunity to retain the information in the form you ultimately want to keep either at the same time or instead.

Well that gets into a long discussion. I am building this project for a friend and there are a number of requirements and personal requests that need to be satisfied.

[dunfiddlin]

The client is a cross we all have to bear unfortunately but I can't see any reason to object to this way of proceeding especially if it's covert!

[victorb17]

I wish I could, but the listview is front and center on the app. There would be no way around it.

[.paul.]

use LINQ. it'll be much faster.

Public Class Form1

    Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
        previewtextbox.Lines = listview.CheckedItems.Cast(Of ListViewItem).Select(Function(lvi) lvi.Text).ToArray
    End Sub

End Class

to use LINQ to check all items:

Public Class Form1

    Private Sub Button1_Click(sender As System.Object, e As System.EventArgs) Handles Button1.Click
        Array.ForEach(Enumerable.Range(0, listview.Items.Count).ToArray, (Sub(x) listview.Items(x).Checked = True))
    End Sub

End Class

[Inferrd]

I tried your code with a ListView containing 50,000 checked items, and I was amazed at how slow it was. It had only managed to append the first 100 items in 51 seconds. This extrapolates to over 7 hours to process all the items (assuming the rate remains constant; I suspect it may become slower as it proceeds. Don't know).

The problem seems to stem from executing the statements Main.listview.CheckedItems.Count and Main.listview.CheckedItems(num).Text every iteration of the loop.


The answer is simple though. Just do as JMC originally advised in Post #3.

You should be using a For each loop and doing one of three things: ..........

Doing so reduced the time for appending 50,000 checked items with a StringBuilder to under 2 seconds.

[victorb17]

I tried your code with a ListView containing 50,000 checked items, and I was amazed at how slow it was. It had only managed to append the first 100 items in 51 seconds. This extrapolates to over 7 hours to process all the items (assuming the rate remains constant; I suspect it may become slower as it proceeds. Don't know).

The problem seems to stem from executing the statements Main.listview.CheckedItems.Count and Main.listview.CheckedItems(num).Text every iteration of the loop.


The answer is simple though. Just do as JMC originally advised in Post #3.



Doing so reduced the time for appending 50,000 checked items with a StringBuilder to under 2 seconds.

Hey I must have missed the for each part. I have this code:

     For Each item As ListViewItem In Main.holdbox.CheckedItems

            sb.Append(item.Text + vbCrLf)
            ProgressBar1.Value = num
            num = num + 1

        Next

        sb.Remove(sb.Length - 1, 1)
        previewtextbox.Text = sb.ToString

It is blazing fast! Almost instant! Thanks a ton guys. I learned something new today!

[victorb17]

I tried your code with a ListView containing 50,000 checked items, and I was amazed at how slow it was. It had only managed to append the first 100 items in 51 seconds. This extrapolates to over 7 hours to process all the items (assuming the rate remains constant; I suspect it may become slower as it proceeds. Don't know).

The problem seems to stem from executing the statements Main.listview.CheckedItems.Count and Main.listview.CheckedItems(num).Text every iteration of the loop.


The answer is simple though. Just do as JMC originally advised in Post #3.



Doing so reduced the time for appending 50,000 checked items with a StringBuilder to under 2 seconds.

How did you know the problem was with those specific lines?

[.paul.]

accessing properties of your listview in a loop slows execution considerably

[jmcilhinney]

accessing properties of your listview in a loop slows execution considerably

Yes, it's always bad practice to use the same complex expression more than once in one block of code. Whenever you need to use a property value or method result more than once, you should always evaluate it once only and assign the result to a local variable, using that variable multiple times there after.