Calculating Progress

[Dnereb]

Hi All,

Situation:
I'm writing a function to load items into a list or combobox from a textfile in a vb 6.0 class.
But i want to be able to show the loading progress (in a event) in case of a large amount of items.

Problem:
How to calculate the progress percentage without reading the file in advance.
I can get the LOF(#1) (size in bytes) but this won't tell me how many items there are or how large each item is because it includes some overhead for each string (vbCr or vbLf or vbCrLf)
and some overhead for the file itself.
does anyone knows a neat trick to solve this??

The Snippet of Code it's about

RaiseEvent Progress(0, CtrlName, "LoadItemsFromTextFile")
        Percent = Total \ 100
        If Percent < 0.5 Then Percent = 0.5                     'At small files evaide division by zero.
        Do Until EOF(1)
            Line Input #Fhdl, Item
            Box.AddItem Item
            Processed = Processed + Len(Item) * 1.2          ' 1 char = 2 bytes ,vb werkt intern met unicode)
            If (Processed \ Percent) > 100 Then                 ' PROBLEM ZONE: Restart progress if it exeeds 100 percent. 
               Processed = 0
            End If
            RaiseEvent Progress(Processed / Percent, Box.Name, "LoadItemsFromTextFile")
            
        Loop

Note: all variables events etc... are declared

Thanks in advance.

[si_the_geek]

Using Len is a good solution.. and assuming that the file uses CrLf as a line delimiter, you can simply add 2 to it (eg: .. + Len(Item) + 2 )

The percentile (0 to 1) done would then be: Processed / Total
..and the percentage done would be: (Processed / Total) * 100

edit: an optimised version of the percentage would be to store Total/100 into a Single variable before the loop (called sngPerc ?), then use: Processed / sngPerc

[Dnereb]

Thanks!

The remark just add 2 to your len() dropped a coin.

I've decided to determine the first delimiter through binary access
and search for vbLf, vbCr or vbCrLf and assume the delimiter stays consistent through the file.

Percent is a double in the original code I will change it to a single.

[si_the_geek]

Thanks!

The remark just add 2 to your len() dropped a coin.

I've decided to determine the first delimiter through binary access
and search for vbLf, vbCr or vbCrLf and assume the delimiter stays consistent through the file.

Ah, that's a good idea!

Percent is a double in the original code I will change it to a single.

To be honest it wont make much difference.. just that a Double will be more accurate for "larger" files.

[Code Doc]

Pseudocode:
1) Open the file in binary mode.
2) Read the whole file in as a string.
3) Use INSTR() to count the number of VbCrLF characters in the file to get the number of lines before you start.
4) Close the file.
5) Open it back up for Input.
6) The progress bar can then be incremented at a uniform rate based upon the average (mean) line length (LOF/Count) as lines are read in one at a time.

Just an idea to consider.

[si_the_geek]

That's valid too.. although I would use Split instead of steps 3-5.

The only problem is that with large files it can cause a major slowdown (as the physical memory is used up, and virtual memory needs to be used instead).


For 'smaller' files that method should be quicker than using the Input methods.

[Code Doc]

Si said, "The only problem is that with large files it can cause a major slowdown (as the physical memory is used up, and virtual memory needs to be used instead)."
----------------
Agree. I would not recommend using this unless the file is under 40 Mb. My reason for using INSTR() rather that Split() is because I understand it better and it's been around longer. I'm not sure which one is faster.

I can assure the OP that the binary read will run like lightning compared to the Line Input read, so that he will hardly notice that it occurred if the RAM is available.

I think what I like about my idea is that the progress bar will move along at a uniform rate and thus be somewhat immune to huge differences in line lengths. Imbedded images in the file could make this variation enormous.

[si_the_geek]

The Split version I suggested would be significantly quicker than InStr, as the file is only read once (which is the slow part).

The downside to it is that it uses twice as much memory (as the data is stored in the string, and the array).

Assuming that the lines take equal time to add to the control, a simple For loop would do for adding the items and updating the progress (as UBound will let you know how many items there are).

[Dnereb]

I don't want to read the whole file in at once before loading it into the list or combobox. The Time consumed reading is relativly little compared to the additem an the line variations will vary between 5 charcters (1 small word)
and 255 characters (way to long for an item in a combobox or listbox)
The size of these files will stay below 40MB (I don't believe a combobox or listbox can store such amounts of data but I'm not sure about that.)
Anyways I want to have an accurate progress (reading the file included) and robust code so I can reuse this class without worries. The files can be delimited with vbLf, vbCr or vbCrLf (Ansii or Unicode in all cases)
The files can be produceded by word/notepad/a web page/a database etc...
(Csv files will be split and use the array variant to load values)
And that routine accepts a 1 dimensional array to load items is already finished.

But thanks for your posts, I like to hear other points of view.

[18experience]

The Time consumed reading is relativly little compared to the additem

If that is true, first write the algorithm that adds all the items. Then, just copy it and replace the AddItem line with ++Count, or whatever, and run it before you add the items. Then change the AddItem algo to keep up with how many have been added, and display the percentage. The only way to know how many items you will be adding is to go through the file first.

Although, I must say this sounds a little strange to me. It will slow down the program (at least a little). However, if you are confident that it won't slow it down significantly, and the user knowing the status of the loading is that important, then go for it.

[Dnereb]

I do not know how I can optimize the additem propperty of a VB6.0
combobox nor listbox control in VB6.0

I suspect C++ is needed to write controls with optimized additems functionality, if possible.