[RESOLVED] Reading arrays from a data file

[CodingJoe]

I'm trying to read a data file with 8 columns of data into an array in VB 6.0.

Here is a cut of the code I've started. How do I convert this to one that will read individual columns into 8 separate arrays?

Private Sub Command1_Click()
    
    Dim TextLine$, Filename$
    Dim FileHandle As Integer
    
    Filename$ = "C:\data.txt"
    
    If Dir(Filename$) = "" Then Exit Sub
    
    FileHandle = FreeFile
    
    Open Filename$ For Input As #FileHandle
    
    Do While Not EOF(FileHandle)
        Line Input #FileHandle, TextLine$
    Loop
    
    Close #FileHandle
    
End Sub

Thanks in advance for your help.

[Jim Davis]

You can use the Split() method, that is will split up the string data, into array elements, by using any delimiter. The delimiter could be a one byte long letter, but you can also use longer strings also.

For example, here is a Data, that will arrive into TextLine$.

1   2   3   4   5   6

There you can see, each data is delimited by 3 spaces. To retrieve this into an array, for faster processing, you can do the following.

Dim aDataIn() As String

'....

    Do While Not EOF(FileHandle)
        Line Input #FileHandle, TextLine$
        aDataIn = Split(TextLine$,"   ")
    Loop

Now, the aDataIn will contain one row of data, but every element will be a coulmn.

There you can also extend this method, to push each elements into individual arrays, if you say so.

Private Type tElements
  Elements() As String
End Type

Dim aArrays() As tElements

'....


    ReDim aArrays(0)
    ReDim aArrays(0).Elements(0)

    Do While Not EOF(FileHandle)
        Line Input #FileHandle, TextLine$
        aDataIn = Split(TextLine$,"   ")
        If uBound(aArrays) < uBound(aDataIn) Then Redim Preserve aArrays(uBound(aDataIn)) 'raise the width, aligned to the number of columns.
        
        For i = 0 To uBound(aDataIn)
          Redim Preserve aArrays(i).Elements(uBound(aArrays(i).Elements)+1) 'raise the number of elements by one, for each column.
          aArrays(i).Elements(uBound(aArrays(i).Elements)) = aDataIn(i) 'place the element in this row/column
        Next i 
    Loop

Now you have got the whole file loaded into cells, that have column and row coordinates.

-> aArray(Column).Elements(Row)

[dilettante]

Questions like this are easier to answer if a few more details are available first.

For example: Is this data file something your program will create and then later read itself, or is the data produced by some other program in a specific pre-existing format you must handle?

[CodingJoe]

This is a pre-existing data file that I will read in as separate variables, perform calculations and write back out into a separate file.

[Jim Davis]

You may want to show a sample, to see what else should be done to read the cells.

[CodingJoe]

Good idea.

Here is a sample of the data file I'm trying to read.

12.443    1332.452  1   231.87334   -342.98344   23.43212     -0.09231   0.008329

This is just the first line of the data file. The rest of the file have data with the similar format.

[Jim Davis]

Hmm, the size of the delimiters are vary. It is might be some kind of an aligned text.

Use the Replace() method, to reduce the spaces to one, per values.

        Line Input #FileHandle, TextLine$

        Do While inStr(TextLine$, "  ") > 0 'remove the double spaces, make them equally one space per element
          TextLine$ = Replace$(TextLine$, "  ", " ")
        Loop

        aDataIn = Split(TextLine$," ")

There is also a small bug in my previous post, that is may cause an array error...

Just replace this line: If uBound(aArrays) < uBound(aDataIn) Then Redim Preserve aArrays(uBound(aDataIn))

... to this:

        If UBound(aArrays) < UBound(aDataIn) Then
          For i = UBound(aArrays) To UBound(aDataIn)
            ReDim Preserve aArrays(i) 'extend the number of columns
            ReDim Preserve aArrays(i).Elements(0) 'initialize the row in each column
          Next i
        End If

To access the values, you can use the cSng() or cDbl() converters, these are produce floating point numbers in single or double precision.

-> Debug.Print cDbl(aArrays(0).Elements(5)) <- Will show you the fifth element in the first column

[CodingJoe]

Okay, I've added the changes you suggested. Here is the code as it stands now:

Private Sub Command1_Click()
    
    Dim TextLine$, Filename$
    Dim FileHandle As Integer
    Dim aDataIn() As String
        
    Filename$ = "C:\data.txt"
    
    If Dir(Filename$) = "" Then Exit Sub
    
    FileHandle = FreeFile
    
    Open Filename$ For Input As #FileHandle
    
    Do While Not EOF(FileHandle)
        Line Input #FileHandle, TextLine$
        
        Do While InStr(TextLine$, "  ") > 0 
          TextLine$ = Replace$(TextLine$, "  ", " ")
        Loop

        aDataIn = Split(TextLine$, "      ")
    Loop
    
    Private Type tElements
        Elements() As String
    End Type

    Dim aArrays() As tElements

        ReDim aArrays(0)
        ReDim aArrays(0).Elements(0)
    
        Do While Not EOF(FileHandle)
            Line Input #FileHandle, TextLine$
            aDataIn = Split(TextLine$, "      ")
        If UBound(aArrays) < UBound(aDataIn) Then
          For i = UBound(aArrays) To UBound(aDataIn)
            ReDim Preserve aArrays(i) 'extend the number of columns
            ReDim Preserve aArrays(i).Elements(0) 
          Next i
        End If
            For i = 0 To UBound(aDataIn)
                ReDim Preserve aArrays(i).Elements(UBound(aArrays(i).Elements) + 1)
                aArrays(i).Elements(UBound(aArrays(i).Elements)) = aDataIn(i)
            Next i
        Loop
        
    Close #FileHandle
        
End Sub

Apparently, I've not placed the "Private Type Elements" block correctly as I'm getting the compile error "Invalid inside procedure".

What am I doing wrong?

[LaVolpe]

Correct, the placement for that is at the top of your form in the Declarations section.

Also, you should be Splitting on a single space, not the 6 or 7 spaces you currently have coded.

Last but not least, it appears you have 2 routines combined into one?

[Jim Davis]

sorry my posts are not copy paste friendly codes

Private Type tElements
  Elements() As String
End Type
 
Private Sub Command1_Click()
    
    Dim TextLine$, Filename$
    Dim FileHandle As Integer
 
    Dim aArrays() As tElements, aDataIn() As String, i As Long
 
    
    Filename$ = "C:\data.txt"
    
    If Dir(Filename$) = "" Then Exit Sub
    
    FileHandle = FreeFile
    
    Open Filename$ For Input As #FileHandle
    
    ReDim aArrays(0)
    ReDim aArrays(0).Elements(0)
 
    Do While Not EOF(FileHandle)
        Line Input #FileHandle, TextLine$
 
        Do While inStr(TextLine$, "  ") > 0 'remove the double spaces, make them equally one space per element
          TextLine$ = Replace$(TextLine$, "  ", " ")
        Loop
 
        aDataIn = Split(TextLine$," ")
 
        If UBound(aArrays) < UBound(aDataIn) Then
          For i = UBound(aArrays) To UBound(aDataIn)
            ReDim Preserve aArrays(i) 'extend the number of columns
            ReDim Preserve aArrays(i).Elements(0) 'initialize the row in each column
          Next i
        End If
        
        For i = 0 To uBound(aDataIn)
          Redim Preserve aArrays(i).Elements(uBound(aArrays(i).Elements)+1) 'raise the number of elements by one, for each column.
          aArrays(i).Elements(uBound(aArrays(i).Elements)) = aDataIn(i) 'place the element in this row/column
        Next i 
    Loop
 
    
    Close #FileHandle
    
End Sub

Now this one is the complete code.

[CodingJoe]

Okay, that seems to work. It didn't give any errors.

What is the easiest way to check this? Should I just create another Private Sub to write the array back out?

[Jim Davis]

After you done your process on the values, you can simply write it back, theres no need to create a sub for this purpose.

Dim i As Long, j As Long, sOut as String

  Open "c:\output.txt" for Output As #2

  For i = lBound(aArray(0).Elements) to UBound(aArray(0).Elements)
    For j = lBound(aArray) to uBound(aArray) - 1
      Print #2, aArray(j).Elements(i),       'store the values as aligned text.
    Next j 
    Print #2, aArray(j).Elements(i)          'store the final column, and make a new line
  Next i

  Close #2

[CodingJoe]

Alright. Here is the code I've got now.

Private Sub Command1_Click()
    
    Dim TextLine$, Filename$, Outname$
    Dim FileHandle, OutHandle As Integer
    
    Dim aArrays() As tElements, aDataIn() As String, i As Long
    Dim j As Long, sOut As String
            
    Filename$ = "C:\data.txt"
    Outname$ = "C:\test.txt"
    
    If Dir(Filename$) = "" Then Exit Sub
    
    FileHandle = FreeFile
    
    Open Filename$ For Input As #FileHandle
    Open Outname$ For Output As #OutHandle
    
    ReDim aArrays(0)
    ReDim aArrays(0).Elements(0)
    
    Do While Not EOF(FileHandle)
        Line Input #FileHandle, TextLine$
        
        Do While InStr(TextLine$, "  ") > 0 'remove the double spaces, make them equally one space per element
          TextLine$ = Replace$(TextLine$, "  ", " ")
        Loop

        aDataIn = Split(TextLine$, " ")
            
         If UBound(aArrays) < UBound(aDataIn) Then
             For i = UBound(aArrays) To UBound(aDataIn)
                 ReDim Preserve aArrays(i)
                 ReDim Preserve aArrays(i).Elements(0)
             Next i
        End If
        
        For i = 0 To UBound(aDataIn)
            ReDim Preserve aArrays(i).Elements(UBound(aArrays(i).Elements) + 1)
            aArrays(i).Elements(UBound(aArrays(i).Elements)) = aDataIn(i)
            Print OutHandle, aArrays(i).Elements(i)
        Next i
        
        For i = LBound(aArrays(0).Elements) To UBound(aArrays(0).Elements)
            For j = LBound(aArrays(i)) To UBound(aArrays(i))
            Print #OutHandle, aArrays(j).Elements(i),       'store the values as aligned text.
            Next j
            Print #OutHandle, ""          'make a new line
        Next i
        
  Loop
        
    Close #FileHandle
        
End Sub

When I run this it gives me a compile error: "Expected array" for the following line:

            For j = LBound(aArrays(i)) To UBound(aArrays(i))

[Jim Davis]

Yeah that was my mistake. I have fixed my post above, just check out again.

But. I'm not sure why are you write back all the values instantly. Don't you want to process the whole database first, then write it back in a one move?

[CodingJoe]

Okay, Jim. One last try here before I head out for the weekend.

I ran the code. It creates the output file, but nothing is in it. I've pasted the code one more time here.

Private Sub Command1_Click()
    
    Dim TextLine$, Filename$
    Dim FileHandle As Integer
    
    Dim aArrays() As tElements, aDataIn() As String, i As Long
    Dim j As Long, sOut As String
            
    Filename$ = "C:\data.txt"
    Open "C:\output.txt" For Output As #2
    
    If Dir(Filename$) = "" Then Exit Sub
    
    FileHandle = FreeFile
    
    Open Filename$ For Input As #FileHandle
    
    ReDim aArrays(0)
    ReDim aArrays(0).Elements(0)
    
    Do While Not EOF(FileHandle)
        Line Input #FileHandle, TextLine$
        
        Do While InStr(TextLine$, "  ") > 0 'remove the double spaces, make them equally one space per element
          TextLine$ = Replace$(TextLine$, "  ", " ")
        Loop

        aDataIn = Split(TextLine$, " ")
            
         If UBound(aArrays) < UBound(aDataIn) Then
             For i = UBound(aArrays) To UBound(aDataIn)
                 ReDim Preserve aArrays(i)
                 ReDim Preserve aArrays(i).Elements(0)
             Next i
        End If
        
        For i = 0 To UBound(aDataIn)
            ReDim Preserve aArrays(i).Elements(UBound(aArrays(i).Elements) + 1)
            aArrays(i).Elements(UBound(aArrays(i).Elements)) = aDataIn(i)
        Next i
        
        For i = LBound(aArrays(0).Elements) To UBound(aArrays(0).Elements)
            For j = LBound(aArrays) To UBound(aArrays) - 1
            Print #2, aArrays(j).Elements(i),       'store the values as aligned text.
            Next j
        Next i
        
  Loop
        
    Close #FileHandle
        
End Sub

By the way, I really appreciate your help with this. I'm slowly learning how to use VB. Thanks a lot!

[Jim Davis]

I just presumed, you want to load the whole file, before you start your process, then after that you write it back. Well, your code is a bit confused right now.

Here is it how should it work, imho.

Private Sub Command1_Click()
    
    Dim TextLine$, Filename$
    Dim FileHandle As Integer, OutHandle As Integer
    
    Dim aArrays() As tElements, aDataIn() As String, i As Long
    Dim j As Long, sOut As String
            
    Filename$ = "C:\data.txt"
    If Dir(Filename$) = "" Then Exit Sub
    
    FileHandle = FreeFile
    
    Open Filename$ For Input As #FileHandle
    
    ReDim aArrays(0)
    ReDim aArrays(0).Elements(0)
  
    'First, read the whole file, and fill the arrays.  
    Do While Not EOF(FileHandle)
        Line Input #FileHandle, TextLine$
        
        Do While InStr(TextLine$, "  ") > 0 'remove the double spaces, make them equally one space per element
          TextLine$ = Replace$(TextLine$, "  ", " ")
        Loop

        aDataIn = Split(TextLine$, " ")
            
         If UBound(aArrays) < UBound(aDataIn) Then
             For i = UBound(aArrays) To UBound(aDataIn)
                 ReDim Preserve aArrays(i)
                 ReDim Preserve aArrays(i).Elements(0)
             Next i
        End If
        
        For i = 0 To UBound(aDataIn)
            ReDim Preserve aArrays(i).Elements(UBound(aArrays(i).Elements) + 1)
            aArrays(i).Elements(UBound(aArrays(i).Elements)) = aDataIn(i)
        Next i
  Loop
  Close #FileHandle

  'Do your processing code here. 

  '...
  '...



  'After that, you can write back the whole array by a simple move.

  OutHandle = FreeFile
  Open Outname$ For Output As #OutHandle

  For i = LBound(aArrays(0).Elements) To UBound(aArrays(0).Elements)
    For j = LBound(aArrays) To UBound(aArrays) - 1
      Print #OutHandle, aArrays(j).Elements(i),       'store the values as aligned text.
    Next j
    Print #OutHandle, aArrays(j).Elements(i) 'store the last column as well!!!
  Next i

       
  Close #OutHandle
        
End Sub

But, if you just want to process the data file line by line, so you dont need the whole columns for any processing, theres no need this kind of complex data handling, it could be even more simple.

[CodingJoe]

Thank you Jim.

From what I see here, it makes sense. I'll take a longer look at this in a couple of days.

You're right, though. I do want to read in the entire file, do the processing and write the new data back out. I was just hastily trying to see if the data was being read properly. Is there a quick and dirty way of doing this by checking say one part of the array like aArrays(1).Elements(1) and printing it out to the screen in a MsgBox?

Don't feel compelled to answer that now (unless you want to). I'm signing out for the weekend.

Again, thank you so much for your help. This will prove valuable for me as I'm sure I'll be doing more of this type of processing in the near future.

[Jim Davis]

You can use the Debug object's Print method in the IDE (while you execute your program, in vb6 but not as standalone), that is write to the 'Immediate Window' you can turn it to visible (if you dont see already), by click on the view menu and selecting it.

        For i = 0 To UBound(aDataIn)
            ReDim Preserve aArrays(i).Elements(UBound(aArrays(i).Elements) + 1)
            aArrays(i).Elements(UBound(aArrays(i).Elements)) = aDataIn(i)
            Debug.Print "Col(" & Cstr(i) & ").Row(" & Cstr(UBound(aArrays(i).Elements)) & ") = " & aDataIn(i)
        Next i

Sure, i'm glad to helping you. Just click on the Thread Tools menu up here in this thread, and select the Mark Thread resolved, to make the thread [Resolved].