[RESOLVED] Log Reader Parser

[TCarter]

So, I am trying to make a simple log parser that pulls info from a constant (real time updating) log file.

it is CONSTANTLY being updated, and i'm having issues...


Here's what I got so far:

        Dim sFilename As String = My.Settings.chatlogloc
        Debug.Print(sFilename)
        Dim sFileReader As StreamReader
        Dim sInputLine As String
        Try
            Dim x As Integer
            Dim s As String
            Dim y As Integer
            Dim c As String
            If Dir(sFilename.ToString) <> "" Then
                sFileReader = System.IO.File.OpenText(sFilename)
                sInputLine = sFileReader.ReadLine()
                Do Until sInputLine Is Nothing
                    sInputLine = sFileReader.ReadLine()
                    x = InStr(sInputLine, ": ")
                    y = InStr(sInputLine, vbNewLine)
                    c = Mid(sInputLine, 1, x - 1)
                    s = Mid(sInputLine, x + 1)
                    s = s.Replace(" : ", "")
                    Debug.Print(s)
                    If sInputLine Is Nothing Then
                        'Clear the log file.
                        Debug.Print("End of FiLE")
                        My.Computer.FileSystem.DeleteFile(My.Settings.chatlogloc)
                    End If
                Loop
                sFileReader.Close()
                sFileReader.Dispose()

            End If
        Catch ex As Exception
            'Do nothing
            Debug.Print(ex.ToString)
        Finally
            sFileReader.Close()
            My.Computer.FileSystem.DeleteFile(My.Settings.chatlogloc)
        End Try

Basically, its messing up when its time to rescan it. I can't use the 'New' on it, as it throws a error on the streamreader.



And if it's throwing an error with that, i'd be curious what other errors it will throw.



The file doesnt necessarily HAVE to be deleted, but it DOES have to be cleared out. (so the program doesnt have to reparse all the lines).


Any assistance would be helpful, as well as any generic improvements. (i'm new to this readbyline stuff.)



Thanks in advance guys!

[stanav]

How big is the log file? If it's relatively small (let's say, few MB's in size) then you can read the whole thing into memory and then clear the file immediately after reading it.
Something like this:

Dim sFilename As String = My.Settings.chatlogloc
'Read all the lines in the log file
Dim logLines() as String = System.IO.File.ReadAllines(sFilename)
'Clear the log file
System.IO.File.WriteAllText(sFileName, String.Empty)
'Now work on the log lines
Dim x As Integer
Dim s As Strin
Dim y As Integer
Dim c As String
For each line as string in logLines
     x = InStr(sInputLine, ": ")
     y = InStr(sInputLine, vbNewLine)
     c = Mid(sInputLine, 1, x - 1)
     s = Mid(sInputLine, x + 1)
     s = s.Replace(" : ", "")
     Debug.Print(s)
Next

I noticed that you have the variables s, x, y, c but you're doing anything with them in the loop except assigning some value to them. So the values will be overwritten in the next iteration of the loop and you end up having only the values from the last iteration. If that's the case (you want only the last values) then don't use loop. Just access the last element of the logLines array directly and parse out the values.

[Sitten Spynne]

Your post leaves out some information but implies some stuff that concerns me.

What is the exception that you are getting? I have a hunch that you get the exception because you try to delete the file while whatever updates it is writing to it, but I could be wrong. Knowing the exception that you are getting would definitely help narrow down the problem.

[TCarter]

sorry for the delayed response....

the x, y, c, s was pulled from another app of mine, as I forgot exactly how to use Mid. Hence why some of which was used, others were not.

The log file is for a game, and the game client basically writes to it each time a message is sent, damage is done to me, or anyone around me. and with the various chat channels, guild chat, etc... It updates each time the message comes in (so..real time)

My thing is, i can delete said file, the game will then recreate it. cool.

I figured if I clear it at the end of each 'parse', it'll just see "new" text and such.

The original exception was the fact that I 'closed' the file, then when i tried to rerun the argument, it was giving me an issue telling me to use the 'New' statement... and such..

[Grimfort]

I have a log parsing app similar to this. The key is to remember the last position you were at when you previously read it, so start at 0, read in data until you hit the end and remember the current byte position. Next time start from there, read on till end, save location etc etc. I've not got the code to hand to show exactly how its done. The only thing you have to do is check the file has not been rewritten, in this case if you know its always being appended to you have no problem.

[TCarter]

In the code of:

Dim sFilename As String = My.Settings.chatlogloc
'Read all the lines in the log file
Dim logLines() as String = System.IO.File.ReadAllines(sFilename)
'Clear the log file
System.IO.File.WriteAllText(sFileName, String.Empty)
'Now work on the log lines
Dim x As Integer
Dim s As Strin
Dim y As Integer
Dim c As String
For each line as string in logLines
     x = InStr(sInputLine, ": ")
     y = InStr(sInputLine, vbNewLine)
     c = Mid(sInputLine, 1, x - 1)
     s = Mid(sInputLine, x + 1)
     s = s.Replace(" : ", "")
     Debug.Print(s)
Next

given in Post #2,

I had to change: ReadAllLines to ReadAllText, as with ReadAllLines it gives the error of: Value of type '1-dimensional array of String' cannot be converted to 'String'.

Now, when I ran it, it took the same text, and looped it. 10x

So, I took out the loop(for each), and it displays correctly, except it posts the lines in a single string.

Also, had to make minor changes to the code, as with the other code, it was displaying 1 character per line in debug...





This is what i'm using now:

            Dim sFilename As String = My.Settings.chatlogloc
            Debug.Print(sFilename)
            Dim sFileReader As StreamReader
            Dim sInputLine As String
            'Read all the lines in the log file
        Dim logLines As String = System.IO.File.ReadAllLines(sFilename)
            'Clear the log file
            System.IO.File.WriteAllText(sFilename, String.Empty)
            'Now work on the log lines
            Dim x As Integer
            Dim s As String
            Dim y As Integer
            Dim c As String
        x = InStr(logLines, " : ")
            'y = InStr(line, vbNewLine)
            'c = Mid(sInputLine, 1, x - 1)
        s = Mid(logLines, x + 1)
        logLines = logLines.Replace(" : ", "")
            'Debug.Print(s)
        Debug.Print("++" & logLines)

[minitech]

A lot of that looks pretty redundant doesn't this:

'Read all the lines in the log file
Dim logLines() As String = IO.File.ReadAllLines(My.Settings.chatlogloc)
'Clear the log file
IO.File.WriteAllText(sFileName, String.Empty)
'Now work on the log lines
For Each line As String in logLines
     s = sInputLine.Substring(sInputLine.IndexOf(":"c) + 2))
     Debug.Print(s)
Next

Do about the same thing?
For clarification, it should replace everything from Try to Catch, I think.

[TCarter]

I resolved this with the help of a friend. Thanks guys for help and a great base.