Where "Flos'tok" and "Tharan Cedrax" and "Infiltrator" can be any of a billion possibilities.
I need to glean 3 different bits of info from them
I first need to get "Flos'tok" no matter which of the 3 strings are presented.
I know it's not possible if the [Infiltrator {1554550527885312}:111091844024]
is presented but that is not an issue since that string can only happen later so for the 1st question
IF [@Flos'tok:Tharan Cedrax {493285583880192}] is the string passed I need to Split "Flos'tok" and "Tharan Cedrax" and ignore anything else as in the "{493285583880192}"
So I can assign
Label1.text = "Flos'tok"
and
Label2.text = "Tharan Cedrax"
1st problem is there is a 50/50 chance the initial string will just be
[@Flos'tok]
Which then I just need to assign
Label1.text = "Flos'tok"
And when it finds
[@Flos'tok:Tharan Cedrax {493285583880192}]
Then just assign
Label2.text = "Tharan Cedrax"
Now Tharan Cedrax and Flos'tok can and will be any number of names, I'm just using them as an example
So my real 1st question is how to strip and remove the {} and anything in between
something like Dim StrValue as String = Regex.Replace(strArr(1), "{^}*", "")?
So after I get past all that I should have 2 populated labelx.text
using the info above it would be
Dim Checker1 = Label1.text
Dim Checker2 = Label2.text
If NOT Checker1.Contains(Label1.Text) OR NOT Checker2.Contains(Label2.Text) Then
'it didn't contain either so it is [Infiltrator {1554550527885312}:111091844024]
Label3.Text = "Infiltrator"
'without any of the {1554550527885312}:111091844024
End If
And get "Flos'tok" from "@Flos'tok:Tharan Cedrax {493285583880192}" and "Tharan Cedrax" from "@Flos'tok:Tharan Cedrax {493285583880192}"
and NOT include "{493285583880192} or "{1554550527885312}:111091844024" in either and if it's neither use the name provided. And assign each value to a Labelx.text
I know I can split Flos'tok:Tharan Cedrax to get Label1 and label 2 but it is not always there first, as many times as not it will just be Flos'tok first and then the next read will give me Flos'tok:Tharan Cedrax and the next read may be Infiltrator {1554550527885312}:111091844024
I hope this makes sense because it took forever to write
I am a real newbie so please treat your response as such.
so I guess I have the regex correct for stripping out the "{493285583880192}" but then it leaves "111091844024" if {1554550527885312}:111091844024] is presented.
The text file dosen't look terribly complicated. All info seem to be contained within brackets and can easily be traversed.
Your description of what you need however, seems very complicated (to me at least). I'm sure if you make a description, that is a bit more clear, about what you need (without labels, textboxes etc.), someone can solve this problem for you pretty quickly.
Something like:
If field 2 contains a name preceeded by a '@' and that name is repeated in the 3rd field followed by a ':', I need the Id contained in the brackets.
For instance early in your OP you split [@Flos'tok:Tharan Cedrax {493285583880192}] into 2 labels, whereas later in the post you split [@Flos'tok:Tharan Cedrax {493285583880192}] into just one label. You need to clearly specify if you need the 1st name, 2nd name, ID, or any/all combinations from 1st, 2nd, .. 5th fields and if they data you need to extract depends on the contents (ie. for instance only need fields with Stealth).
Tom
In truth, a mature man who uses hair-oil, unless medicinally , that man has probably got a quoggy spot in him somewhere. As a general rule, he can't amount to much in his totality. (Melville: Moby Dick)
I need to extract each [] into a usable bit of data
things I do NOT need are anything between a { and } or between a { and } followed by a : if it is numerical as they really don't have any info I need at this time
What I eventually need to do is write everything pertaining [using the bottom log set] to Lilulan into 1 text file anything pertaining to Elara Dorne in a different text file and anything pertaining to anything that is NOT Elara Dorne or Lilulan into a 3rd text file
as the game that writes these files progresses I need to update each of the 3 text files
[this is the main toon doing damage to the infiltrator]
so as I go I will have 3 text files [or more], one with something like
[using the 1st log example]
Flos'tok-2012-07-23-01.txt and in it
This way I can separate each line into it's own category
Category A is the main toon
Category B is the companion
Category D is the bad guy
and each category will have
Time, Who did the damage, who it did the damage to, what kind of damage, how much damage internal, and how much damage external
that is IF it is damage, as you see there are a lot of other things going on such as healing, removing effects, etc, each of those need to also be accounted for in each of the text files
Ultimately I need to know who did what to whom and for how much.
I need to be able to manipulate the data to get information such as
"average damage per ability and how often it crits so i can tell what crits and what does not. "
That kinda of info
Does this I hope clear any of my confusing questions up [it's REALLY late and I am really tired so my brain is working even worse than normal]
one of the problems I see is not EVERYTHING is wrapped in [] , some are wrapped in () and others in <>
[this is not my doing, blame the goof balls who wrote the SWTor client ]
Now there are a possible 8 toons each with a companion, so I had planned on creating a directory for each toon to store each set of files
Dir structure would then be something like
[CombatLogs]
holds all the game created logs
[CombatLogs\ReaderFiles]
holds all the subdirs for the toons and in each say
[CombatLogs\ReaderFiles\Lilulan]
Lilulan-date-1.text
Lilulan-date-2.text
Elara Dorne-date-1.text
Elara Dorne-date-2.text
BadGuy-date-1.text
BadGuy-date-2.text
[CombatLogs\ReaderFiles\Flos'Tok]
Flos'Tok-date-1.text
Flos'Tok-date-2.text
Tharan Cedrax -date-1.text
Tharan Cedrax -date-2.text
BadGuy-date-1.text
BadGuy-date-2.text
[CombatLogs\ReaderFiles\AnotherToon]
Toon-date-1.text
Toon-date-2.text
Companion-date-1.text
Companion-date-2.text
BadGuy-date-1.text
BadGuy-date-2.text
etc, I will create the dirs and files on the fly as needed
Anyway if needed, after some sleep, I can try to explain even clearer.
I made a small example to get you started. I'll probably expand a bit upon it tonight, when I get a little time in front of the computer.
{Sub deleted - better version below. The one listed here initially had some minor flaws like including brackets with fields.}
A regex might be handy to split up the current line into fields, but using regular expressions is generally much less efficient than using the built-in string functions. I used them in the example because they make for short and easily readable code (and because you used regex'es yourself). But if you plan on using this code intensely, I suggest trying to make some optimizations.
Regards Tom
#EDIT: Forgot to mention: your explanation was much better this time around.
I'll try to expand the code a bit tonight.
And one last thing: is it safe to assume, that any file with any amount of data will fit into memory, or is there a neccessity to have say 16 or 17 files open at the same time while data is processed?
Last edited by ThomasJohnsen; Jul 23rd, 2012 at 12:34 PM.
In truth, a mature man who uses hair-oil, unless medicinally , that man has probably got a quoggy spot in him somewhere. As a general rule, he can't amount to much in his totality. (Melville: Moby Dick)
The sub below will parse lines on the form you specified, and extract all information. Nothing is saved to files though - you will have to implement that yourself or specify a bit more info. I have used heavily commenting to avoid having to explain here, what the sub does.
I'll say once again though, that using Regex isn't the most effective way, so if this is for extreme workloads or a commercial product, you may wan't to upgrade the performance by using something other than regular expressions.
vb Code:
Private Sub parse_file(ByVal sFileName As String)
Dim lines As String() = File.ReadAllLines(sFileName)
'Will parse lines of fields [<field>]+ (<opt_a>)? <<opt_b>>?
Console.WriteLine(String.Format("Line did not parse: '{0}'.", line))
End If
Next
End Sub
Regards Tom
Last edited by ThomasJohnsen; Jul 23rd, 2012 at 12:43 PM.
In truth, a mature man who uses hair-oil, unless medicinally , that man has probably got a quoggy spot in him somewhere. As a general rule, he can't amount to much in his totality. (Melville: Moby Dick)
I made a small example to get you started. I'll probably expand a bit upon it tonight, when I get a little time in front of the computer.
{Sub deleted - better version below. The one listed here initially had some minor flaws like including brackets with fields.}
A regex might be handy to split up the current line into fields, but using regular expressions is generally much less efficient than using the built-in string functions. I used them in the example because they make for short and easily readable code (and because you used regex'es yourself). But if you plan on using this code intensely, I suggest trying to make some optimizations.
Regards Tom
#EDIT: Forgot to mention: your explanation was much better this time around.
I'll try to expand the code a bit tonight.
And one last thing: is it safe to assume, that any file with any amount of data will fit into memory, or is there a neccessity to have say 16 or 17 files open at the same time while data is processed?
I'll play with your sub below shortly, but to answer your question about open files, there really should only be the need to have open at most 4 files
The file being written by the game
the main toon dump
the companion dump
the enemy dump
I think i'd be fine with all data not being main toon or companion being dumped to a single file, instead of a file for each enemy.
as it is with the 2 or 3 loops and one file being open it already seems to eat cpu and it grows the longer the app is run
I'm not sure I need them in memory other than using them as say a flat file to run say SQL queries against them to come up with some statistics and whatnot.
thanks a ton for this help, I'm hoping to grow my knowledge in this project and other than it just being a fun thing to do I am pretty sure a lot of people who play SWTor will find the app useful and that is good enough for me
One thing I need to point out is there will only be 1 toon and [after like 10 levels in game] one companion loaded at any given time. but there could be as many as 5 or 6 bad guys
Last edited by thequestor; Jul 23rd, 2012 at 01:40 PM.
Reason: forgot something
I've plugged it and passed a couple files through it, but it doesn't seem to really do anything
one thing is it was missing an end if but that was an easy fix, though I may have put it in the wrong place.
One positive thing came to light though. While I was playing around with your function I noticed something odd in my timer code
Code:
Private Sub Timer1_Tick(sender As System.Object, e As System.EventArgs) Handles Timer2.Tick
And when I fixed it to say Handles Timer1.Tick it dropped almost 10% cpu usage lol.
ok out of frustration with my lack of skills I started over
everything so far is good except I am stuck with a needed loop
Code:
Public Function GetToonName(ByVal WhichToon)
If WhichToon = 1 Then
Dim ToonInfo As String = ReadSpecifiedLine(Label6.Text, 1)
Dim str As String
Dim strArr() As String
Dim count As Integer
str = ToonInfo
strArr = str.Split(New Char() {"]"c})
For count = 0 To strArr.Length - 1
Try
Dim strValue
strValue = Replace(Replace(Regex.Replace(strArr(1), "\{.+?}]*?", ""), "@", ""), "[", "")
Label12.Text = strValue 'debug output
Label17.Text = Label12.Text
If Label7.Text = 0 Then
Label17.Text = "Loading"
End If
Catch Ex As Exception
LogConsole.Text = Ex.Message
End Try
Next
End If
If WhichToon = 2 Then
'this one is funky.
'I need to check for a : and if it doesn't have one, read the next line, and if it doesn't have one the next until it does
'up to line 20 and then give up and list as no companion
Dim ToonInfo As String = ReadSpecifiedLine(Label6.Text, 9 - 1)
Dim str As String
Dim strArr() As String
Dim count As Integer
str = ToonInfo
strArr = str.Split(New Char() {"]"c})
For count = 0 To strArr.Length - 1
Try
LogConsole.Text = strArr(1)
Dim strValue
strValue = Replace(Replace(Regex.Replace(strArr(1), "\{.+?}]*?", ""), "@", ""), "[", "")
Label13.Text = Replace(strValue, Label12.Text + ":", "") 'debug output
Label16.Text = Label13.Text
Catch Ex As Exception
LogConsole.Text = Ex.Message
End Try
Next
End If
' - strAryWords is now an array
Return 0
End Function
The first part works fine since the 1st line always returns the needed @ToonName
BUT @ToonName:CompanionName may be on the 2nd or the 20th line
I have tried a IF strValue.Contains(":") Then
and it does see the difference, but what I need to figure out is how to loop through lines 2 through say 20 until it finds a : and then use that
The code I am using to read specific lines in the text file is
Code:
Public Shared Function ReadSpecifiedLine(file As String, lineNum As Integer) As String
'create a variable to hold the contents of the file
Dim contents As String = String.Empty
'always use a try...catch to deal
'with any exceptions that may occur
Try
Dim logFileStream As New FileStream(file, FileMode.Open, FileAccess.Read, FileShare.ReadWrite)
Using stream As New StreamReader(logFileStream)
contents = stream.ReadToEnd().Replace(vbCr & vbLf, vbLf).Replace(vbLf & vbCr, vbLf)
Dim linesArray As String() = contents.Split(New Char() {ControlChars.Lf})
'Make sure we have ana ctual array
If linesArray.Length > 1 Then
'Make sure user didnt provide number greater than the number
'of lines in the array, and not less than 0 (zero) Thanks AdamSpeight2008
If Not lineNum > linesArray.Length AndAlso Not lineNum < 0 Then
Return linesArray(lineNum)
Else
'Failed our check so return the first line in the array
Return linesArray(0)
End If
Else
'No array so return the line
Return contents
End If
End Using
Catch Ex As Exception
Return Ex.ToString()
End Try
End Function
and it seems to work peachy
I have tried something along the lines of
Code:
IF NOT strArr(1).Contains(":") Then
Dim i
For i = 1 to 20
ToonInfo = ReadSpecifiedLine(Label6.Text, i -1)
Next
End If
and whilst it does loop through the specified number of lines [or till the end of file if there aren't 20 lines yet] I can't figure out to use it when it does find it and then stop the loop or if it never does find it just to list it as say "N/A"
I've plugged it and passed a couple files through it, but it doesn't seem to really do anything
Maybe I should've explained better. I designed it to do absolutely nothing - so apparently it works . It just parses the file and splits up every line into fields and elements - if you read through the comments, you can see exactly which variables contain the info you need. Then you can choose what to do with the info yourself (ie. save, ignore, use for calculations etc.). If the parser encounters an illegal line, it will output it to Console window - if you find such a line, don't hesitate to mention it here, since the parser may have to be modified to accomodate some lines, that contain info, you haven't mentioned here (all lines you included in this thread parses without problems).
I think the comments should explain everything, but if you have some questions, feel free to ask, and I'll try to explain best I can. I'll try to find the time tonight, to make a much faster version, but it will still only be a parser, into which you'll have to put your own data-handling. I can easily put in some data-handling of my own, but it will mostly be based on guesses about how you would like the info stored/used, since I don't know anything about neither the game nor the spells/events/abilities mentioned
The parser takes each line and strips out the relevant bit into 5 fields (enclosed in brackets) and 2 optional fields (enclosed in () and <>). The first field is parsed as a time with thousands included. The 2nd and 3rd fields are parsed as name-fields, containing either a player, a player controlled companion or an opponent (monster). The 4th field is parsed to be either empty or contain an effect with attached id. The 5th field is parsed as an event connected to an ability, both with id. The comments will explain exactly which field/variable contain which info in the current line.
Oh and one last thing - if I get the time to make a faster version tonight, it will be based on a series of assumptions about the data:
1) There are exactly 5 fields enclosed in [] in each line.
2) There may be 2 additional fields enclosed in () and <> in each line.
3) All fields (including optional) are seperated by exactly one space.
4) The 2nd and 3rd fields will allways be name-fields.
5) A name-field will allways be one of [@player], [@player:companion_{id}] or [monster_{id1}:id2] where _ denotes a space and no other whitespace is included in the fields.
6) The 4th field will allways be an effect-field.
7) An effect field will allways be one of [effect_{id}] or [], where _ again denotes a space and no other whitespace is included in the field.
8) The 5th field will allways be an event-field.
9) An event field will allways contain [event_{id1}:_ability_{id2}], where _ again denotes a space and no other whitespace is included in the field.
10) The optional field in () will be one of (), (amount) or (amount_descriptor_{id}), where _ again denotes a space and no other whitespace is included in the field (this may not be an optional field, since it's included in every line, but why they didn't enclose it in []'s like the other fields if that's the case is beyond me).
11) The optional field in <> will be one of <>, <amount>.
This is most definately one of the oddest formats I've ever seen. They use alot of enclosing that could (and should IMO) have been omitted, and they omit enclosing, when it would be really handy (for instance when dealing with names since spaces are used as seperators and may also be parts of a name). Either straight out seperation by for instance ';' or an XML format would definately be preferrable.
Regards Tom
In truth, a mature man who uses hair-oil, unless medicinally , that man has probably got a quoggy spot in him somewhere. As a general rule, he can't amount to much in his totality. (Melville: Moby Dick)
I have now made a much faster version. It can parse a single line at a time or an entire file, whichever you prefer. Besides that, I'll explain in the following post, since this post probably will exceed it's allowed length, if I add to much besides the code:
vb Code:
Private Sub parse_textfile(ByVal sFilename As String)
'Instead of reading all lines at the beginning of the sub, I chose to use a StremReader
'this time and read one line at the time. The lines are passed on to the sub parse_line,
'where they are parsed and the contents dealt with.
Console.WriteLine(String.Format("Illegal optional field 2: '{0}' in line '{1}'.", f_opt_1, sLine))
Exit Sub
End Try
'The contents of the current line (as formatted by the append methods) is now stored in sSavedata.
'You can use this to save to a file, write in a textbox or whatever you prefer.
'This sub just writes it to the console window.
Console.WriteLine(sSavedata.ToString)
End Sub
Regards Tom
Last edited by ThomasJohnsen; Jul 24th, 2012 at 02:40 PM.
Reason: Updated code with comments
In truth, a mature man who uses hair-oil, unless medicinally , that man has probably got a quoggy spot in him somewhere. As a general rule, he can't amount to much in his totality. (Melville: Moby Dick)
The sub parse_textfile should need no explanation.
The sub parse_line is where all the action is taking place:
1) The optional field in <>s is stripped away and stored.
2) The optional field in ()s is stripped away and stored.
3) The rest of the line is split into fields within []s.
From this point on the parsing begins. All data will be stored in a stringbuilder in the format of your choosing. This can later be used to save the data to a file, textbox or similar. All data in the stringbuilder is seperated by a pipe ('|') (this is my choosing, since you indicated a wish to use this as a seperator in an earlier post - you can change it to anything you like).
4) Verify that the line contains exactly 5 fields in []s.
5) Parse the time field and store it in the format Time:'<time>'+Thousands:'<thousands>'.
6) Parse the first name-field and store it as Companion:'<companion>', Player:'<player>' or Monster:'<monster>'.
7) Parse the second name-field and store it as Companion:'<companion>', Player:'<player>' or Monster:'<monster>'.
8) Parse the effect field and store it as Effect:'<effect>', unless it's empty, in which case nothing is added to the stringbuilder.
9) Parse the event field and store it as Event:'<event>',Ability:'<ability>'.
10) Parse the optional field in ()s and store is as (<amount>) if it contains an amount, (<amount>,<descriptor>) if it contains a descriptor and store nothing, if this field is non-existant or empty.
11) Parse the optional field in <>s and store it as <<amount>> if it contains an amount. If it is empty of non-existant nothing is saved to the stringbuilder.
All the formatting of fields is done with String.Format, which allows you to specify exactly in which way you prefer to have the values saved to the stringbuilder. For instance a companion is stored as Companion:'<companion>' due to the String.Format("|Companion:'{0}'", ...). If you for instance prefer a ';' as seperator char and the format C:<companion> just use String.Format(";C:{0}", ...). I have added a comment starting with '-> every place that a String.Format is called for a field, so you can quickly locate them and add your own format.
At present, the stringbuilder contents will just be written into a console window for every line parsed (so you can verify, that it works).
Regards Tom
#EDIT: You should note, that no rigorous testing of the code has been done, since I only had the few lines, you supplied as a testing base.
#EDIT2: Just browsed through your code. Your subs look alot like mine, so it shouldn't be too hard to understand, what's going on.
Your function ReadSpecifiedLine should be erased and never written again though - it will read the entire file and sort it into an array of lines only to return a single line. This means that reading a small 50k file with average line length of 50 chars will be the equivalent of reading a 50M file (or almost at least).
#EDIT3: An example of the data the code outputs presently:
as you can see, the effect field is sometimes empty due to odd content like [ {2531161666486272}]. I cannot say if this is intentional.
Last edited by ThomasJohnsen; Jul 24th, 2012 at 02:41 PM.
In truth, a mature man who uses hair-oil, unless medicinally , that man has probably got a quoggy spot in him somewhere. As a general rule, he can't amount to much in his totality. (Melville: Moby Dick)
I think I understand this, but it brought me to a new issue I can't remember how to get around.
Code:
Private Sub Form1_Load(sender As System.Object, e As System.EventArgs) Handles MyBase.Load
CheckLogDir() 'check that the game's log dir exists, if not ask to make it
CheckLogsDataDir() 'check that the apps's log dir exists, if not ask to make it
Timer1.Start() 'check for client running
RichTextBox1.Text = Me.IsSWTORRunning.Text.ToString
End Sub
'--------------------------------------------
Public Sub Timer1_Tick(sender As System.Object, e As System.EventArgs) Handles Timer1.Tick
Me.IsSWTORRunning.Text = IsProcessRunning("swtor")
End Sub
'--------------------------------------------
Public Function IsProcessRunning(name As String) As Boolean
'here we're going to get a list of all running processes on the computer
For Each clsProcess As Process In Process.GetProcesses()
If clsProcess.ProcessName.StartsWith(name) Then
'process found so it's running so return true
Return True
End If
Next
Return False
End Function
'--------------------------------------------
When I run timer1 it does what it should and returns a value to IsSWTORRunning.Text as it should, either true or false
but when I try to see the value of IsSWTORRunning.Text after the timer.start in the form1_load it doesn't see the value and just returns a value of IsSWTORRunning
how do I go about setting a variable in the timer that can be read anywhere after the timer is launched?
Once I get past this, I will plug in your awesome bit of code and see what awesome things I can do with it.
[I am way to stuck in classic asp mind frame so I have to unlearn all that bad to try to learn this not so bad]
I guess I am asking how do I set a variable in Function IsProcessRunning that can be read from anywhere in the form?
Ok I plugged it in and ran it, and it does seem to function, but I haven't had much time to kick the tires. But one thing it does seem to do is lock the form.
I do not know how to call it other than from one of my other functions which is run via a timer and as soon as your function is called and all kinds of info is being passed to the console in visual studio, it seems to go crazy and pretty much lock my form in place, the other timers still function [ie the clock and other little doodaddles I have running, but it seems this function is spewing too much info for the form to handle, since it is being called from the timer [2 times a second or 500 ms] it really seems to overload.
How can I use this in a sane manner?
Also due to the game locking the file I needed to add this
Dim logFileStream As New FileStream(sFilename, FileMode.Open, FileAccess.Read, FileShare.ReadWrite)
Dim f As New StreamReader(logFileStream)
I kinda figured you'd have to run this extensively, which is why I made a faster version. I imagine this will be running alongside the game (ie. it is supposed to read a logfile, as it is being generated). This is why I implemented the parse_line sub to handle single lines at a time. You need to be able to access only the most recently added lines in the logfile and parse only those - parsing the entire logfile twice every second will indeed output loads and loads of data (most of which is repetiotions).
You could implement a reader that keeps track of how many bytes have been read so far (and hopefully added to your own log-files). When the timer activtes, the reader only begins a parsing, if the logfile is larger than the recorded number of bytes. This could be implemented as a BGW. It could be wise to ignore small size changes and wait until a decent-sized chunk has been added before invoking parsing.
Which type of stream-reader you use should be completely irrelevant. As long as the parse_line gets a unicode string, it will do as expected.
If you aren't able to do the things described, you can start a new thread(s), and I'm sure there are many here (including myself) that are willing to help you.
When a thread gets to this length, not many reads it to the end, so you wont get many responses except from me (which can be annoying for you, since we are in entirely different time-zones).
Regards Tom
In truth, a mature man who uses hair-oil, unless medicinally , that man has probably got a quoggy spot in him somewhere. As a general rule, he can't amount to much in his totality. (Melville: Moby Dick)
Update, This code is great, but my ability to use it is really hampered by not knowing how to use it well
my app is run while the game is running and is always being updated 2 times a second. This code seems to parse the entire file on each read. So when I use it to write to a richtextbox [or other field] and use the onchange event to write it to a text file, it writes the whole text file, then the whole text file, etc, so In a matter of minutes the new text file is many megs in size with repeated info. how do I get around this, so it only reads / writes the last line of the logfile?
Also I have attached my combat log dir as a zip file so you can have all of the data to run against you could possibly need
In regards to the time zone thing it's actually perfect I tend to get most of my code time about 12am - 3 or 4am CST which is mid day for you no?
Also I find it easier to deal with one person than 20, so if it is no bother to you than I am content with working out my issues in a slow and methodical process with somebody who seems to know how to deal with newbies
Lot of changes this time. First I had to change the parser quite a bit. I got rid of the parse_textfile sub that you obviously aren't going to need. The parse_line sub has been changed in the following way:
1) The 2nd and 3rd field can now be empty (this was allowed in some of the logs you attached).
2) Amounts can now be postscripted by an asterisk, Amount* (but I haven't got a clue about what this means).
3) Instead of reporting errors to the console window, I changed the sub to a function returning True if parsing was successfull and False if parsing failed. This was neccessary to accomodate the rest of the changes.
The complete function is listed below:
vb Code:
Public Function parse_line(ByVal sLine As String) As Boolean
'The contents of the current line (as formatted by the append methods) is now stored in sSavedata.
'You can use this to save to a file, write in a textbox or whatever you prefer.
'This sub just writes it to the console window.
'Console.WriteLine(sSavedata.ToString)
Return True
End Function
--> This post will be continued in the following post.
In truth, a mature man who uses hair-oil, unless medicinally , that man has probably got a quoggy spot in him somewhere. As a general rule, he can't amount to much in his totality. (Melville: Moby Dick)
I added some controls to my form to allow for the changes:
1) A timer, logfile_timer, that has an interval of 1000 and is disabled at startup.
2) A background-worker, logfile_worker, that will handle all the parsing.
3) A progress bar, logfile_progress, to follow the progress of the parsing (this is not a neccessity and you can freely remove it, when you no longer need it).
The timer in 1) is the timer you call Timer1 in your code.
I added the following to the main windows load event (or another sub you use for initialization).
I also had to add some locals to the main window form:
Code:
Private logfile_name_and_path As String
Private errorfile_name_and_path As String
Private logfile_last_recorded_position As Long = -1
Private Const logfile_sizechange_before_parsing As Long = 1024
You need to set the logfile_name_and_path to the logfile, you currently wish to parse, and the errorfile_name_and_path to a file, in which parsing errors will be recorded. The constant logfile_sizechange_before_parsing ensures that we don't start the parser up for any small change but rather waits until a substantial amount of data has been added.
-----------------------------------------------------------------
Now for the events:
In the tick event for the timer from 1):
vb Code:
Private Sub logfile_timer_Tick(sender As Object, e As EventArgs) Handles logfile_timer.Tick
'Disable timer while we are parsing.
'This to prevent having 2 parsings running on the same data at the same time.
logfile_timer.Enabled = False
'Retrieve info about the logfile.
Dim logfile_info As FileInfo = FileIO.FileSystem.GetFileInfo(logfile_name_and_path)
'If the size of the logfile has grown in size by more than a specified amount, parse it.
If logfile_last_recorded_position + logfile_sizechange_before_parsing <= logfile_info.Length Then
Private Sub logfile_worker_progress(ByVal sender As Object, ByVal e As ProgressChangedEventArgs)
'You can display the progress, if you choose.
'For instance in a progress bar.
logfile_progress.Value = e.ProgressPercentage
End Sub
Private Sub logfile_worker_completed(ByVal sender As Object, ByVal e As RunWorkerCompletedEventArgs)
'Restart timer, since the worker is now done with the parsing, and a new parsing can begin.
logfile_timer.Enabled = True
logfile_timer.Start()
'Reset progress bar.
logfile_progress.Value = 0
End Sub
And that should be it. Every second the parsing will start if the logfile has increased in size by more than 1024 bytes. It will only parse the lines that have been added since last parsing. Nothing parsed is saved to file - you will have to implement that yourself in the parse_line sub at the very bottom
Regards Tom
#EDIT: Oh and forgot to mention - now is afternoon for me (4:30 pm). I tend to get most of my programming done in the evenings, but I wanted to get this done in a hurry, in case you were anxiously waiting for it
#EDIT2: The parser will parse roughly 500k in 0.1 secs (with reporting to progress bar and no saving of parsedata), but the background worker has a little startup overhead, so you will probably register it as a 0.5-1.0 secs. A parsing this large will ofc only occur if you have been playing a long time without having the parser running. If the parser is running continuously while you play, you shouldn't be able to register it running at all.
#EDIT3: I allways have to edit my post an awful lot of times. I blame old age
#EDIT4: You ofc have to enable and start the timer for parsing to begin (which I forgot to mention - old age again).
Last edited by ThomasJohnsen; Jul 25th, 2012 at 09:44 AM.
In truth, a mature man who uses hair-oil, unless medicinally , that man has probably got a quoggy spot in him somewhere. As a general rule, he can't amount to much in his totality. (Melville: Moby Dick)
1st off I want to thank you for putting your time and effort into MY issue, this says loads about you as a person. Thank you.
Right now the damn game servers are down AGAIN [rant for different day] so I am really unable to progress but I did have some time to play with this new code and for the life of me I can't get it to do anything at all . No errors, no output, no anything
I'm going to wrap a bunch of my code around a new variable [dim DamnServersDownAgain as Boolean = True] so I can continue to work with some of my last log files [right now I can't as It looks to both game client being running and a toon actually loaded before it does any of its things. sigh] And I am kinda of tired tonight so I may just do something crazy like go to bed before 4am.
And don't get me started on old, I feel 98 years old tonight when my body is only 1/2 that which puts me in the "not ancient, but not exactly young" category myself :P
ok it seems I missed an entire post and in it. seems to explain WHY I can't get it to do anything, at all!
Because I am missing 1/2 the code sigh. man I hate my brain.
Last edited by thequestor; Jul 26th, 2012 at 03:00 AM.
Reason: my brain is older than yours
man oh man, if you haven't already put some time into this I would have just given up trying to get this code into my form
nearly everything errors on insert. I'm going to try something instead. here is my whole project
That said one thing is already an issue even without trying to add background events and progress bars and timers and whatnot
You need to set the logfile_name_and_path to the logfile, you currently wish to parse, and the errorfile_name_and_path to a file, in which parsing errors will be recorded. The constant logfile_sizechange_before_parsing ensures that we don't start the parser up for any small change but rather waits until a substantial amount of data has been added.
you do know that the logfile_name will be different on each load and each subsequent changing of toon in game no? so there is no way I can give it a name until the form actually knows what the name is later?
there are days I just want to quit and today is one of them. GAH why can't I just download the knowledge like in the matrix movies. WHY WHY!!!
because this forum only has a 250k max for rars I am just tossing it on my website and the link to it is here
if you could plug in the places that need plugged in I can take it from there and see what I need to do to make it work for me.
I am a horrible coder but an ok hacker. no idea why, but I can mangle just about anything already working to do what I need but to make something from scratch is just SOOOOOOOOO alien
I will have a look at it tonight. Maybe I should try the game and have my code running beside the game on my computer to test it out under proper circumstances (this is ofc. assuming, that the game is free).
But there's no need to be gloomy - we will get this up and running before the weekend is over
Tom
Oh and by the way - the fact that logfiles may change during game is a problem atm., since I wasn't aware of that, but it's most certainly solveable with very few changes - so no biggie
In truth, a mature man who uses hair-oil, unless medicinally , that man has probably got a quoggy spot in him somewhere. As a general rule, he can't amount to much in his totality. (Melville: Moby Dick)
the game does have a free to level 15 thing going on so yes you can
it's a rather large download though [like 6 gigs]
kinda sucks for me since I had to pay 85 bucks for the damn game and now people get the client for free
I wrote and used a lot of time on the file name detection which basically just looks for and loads the newest file in the directory. I even had to write a little function to create a 0 byte file on app load to always reset the toon and companion names to n/a
But you will see that in my project vb
oh and the site is down right now as well as the game servers [don't get me started on that issue either please :P] but when they come back up [around 6AM CST] the url is http://www.swtor.com
you can get the client there
Last edited by thequestor; Jul 26th, 2012 at 04:49 AM.
Reason: too old for this stuff :P
the game does have a free to level 15 thing going on so yes you can
it's a rather large download though [like 6 gigs]
kinda sucks for me since I had to pay 85 bucks for the damn game and now people get the client for free
I wrote and used a lot of time on the file name detection which basically just looks for and loads the newest file in the directory. I even had to write a little function to create a 0 byte file on app load to always reset the toon and companion names to n/a
But you will see that in my project vb
oh and the site is down right now as well as the game servers [don't get me started on that issue either please :P] but when they come back up [around 6AM CST] the url is http://www.swtor.com
you can get the client there
...and here I thought this was some small niche game. How developers creating such a huge and great looking game, can make such an odd logfile design is beyond me. Perhaps they didn't intend for it to be parsed and read
Will download tonight and try my skillz with a lightsabre .
Tom
#EDIT: Damn this is taking a looooong time to download. For some reason the download only progresses at under an eighth of the speed of my connection - maybe the servers are busy. But downloading several GBs at 1.5MBs speed tops takes a while
#EDIT2: And now it's at 500k and stays steady there
Last edited by ThomasJohnsen; Jul 26th, 2012 at 01:00 PM.
In truth, a mature man who uses hair-oil, unless medicinally , that man has probably got a quoggy spot in him somewhere. As a general rule, he can't amount to much in his totality. (Melville: Moby Dick)
hey when you do sign up make sure to use "TheQuestor" as a referrer so I get my special speeder :P
Oh sorry - when you posted this I was asleep. The download stopped at 11:30 pm, and I tried it real quick (well - had to see the videos ofc), before heading to bed at midnight (had to be up at 4 am).
I have already created some smuggler dude with red skin.
If it's not to late, I will add you as referrer today .
Regards Tom
In truth, a mature man who uses hair-oil, unless medicinally , that man has probably got a quoggy spot in him somewhere. As a general rule, he can't amount to much in his totality. (Melville: Moby Dick)
Well I have now tried several sessions playing smuggler and sith (sith is by far the most fun - love the lightsabre and those horns).
I have made a small application that runs in the background.
It will let you specify a date with a datetimepicker, and depending on your pick, will show a list of possible combatlogs by time and player. This list will automatically be refreshed if you pick a new date or Starwars has started/stopped running.
You can click any comabtlog and choose to either parse it offline or online (requires Starwars to be running). An offline parsing will just take a combatlog and parse it to a textfile. Offline parsings report progress in a progressbar but cannot be cancelled (I did want to implement that, but of all the parsings I made, none lasted for more than ~half a second, so theres not much need - besides it's easy to implement - you can use the online parsing as a template if you want to try). An online parsing will not report progress, but a textbox indicates, that it's running (ie. text changes every time it parses). It can be cancelled with a press of a button. Online parsing will only activate when a certain amount of bytes has been added to the current combatlog - it then parses those and waits until another chunk is ready. You can specify the size of the chunk, but you probably don't wan't to choose too high a number, if you wan't to use this parser as a helping tool, that reports average damage or whatever.
A textbox will indicate if Starwars is running or not (updated on a 5 sec basis, so there may be a little delay).
All parsings go to textfiles in your documents named after the player and the time of the combatlog (similar to the system used for the actual combatlogs). All parse-errors go to an error-file in the same location. Another parsing of the same combatlog will overwrite the old parse-file. The error-file will never be erased, but allways appended to. I haven't tested performance of the parser, but based on the file-sizes I have tried, I estimate it to parse >200k per second.
And that's about it. Some of the last code got a bit messy (ie. not many comments), since I didn't have alot of freetime to code today and wanted to get this ready for you. Hopefully you'll be able to decipher it.
Drop me a PM on how you wan't it.
Regards Tom
In truth, a mature man who uses hair-oil, unless medicinally , that man has probably got a quoggy spot in him somewhere. As a general rule, he can't amount to much in his totality. (Melville: Moby Dick)
Hopefully you'll be able to use my code; either as is or as inspiration for your own project
Regards Tom
In truth, a mature man who uses hair-oil, unless medicinally , that man has probably got a quoggy spot in him somewhere. As a general rule, he can't amount to much in his totality. (Melville: Moby Dick)
what version of VS was this written in? It won't load in 2010
It was written in 2011 beta.
I'm afraid it's the only one I have atm. My 2010 Express is C# only.
I will download 2010 express VB and try to convert.
Meanwhile you can test the compiled version in Debug/bin.
Or you can download 2011 beta, if you wish.
Tom
#EDIT: That took a while - it wanted a couple of upgrades and a reboot. But here is a 2010 Express version Attachment 89875.
Last edited by ThomasJohnsen; Jul 29th, 2012 at 04:04 AM.
In truth, a mature man who uses hair-oil, unless medicinally , that man has probably got a quoggy spot in him somewhere. As a general rule, he can't amount to much in his totality. (Melville: Moby Dick)
looks like I have a few days of hacking ahead of me now. I like some of what I see a lot, some of what I see ok, and though it's really awesome what you have done in just a couple of days, there is a lot I want to add for the realtime display part Plus I need to rework where it places and handles the files. It also seems to filter out things like looting and stuff.
BUT what is there is more than I could have ever hoped to do myself and for this I am very grateful and it is going to be the basis of something really cool [not that it isn't already that]
You any good with DX9/10/11 overlays? :P
just kidding
ok not really kidding.
sorry about going on and on but I feel almost drunk after watching a real mindscrew movie called Chronicle. It's almost like a real live anime movie and actually pretty freakin awesome! Worth the price of the disk or rental for sure!
Anyway I am dead beat and need sleep, will begin the hack session later today
Never worked with it much.
Use XNA quite alot these days though, and am getting to be quite familiar with a subset of blender. Have allways been fascinated with linear algebra since even before university, and wrote my first Phong shader some 24 years ago. I was proud as hell and showed it off to friends/family - though none of them were particularly impressed by the red round thing on the screen with a glossy spot . The name Perlin, early Pixar, ILM and some of the early images from Cornell still give out good vibes
Anyways - please PM me in the likely event of questions/errors.
There's no need to keep reviving this thread + I do not subscribe and will probably respond quicker to a PM.
Regards Tom
In truth, a mature man who uses hair-oil, unless medicinally , that man has probably got a quoggy spot in him somewhere. As a general rule, he can't amount to much in his totality. (Melville: Moby Dick)
Reply With Quote
Originally Posted by ThomasJohnsen
.
