Increasing com port reading efficiency
So I just completed my first test of reading data coming through a com port (usb gps, using usb->com driver). However, I'd like to pick some minds about the actual process that may be better than what I am doing.
When I connect to my com port (hardcoded for now), I hook the data received event handler. Inside this method, I did a .ReadExisting() with output going to a listbox. I noticed that the com port is sending data very fast which is awesome and expected. I noticed that the data is not sent in complete lines.
I then made a class level stringbuilder and kept appending the .ReadExisting() strings to the stringbuilder. After each append, I attempted to check the new text of the stringbuilder for "GPGGA" and removed it. If already removed once, and found "GPGGA" again, we have a complete line. I realize this is not a very correct way to test for a complete line... After a complete line is found, I parse the data into a latitude/longitude (with some code help from codeproject). The lat/long code I'm using has splits, converstions to doubles, and more conversions back to strings...
After appending, testing, and parsing, I seem to get results about 1/sec. I know the gps puck sends complete lines more often than that. I also know that by setting a breakpoint after a while, the stringbuilder is appending faster than it's tested/parsed.
I've read a couple places that a good way is to read the incoming received data into a byte buffer and test that, having a separate method to actually do the parsing. I don't really want to have a thread constantly running just waiting to parse data. However, a constantly running thread to do the testing and parsing might not be a bad idea.
The only problem, I don't know how to work with bytes, or the .Read() with arguments for a byte,count,etc. Does anyone have any good leeway for a more efficient way of going about this?
Re: Increasing com port reading efficiency
You need to know the protocol that your serial device use... Once you have that, you can determine what the 'record' terminator is and go after that.... It could be a fixed number of bytes, it could be a special character such as EOT... You can set you serial port receive threshold to x bytes, and read the bytes into a buffer. Test if you have collected enough bytes in the buffer to form a record and if you do, process that record, remove only the processed byte from the buffer and leaving the incomplete one untouched. You definitely will need to synclock the buffer so that when you read it, the adding has to wait and vice versa...
Re: Increasing com port reading efficiency
The buffer is the most versatile and efficient way to go.
There's nothing wrong with using a thread - remember, the DataRecieve method runs on a separate thread.
Re: Increasing com port reading efficiency
Here are some ideas. I don't like the string concatenation but since stringbuilder doesn't have an indexof you would need to use .ToString. The DataReceived event reads the data and adds it to the stream. The checker thread looks for complete sentences and adds them to a list that is processed in a timer that runs every .10 secs. The timer is on the UI so updating controls is easy.
How fast is the GPS connection? 9600, 19200...
Code:
Private Sub SerialPort1_DataReceived(sender As Object, _
e As IO.Ports.SerialDataReceivedEventArgs) _
Handles SerialPort1.DataReceived
Threading.Monitor.Enter(dataStrmLock)
dataStrm &= SerialPort1.ReadExisting
Threading.Monitor.Exit(dataStrmLock)
checkIt.Set()
End Sub
Dim checkIt As New Threading.AutoResetEvent(False)
Dim dataStrm As String
Dim dataStrmLock As New Object
Dim sentences As New List(Of String)
Dim sentenceLock As New Object
Public Sub checker()
Dim found As Boolean = False
Do
checkIt.WaitOne()
Do
found = False
Threading.Monitor.Enter(dataStrmLock)
Dim sPos As Integer = dataStrm.IndexOf("$")
Dim ePos As Integer
If sPos >= 0 Then
ePos = dataStrm.IndexOf(ControlChars.CrLf, sPos)
If ePos > sPos Then
Threading.Monitor.Enter(sentenceLock)
sentences.Add(dataStrm.Substring(sPos, ePos - sPos))
Threading.Monitor.Exit(sentenceLock)
found = True
dataStrm = dataStrm.Remove(0, ePos + 2)
End If
End If
Threading.Monitor.Exit(dataStrmLock)
Loop While found
Loop
End Sub
Private Sub Timer1_Tick(sender As Object, e As EventArgs) Handles Timer1.Tick
Do While sentences.Count > 0
Threading.Monitor.Enter(sentenceLock)
Dim aSentence As String = sentences(0)
sentences.RemoveAt(0)
Threading.Monitor.Exit(sentenceLock)
'process the sentence here
Debug.WriteLine(aSentence)
Loop
End Sub
Private Sub Form1_Shown(sender As Object, e As EventArgs) Handles Me.Shown
Dim t As New Threading.Thread(AddressOf checker)
t.IsBackground = True
t.Start()
Timer1.Interval = 100
Timer1.Start()
End Sub
Re: Increasing com port reading efficiency
Quote:
How fast is the GPS connection? 9600, 19200...
I honestly must've overlooked that fact in my testing. I guess by default it was connecting at 9600. I thought for some reason that number was how fast the serial unit updates through the com port (not thinking it could be set via the com port). The unit itself states 4800 - 115200 (1hz - 5hz).
Quote:
You need to know the protocol that your serial device use
Does this just make it 'seamless' to do. It can be done efficiently without knowing the protocol right?
Quote:
The buffer is the most versatile and efficient way to go.
As with the above, does the buffer only really work if we know the protocol and the byte amounts? Or is this something we can set and keep checking the incoming and be more efficient than using strings/etc? Remember, i've yet to work with bytes, but started reading up on them...
dbasnett - thanks for the example. Some threading terms I've not yet heard of or used (ie: Threading.Monitor). Doing some research on them. The example seems like how I imagined it should work, maybe I'll give something to that effect a try.
The gps unit is a gm-2 iblue (apparently disco'd for a while). I've had it about 2 years and just started toying with it. Cheers!
Re: Increasing com port reading efficiency
Quote:
Originally Posted by
detlion1643
I honestly must've overlooked that fact in my testing. I guess by default it was connecting at 9600. I thought for some reason that number was how fast the serial unit updates through the com port (not thinking it could be set via the com port). The unit itself states 4800 - 115200 (1hz - 5hz).
Does this just make it 'seamless' to do. It can be done efficiently without knowing the protocol right?
As with the above, does the buffer only really work if we know the protocol and the byte amounts? Or is this something we can set and keep checking the incoming and be more efficient than using strings/etc? Remember, i've yet to work with bytes, but started reading up on them...
dbasnett - thanks for the example. Some threading terms I've not yet heard of or used (ie: Threading.Monitor). Doing some research on them. The example seems like how I imagined it should work, maybe I'll give something to that effect a try.
The gps unit is a gm-2 iblue (apparently disco'd for a while). I've had it about 2 years and just started toying with it. Cheers!
If it is running at those slow speeds your program should not have trouble keeping up.
Re: Increasing com port reading efficiency
I'm just going to put this out there in case it might help anyone else. This is a snippet of what my gps puck was returning. I eyed the lines to notice that \r\n would be the line ending/beginning.
Code:
$GPGSA,A,3,31,13,29,20,32,06,30,16,23,03,,,1.14,0.83,0.79*0B\r\n
$GPGSV,3,1,11,30,80,059,18,16,66,206,33,23,48,308,19,31,44,069,16*72\r\n
$GPGSV,3,2,11,51,34,217,28,20,31,245,19,32,31,215,29,06,22,157,22*75\r\n
$GPGSV,3,3,11,13,21,312,19,29,16,045,15,03,12,175,20*48\r\n
$GPRMC,010337.000,A,4204.300438,N,08006.947251,W,0.19,135.61,080213,,,D*73\r\n
$GPGGA,010338.000,4204.300398,N,08006.947227,W,2,10,0.83,281.632,M,-33.777,M,0000,0000*58\r\n
$GPGSA,A,3,31,13,29,20,32,06,30,16,23,03,,,1.14,0.83,0.79*0B\r\n
$GPGSV,3,1,11,30,80,059,18,16,66,206,33,23,48,308,19,31,44,069,16*72\r\n
$GPGSV,3,2,11,51,34,217,28,20,31,245,19,32,31,215,30,06,22,157,22*7D\r\n
$GPGSV,3,3,11,13,21,312,20,29,16,045,15,03,12,175,21*43\r\n
$GPRMC,010338.000,A,4204.300398,N,08006.947227,W,0.02,135.61,080213,,,D*7A\r\n
$GPGGA,010339.000,4204.300409,N,08006.947231,W,2,10,0.83,281.638,M,-33.777,M,0000,0000*5B\r\n
$GPGSA,A,3,31,13,29,20,32,06,30,16,23,03,,,1.14,0.83,0.79*0B\r\n
$GPGSV,3,1,11,30,80,059,18,16,66,206,33,23,48,308,19,31,44,069,17*73\r\n
$GPGSV,3,2,11,51,34,217,28,20,31,245,20,32,31,215,30,06,22,157,23*76\r\n
Now, I've been looking through some current parsers and what each piece of the lines mean. I still have to figure that part out, but everyone in this thread has been a great help so far.
Re: Increasing com port reading efficiency
I'm probably just nitpicking now, but does it make any sense to fire an event for each complete line/sentence found? Or is that not efficient with how much it will be firing?
I have set up the parsing part already and fire events based on each 'piece' of information parsed.
Re: Increasing com port reading efficiency
If you are using something like I posted in post#4 you don't need to fire an event. When you have a sentence the first thing to do is to calculate the checksum and check that it is the same as what is in the sentence.
Re: Increasing com port reading efficiency
Quote:
If you are using something like I posted in post#4 you don't need to fire an event.
That's why I mentioned I'm probably just nitpicking now. Timer, thread, event? I guess at the point in which you get a complete sentence, how to begin parsing sounds like it doesn't really matter which method is chosen.
Re: Increasing com port reading efficiency
Quote:
Originally Posted by
detlion1643
I'm just going to put this out there in case it might help anyone else. This is a snippet of what my gps puck was returning. I eyed the lines to notice that \r\n would be the line ending/beginning.
Code:
$GPGSA,A,3,31,13,29,20,32,06,30,16,23,03,,,1.14,0.83,0.79*0B\r\n
$GPGSV,3,1,11,30,80,059,18,16,66,206,33,23,48,308,19,31,44,069,16*72\r\n
$GPGSV,3,2,11,51,34,217,28,20,31,245,19,32,31,215,29,06,22,157,22*75\r\n
$GPGSV,3,3,11,13,21,312,19,29,16,045,15,03,12,175,20*48\r\n
$GPRMC,010337.000,A,4204.300438,N,08006.947251,W,0.19,135.61,080213,,,D*73\r\n
$GPGGA,010338.000,4204.300398,N,08006.947227,W,2,10,0.83,281.632,M,-33.777,M,0000,0000*58\r\n
$GPGSA,A,3,31,13,29,20,32,06,30,16,23,03,,,1.14,0.83,0.79*0B\r\n
$GPGSV,3,1,11,30,80,059,18,16,66,206,33,23,48,308,19,31,44,069,16*72\r\n
$GPGSV,3,2,11,51,34,217,28,20,31,245,19,32,31,215,30,06,22,157,22*7D\r\n
$GPGSV,3,3,11,13,21,312,20,29,16,045,15,03,12,175,21*43\r\n
$GPRMC,010338.000,A,4204.300398,N,08006.947227,W,0.02,135.61,080213,,,D*7A\r\n
$GPGGA,010339.000,4204.300409,N,08006.947231,W,2,10,0.83,281.638,M,-33.777,M,0000,0000*5B\r\n
$GPGSA,A,3,31,13,29,20,32,06,30,16,23,03,,,1.14,0.83,0.79*0B\r\n
$GPGSV,3,1,11,30,80,059,18,16,66,206,33,23,48,308,19,31,44,069,17*73\r\n
$GPGSV,3,2,11,51,34,217,28,20,31,245,20,32,31,215,30,06,22,157,23*76\r\n
Now, I've been looking through some current parsers and what each piece of the lines mean. I still have to figure that part out, but everyone in this thread has been a great help so far.
Collecting data is one thing and making that data useful is another. To accurately parse a "sentence", you need to know what each piece of information means. You can only guess so much, and if you use these values for further calculations, it won't take much to steer your results way off the real target. If possible, I suggest to contact the device manufacturer to see if they can provide any documentation for you...
Re: Increasing com port reading efficiency
NMEA is an industry standard, is it not? The amount of info in each 'piece' might differ, but isn't the layout of each sentence the same.
IE: Split sentence at comma, each result is a certain piece of info. Result(1) is always latitude, Result(2) is always longitude, and so forth, no matter how much is contained within those Results. I know 1 and 2 aren't lat/long, it's an example.
I've been reading up on the contents of NMEA strings and it seems to point to this at least.
Re: Increasing com port reading efficiency
Quote:
Originally Posted by
detlion1643
NMEA is an industry standard, is it not? The amount of info in each 'piece' might differ, but isn't the layout of each sentence the same.
IE: Split sentence at comma, each result is a certain piece of info. Result(1) is always latitude, Result(2) is always longitude, and so forth, no matter how much is contained within those Results. I know 1 and 2 aren't lat/long, it's an example.
I've been reading up on the contents of NMEA strings and it seems to point to this at least.
Check this out
http://home.mira.net/~gnb/gps/nmea.html#top
Re: Increasing com port reading efficiency
Here are some classes that I used
Code:
Public Class GPSSystemFixData
Property wasError As Boolean
'$GPGGA - Global Positioning System Fix Data
Property fixTaken As DateTime '123519 UTC
Property clockDiff As TimeSpan
Property clockWithinSecond As Boolean
Property Latitude As Decimal '4807.038
Property NS As String 'N
Property Longitude As Decimal '01131.000
Property EW As String 'E
Property fixQuality As Integer '0 = invalid, 1 = GPS fix (SPS), 2 = DGPS fix, 3 = PPS fix, 4 = Real Time Kinematic, 5 = Float RTK
' 6 = estimated (dead reckoning) (2.3 feature), 7 = Manual input mode, 8 = Simulation mode
Property numSatellitesTracked As Integer
Property HDOP As Decimal 'Horizontal dilution of position
Property Altitude As Decimal 'Altitude, in Meters, above mean sea level
Property geoidHeight As Decimal 'Height, in Meters, of geoid (mean sea level) above WGS84 ellipsoid
End Class
Public Class GPSActiveSatelliteDOP
Property wasError As Boolean
'$GPGSA - GPS DOP and active satellites
Property AutoManual As String
Property mode As String
Property PRN As New List(Of String)
Property PDOP As Decimal
Property HDOP As Decimal
Property VDOP As Decimal
End Class
Public Class GPSTransit
Property wasError As Boolean
'$GPRMC - Recommended minimum specific GPS/TRANSIT data
Property fixTaken As DateTime '123519 UTC
Property positionValid As Boolean
Property Latitude As Decimal '4807.038
Property NS As String 'N
Property Longitude As Decimal '01131.000
Property EW As String 'E
Property speed As Decimal 'knots
Property course As Decimal
Property variation As Decimal
Property variationEW As String
Property modeIndicator As String 'Mode indicator, (A=Autonomous, D=Differential, E=Estimated, N=Data not valid)
End Class
Public Class GPSSatellitesInView
Property wasError As Boolean
Property numOfSatellites As Decimal
Property PRN As String
Property elevation As Double
Property azimuth As Double
Property SNR As Double
End Class
As I recall I processed the sentences and updated instances of these classes in the background thread. In a timer that ran 10 times per second I updated a form with the data.
