Program hang when running outside of VS

[stanav]

Hi,
I hope someone can help explain this very weird behavior that I'm running into. I have a small program that communicates with a testing device over the serial port. The program will send some commands to the tester and receive some test data back. That test data is saved to a database and displayed on the UI. During debugging in Visual Studio 2015, everything works correctly as designed. However, when the program is released and tried to run as a standalone app, it will hang when trying to update the UI with the test data (the test data received is all valid and successfully saved to the database, but the program hangs when trying to display that data on the UI when calling "PopulateForm"). I'm stuck and unable to determine the cause of this behavior.
In summary:
- Program runs properly during debugging in VS
- It will hang when running outside of VS
- Narrow down to this line when it gets stuck: Me.BeginInvoke(New UpdateUI_Delegate(AddressOf PopulateForm))
- When the execution reaches that code line, the program becomes unresponsive and needs to be killed via task manager.

What I have tried:
- Rewrite that code line using different ways, i.e. Me.BeginInvoke(Sub() PopulateForm())... This made no difference
- Tried using Invoke instead of BeginInvoke: made no difference.
- Spinning off another thread to perform the PopulateForm task - also made no difference
- Set Form.CheckForIllegalCrossThreadCalls = False then call PopulateForm() without Invoke or BeginInvoke: this works but I try to avoid it if there is a more proper way to do it.

Here is the relevant code:

Private Delegate Sub UpdateUI_Delegate()

 Private Sub RS232_DataReceived(sender As Object, e As SerialDataReceivedEventArgs) Handles RS232.DataReceived
        If Not (e.EventType = SerialData.Eof) Then
            Dim incomingData = Me.ReadIncomingSerialData()
            If incomingData IsNot Nothing Then
                ProcessIncomingSerialData(incomingData.ToArray)
            End If
        End If
    End Sub

 Private Sub ProcessIncomingSerialData(ByVal rx_buf As Byte())
        Me.StatusUpdate("Process incoming serial data...")

        Dim x As Integer = rx_buf.Length
        Dim y As Integer = 0
        Dim crc As Byte = 0

        'code removed....

            Select Case rx_buf(0)   
                Case UPLOAD_LOG_DATA_COMMAND 

                    Me.StatusUpdate("Received UPLOAD_LOG_DATA_COMMAND")

                   'code removed...

                    Me.StatusUpdate("Reply_To_ATE(ASCII_ACK)")
       
                Case REQUEST_EE_STRING_COMMAND

                    Me.StatusUpdate("Received REQUEST_EE_STRING_COMMAND")

                   ' code removed....
                 
                Case SAVE_RECORD_COMMAND

                    Me.StatusUpdate("Received SAVE_RECORD_COMMAND")
                    ' Save data record to database
                    '         
                    If SaveDataRecords(ErrorMessage) Then
                        Me.StatusUpdate(String.Format("Data records for serial number '{0}' saved to database.", Me.lblSerialNum.Text), Color.Blue)
                        
                        Me.BeginInvoke(New UpdateUI_Delegate(AddressOf PopulateForm))

                        Reply_To_ATE(ASCII_ACK)
                        Me.StatusUpdate("Reply_To_ATE(ASCII_ACK)")

                    Else
                        Reply_To_ATE(ASCII_NACK)
                        Me.StatusUpdate("Reply_To_ATE(ASCII_NACK)", Color.Red)
                    End If

                Case UPLOAD_EE_STRING_COMMAND
                    Me.StatusUpdate("Received UPLOAD_EE_STRING_COMMAND")
                   
                   'code removed

            End Select
        Else
            Reply_To_ATE(ASCII_NACK)
            Me.StatusUpdate("Reply_To_ATE(ASCII_NACK): the value of CRC is incorrect.", Color.Red)
        End If

    End Sub

 Sub PopulateForm()

        Me.StatusUpdate("Populating form...")
        Try
            Me.lblTestDate.Text = ATEData.TestDate.ToString("G")
            Me.lblSerialNum.Text = ATEData.SerialNumber.Substring(0, SN_LENGTH)
            Me.txtVoltageSupply.Text = ATEData.Supply_Voltage.ToString("F3")
            Me.txtVoltageRawMin.Text = ATEData.MinRawTreadleVoltage.ToString("F3")
            Me.txtVoltageRawMax.Text = ATEData.MaxRawTreadleVoltage.ToString("F3")
            Me.txtVoltageRawDelta.Text = (ATEData.MaxRawTreadleVoltage - ATEData.MinRawTreadleVoltage).ToString("F3")
            Me.txtVoltageMVMMax.Text = ATEData.MaxNVMTreadleVoltage.ToString("F3")
            Me.txtVoltageNVMMin.Text = ATEData.MinNVMTreadleVoltage.ToString("F3")
            Me.txtMotorLoadLow.Text = ATEData.Low_Motor_I.ToString("F1")
            Me.txtMotorLoadMed.Text = ATEData.Med_Motor_I.ToString("F1")
            Me.txtMotorLoadHigh.Text = ATEData.High_Motor_I.ToString("F1")
            Me.txtVoltagePBReleased.Text = ATEData.PB_Released_Voltage.ToString("F3")
            Me.txtVoltagePBPressed.Text = ATEData.PB_Pressed_Voltage.ToString("F3")
            Me.txtLEDLoadX.Text = ATEData.LoadCurrentX.ToString("F1")
            Me.txtLEDLoadY.Text = ATEData.LoadCurrentY.ToString("F1")
            Me.txtLEDLoadZ.Text = ATEData.LoadCurrentZ.ToString("F1")
            Me.rtbNVMData.Text = Module1.ConvertToHexString(ATEData.NVMData)
            Me.txtCRCHeaderNVM.Text = Conversion.Hex(CRC_Data.NVM_Header)
            Me.txtCRCModuleNVM.Text = Conversion.Hex(CRC_Data.NVM_Module)
            Me.txtCRCHeaderCalculated.Text = Conversion.Hex(CRC_Data.Calculated_Header)
            Me.txtCRCModuleCalculated.Text = Conversion.Hex(CRC_Data.Calculated_Module)
            Me.dgvATETestResults.DataSource = Me._dsLinemasterSoftware.Tables("TableName")
            Me.txtSerialNumber.Select()
            Me.StatusUpdate("Finished populating form.")
        Catch ex As Exception
            Me.StatusUpdate("Failed to populate form. " & ex.Message, Color.Red)
        End Try

    End Sub

[sapator]

All I can think of is that since BeginInvoke run asynchronously, in the debug you give it time to complete and execute correctly while on runtime it goes instantly to the values without been able to get them.
If there is a BeginInvoke(async method or can be turned to taskof it would be preferred as you could control it with an await.
Having said that, I'm speculating , the issue might be elsewhere.

[stanav]

The values are readily available at the time PopulateForm is called because I can verify that they were saved to the database. The same behavior occurs if I use Invoke (non-asynchronous) instead. However, if I set the Form.CheckForIllegalCrossThreadCalls to False and call the PopulateForm directly then it works fine. Thus, I narrowed this down to the BeginInvoke or Invoke call but was unable to dig any deeper.

[dbasnett]

There is a lot missing from the code posted. Like what is ATEData and where is it being populated? What is Me.StatusUpdate? In the data being received can you have a x1A character?

Try your beginginvoke like this,

        Me.BeginInvoke(Sub() PopulateForm())

[stanav]

@dbasenett:
- The removed code was intended since it was irrelevant to the issue.
- Me.StatusUpdate is just a sub to update the UI letting the user know the progress of the current test, which is working fine. There is no issue with the data received either. The problem comes in when I try to display that data to the UI. And it is only problematic when the program runs outside of VS.
- I already tried what you suggested before posting this, and it made no difference.
- I also mentioned that if I set the Form.CheckForIllegalCrossThreadCalls = False then it works fine, but this is hacking and I try to avoid this.

Any other suggestions?

[dbasnett]

@dbasenett:
- The removed code was intended since it was irrelevant to the issue.
- Me.StatusUpdate is just a sub to update the UI letting the user know the progress of the current test, which is working fine. There is no issue with the data received either. The problem comes in when I try to display that data to the UI. And it is only problematic when the program runs outside of VS.
- I already tried what you suggested before posting this, and it made no difference.
- I also mentioned that if I set the Form.CheckForIllegalCrossThreadCalls = False then it works fine, but this is hacking and I try to avoid this.

Any other suggestions?

So Me.StatusUpdate is also checking for Me.InvokeRequired? You said,"Narrow down to this line when it gets stuck: Me.BeginInvoke(New UpdateUI_Delegate(AddressOf PopulateForm))", and I find that hard to believe since the BeginInvoke documentation says, "The delegate is called asynchronously, and this method returns immediately."

Looking at your code I suspect that you could have PopulateForm running simultaneously though using Me.Invoke should have cured that.

How much data do you expect when you Me.ReadIncomingSerialData? The code I'd most like to see is Me.ReadIncomingSerialData.

What version of .Net Framework are you using?

[stanav]

@dbasenett:
- Yes, StatusUpdate also checking for Me.InvokeRequired since it can be called from a different thread other than the main UI thread.

 Public Sub StatusUpdate(ByVal message As String, Optional ByVal textColor As Color = Nothing)
        If Me.rtbStatus.InvokeRequired Then
            Me.rtbStatus.Invoke(Sub() Info_Update(message, textColor))
        Else
            Info_Update(message, textColor)
        End If
    End Sub

    Private Sub Info_Update(ByVal message As String, Optional ByVal textColor As Color = Nothing)   
        'Clear status if has more than 100 lines
        If rtbStatus.Lines.Count > 100 Then
            Me.rtbStatus.Clear()
        End If

        Dim dte As String = Date.Now.ToString("G")
        Dim line As String = String.Format("{0} : {1}{2}", dte, message, Environment.NewLine)
        With Me.rtbStatus
            .SelectionStart = 0
            .SelectionLength = 0
            .SelectedText = line
            .Select(0, line.Length)
            If textColor = Nothing Then
                textColor = .ForeColor
            End If
            .SelectionColor = textColor
        End With

    End Sub

- Incoming data varies but is expected to be between 75 to 128 bytes.

 Private Function ReadIncomingSerialData() As List(Of Byte)
        Dim rx_buf As New List(Of Byte)
        Dim x As Integer = 0
        Dim crc As Byte = 0
        Dim byte_count As Byte = 3

        ' Receive the whole data stream
        While x < byte_count
            rx_buf.Add(RS232_ReadByte(1000))
            If (rx_buf.Item(0) >= REQUEST_PRESENCE And rx_buf.Item(0) <= UPLOAD_LOG_DATA_COMMAND) Then
                If (x = 1) Then
                    byte_count = byte_count + rx_buf.Item(x)
                End If
            Else
               ' Corrupted or incomplete data. Read all existing and discard it.
                RS232.ReadExisting()
                Return Nothing
            End If
            x = x + 1
        End While
        Return rx_buf
    End Function

- The program targets .Net Framework 4.5.2

Thanks.

[dbasnett]

And this code RS232_ReadByte(1000)? In the data received event handler you have this, If Not (e.EventType = SerialData.Eof) Then. Why? Does the device send x1A for control purposes?

Are you expecting three byte messages?

Does the device transmit continuously or is it polled?

"It will hang when running outside of VS" - my best guess is that you have code that is running on the UI that Invokes other code. Create a form with a button and label with default names and add this,

    Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
 
        'This is an example of a deadlock
        '  because the Button1.Click code IS running on the UI and 
        '  the task is waited

        Dim t As Task
        t = Task.Run(Sub()
                         Me.Invoke(Sub()
                                       Label1.Text = "*"
                                   End Sub)
                     End Sub)
        t.Wait()
        Stop 'will never reach here
    End Sub

[dbasnett]

What follows will not be a surprise to long term members. My first serial port program in .Net (2.0) was to interpret GPS NMEA sentences. The device I had transmitted sentences on a continuous basis. Over the years I have refined my thinking about how the serial port should be dealt with, and I always assume the device at the other end will be transmitting continuously. The code might be overkill for devices that have to be polled.

The basic structure is
Get all bytes from port
Create protocol messages
Process protocol messages

This is accomplished using three tasks. I do know that the code is different that what is normally presented.

The sample program connects to a device that transmits data continuously at 400,000 bps. What I know is that this code is keeping up with the device meaning the UI is updating and not freezing. I have let it run for over 200,000 messages. The CPU is averaging about 2% with 250 - 400MB of memory. My opinion is that this pattern should be followed for all serial port apps, it has always worked for me.

The code needs a form with one button, two labels, and a richtextbox. Use default names.

Public Class Form1
#Region "Start / form stuff"
    Private Sub Form1_Shown(sender As Object, e As EventArgs) Handles Me.Shown
        OpenPort()
    End Sub

    Private Sub Form1_FormClosing(sender As Object, e As FormClosingEventArgs) Handles Me.FormClosing
        'close is done this way so UI activity is completed
        '  i.e. PopulateForm
        Static first As Boolean = True
        If first Then
            first = False
            DoClose()
            e.Cancel = True 'cancel this close
        End If
    End Sub

    Private Async Sub DoClose()
        ClosePort()
        Dim tsk As Task = Task.Run(Sub()
                                       While Threading.Interlocked.Read(closedCT) < 3L
                                           Threading.Thread.Sleep(25)
                                       End While
                                   End Sub)
        Await tsk
        Me.Close()
    End Sub

    Private WithEvents RS232 As New IO.Ports.SerialPort 'the port
    Private isStop As New Threading.ManualResetEvent(False) 'false until ClosePort
    Private closedCT As Long = 0L 'count of task endings
    Private readTask As Task 'task to accumulate bytes from port
    Private protoTask As Task 'task to enforce protocol
    Private procMessTask As Task 'task to process messages

    Private Sub OpenPort()
        'start tasks
        readTask = Task.Run(Sub() ReadRS232()) 'task to read data
        protoTask = Task.Run(Sub() GetMessages()) 'task that knows protocol
        procMessTask = Task.Run(Sub() ProcMessages()) 'task that knows messages
        'settings as needed
        RS232.PortName = "COM4"
        RS232.DataBits = 8
        RS232.StopBits = IO.Ports.StopBits.One
        ' RS232.BaudRate = 9600 'my device is USB attached so this has no meaning
        RS232.Open()
        RS232.DtrEnable = True
    End Sub

    Private Sub ClosePort()
        isStop.Set() 'so the tasks stop
        haveData.Set() 'pretend we have data
        RS232.Close() 'close the port
    End Sub

    Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
        ''  send data code if needed

    End Sub
#End Region

#Region "serial port message processing"
    Private haveData As New Threading.AutoResetEvent(False) 'DataReceived event fired
    Private protoCheck As New Threading.AutoResetEvent(False) 'check protocol control
    Private haveMessages As New Threading.AutoResetEvent(False) 'message control
    Private inBuffLock As New Threading.AutoResetEvent(True) 'lock for buffer access
    Private inBuff As New List(Of Byte) 'accumulated bytes from port
    Private MessageQ As New Concurrent.ConcurrentQueue(Of MyMessage)

    Private Sub RS232_DataReceived(sender As Object,
                                    e As IO.Ports.SerialDataReceivedEventArgs) Handles RS232.DataReceived
        'nothing else in this method, fire and forget
        haveData.Set()
    End Sub

    Private Sub ReadRS232()
        'reads bytes and adds to buffer
        'no changes needed
        While Not isStop.WaitOne(0)
            haveData.WaitOne() 'see RS232_DataReceived
            If isStop.WaitOne(0) Then
                protoCheck.Set()
                Exit While
            End If
            'read all data available
            Try
                Dim br As Integer = RS232.BytesToRead 'num bytes to read
                Dim buf(br - 1) As Byte
                br = RS232.Read(buf, 0, br) 'read returns num bytes read
                If br <> buf.Length Then 'rarely but have seen it happen
                    Array.Resize(buf, br)
                End If
                inBuffLock.WaitOne() 'lock
                inBuff.AddRange(buf)
                inBuffLock.Set() 'unlock
                protoCheck.Set() 'wake GetMessages up
            Catch ex As Exception
                'hmmmmmm
            End Try
        End While
        Threading.Interlocked.Increment(closedCT)
    End Sub

    Private Sub GetMessages() 'protocol
        While Not isStop.WaitOne(0)
            protoCheck.WaitOne()
            ' method ReadRS232 has signalled

            ' your protocol code >>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>>
            ' changes needed based on protocol
            ' sample made up stuff
            Const maxSZ As Integer = 14
            While inBuff.Count >= maxSZ
                If isStop.WaitOne(0) Then Exit While
                Dim msg As List(Of Byte)
                Dim msgSZ As Integer = 0
                inBuffLock.WaitOne() 'lock
                Select Case inBuff(0)
                    Case Is <= 64
                        msgSZ = 4
                    Case Is <= 128
                        msgSZ = 8
                    Case Is <= 240
                        msgSZ = 10
                    Case Else
                        msgSZ = maxSZ
                End Select
                If msgSZ > 0 AndAlso msgSZ <= maxSZ Then
                    msg = inBuff.GetRange(0, msgSZ)
                    inBuff.RemoveRange(0, msgSZ)
                    MessageQ.Enqueue(New MyMessage(msg))
                End If
                inBuffLock.Set() 'unlock
            End While
            haveMessages.Set()
            ' your protocol code <<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<<
        End While
        haveMessages.Set()
        Threading.Interlocked.Increment(closedCT)
    End Sub

    Private Class MyMessage
        Public message As String = ""
        Public length As Integer = 0
        Private Shared messCTR As Long = 1L
        Public messageID As Long = 0L

        Public Sub New(mess As List(Of Byte))
            Me.messageID = messCTR
            messCTR += 1L
            Dim sb As New System.Text.StringBuilder
            For Each b As Byte In mess
                sb.Append("x")
                sb.Append(Convert.ToString(b, 16).PadLeft(2, "0"c))
                sb.Append(ControlChars.Tab)
            Next
            Me.message = sb.ToString
            Me.length = mess.Count
        End Sub
    End Class

#Region "protocol specific"
    Private Sub ProcMessages()
        While Not isStop.WaitOne(0)
            haveMessages.WaitOne()
            While MessageQ.Count > 0
                If isStop.WaitOne(0) Then Exit While
                Dim mess As MyMessage
                If MessageQ.TryDequeue(mess) Then
                    PopulateForm(mess)
                Else
                    Threading.Thread.Sleep(0)
                End If
            End While
        End While
        Threading.Interlocked.Increment(closedCT)
    End Sub

    Private Sub PopulateForm(msg As MyMessage)
        If Not isStop.WaitOne(0) Then
            If Me.InvokeRequired Then
                Me.Invoke(Sub() PopulateForm(msg))
            Else
                Threading.Interlocked.Decrement(closedCT)
                RichTextBox1.AppendText(msg.messageID.ToString.PadLeft(9, " "c))
                RichTextBox1.AppendText(ControlChars.Tab)
                RichTextBox1.AppendText(msg.length.ToString)
                RichTextBox1.AppendText(ControlChars.Tab)
                RichTextBox1.AppendText(msg.message)
                Label1.Text = inBuff.Count.ToString("N0")
                If RichTextBox1.Lines.Length > 2000 Then
                    Dim ln As List(Of String) = RichTextBox1.Lines.ToList
                    ln.RemoveRange(0, 500)
                    RichTextBox1.Lines = ln.ToArray
                End If
                RichTextBox1.AppendText(ControlChars.Cr)
                RichTextBox1.ScrollToCaret()
                Label2.Text = RichTextBox1.Lines.Length.ToString("n0")
                Threading.Interlocked.Increment(closedCT)
            End If
        End If
    End Sub
#End Region
#End Region
End Class

[stanav]

Thanks for the suggestion and the sample code, Dbasenett. I will study your method of dealing with serial port data, and find a way to apply that to my program.