PictureBox.SetPixel Timing

[VB-MCU-User]

Hi. I'm receiving through a USB connection the data for 16 image pixels at a time. The data comes in as RGB565 in two bytes per pixel for a 160 x 120 pixels image.

If I receive the pixels data with an interval of 1 millisecond between data packets, my vb.net program works fine displaying the image. My USB device can send data a lot faster than this, so if I try to decrease the interval below 1 mS (speed up the data transfer), the display process of the image in the PictureBox starts losing data and it won't display the whole picture. This makes me think that my vb.net program is taking too long to display the image.

How do I speed up my code? What are the lines in the code that are taking more time to process? I remember there is a way to run the code in a second thread. Is this going to help? Any ideas? Thanks.

For IvarImage = 0 To 15       'PROCESS DATA FOR 16 PIXELS FOR IMAGE IN PICTUREBOX. RGB565 DATA.

   '248 = b11111000, SHIFT 3 SPACES TO THE RIGHT.
   RedValueImage = (ALL64.Item(DevNum).Get_SquaresArray(((IvarImage * 2) + 1)) And 248) >> 3
   '7 = b00000111, SHIFT 3 SPACES TO THE LEFT. 224 = b11100000, SHIFT 5 SPACES TO THE RIGHT.
   GreenValueImage = ((ALL64.Item(DevNum).Get_SquaresArray(((IvarImage * 2) + 1)) And 7) << 3) +
      ((ALL64.Item(DevNum).Get_SquaresArray((IvarImage * 2)) And 224) >> 5)
   '31 = b00011111.
   BlueValueImage = ALL64.Item(DevNum).Get_SquaresArray((IvarImage * 2)) And 31

   'NEED TO SCALE UP COLOR VARIABLES.
   RedValueImage = RedValueImage * 255 / 31            '31 CORRESPONDS TO 5 BITS FOR RED
   GreenValueImage = GreenValueImage * 255 / 63        '63 CORRESPONDS TO 6 BITS FOR GRREN
   BlueValueImage = BlueValueImage * 255 / 31          '31 CORRESPONDS TO 5 BITS FOR BLUE

   Dim pixelColor As Color = Color.FromArgb(RedValueImage, GreenValueImage, BlueValueImage)
   bmpImage.SetPixel(xImage, yImage, pixelColor)

   xImage = xImage + 1         'NEXT POINT. IMAGE SIZE IS 160 X 120 PIXELS.
   If xImage > 159 Then
      xImage = 0
      yImage = yImage + 1
      If yImage > 119 Then
         yImage = 0
      End If
   End If

Next

' Display the generated image in a PictureBox. 16 pixels at a time.
pbxCameraImage.Image = bmpImage

[dday9]

Using the SetPixel method is probably your main issue. Even then, this code shouldn't be running on the UI thread, which I believe you're doing based on this:

If I receive the pixels data with an interval of 1 millisecond between data packets, my vb.net program works fine displaying the image.

The process should be this:

1. Start a background worker
2. In the BackgroundWorker.DoWork event:
   
   1. Loop while the number of received bytes is less than the number of expected bytes (160 x 120 x 2 = 38,400)
      
      1. Attempt to read data from the USB.
      2. Perform the logic to build the image.
      3. Call ReportProgress based on number of received bytes compared to the number of expected bytes along with a copy of the working image.
         
         1. E.g. you've received 19,200 bytes.
         2. Then you would pass 19,200 * 100 / 38,400 as the percentProgress argument.
         3. And then call Bitmap.Clone on your working image and pass that as the userState argument.
3. In the BackgroundWorker.ProgressChanged event:
   
   1. Update the PictureBox.Image to be the current state of the ProgressChangedEventArgs.UserState (set back in 2.a.III.c).
4. In the BackgroundWorker.RunWorkerCompleted event:
   
   1. Check if the number of received bytes matches the number of expected bytes.
   2. If it doesn't, then display an error of some sort indicating a timeout failure.

There are a few things implied in these instructions, for instance you should setup timeout logic to exit your loop prematurely (you don't want to potentially loop forever) and you should also be checking the CancellationPending to see if the operation should be cancelled.

My guess would be that if you changed your business logic to implement this, then the issues with SetPixel will go away. If it doesn't, then you could look into leveraging LockBits and UnlockBits, but to be honest I don't think you'll need to.

[VB-MCU-User]

Hmm, I would be accessing a UI property in DoWork. That would be bmpImage.SetPixel. I don't think the BackgroundWorker is going to work here.

Essential Rules for DoWork

No UI Controls: Never access UI properties (e.g., changing Label.Text) directly inside DoWork. This causes cross-thread errors.

Parameters & Results: Use .RunWorkerAsync(data) to pass input to DoWork via e.Argument. Return data to the UI using e.Result.

Cancellation: Regularly check worker.CancellationPending inside your loops to enable graceful cancellation.

[VB-MCU-User]

What about this method below? It says that it can run processor-heavy code on a background thread.

If your goal is to push processor-heavy code onto a completely different background thread so your app stays responsive, you should use Task.Run or the Thread class.

' This spins up a background thread to execute the code
Task.Run(Sub()
' Put your heavy workload or API calls here
End Sub)

[.paul.]

A BGW would work, because you don’t need to access any form controls until the worker ends, then you set the picture in the worker completed event, which runs on the primary thread, thereby no cross thread operations…

[dday9]

You don't modify the UI in the DoWork method, you do it in the ProgressChanged.

Reread my suggestion.

[VB-MCU-User]

You don't modify the UI in the DoWork method, you do it in the ProgressChanged.

Reread my suggestion.

Does the ProgressChanged work in the main UI thread or in a background thread?

[dday9]

ProgressChanged runs on the thread that created the BackgroundWorker. Assuming your creating the BackgroundWorker via the designer or from an event on the form (or child control), then this would mean the UI thread created it.

[VB-MCU-User]

A BGW would work, because you don’t need to access any form controls until the worker ends, then you set the picture in the worker completed event, which runs on the primary thread, thereby no cross thread operations…

I think that what takes processing time is the bmpImage.SetPixel line. If I cannot put that line in a BGW thread then what is the purpose of using it?

[.paul.]

I think that what takes processing time is the bmpImage.SetPixel line. If I cannot put that line in a BGW thread then what is the purpose of using it?

The purpose is not to speed up the operation. The purpose is to free up the primary (UI) thread…

[dday9]

I'm not in a position to test this because I don't have RGB565 data coming in from a USB, but I'm fairly certain that my exact plan back in post 2 will work using this code:

Option Strict On
Imports System.ComponentModel

Public Class Form1

    Private Const IMAGE_WIDTH As Integer = 160
    Private Const IMAGE_HEIGHT As Integer = 120
    Private Const BYTES_PER_READ As Integer = 2
    Private Const TOTAL_EXPECTED_BYTE_COUNT As Integer = IMAGE_WIDTH * IMAGE_HEIGHT * BYTES_PER_READ
    Private Const TIMEOUT_IN_SECONDS As Integer = 10

    Private _bytesReceivedCount As Integer = 0
    Private WithEvents _workerReadUsb As New BackgroundWorker() With {
        .WorkerReportsProgress = True,
        .WorkerSupportsCancellation = True
    }

    ' control events
    Private Sub ButtonToggleUsbRead_Click(sender As Object, e As EventArgs) Handles ButtonToggleUsbRead.Click
        If (_workerReadUsb.IsBusy) Then
            _workerReadUsb.CancelAsync()
            ButtonToggleUsbRead.Text = "Read"
        Else
            ResetState()
            ButtonReset.Enabled = False
            ButtonToggleUsbRead.Text = "Cancel"
            _workerReadUsb.RunWorkerAsync()
        End If
    End Sub

    Private Sub ButtonReset_Click(sender As Object, e As EventArgs) Handles ButtonReset.Click
        ResetState()
    End Sub

    ' worker events
    Private Sub _workerReadUsb_DoWork(sender As Object, e As DoWorkEventArgs) Handles _workerReadUsb.DoWork
        Dim imageFromUsb As New Bitmap(IMAGE_WIDTH, IMAGE_HEIGHT)
        Dim startTime As Date = Date.Now
        Try
            While _bytesReceivedCount < TOTAL_EXPECTED_BYTE_COUNT
                If (_workerReadUsb.CancellationPending) Then
                    ' exit prematurely
                    e.Cancel = True
                    Exit While
                End If

                If (Date.Now.Subtract(startTime).TotalSeconds > TIMEOUT_IN_SECONDS) Then
                    e.Result = _bytesReceivedCount
                    Exit While
                End If

                ' TODO: fill in existing logic reading from USB
                '       this is also where you use SetPixel to draw on imageFromUsb
                '       omitting this section for brevity and the fact that I can't feed in 2bytes of RGB565 data from a USB

                Dim percentProgress As Integer = Convert.ToInt32(_bytesReceivedCount * 100 / TOTAL_EXPECTED_BYTE_COUNT)
                Dim imageCopy As Bitmap = DirectCast(imageFromUsb.Clone(), Bitmap)

                _workerReadUsb.ReportProgress(percentProgress, imageCopy)
            End While

            e.Result = _bytesReceivedCount
        Finally
            imageFromUsb.Dispose()
        End Try
    End Sub

    Private Sub _workerReadUsb_ProgressChanged(sender As Object, e As ProgressChangedEventArgs) Handles _workerReadUsb.ProgressChanged
        ProgressBarUsb.Value = Math.Min(100, Math.Max(0, e.ProgressPercentage))

        Dim newImage As Bitmap = DirectCast(e.UserState, Bitmap)
        PictureBoxUsb.Image?.Dispose()
        PictureBoxUsb.Image = newImage
    End Sub

    Private Sub _workerReadUsb_RunWorkerCompleted(sender As Object, e As RunWorkerCompletedEventArgs) Handles _workerReadUsb.RunWorkerCompleted
        If (e.Cancelled) Then
            MessageBox.Show("You successfully cancelled the process.", "Cancelled", MessageBoxButtons.OK, MessageBoxIcon.Stop)
        ElseIf (e.Error IsNot Nothing) Then
            MessageBox.Show(e.Error.Message, "Exception", MessageBoxButtons.OK, MessageBoxIcon.Error)
        Else
            Dim finalReceivedBytes As Integer = Convert.ToInt32(e.Result)
            If (finalReceivedBytes <> TOTAL_EXPECTED_BYTE_COUNT) Then
                MessageBox.Show($"Unable to read the entire image because the operation timed out after {TIMEOUT_IN_SECONDS} seconds.", "Timeout", MessageBoxButtons.OK, MessageBoxIcon.Error)
            Else
                MessageBox.Show("Operation completed successfully", "Complete", MessageBoxButtons.OK, MessageBoxIcon.Information)
            End If
        End If
        ButtonReset.Enabled = True
    End Sub

    ' private methods
    Private Sub ResetState()
        _bytesReceivedCount = 0
        ButtonToggleUsbRead.Text = "Read"
        PictureBoxUsb.Image = Nothing
        ProgressBarUsb.Maximum = 100
        ProgressBarUsb.Value = 0
    End Sub

End Class

The controls used are:

• ButtonToggleUsbRead: Button with text set to "Read"
• ButtonReset: Button with text set to "Reset"
• ProgressBarUsb: ProgressBar
• PictureBoxUsb: Size set to 160x120