[RESOLVED] What is it with threads and performance?

[bergerkiller]

Hello everyone.

Not really asking for code, but I'm interested in knowing the basics of "thread performance".

For example, you try to copy a bitmap in a pixel by pixel operation. (Is slow, but it is a nice example)

        Dim IMG As Bitmap = Image.FromFile("c:\test.png")
        Dim COPY As New Bitmap(IMG.Width, IMG.Height)
        For X As Integer = 0 To IMG.Width - 1
            For Y As Integer = 0 To IMG.Height - 1
                COPY.SetPixel(X, Y, IMG.GetPixel(X, Y))
            Next
        Next

This litterally takes ages on a 1280 x 1024 bitmap, but what if I would use 1024 separate threads performing the copy of a single line?:

    Private IMG As Bitmap
    Private COPY As Bitmap
    Private started As Boolean
    Private line As Integer
    Private finishcount As Integer

    Private Sub CopyLine()
        Dim Y As Integer = line
        started = True
        For X As Integer = 0 To IMG.Width - 1
            Dim succesfull As Boolean = False
            Do While succesfull = False
                Try
                    COPY.SetPixel(X, Y, IMG.GetPixel(X, Y))
                    succesfull = True
                Catch
                End Try
            Loop
        Next
        finishcount += 1
    End Sub

    Private Sub CopyImage()
        IMG = Image.FromFile("c:\test.png")
        COPY = New Bitmap(IMG.Width, IMG.Height)
        finishcount = 0
        For Y As Integer = 0 To IMG.Height - 1
            line = Y
            started = False
            Dim t As New Threading.Thread(AddressOf CopyLine)
            t.IsBackground = True
            t.Start()
            Do While started = False
                Threading.Thread.Sleep(50)
            Loop
        Next
        'all threads launched, now waiting for it to finish
        Do While finishcount < IMG.Height
            Threading.Thread.Sleep(500)
        Loop
        MessageBox.Show("Copying finished.", "Finished.", MessageBoxButtons.OK)
    End Sub

This is, of course, very buggy and causes a lot of violation errors etc. But that doesn't matter right now. How come the multi-threading way is so much faster than the regular routine? Is the processor not running on full speed with the first code, and is it working harder on the second?

I find it weird to see this type of "threading performance increase", as if every single thread gets their own piece of processor speed allocated like it is a new process. Is there anyone who knows more of this?

[stanav]

If you have a single core/thread CPU, multithreading will not complete the task any faster. In fact it will take longer to finish the same task. If you have multicore/multithread CPU then the task will be done faster.

[BlindSniper]

Well theoretically if you only had one processor doing them in linear or parallel would take Exactly the same time since the processor can only do X amount of calculations per second and you only have Y amount of calculations that need to be done. But if you have a dual or quad core processor theoretically it could be done faster because then you have true parallelism .

[bergerkiller]

Ah so the fact that I have a dual-core processor could explain why it is faster when using threads? Then still...why is it like 10x as fast?

EDIT

BTW: I added a picturebox which was updating from the COPY bitmap (very buggy, took a while to make it work), and it is filling the empty pixels nicely parallel.
The image looked sorta like this, dots stand for filled pixels:
..............-----------------
...........-------------------
.........--------------------
.......----------------------
etc.

[stanav]

Well theoretically if you only had one processor doing them in linear or parallel would take Exactly the same time since the processor can only do X amount of calculations per second and you only have Y amount of calculations that need to be done. But if you have a dual or quad core processor theoretically it could be done faster because then you have true parallelism .

The key term here is "theoretically". Mutithreading requires slightly more overheads than single threading and thus with a single core/thread (non-hyper threading) CPU, it'll take longer to complete the same task when multithreaded.

[Shaggy Hiker]

There are books on the subject, and some of them are quite thorough. I have a good one around here somewhere, and would figure out where if the title would be of interest to you.

The processing time on the CPU is chopped up into slices if you have multiple processes running (and you do on any modern OS, though maybe not quite all the time). Each process gets a slice in turn, loads what it needs to load into the registers, processes for the length of its slice, then unloads its items to make way for the next process. It doesn't quite happen like that in reality, but I would say that it is a good way to visualize the situation because of the loading and unloading steps. Those are the context switches, and they are not free. However, many processes have time periods when they are doing nothing and don't even need their slice of processing time. For example, any user interface has HUGE amounts of time that nothing is happening, normally. Even a fast typist is slow by computer standards. Therefore, there are times when the UI is sitting around waiting for the next keystroke. When the OS offers it a slice of processing time, it pretty much says no, so the OS goes on to the next process.

In your case, the loop you have shown is probably running in such a fashion that the process probably doesn't have any down time, so it takes advantage of every slice of processor time that it can get. However, the situation becomes considerably different when you have multiple cores, as there are now multiple processors to spread the load across such that instead of each process sharing slices on a single processor, they can have shares on whichever processor happens to be available. At the very least, there are more slices of time, and they are simultaneous, so multiple processes can be running at the same time. There is still a cost for context switching, but there may well be less context switching going on, and you are still running two things at the exact same time, whereas with a single core you really aren't running multiple processes at the same time, you are just interleaving them such that they appear to be running at the same time when they are not.

[Shaggy Hiker]

That may be a new record in slow for me. Three posts, one including a goodly snippet, all posted while I was writing a relatively short reply.

[bergerkiller]

Hehe, well that could indeed be an explanation. Maybe the processor is "idle" all the time, and is slowly, by it's standard, processing the commands of my program. This, I guess, to prevent a fatal crash of the OS it is running on.

Downside is that .NET does not have some sort of "give me some more CPU time OS" call, which could be quite a problem when your doing some serious calculations...

Then about balancing...of course you could make X amount of threads performing your enormous amount of calculations, but does this eventually freeze the computer halting it to a fatal stop? I bet those single core computers would not be happy with 1000 threads running at once.

[Shaggy Hiker]

There is certainly a theoretical maximum number of threads beyond which performance will just plain suffer, regardless of the CPU, but it really shouldn't matter how many threads you have. Each one gets a slice in order. If you have a thousand threads, then thread A will get one slice in every 1000, which may mean that it will appear to slow to a crawl.

By the way, modern OS pretty much make sure that the CPU isn't sitting around unused to terribly often. If you watch the system monitor that tracks CPU usage (you will see two CPU on a dual core), you will see that the line isn't flat, even when you aren't doing anything. Basically, when you aren't doing anything, your OS gets bored, and goes looking for naughty bits.

[bergerkiller]

Ah ok that really did help a lot in my understanding of Threads
While your program is running in a single thread, it is running command by command and is only using a single CPU core.

When using multiple threads, it will appear to gain performance, but actually it is just eating up the performance of other programs and itself, therefore making the system a bit unstable. Modern OS can deal with this and will evenly divide the CPU time to all programs, giving a bit more to the hungry process.

So, I guess you would have to seek a balance at which your program runs best?
Like, for every routine that can be done parallel to your application (checking for updates, updating internal global values) you should do them parallel.

[MonkOFox]

Say you had a bitmap file that was read in (stored into a bitmap object) and you had a two dimensional array of the same row/col count (of type pixel of course).

if you had a thread execute and read the pixels on each row of the bitmap to (column.count - 1) and put that pixel in the corresponding row,col of the [][], that wouldn't be faster than doing a single for i... for j... next next loop?

Is the bitmap file locked when a thread attempts to read a pixel?

Justin

I'm not saying it wouldn't require more resources, but surely it would copy the pixels from the bitmap object into the [][] faster.

[bergerkiller]

Could be locked, but I used an internal graphics object to draw a blue rectangle onto a bitmap surface. Then disposed the graphics object.

I know of the locking bit, have had some trouble with it in the past

The thing explained here is, why "parallel line processing" was processing faster than processing the same deal in a single thread. (main loop)

Like, you launch 100 different programs to write a single line to a single file, instead of writing 100 lines to the file from a single program.

EDIT

So yeah, ignore the "IMG = Image.FromFile("c:\test.png")" part, only added it to keep the post short.

[Shaggy Hiker]

When using multiple threads, it will appear to gain performance, but actually it is just eating up the performance of other programs

That is right.

therefore making the system a bit unstable.

Why unstable? It might slow it down, but it shouldn't affect system stability.

Modern OS can deal with this and will evenly divide the CPU time to all programs, giving a bit more to the hungry process.

The hungry process gets no extra time slices, normally. If other processes are idle and skip their slices, then the processes that do not skip their slices do get more total processing time, but that isn't the doing of the OS.

By the way, there is a way to set thread priority, but it is a dangerous thing. Lower priority threads don't just get less time than higher priority threads, they get NO time until all the higher priority threads complete (except that the OS will occasionally boost the priority of some threads for a variety of reasons).

So, I guess you would have to seek a balance at which your program runs best?

Always.

Like, for every routine that can be done parallel to your application (checking for updates, updating internal global values) you should do them parallel.

Those are excelent candidates for threading. Be careful with updating internal global values, though, as you can never be certain when threads will switch (there are rules, but they are effectively meaningless to the coder, since they apply at a level that is often lower than a single line of code such that the context can switch from one thread to another inside the block of machine instructions generated from a single line of VB. This means that the context can appear to switch inside a single line of code). Therefore, some care has to be taken to avoid race conditions when altering global variables accessed by multiple threads. For instance, if your main thread has:

If someGlobal = 1 Then

the code could check that the global equaled 1, and before a single further line of code executed, another thread could alter someGlobal such that it no longer was 1, making the condition false immediately after it had just evaluated to True. That can be a disaster.