Multithreading, Multitasking, Multiprocessing

Until sometime back until multicore processors came into existance, multithreading was not considered good w.r.t. CPU utilization optimization. In most situations, single threaded applications performed better because the CPU can do only one task at a time, while multithreading adds overhead of managing multiple threads along with performing the intended task.

But now with the advent of multicore processors that no longer holds true. Single threaded applications will use the power of only one CPU optimally while the other CPUs may remain idle. That means the situation has now reversed. When coded well, now multithreaded applications will perform far better as compared to single threaded application.

I just wanted to start this discussion and have your ideas and views on maximizing the CPU utilization in multicore processors, as there are no good articles on this topic here yet.

Quote:

---

Originally Posted by Pradeep1210

Until sometime back until multicore processors came into existance, multithreading was not considered good w.r.t. CPU utilization optimization. In most situations, single threaded applications performed better because the CPU can do only one task at a time, while multithreading adds overhead of managing multiple threads along with performing the intended task.

---

This was never true, even on single processor systems.

However there were guidelines stating that it usually didn't buy you much to use too many threads. I think the recommended formula was roughly 2 threads per execution unit (i.e. CPUS, cores, hyperthreads).

The main reason for multithreading is blocking operations, like waiting for a disk I/O to complete or data to be returned over the network. Consuming additional execution units is only secondary for most applications since an application usually doesn't "own the machine" but needs to share it with many other processes.


Things have changed as raw CPU speeds have stopped increasing in favor of adding execution units. This may be the main force driving programmers to look at multithreading today but caution is still indicated, especially for server applications. There more power should equal more users.

For a desktop machine when I buy more cores or more CPUs I want the machine to be more responsive. I do not need Joes's Slick Freeware Tool to try to soak every available CPU cycle, I expect the "cores instead of raw speed" to act as a throttle on Sad Programmer Joe's code.

Bad multithreading can give even worse performance than no multithreading at all. In highly parallell solutions, you will need to synchronize threads, and use locks to make sure specific data is not accessed by multiple threads simultaneously. This is the pitfall of multi-threading, as you might end up with threads spending most of their time waiting for eachother to complete tasks. Accessing locks is not a cheap operation.

My thesis tutor (supervisor?) is really knowledgeable in the field of non-blocking synchronization, you might find some of his publications an interesting read: http://www.adm.hb.se/~hsu/

Good point. For an intro see the old article What Every Dev Must Know About Multithreaded Apps.

There seems to be one type of program where multithreading on a single core system is never a good idea. That type of program is one that consists of no user interface and virtually no IO operations. Not a common type of program, but I have written a couple of them. Basically, the programs (GA systems are one example) are essentially very long running calculations. There is no intrinsic pause during the normal course of operations for the procedure, so the overhead of context switching makes a multithreaded operation slower than a single threaded operation...as long as they are sharing the same execution unit.

However, most programs deal with human interactions. The fastest typist takes an eternity to enter a single character, as far as the computer is concerned. Therefore, there are plenty of wasted cycles in a normal program that could be put to better use on other operations.

That's where priority schemes come in, but the regular process priority scheme in Windows doesn't seem adequate.

Process Lasso looks interesting, an example of a tool designed to help manage priority on a dynamic basis.

"Soaker" programs that would tend to eat every available cycle may benefit from coding in a call to Sleep passing 0 every so many iterations (assuming loop-driven code). This will give up the process' timeslice early, giving other processes more opportunities to run. Tuning the "so many iterations" is more art than sceience though.