Are You Falling Behind? The Multicore, Concurrency Question...

Are you looking seriously at concurrent or paralyzed procedures within your applications yet? Do you know what impact using such techniques is going to have on your development cycle or your applications?

If you are not at least considering the impact of multi-core processors on your application development, then it might be a good time to start. If you look at the system offered today, you'll find that many now come with dual core chips. On higher-end desktop systems quad-core is becoming the norm. In 2010 you'll see AMD and Intel start migrating the high-end towards six-core chips.

The number of cores is going to continue to grow. In fact, in 2010 Intel is expected to release a 48-core chip to researches to begin using. According to BBC News, Intel has already developed an x86 based 48-core chip dubbed the Single-Chip Cloud Computer (SCC).

If you are not adjusting your applications to take advantage of multi-core, then you could be wasting nearly 80% of the power of a six core chip. On a 48 core chip you'd barely be touching the power if you don't adjust your application.

Of course, you can argue that 48 core chips are only for the high-end user and not main stream. Of course the same argument was made with dual core chips just a few years ago. The other old argument is that you'll never use all the power of the many cores. Again, a similar argument was made a few years ago about disk space. Nobody would ever need a terabyte of data storage on their machines. They could never use it all. Yet today, people are buying Terabyte drives for under $100. More importantly, they are using the space. Of course, to process all that data, what would be better than applications using multi-core chips to process it concurrently!

Are you already considering the impact of multi-core on your applications?



More...

I am failling behind in the knowledge. I currently do not do any development that really requires any concurrency so I haven't paid alot of attention to it. I remember being excited about threading when .NET came out, but that quickly fell to the side as there was just no call for me to ever work with it.

Currently though I am rushing to really learn it because I have to give a presentation for an IT Users group at a local college in January on .NET development and I want to cover the importance of concurrency.

I use a small amount of concurency in my work, and a HUGE amount of concurrency in hobby projects. There are some business apps that lend themselves, in whole or in part, to mutli-processing, but there aren't that many. What is the major slowdown in most business apps? The user! Either they are slow in entry, or slow in understanding an interface, which could be the fault of the interface. Looking back on my current app, the only way concurrent processing could speed things up is if the user had four hands and a whole brain (that's not fair, my users are actually very smart, but they would also appreciate a little flak). Where there was a slowdown, threading helped, but never more than one other thread, as it turned out.

On the other hand, I work with genetic algorithms and distributed systems as a hobby, and that is an area where multi-processing makes a HUGE difference. My current project will have a massive series of GAs all running lengthy, simultaneous, tests to train different parts of a single large GA. I already have about 8 cores that are available for use, and when the program is ready, I will add more. At a rough calculation, I could easily make use of 48 simultaneous cores, so the program is being designed from the ground up to allow different cores to run different tests. The more cores I can give the process, the more it can do. Also, what I am doing with the GA makes a certain amount of sense for the future: Robotics. Right now, it doesn't matter how smart an AI can be because the computers that could run it in the domestic arena don't exist. Work seems to be focusing on simulating clever behavior that can work within existing hardware limitations. Once you get a large number of surplus cores, you can run numerous simultaneous processes. My goal is a robot that has one working self, while all the other cores are busy training the genome for that self by dreaming. A dream is a fictional world that may be based on the real world...or not, with fictional starting conditions, that will train parts of fictional selfs. The learning gained from the dreams will be synthesized into the full working genome that drives the self. That's an example of an "embarrassingly parallel" problem, yet it both benefits from a nearly infinite number of cores (the more cores, the more dreams, or the more simultaneous tests in the same dream, and therefore, the faster the learning). As we try to create more interesting domestic hardware, those multiple cores will become increasingly valuable.

Therefore, once again, what is said to be "only needed for servers and high-end users", a statement that has been made since, at least, the 486, will prove to be totally inaccurate.

Using concurenncy makes sense for some areas but it's not a panachea. My experience is in systems that deal with realtime payments. On the server side it's important to have enough CPU power and in that respect concurrency helps. However, I've yet to experience a bottleneck in a payment system because of the CPU - it's always the disk and, to a lesser extend, the network. For every transaction that comes in over the network such systems have to go to the database at least one time to read something, do a trivial amount of in-memory processing, go to the database once more to write something and send the response back to the network. This is a bit oversimplified, but 95% of its time a payment system is waiting for the database. Of the rest, half is spend sending and receiving data over the network and the other half is spend on the CPU.

Quick true story: back in the 90s, my company had installed a system for the stock exchange. The growth was incredible and in a couple of years, after a series of hardware upgrades which ended when the customer purchased the largest and most powerful server available, the system begun to show sings of a serious bottleneck when the market opened. The situation was resolved by installing a 200MB solid state disk.

As Shaggy said, multi-core is not that relevant for business front-ends either. The computer is waiting for eons for commands or feedback from a user and I've yet to encounter a single scenario when having more cores would speed up the user's work in a measurable way.

The infrastructure (OS and server processes) makes use of additional cores and in that respect everything runs quicker so having multiple cores cannot hurt. It's just that multiple cores are not relevant to all tasks. My XP installation did run faster when I switched from a Core-Duo to a Quad-Core but not twice as fast - not even close. But when I added a Ramdisk I could see the difference without even measuring.

Having said that I'm certain that other, more CPU-intensive applications, can benefit from multiple cores. A lot of work has been done to allow modelling such applications in a way that they can take advantage of more cores which is a reasonable strategy given the fact that hardware manufacturers seem geared towards creating chips with more cores and not faster cores.

I'm amused that the RamDisk is back. I expect that you are old enough to remember them from the days of the PC/AT when DOS only allowed direct addressing up to 1MB, so memory above that could be turned into a virtual disk to speed up drive access. With drives as small as they were, it was possible to load a significant portion of total HD storage into the extended memory space. This was especially true when programs were run off floppies, which were always slow.

What I would like to see is flash memory added to the motherboard. When the computer shuts down, it could store an image of the current state of the RAM, including the processor cache, into the flash memory, then restore it on power up. I just built a computer that came with a mini-OS on the motherboard. The mini-OS allows the user to do some web surfing, gaming, and so forth without going through the full boot. That is similar to where I would like to see OSs go, but not quite there.

Quote:

---

Originally Posted by Shaggy Hiker

I expect that you are old enough to remember them from the days of the PC/AT when DOS only allowed direct addressing up to 1MB

---

I am but let's not make a big deal out of it. :D

I do remember those bad old days. I had an IBM PS/2 with an MCGA with 64kb of memory and I coded a crazy assembly hack to use 32kb of those for storing data structures without going to the floppy or disk. That extra 32kb seemd like a considerable amount of memory at that time.

I think that an SSD and an OS that supports hibernation would be a good alternative to what you're looking for. The perfect SSD would utilize DDR2/DDR3 memory, provide no less than 64GBs of storage, have a built-in battery, come with a disk that would be used to dump the memory when the system powers off, so in case the battery is exausted you don't loose the contents of the RAM and cost no more than €300. Unfortunately we're not quite there yet, but things are improving as you can see here.

I'd rather not use hibernation. I shut off my systems at the wall whenever I power down, so a battery system would have to have reasonably long duration (though all it would be doing would be maintaining RAM, which would take very little battery power).

I remember RAM disks, too.

Hibernation does not involve any power use.

Everything other than shutting the system off at the wall involves power use. Latent power consumption for a computer that is shut off can be appreciable, especially if you have some good speakers with the system. Those suckers can draw pretty well, even when turned off.

If you haven't already, have a look at the Parallel Extensions framework for .NET. It's in a CTP or Beta right now, but it's going to be part of .NET 4.0. And it's really easy (which can be disastrous if you don't understand it right), but the gist is, you can change your for or foreach loops to Parallel.For and Parallel.ForEach loops, then specify the code to execute in an inline delegate.

Parallel.For(0,100,(i) => { your code here });

Further, they've introduced ConcurrentDictionary and ConcurrentBag (corresponding to Lists, but when you take something from the bag, you don't know what you're getting) which you'd use when collecting or modifying data in these parallel loops.

The Parallel.For and .ForEach will automatically scale to the number of cores that you have. And I've seen the results on a 32 core machine we have, it works as advertised :D

You can obviously see the disadvantage here - you'll get developers who attempt to use it without properly understanding some of the important concepts that could make all the difference.

I have been looking forward to that feature for some time. Fortunately, I can wait until the full release.

I didnt know a thing about how these parallel functions actually worked (ie how you would actually use them) until recently, but I found this rather useful video on the subject:
http://www.rocksolidknowledge.com/Sc...allelLoops.wmv
I'm sure there are other such videos on the subject but this one made it nice and easy to understand and see why/how to use the parallel stuff for me

Ok this is an interesting topic.

How does this fit into web-development? I assume the process is the same, but the application and practicality is not all that common?

When you say web-development are you talking about the server side or the client side? I dont see any reason why implementing parallel functions and loops etc in the server side part of a web app is any different to a normal desktop app. On the client side though, the web page, I dont think you can do anything to take advantage of it can you?

Quote:

---

Originally Posted by chris128

On the client side though, the web page, I dont think you can do anything to take advantage of it can you?

---

You can't, at least not with .Net parallels anyways but then again what on earth would you need parallel processing in a web browser for??

It'd help if you wanted to execute a long running process but I don't think it'll be used a lot otherwise in web development. It would get used a lot in web service development.

Quote:

---

I just built a computer that came with a mini-OS on the motherboard. The mini-OS allows the user to do some web surfing, gaming, and so forth without going through the full boot.

---

Sounds a lot like sugar on a stick.

Quote:

---

but the gist is, you can change your for or foreach loops to Parallel.For and Parallel.ForEach loops, then specify the code to execute in an inline delegate.

---

Boy, can't wait for this to come out. I've written some pretty intensive loops that I'd love to stick on another thread (just the loop) to avoid app freeze and to allow non sequential code processes to finish up.

You can do that already with normal threading. The point about Parallel is that each iteration of the loop would be on its own thread. This is great if each iteration is independent of the others, but a disaster if the order of loop execution needs to be preserved.

Quote:

---

You can do that already with normal threading.

---

No, I know that. I must have misunderstood his point. I thought this was supposed to be an easier way of initiating multiple threads.