STM.NET - thread safe programming without SyncLock :)

STM.NET - http://msdn.microsoft.com/en-gb/devlabs/ee334183.aspx

Anyone played with it? The concept sounds good to me - I hate threading issues and SyncLocks and Mutexes etc.
Basically from what I understand, if you had a program where a shared object is accessed from multiple threads at the same time, instead of having to SyncLock or whatever and choose what to lock on etc you would simply stick this at the start of the code that needs to be isolated so that it can only run in one thread at a time:

Code:

---

Atomic.Do()=>

---

Apparently it provides much more than just a replacement for SyncLock but I havent really delved into the details yet, was just wondering if anyone else has had a look at this and what they thought?

EDIT: Here is Microsoft's description of why they are creating this technology (taken from http://download.microsoft.com/downlo...User_Guide.pdf):

Quote:

---

By now you’d be hard-pressed to find anyone in the industry who hasn’t heard about the coming crisis in software performance. The rate of increase in processor speed has slowed dramatically and is nowhere near the doubling of speed that we saw every two years during the last twenty years.

That’s the bad news; the good news is that Moore’s law is by no means dead. The hardware industry is still able to follow the path that Moore predicted. The density of transistors, in an integrated circuit, still doubles every two years. The industry has, however, run out of effective ideas for turning those transistors into faster processors. Therefore, they are using the extra transistors to create more processors, building chip multiprocessors (CMP) with many processors on a single chip. Those chips are also known as multi-cores, or many-cores.
This is difficult news for the software industry. We can no longer add new features to our applications and rely on increasing hardware capacity to make evermore complex software perform acceptably. To state things more positively, we can also no longer rely just on better hardware to bring applications that were too expensive in the past to become feasible today (as was the case with, for example, speech recognition). Now, to take advantage of future hardware capacity, we have to fundamentally change the way we program, to exploit the parallel capacity of this new hardware.

Parallel programming is notoriously difficult. Researchers have been working for decades on systems to make parallel programming easier, yet despite these efforts, parallel programming has not yet become popular with all developers. Some amount of skepticism about whether we’ll succeed now is therefore justified – but note that in the past parallel programming was a niche market, if only because of the expense of multiprocessor computers. The hardware industry has the capability to produce chip multiprocessors whose core counts double in time-frames similar to those we have been used to for processor speeds – whether they will do so depends on whether anyone will buy them which, in turn, depends on whether there is any software that utilizes their power to do interesting things that couldn’t be reasonably done on a single-processor machine. If we succeed in creating such software, the virtuous cycle of new hardware and software features and applications will continue.

In order to enable this virtuous cycle Microsoft is making an effort to make effective parallel programming easier. There are many competing ideas on how this might be accomplished. Some are fairly radical: “get everyone to use pure functional languages,” for example. Here we focus on one of these ideas, Transactional Memory. Compared to some other ideas in this space, it is evolutionary rather than revolutionary; programming with transactions will feel fairly similar to programming today, just simplified.

---

Chris

Seems like a good concept to me. If it becomes a success we will be able to do true multiprocessing at our control.

It says it's only available in C#. I'd have loved to try it in VB. Maybe they will extend that to VB in the future.

Yeah I think at the moment it is very much an experiment. If a lot of people get on board and find it really useful then I guess they may consider adding it to the next version of the framework but at the moment I dont think there are any plans to actually bring it into production, which is a shame.

It's ok, it's the same kind of thing the Parallel library does with multithreaded loops. Multithreaded programming really isn't that tough, you just have to stop thinking linearly and tell yourself "Ok, now at any time, I might have more than one process trying to hit this at the same time. What does that do to the variables and how my program works?"

Have a read of the Programming Guide I linked to if you get chance, they do mention the parallel stuff in .NET 4 in there but I think this goes beyond what that can do (if it didnt, they wouldnt bother publishing it surely)

Quote:

---

Originally Posted by Jenner

It's ok, it's the same kind of thing the Parallel library does with multithreaded loops. Multithreaded programming really isn't that tough, you just have to stop thinking linearly and tell yourself "Ok, now at any time, I might have more than one process trying to hit this at the same time. What does that do to the variables and how my program works?"

---

Multithreading and Multiprocessing though similar are two very different concepts. You make two threads interact on a single processing system because the threads are separated logically only. They are not limited by the hardware. So multithreading is just a conceptual multiprocessing. At one time only one thread (process) is running in real.
But while true multiprocessing (parallel processing) using multiple processors, you can't make the processes interact that way because they are separated by hardware boundaries since they are running on two different processors.

I thought Windows itself controlled that boundary and managed it transparently. Example: If I make 50 threads in a program, Windows will decide which physical CPU core to process them on based on core load and grant each thread processing time as it determines appropriate.

Thus, with my 50 threads, Windows may decide 30 of them to run on core 1 and the other 20 on core 2 (in no particular order). Then time-share process each of those threads on their independent cores.

Quote:

---

Originally Posted by Jenner

I thought Windows itself controlled that boundary and managed it transparently. Example: If I make 50 threads in a program, Windows will decide which physical CPU core to process them on based on core load and grant each thread processing time as it determines appropriate.

Thus, with my 50 threads, Windows may decide 30 of them to run on core 1 and the other 20 on core 2 (in no particular order). Then time-share process each of those threads on their independent cores.

---

Yep.. you are right. Though there are ways to hint windows which processor to give priority to.

That's what all this discussion is about. It is about a framework that gives us ability to control multiple processors ourselves.

Ok, so you're saying it's a pipeline to Windows to tell it discretely "run this on this core"?

Thus, If I'm building a game engine, I can detect the engine is running on a quad-core and can tell it "run the game loop on core 1, run the enemy AI processing on core 2, run the sound processing on core 3 and run the physics engine off core 4"?

Seems like a bit of overkill. I'd rather they concentrate on optimizing core utilization in Windows and leave it transparent.

Maybe I'm biased because I'm very used to multithreading at this point and don't care to see yet another Microsoft re-invention of a perfectly good wheel like they did with LINQ to SQL (i.e. ADO.NET works perfectly fine, why make double-standards?!). I have programs at startup that utilize no less than 5 threads to control things like pump recirculation, automatic backup timing, continuous serial reads for scales hooked up to the serial ports, motor-control routines, and error check routines to make sure all the hardware is up and functional and the moment something does or looses contact to put everything into a safe-mode. Whenever I command any of the hardware to do anything critical in these industrial programs, I spin it to a thread. Anytime I have a large amount of data to push-pull from the database, it's on another thread.

I dont think this STM.NET has anything to do with telling a thread which processor/core to run on...

No you got me wrong. From what I read there, it just provides a way to specify a block of code as fully independent from others. So the runtime can optimize it to put it on any processor as it deems fit as it knows that that code block will not cross the boundaries.

Ahh... ok, I get you now. It's a way to tell .NET that it CAN make a spearate thread out of a code-block if it wants and run it on a separate core. Basically, threading without having to set up the thread. I'm still foggy on what the difference is between this and the Parallel library is though. I mean, does this automatically handle delegation between form controls and these independent code blocks for example?

If I got a ListView that is populating itself with a loop as it reads entries from a database, can I just Atomic.Do()...Loop that loop and call it a day and .NET will automatically split it off to a separate thread and pass results to an on-the-fly delegate function to handle the ListView1.Items.Add() bit? I mean, I can do the exact same thing with Parallel.For() if I split my loop code to a separate subroutine, but I still have to do my ListView via a delegate call that is invoked from the GUI thread.

I suppose I could block-off non-looping chunks of that don't interact with the GUI thread, but most things still run too sequentially. Even if when you click "button 1" to start a chunk of totally independent code that calculates some numbers and stores them in a class-wide variable, there's no guarantee that when the user clicks "button 2" that starts code to further use those variables, that the first process is done processing it unless there was some kind of Atomic.FinishAllProcessing() or something to that effect.

Yeah I think this Atomic.Do would mean you dont have to use delegates to pass things across thread boundries but I'm not 100% sure. You can basically think of it as like a global synclock from what I understand. I keep meaning to finish reading the guide but as its not planned to be put into any production version of the framework anytime soon I'm loosing interest :(

You know how I said it was much more than a replacement for SyncLock... here's a snippet from one of the developers on the STM team:

Quote:

---

Atomic blocks provide isolation and failure atomicity. To provide these properties the runtime instruments the code running inside atomic blocks. The code is instrumented to ensure that any changes that are made inside a transaction can be rolled back in case of a conflict or failure. Verifiable managed code can be instrumented in a straight forward manner by the JIT in the CLR version of STM.NET

---

Quote:

---

Providing database-style failure atomicity is an extremely powerful tool. This cannot be stressed enough. It provides a much more robust mechanism for handling failures, instead of the fragile and untested “catch” handlers that developers use today with manual error recovery code. STM provides here a great deal of value which is not only available to concurrent code, but to any code that requires a high level of reliability.

---

Sounds interesting...
but one rather large problem:

Quote:

---

As you can imagine the cost of all the instrumentation and fine-grained locking is significant. We are currently measuring anywhere between 2x and 7x serial slowdown compared to lock-based code. It should be noted though that our current STM implementation hasn’t undergone a significant amount of optimization. There is still a lot of room for improvement and of course hardware support, if and when it becomes available, will dramatically change the picture.

---

Huh, interesting. Yea, I can kinda see it's point and the aim at what they're trying to do. They're trying to "black-box" chunks of code that could be run as an independent thread with full error handling and reversibility. It must record the initial states of all variables used and if needed, can roll-back to them on failure. I admit, that would be a cool feature. I don't know where I'd use it though.

Yeah that was exactly my thought - cool feature but dont know where I would actually use it.