multi threaded programming

I created a program that allows me to choose how many threads to use to send requests to a web service. But I've noticed the numbers don't add up when done in a certain way. When I open only one copy of my application and select 30 threads to start sending request, in 15 minutes, the request per second is 0.09. But when I open two copies of my application, and select 15 threads for each (so that's 30 total threads), I am able to get more request with a request per second of 0.12. Is it my design or is this just how it is? The more threads you use, the slower it is?

This really isn't what multithreading is for. In your case you are creating too many threads and killing yourself on threading overhead and synchronization.

If you have a legit need to run this many Web Service calls in parallel use async requests, a much more efficient use of resources than multithreaded blocking calls. The real answer is probably to rip and redesign the Web Service to handle multiple requests and return multiple response-sets per call. The "Let's pretend Web Services are procedure calls" style of architecture is very inefficient. It is like making a separate database query for every row you need to get back - only worse.

Quote:

---

Originally Posted by dilettante

This really isn't what multithreading is for. In your case you are creating too many threads and killing yourself on threading overhead and synchronization.

If you have a legit need to run this many Web Service calls in parallel use async requests, a much more efficient use of resources than multithreaded blocking calls. The real answer is probably to rip and redesign the Web Service to handle multiple requests and return multiple response-sets per call. The "Let's pretend Web Services are procedure calls" style of architecture is very inefficient. It is like making a separate database query for every row you need to get back - only worse.

---

Actually, this is for testing purpose. I am testing for the response time of the web service. The reason for using multiple threads to make requests is to simulate a situation where multiple clients (in my case, 30 clients) are trying to make requests. I thought I could simulate 30 clients by creating 30 threads (within the same process). But I've noticed that if I was to split the threads by half, one process will run 15 threads and another process run the other 15 threads, it performs a little better than if I ran 30 threads in a single process. I know you mentioned about async requests, but let's assume that is not an option. Is this simply just how threads behave when too many are created within the same process?

How many processor cores do you have?

Quote:

---

Originally Posted by Shaggy Hiker

How many processor cores do you have?

---

Yeah, and do you have a thread affinity issue here, i.e. are all of the threads running in the same execution unit (CPU, core, or hyperthread)?

Inverting your "requests per second" metrics I get 8.3 vs 11.1 seconds per request which easier to visualize. That is a substantial difference.

Quote:

---

Originally Posted by dilettante

Yeah, and do you have a thread affinity issue here, i.e. are all of the threads running in the same execution unit (CPU, core, or hyperthread)?

Inverting your "requests per second" metrics I get 8.3 vs 11.1 seconds per request which easier to visualize. That is a substantial difference.

---

Shaggy: I'm running a machine that is dual core.

dilettante: I'm not sure what you mean by "inverting your 'request per second' metics". Did you create your own test? I'm running on one process (30 threads) vs 2 process (15 threads each). I guess to make it simpler, is there a way for me to code it so that my one process (30 threads) has similar results (request per second) as the 2 process (15 threads each)?

Regarding using async: the reason I can't use this is because the whole point of the program is to test the waiting time between request and response. The thread should not do anything else until it gets the response.

Your 0.09 requests per seconds is the same as 11.1 seconds per request. Inversion is just simple arithmetic. All I was saying is I agree there is quite a difference between your one-copy and two-copy results.

Using async requests doesn't mean you'd be "doing anything else" while waiting for responses. It just allows you to have overlapping requests in parallel with no need for multithreading.

As I asked before, do you have thread-affinity issues? Dual core and two processes are faster than one. Sounds like a possibility.

Quote:

---

Originally Posted by dilettante

Your 0.09 requests per seconds is the same as 11.1 seconds per request. Inversion is just simple arithmetic. All I was saying is I agree there is quite a difference between your one-copy and two-copy results.

Using async requests doesn't mean you'd be "doing anything else" while waiting for responses. It just allows you to have overlapping requests in parallel with no need for multithreading.

As I asked before, do you have thread-affinity issues? Dual core and two processes are faster than one. Sounds like a possibility.

---

So you're saying that if my processor was only a single core, the two process (with 15 threads each) would have same result as one process (with 30 threads)?

Perhaps pretty close, or worse.

That's what I would expect, too.

One thing that I have yet to get into (though I may be doing so soon thanks to some items at work), is thread affinity and maximizing utilization of multiple cores. What you have shown is pretty interesting, though it may not be a surprise to somebody more familiar with how Windows allocates multiple cores. It sounds like each app is being assigned to a single core, regardless of the number of threads. Naturally, the OS has a bunch of overhead stuff to be running, as well, so your apps don't get 100% of the time on any core, but if each app is tied to a single core, then all the threads associated with that app will run on a single core. Therefore, a single instance running 30 threads is not going to do as well as two instances, each running 15 threads, because in the second case you are using both cores, whereas in the first case you are using only a single core. I could probably come up with some reason why the performance of the two apps isn't twice as fast as the single app, too, but it would just be speculation.

I'd like to see how the same thing performs on a quad core. Four apps, each running 10 threads, should be considerably faster than one app running 40 threads. That would pretty much cover it, but some googling is probably a whole heckuva lot cheaper than a quad core computer.

I bought my quad-core system in 2007 for $400, whole system with 2GB RAM a 500GB hard drive and Windows on it. It has been a good investment.

One issue here is that the test is testing Web Service response time. Most of the time should be waiting for responses so I'm not sure why any of this matters very much. Surely nobody uses Web Services within a LAN or a single machine? That's not what they are for.