Results 1 to 14 of 14

Thread: Need advice on server-side issue

  1. #1

    Thread Starter
    Addicted Member
    Join Date
    Dec 2010
    Posts
    213

    Need advice on server-side issue

    Hi, so we're developing a website that will allow user's to pay some money and be able to use a script on our server

    The point is to allow the user to enter their script parameters on the settings page, click save, and then go to bed. The script will run on the server.

    However, say we have 1000 users. If 1000 users were to go to bed that night, then somehow this script would need to run 1000 instances of the script on 1 server.

    Problem is: How can we make this happen without the cost of speed? Because the script's main advantage over our competitors is the script's speed, but with 1000 user's I don't think sequential processing of 1000 users' script runs is speed efficient.

    I heard of multithreading but I'm not sure if this will work because in the end its 1 server, and no matter how much you try to attach the script to an individual process it will still take a lot of time.

    One of the settings of the script is: The user enters a time and the script will run at that time. So if the user enters 3 am, they can go to bed and be rest assured the script will execute as fast as possible at 3 am. Right now, we have our script running at 25ms per iteration (its a while loop that says run script from Time X to Time Y). Each iteration takes 25ms. I haven't tested for multiple users because we don't even know how to schedule all the scripts to run at a certain time. That's what we're trying to develop. Some scheduling manager that will go through all the user's in the database, check their Time's and then execute the script if its time to run it. But do you even think this should be worthy of pursuing? Will it have an effect on the script's speed, say , if 1000 user's were to run it?

    Any ideas on how to do this and preserve speed/efficiency for a multi-user base?
    Last edited by zero_coke; Jun 25th, 2012 at 12:39 AM.

  2. #2
    Super Moderator si_the_geek's Avatar
    Join Date
    Jul 2002
    Location
    Bristol, UK
    Posts
    41,683

    Re: Need advice on server-side issue

    Multithreading isn't about the amount of computers it will run on (one is normal), but things like:
    • whether the code can safely run in isolation from the other threads (eg: it doesn't need to read and write data that is shared with the others [read-only is fine, so is write-only]),

    • the kind of work that is being done (eg: working with a database has periods of the code just waiting for a reply),

    • and in terms of a speed-boost, the amount of logical processors that are available (a server with 1 CPU that has 2 cores = 2 logical processors, 2 CPUs with 4 cores each = 8, etc).


    However, even if you have 'lots' of logical processors available (32? 64?), it wont be enough to ensure that all 1000 copies can run at the same time, so you will still need to work out some kind of scheduling.


    You mention 25ms per iteration, but it isn't clear quite how that works.

    I presume it is multiple iterations per user, but do the iterations happen continuously, or is there a delay until the next one starts?

    Can you accurately predict in advance how long it will run for each user?

  3. #3
    Super Moderator Shaggy Hiker's Avatar
    Join Date
    Aug 2002
    Location
    Idaho
    Posts
    36,287

    Re: Need advice on server-side issue

    I would say that Read-Only is fine, but Write-Only....is risky. For instance, if you were to do something like:

    a += 1

    you might think that you are just writing, because all you are doing is incrementing a, but in fact it is not write-only, and it is not safe. That line requires a read of a prior to the write, and that requires a bit of thought in a multithreaded environment.
    My usual boring signature: Nothing

  4. #4

    Thread Starter
    Addicted Member
    Join Date
    Dec 2010
    Posts
    213

    Re: Need advice on server-side issue

    Quote Originally Posted by si_the_geek View Post
    Multithreading isn't about the amount of computers it will run on (one is normal), but things like:
    • whether the code can safely run in isolation from the other threads (eg: it doesn't need to read and write data that is shared with the others [read-only is fine, so is write-only]),

    • the kind of work that is being done (eg: working with a database has periods of the code just waiting for a reply),

    • and in terms of a speed-boost, the amount of logical processors that are available (a server with 1 CPU that has 2 cores = 2 logical processors, 2 CPUs with 4 cores each = 8, etc).


    However, even if you have 'lots' of logical processors available (32? 64?), it wont be enough to ensure that all 1000 copies can run at the same time, so you will still need to work out some kind of scheduling.


    You mention 25ms per iteration, but it isn't clear quite how that works.

    I presume it is multiple iterations per user, but do the iterations happen continuously, or is there a delay until the next one starts?

    Can you accurately predict in advance how long it will run for each user?
    Thanks for the great reply si!

    Ok, I'll explain what the script does and why we need this.

    We have a script, and it basically takes input a time A and time B and what it does is it executes a while loop from Time A to Time B and that while loop makes API calls to some other website. That's all. Each iteration of this loop costs 25ms (tested with only 1 user using the script on the server).

    The problem is: How do we make the script run from Time A to Time B without user being active in the browser session? We need some sort of cron job scheduler program that will go through all the user profiles, check all the Time A's and Time B's and then if the current time is matched it would simply execute the script for those users. And the second problem is, how do manage to keep the 25 ms per iteration speed when we have 1000 users, as an example, trying to run the script between the exact same Time A and Time B? You can't have 1 script execute, for say, 10 minutes and then have the other user's script execute for 10 minutes and so on. You need to run them at the same time because every user has same Time A and Time B. There needs to be some sort of multithreading but would it be effective in running 1000 user scripts at the same time without costing the per iteration speed of 25ms to go up?

  5. #5
    Super Moderator Shaggy Hiker's Avatar
    Join Date
    Aug 2002
    Location
    Idaho
    Posts
    36,287

    Re: Need advice on server-side issue

    From your description, I would say that the answer is probably YES! This may really be an ideal situation for multithreading. The key question is whether or not those website APIs would ever block. In other words, does the calling thread sit and wait for a reply? If it does, even for a fraction of a second, then multithreading is probably going to work well....up to a point. However, your execution time is fairly minimal already, so you may not be spending much time waiting. The thread scheduler will switch out threads that are blocking, so those delays would be fully utilized by other processes.

    Having said that, I'd also point out that it is unlikely that there is enough waiting going on that 1000 users would all be able to run simultaneously when each one would take so little time on their own. Therefore, you'd probably get some benefit, but quite likely it wouldn't be enough. That may be ok if your 1000 users is a worst case scenario, but if that is fairly likely, then things will probably slow down whether threaded or not. The number of cores available will impact this, as well, but once again, 1000 would probably swamp whatever number of cores you have on a single processor.

    I would expect that a call from user A and a call from user B would only be the same in very rare circumstances. You make it sound a little like they are doing the same thing, in which case you might find the similarities between A and B and reduce the number of actual calls by sharing the results to both A and B. Of course, if there aren't any similarities in the calls, there probably is nothing to be gained there.
    My usual boring signature: Nothing

  6. #6

    Thread Starter
    Addicted Member
    Join Date
    Dec 2010
    Posts
    213

    Re: Need advice on server-side issue

    Hi Shaggy,

    Thanks for the awesome reply. I don't know much about programming to fully explain the situation, but the basic thing we want to do is: Make user pay set amount, let them enter their Start time and End time, and then at that start time a script will make API calls until end time. All of their details are stored in the database, such as Start & End times, API url, etc.

    Now, if we had 1000 users as an example, and they each had the same start and end times, that's what we're trying to figure out. How do we do this? How do we make this work when the user is offline too?

  7. #7
    Super Moderator Shaggy Hiker's Avatar
    Join Date
    Aug 2002
    Location
    Idaho
    Posts
    36,287

    Re: Need advice on server-side issue

    I can see the problem. If the user is paying for a set service, and you have so many concurrent users that they all slow down, they will be getting less and less for their money. Ultimately, this will become a matter of bandwidth. At some point, there will be NO solution that will work other than distributing the load across more processors. When you are talking about 1000 concurrent users, it is probably impossible to have all of that on one box and have it run acceptably.

    You'd have to do some testing to figure out what happens to performance as you add more and more concurrent sessions. Modern languages, including .NET, have well-designed thread pools. You could just queue up a thousand calls and see what happens, but what would probably happen would be unacceptable. As a general rule, the thread pool will create as many concurrent threads as it has processors. However, the thread scheduler is smart enough to take note of threads blocking, and will spin up new threads to keep the CPUs at maximum occupancy. You might find that you get 5-10 threads per CPU core, but I'd be surprised if you did much better than that based on your description, and you would need over 100 threads per core to handle 1000 threads even if you had eight cores.

    Now, you talk about fairly small blocks of time during which each one runs. If those can be consecutive such that the 1000 users are spread across the day in 15 minute increments....I don't know where I came up with 15 minute increments. I thought you had written something about that, but I can't see it. You mention running a loop from time A to time B, but I guess I made up the 15 minutes. How long are these intervals? If they were 15 minutes, and they could run consecutively, then you'd have nearly 100 intervals, so you'd only be looking at 100 concurrent users, which might be reasonable, but some people would be waiting for up to a day to get their time slot scheduled.

    I'd say you are simply going to have to consider a design that scales, and add boxes as needed.
    My usual boring signature: Nothing

  8. #8

    Thread Starter
    Addicted Member
    Join Date
    Dec 2010
    Posts
    213

    Re: Need advice on server-side issue

    Hi Shaggy,

    Ok for example, user 1 sets their time A and time B as: 9:00 am - 9:30 am

    User 2: 9:00 am - 9:30 am

    User 3: 9:00 am - 9:30 am
    .
    .
    .User 1000 9:00 am - 9:30 am

    We need the script to start making blazing fast api calls starting at 9:00 am for all users (who's start time is now) and end on their end times. Each iteration of the API call for 1 single user takes like 25 ms (including analyzing the response of the API call and writing to the database the result for each user). I'm thinking, between 9 am and 9:30 am (for example), there's probably 12,000 calls @ 25ms.

    How do we schedule this? Should I just make a cron job that points to php file every minute and this php file would just goes through everyone's data in the database and looks up their start time, and if it matches the current time then make their API call, analyze response, and call the API of next user, analyze response, and make API call for next user, analyze, etc. ?

  9. #9
    Freelancer akhileshbc's Avatar
    Join Date
    Jun 2008
    Location
    Trivandrum, Kerala, India
    Posts
    7,652

    Re: Need advice on server-side issue

    Hi,

    Just a small doubt. Are you going to run the script in a shared hosting service or in a dedicated server ? Or in your own home/office server ?

    Also, I think making a provision to provide a client app that will do the API calls(from that client app) and also to do the communication with your server would be good. Because this could reduce the load of your server, as the processing is done in the client app. And in server, it would just provide the details to the client(from the db) or store the data(results maybe) from the client app. So, for those who opt for client app, could register for your service and download the app. And this client app would do the rest. But the only downside is, the client app needs to be run in the user's machine when required. So they couldn't turnoff the PC at that time.


    If my post was helpful to you, then express your gratitude using Rate this Post.
    And if your problem is SOLVED, then please Mark the Thread as RESOLVED (see it in action - video)
    My system: AMD FX 6100, Gigabyte Motherboard, 8 GB Crossair Vengance, Cooler Master 450W Thunder PSU, 1.4 TB HDD, 18.5" TFT(Wide), Antec V1 Cabinet

    Social Group: VBForums - Developers from India


    Skills: PHP, MySQL, jQuery, VB.Net, Photoshop, CodeIgniter, Bootstrap,...

  10. #10

    Thread Starter
    Addicted Member
    Join Date
    Dec 2010
    Posts
    213

    Re: Need advice on server-side issue

    Quote Originally Posted by akhileshbc View Post
    Hi,

    Just a small doubt. Are you going to run the script in a shared hosting service or in a dedicated server ? Or in your own home/office server ?

    Also, I think making a provision to provide a client app that will do the API calls(from that client app) and also to do the communication with your server would be good. Because this could reduce the load of your server, as the processing is done in the client app. And in server, it would just provide the details to the client(from the db) or store the data(results maybe) from the client app. So, for those who opt for client app, could register for your service and download the app. And this client app would do the rest. But the only downside is, the client app needs to be run in the user's machine when required. So they couldn't turnoff the PC at that time.

    Yes, we thought of this before, but there are two main reasons why we can't use this approach: 1) The client's PC must be ON as you mentioned and 2) We can't rely on the client's internet speed to do the API calls. We have to do it through our server because the speed is what gives us the advantage over our competitors...

    I'll have to check what kind of server hosting we have and see if we have access to the root, and if we do, maybe installing java and making a back end java scheduling manager to do multithreading will be the only way...

  11. #11
    Freelancer akhileshbc's Avatar
    Join Date
    Jun 2008
    Location
    Trivandrum, Kerala, India
    Posts
    7,652

    Re: Need advice on server-side issue

    I believe, most of the those shared hosting plans discourage something like this that eats up the server resources. A dedicated server might be needed and this might cost you some, I think.

    I own a shared hosting account from Hostgator. And they don't allow passing above 25% CPU usage. Because, since we are in a shared hosting, there are other sites too hosted in the same server. So, CPU usage to the maximum might affect the other sites also.


    If my post was helpful to you, then express your gratitude using Rate this Post.
    And if your problem is SOLVED, then please Mark the Thread as RESOLVED (see it in action - video)
    My system: AMD FX 6100, Gigabyte Motherboard, 8 GB Crossair Vengance, Cooler Master 450W Thunder PSU, 1.4 TB HDD, 18.5" TFT(Wide), Antec V1 Cabinet

    Social Group: VBForums - Developers from India


    Skills: PHP, MySQL, jQuery, VB.Net, Photoshop, CodeIgniter, Bootstrap,...

  12. #12

    Thread Starter
    Addicted Member
    Join Date
    Dec 2010
    Posts
    213

    Re: Need advice on server-side issue

    Yeah thanks for the information. Yes, we'll have to check if we have shared hosting or not and possibly get dedicated hosting if we need to pursue this thing any further.

  13. #13
    Registered User
    Join Date
    Mar 2015
    Posts
    9

    Re: Need advice on server-side issue

    HI Zero, I was curious if a set up like this was possible. Were you able to complete this? Can you also please share some details?

  14. #14
    New Member Tattia's Avatar
    Join Date
    Apr 2019
    Posts
    8

    Re: Need advice on server-side issue

    I have seen such a decision on Github.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Click Here to Expand Forum to Full Width