Handling highly CPU intensive process on server

Question

I have a Windows server machine on Amazon EC2 (m4.2xlarge) which handles web requests, does some processing using an app A and returns the results. Problem is, this app A is highly CPU intensive and ends up taking up to 100% CPU. This creates two problem -

1. Returning results take time [I am absolutely ok with this]
2. Since CPU is being used almost to 100%, my server is not able to handle the subsequent web requests and returns error for web services. [This is the problem] I don't want any web service call to be dropped, them returning results late is alright with me.

I tried reducing CPU affinity for app A, but this isn't helping me much. Any other ideas such that there is enough CPU free to handle the web requests, and remaining is used for processing of app A.

P.S. I am interested in the answer from a general server setup point of view. Not necessarily AWS related.

Answer 1

If your back-end service is using all your CPU, then you have 3 options:

1. Move your application to a server with a stronger CPU. This may let your CPU-intensive process finish faster.
2. Move your application to a server with more CPUs or CPU cores. This will allow your front-end application get some CPU time while your back-end process is working.
3. Separate your processes into separate servers. Put your front-end application on one server and your back-end processes on another server.

Long-term, option 3 allows for the best possibilities:

• It allows you to scale your applications separately. For example, if needed you could add more back-end workers without adding more front-end workers. You can also have a bunch of t2.mediums handling the front-end, with a single m4.2xlarge on the back-end.
• It prevents your back-end process from interfering with your front-end process.