0

I have a django site on webfaction that uses apache + mod_wsgi.

Site is getting around 1000 requests per minute.

But it makes some calculations, so request takes about 5-10 seconds.

I use the following configuration

StartServers 2 MinSpareThreads 10 MaxSpareThreads 25 ThreadLimit 25 ThreadsPerChild 25 MaxClients 75 MaxRequestsPerChild 1000

threads=15 processes=12

The problem is high CPU usage and it takes time to process a simple static page without calculations (looks like Apache queued the request).

So what I want is for Apache to quickly accept requests.

I'm totally lost because of number of parameters, I also don't quite understand what they mean. What do we need StartServers and MaxRequestWorkers for?

Any help and/or explanations will be highly appreciated.

I have 8GB of RAM.

Apache MPM Worker.

mod_wsgi 4.4.21.

Thank you in advance.

Improve this question
0

1 Answer 1

Reset to default
0

StartServers is the number of number of server processes you start with, and MaxRequestWorkers is the number of threads per process. The Webfaction settings should be reasonable in most situations although a thousand requests a minute may need some tuning, but probably mostly in the application.

In normal use httpd would take a request, pass it to mod_wsgi and wait for its return, which should be near instantaneous, so what is actually taking its time is whatever the python script is doing. Your httpd worker threads are therefore in a wait state and will build up as requests come in while other requests are being processed, so even a static page will end up waiting if your threads are occupied.

Look at what the application is doing and for solutions. You may be able to cache queries using memcached or similar. If the time that your application takes to process a request is unavoidable, look at making it asynchronous using a message queue like Celery, so rather than having your web server wait for responses, you can poll for them using browser side scripting.

Splitting the static page serving from the dynamic will also improve the response. If possible you could run multiple sets of workers, or pass static page and object serving to nginx, which is a more common way of handling wsgi.

Another method would be to serve python through a native web server such as tornado or gunicorn and use apache as a reverse proxy, which may improve the backend response although still won't help if processes are causing large numbers of waiting threads.

Improve this answer
6
  • "Your httpd worker threads are therefore in a wait state and will build up as requests come in while other requests are being processed, so even a static page will end up waiting if your threads are occupied." So the question is about optimal combination for this situation. I want static pages (pages without calculations) to get very fast and pages with calculations are already loaded asynchronously. Commented Feb 7, 2018 at 16:27
  • See the last paragraph of my reply. You can also route with apache as long as your static pages are identifiable, such as ending in .html but you would need separate processes to handle static dynamic objects and the response from mod_wsgi if the dynamic handler is still blocking. However, if as you say, your calculations are handled asynchronously then you need to look at a different way to handle the responses on the page. Commented Feb 7, 2018 at 16:57
  • Can you provide some example configuration? Commented Feb 7, 2018 at 17:14
  • In all honesty the simplest answer would probably to serve static objects from another cloud instance. However, you need to stop your application causing httpd threads to wait as no amount of tweaking will stop that, and the answer is most probably to use a browser based front end to poll for responses from the back. Commented Feb 7, 2018 at 22:26
  • What is the difference between Apache number of processes and mod_wsgi number of processes? How do they interact? Commented Feb 8, 2018 at 12:14

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.