2

We have an nginx instance sitting in front of apache 2 (nginx handles static resources, dynamic content requests get proxied to apache) which is primarily using mod_wsgi to server various python/django apps.

We have suddenly seeing a high server load (steadily climbing to more than 50, normally it's less than 1) and are looking for the best ways to track down the cause.

Is there a way to see which virtual hosts are responsible for the apache processes?

Some general tips in diagnosing the cause of high server load would be welcome too.

Edit: Thanks to everyone for their help, it turned out to be caused by a code error and no server reconfiguration was necessary. However, the tips provided will be useful in the future.

Improve this question

2 Answers 2

Reset to default
4

The mod_status module in Apache will show you which virtualhost / request a particular process is handling, but this will only be useful to you if the requests are long-lived. Adding execution time (%D) to your Apache LogFormat is also useful.

More important is tracking down the bottleneck. For this you need to investigate iostat and vmstat (usually installed by default) and a lovely newer tool that combines the two and adds even more stuff, dstat.

Improve this answer
5
  • We have just had dstat installed, but we're not really sure how to use it in this context. Commented Aug 24, 2009 at 16:19
  • Run dstat and take a look at the output: What is the CPU spending its time on? If the "usr" column is where the high numbers are, your process is CPU bound. If it's the "wai" column, your CPU is blocking on IO. If that's the case, take a look at your disk and paging columns. If you see blocks continually being swapped in and out, you're running out of memory. Those are just the two most common / likely scenarios. Commented Aug 24, 2009 at 22:55
  • If it's still not making sense, feel free to edit your question and add 15 or 20 lines of dstat output. Commented Aug 24, 2009 at 22:56
  • it's calmed down at the moment, but when it happens again we'll try Jeff Waugh's suggestion. If that doesn't work I'll add in the dstat output. Commented Aug 25, 2009 at 8:31
  • however, the machine has 20gb of RAM, so I would be suprised if it is a memory problem Commented Aug 25, 2009 at 8:33
1

If you're seeing a lot of action in the 'wa' column under 'cpu' when you run vmstat (ie. a lot of time spent waiting on input/output), then I suspect the problem is actually nginx, not Apache. It may be doing too much I/O because its buffers aren't large enough to handle the proxied content.

Try adding this to nginx.conf:

proxy_buffers 32 4k;

Increase as necessary.

Improve this answer
2
  • This seems like a plausible explanation, the primary function of the server is serving dynamically generated resized images, so almost all files served are going to be large compared to html. Do you have any advice on optimising nginx for this kind of proxying? Commented Aug 25, 2009 at 9:06
  • It's all about getting the proxy buffer settings right... You want the proxy_buffer* settings big enough to avoid hitting the disk as much as possible, and have proxy_temp_path somewhere sensible just in case nginx does have to hit the disk. Commented Aug 25, 2009 at 9:55

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.