Apache processes stuck in "write" state

Question

My apache server is producing lots of processes in "write" state until all slots are filled and the whole webserver hangs. Here's what I got:

root@viking:/# uname -a Linux viking 2.6.32-51-server #113-Ubuntu SMP Wed Aug 21 20:01:09 UTC 2013 x86_64 GNU/Linux root@viking:/# apache2 -V Server version: Apache/2.2.14 (Ubuntu) 

Here's a stack of hanged process:

root@viking:/# cat /proc/14475/stack [<ffffffff81158c39>] poll_schedule_timeout+0x39/0x60 [<ffffffff81158e93>] do_poll+0x233/0x2c0 [<ffffffff81159965>] do_sys_poll+0x155/0x210 [<ffffffff81159c1c>] sys_poll+0x7c/0x110 [<ffffffff81013172>] system_call_fastpath+0x16/0x1b [<ffffffffffffffff>] 0xffffffffffffffff 

This is hot it appears in server-status:

Srv PID Acc M CPU SS Req Conn Child Slot Client VHost Request 0-0 14472 0/1033/1033 W 8.65 965 0 0.0 8.43 8.43 178.129.218.193 ***.com GET /programs/ HTTP/1.0 

What may cause such behavior?

Answer 1

One common cause of this behavior is malicious clients. They connect to your web server, request some data, and then never read the data your server sends them. This eats up your connection slots.

Answer 2

It appeared that it were hanged curl requests to unavailable service. For some reason curl never times out and waits for response forever.