Tips for benchmarking and load testing web server?

Question

I'm trying to benchmark and load test various web server setups (mod_php vs php as fastcgi, max/min worker/thread configuration, and so on). I'm starting with ab, but would be interested in other recommended tools for this job. The web server is a Linux server, but I can run Windows based testing tools from my desktop machine if need be.

I'm also interested in any recommendations as to technique. For example:

• I'm assuming I'll get more useful results running from a separate machine doing nothing else (as opposed to running on the same machine as the web server, or on my desktop machine as a background process). Is this correct?

• Likewise, I figure I can reduce the effects of network issues on the testing by putting that machine in the same data center on a private IP.

• I'll want to setup more than just a single file to get any sort of real test (can I even do this with ab?).

Any other tips?

Answer 1

I like to set up a classroom of computers who run siege.

I crawl the website once using wget

wget -r -l0 -np -O /dev/null http://domainname.com // Note that the -l0 means that it will recurse infinitly, depending on your site, // you might want to change that to something lower, say ... -l3 // This will crawl the site, and throw all the contents away. 

Then I fetch all the unique URLs from the serverlogs by using a oneliner like this:

awk '{ print "http://domainname.com"$7 }' access.log | sort | uniq 

Then siege can be set up to use this file, and will then attempt them in random order.

siege -f siegeurls -i -b 

It can also be configured to send various headers, set useragent and whatnot.

I'd recommend that you take a look at siege. This is the setup I use when I test website-rigs myself.