Reduce system load when using image processing via php-fpm/nginx

Question

We have a PHP image api written in Code Igniter that's based on imagemagic. We have the api behind nginx/php-fpm. We know the Imagemagic part is heavy, but we try to make the overal throughput higher and higher.

We have went over of more than 30 optimization articles about Nginx. We were able to increase output from 300 requests/minute to 350 request/minute by mostly following the articles and doing TCP optimizations.

By this we seriously affected server load time. There is no avg load I can tell you, as this is a new box, and it's Nginx based. We had in past Apache workers (less than 300 request/minute). Now with 350req/m we have an AVG load of 20. Which is way too much, and we are looking to improve.

When we use htop to investigate the processes, usually we see around 20 processes of php-fpm that are heavily using all 16 CPUs, and memory is somewhere at 2GB out of 24GB.

We aim to reduce system load drastically. I would like experienced users to share their knowledge where to look, what to try. I will post whatever numbers you are requesting, but I didn't wanted to full the post with that at the beginning.

How would you proceed to reduce system load but still keep the throughput.

Answer 1

Since we've established that this is a pure CPU hog issue (no iowait) then I really can't see any way of keeping the same throughput without adding more resources. Either tune down the maximum number of PHP processes or add more CPU cores.

Or get the developers to use something else than ImageMagick..

Answer 2

If you have repeated requests you can check fastcgi cache (http://wiki.nginx.org/HttpFastcgiModule) or use varnish in front of nginx.

This will enable you to cache the processed images and not have to do the heavy cpu processes again.

Answer 3

Since your question states that PHP-FPM processes are in TOP cpu, I image you're using ImageMagick PHP libraries (http://php.net/imagick).

Ask your developer(s) to test the scenario where the image processing is accomplished directly by the ImageMagick binary (ex. /usr/bin/[compare, import, display ...].

If using directly the binary works, then use php function exec() and encapsulate the call to ImageMagick binaries, and see how that goes (in terms of CPU and duration).

@additional info: https://stackoverflow.com/questions/4323672/how-do-i-get-imagemagick-installed

Answer 4

Developers ways

1. try to decrease php overhead for image processing.
2. try to use less resources for image processing. Use prepared images or change some operations with less performance requirements

Admin way

1. use more powerful CPUs