Sudden spike in PHP-FPM process CPU usage. How to track down?

Question

I have a magento2 store(with 10609 products) running on an ec2 instance(ubuntu 20.04, 16core and 32GB ram). It is a shared server with staging and live on the same instance.

A few days ago, I got an update from amazon saying high cpu usage. when i tracked down Php-fpm logs, slow logs and access logs i observed that there were sudden purging events in the access logs. when i saw PHP-fpm logs it normal.

access logs

. - - [01/Jan/2023:07:32:22 +0300] "PURGE / HTTP/1.1" 301 178 "-" "-" . - - [01/Jan/2023:07:32:22 +0300] "PURGE / HTTP/1.1" 301 178 "-" "-" . - - [01/Jan/2023:07:32:22 +0300] "PURGE / HTTP/1.1" 301 178 "-" "-" . - - [01/Jan/2023:07:32:22 +0300] "PURGE / HTTP/1.1" 301 178 "-" "-" . - - [01/Jan/2023:07:32:22 +0300] "PURGE / HTTP/1.1" 301 178 "-" "-" . - - [01/Jan/2023:07:32:22 +0300] "PURGE / HTTP/1.1" 301 178 "-" "-" . - - [01/Jan/2023:07:32:22 +0300] "PURGE / HTTP/1.1" 301 178 "-" "-" . - - [01/Jan/2023:07:32:22 +0300] "PURGE / HTTP/1.1" 301 178 "-" "-" [01/Jan/2023:07:32:22 +0300] "PURGE / HTTP/1.1" 301 178 "-" "-" . - - [01/Jan/2023:07:32:22 +0300] "PURGE / HTTP/1.1" 301 178 "-" "-" . - - [01/Jan/2023:07:32:22 +0300] "PURGE / HTTP/1.1" 301 178 "-" "-" . - - [01/Jan/2023:07:32:22 +0300] "PURGE / HTTP/1.1" 301 178 "-" "-" . - - [01/Jan/2023:07:32:22 +0300] "PURGE / HTTP/1.1" 301 178 "-" "-" . - - [01/Jan/2023:07:32:22 +0300] "PURGE / HTTP/1.1" 301 178 "-" "-" . - - [01/Jan/2023:07:32:22 +0300] "PURGE / HTTP/1.1" 301 178 "-" "-" . - - [01/Jan/2023:07:32:22 +0300] "PURGE / HTTP/1.1" 301 178 "-" "-" . - - [01/Jan/2023:07:32:22 +0300] "PURGE / HTTP/1.1" 301 178 "-" "-" . - - [01/Jan/2023:07:32:23 +0300] "PURGE / HTTP/1.1" 301 178 "-" "-" . - - [01/Jan/2023:07:32:23 +0300] "PURGE / HTTP/1.1" 301 178 "-" "-" . - - [01/Jan/2023:07:32:23 +0300] "PURGE / HTTP/1.1" 301 178 "-" "-" . - - [01/Jan/2023:07:32:23 +0300] "PURGE / HTTP/1.1" 301 178 "-" "-" . - - [01/Jan/2023:07:32:23 +0300] "PURGE / HTTP/1.1" 301 178 "-" "-" . - - [01/Jan/2023:07:32:23 +0300] "PURGE / HTTP/1.1" 301 178 "-" "-" . - - [01/Jan/2023:07:32:23 +0300] "PURGE / HTTP/1.1" 301 178 "-" "-" . - - [01/Jan/2023:07:32:23 +0300] "PURGE / HTTP/1.1" 301 178 "-" "-" . - - [01/Jan/2023:07:32:23 +0300] "PURGE / HTTP/1.1" 301 178 "-" "-" . - - [01/Jan/2023:07:32:23 +0300] "PURGE / HTTP/1.1" 301 178 "-" "-" . - - [01/Jan/2023:07:32:23 +0300] "PURGE / HTTP/1.1" 301 178 "-" "-" . - - [01/Jan/2023:07:32:23 +0300] "PURGE / HTTP/1.1" 301 178 "-" "-" 

The configuration file of php is as following:

pm = dynamic pm.max_children = 70 pm.start_servers = 1 pm.min_spare_servers = 1 pm.max_spare_servers = 7 php_value[upload_tmp_dir] = php_value[session.save_path] = php_admin_value[error_log] = /home/logs/ php_admin_flag[log_errors] = on php_admin_value[memory_limit] = 2048M pm.max_requests = 500 request_slowlog_timeout = 10 slowlog = /home/logs/php_slow.log 

The db conf is as following

* Fine Tuning max_connections = 2000 connect_timeout = 50 wait_timeout = 600 max_allowed_packet = 2048M thread_cache_size = 128 sort_buffer_size = 4M bulk_insert_buffer_size = 16M tmp_table_size = 32M max_heap_table_size = 32M default_storage_engine = InnoDB innodb_buffer_pool_size = 10G innodb_log_buffer_size = 16M innodb_file_per_table = 1 innodb_open_files = 400 innodb_io_capacity = 400 innodb_flush_method = O_DIRECT [1]: https://i.sstatic.net/0A4w1.jpg [2]: https://i.sstatic.net/l8Lk9.jpg 

Answer 1

Any entity in the internet may have sent those PURGE / requests to your webserver.

It might be an attempt of Denial of Service attack.

There are several options to try to mitigate the issue:

• If Magento does not use PURGE requests for anything useful, you can deny them in nginx configuration
• If Magento uses those for some purpose, then you can consider implementing rate-limiting to this endpoint, for example 5 requests / minute. The actual number needs to be determined by Magento usage patterns.

• Rate limiting can be implemented locally with your web server or via a service like Cloudflare.

There is no single setting that can solve this issue magically.