Slow Azure Kubernetes serving of static files with Nginx ingress controller

Question

I have an Angular application whose users are experiencing random latency in the loading of pages, so I installed Opentelemetry, Tempo and Grafana to monitor app traces.

The app is deployed in an Azure Kubernetes cluster behind an nginx ingress-controller community edition. From the analysis of traces and spans the slowest calls seem to be servings of static files (.js, .tff, .css) that spend little time in the actual backend service, while a long time coming back to the nginx ingress-controller, or in the ingress-controller itslef. In the screenshot below for example, the call takes one minute, but just 2.73ms in the backend pod.

What could be the cause of this? Has anyone had experience with such an issue, or know a way to further debug it?

Answer 1

There are many factors that may cause slow loading of static files in your Azure Kubernetes cluster with Nginx ingress controller: Check your Nginx ingress controller configuration for cache settings, Health checks, Resource utilization and logs.

Follow below troubleshooting steps :

• Check your Nginx ingress controller configuration again for cache settings. Ensure that js, tff and css files for static contents are cached properly with correct expiration time.

• Make sure you do not have unnecessary overheads by cross-checking the health checks of your Nginx ingress controller

• check Nginx ingress controller resource utilization cpu,memory.

• Make sure the network latency between your application backend pods and Nginx ingress controller pod is to be investigated.

• It may be wise to explore Azure Application insights to capture performance metrics from both the app and Nginx ingress controller.

• If you have changed something in either how you have developed or deployed an application, consider rolling back to a previous stable version just to confirm if there is a problem.