0

I have an Angular website with static assets of around 1.5 mb and gzipped it is around 400 kb, I have nginx as my webserver & reverse proxy to the API server, when I test nginx with Apache benchmark tool, I find huge drop in performance if I test the https site compared to the http (https is 10 times slower) & the cpu utilization & memory is not high at all (cpu 30% memory is only 1 mb!!)

I have been searching for hours & tried all possible enhancements but none worked, as far as I have read https shall not be that much slower on modern web servers (http around 1500 req/sec & https is 46 req/sec for nginx), this is mostly from the Nginx https very high connect time but I have no clue how to solve this.

Can someone advise how to improve this?

(Also to my surprise, Apache performs much better in both cases but doesn't respond if I set concurrent connections to more than 200) & this is not nginx vs apache I am just stating my situation.

Important note:

I am not comparing the 2 web servers that is not the point of this site, but generally they have comparable performance so if https in nginx is 10 times slower than Apache I feel that something is wrong in my Nginx configuration & I want to fix it.

All test are on my windows machine i7 & 16 gb ram.

Nginx http only:

C:\Apache24\bin>ab -n 5000 -c 200 http://localhost:8100/abc/index.html?param=abc This is ApacheBench, Version 2.3 <$Revision: 1826891 $> Server Software: nginx/1.15.4 Server Hostname: localhost Server Port: 8100 Document Path: /abc/index.html?param=abc Document Length: 1099 bytes Concurrency Level: 200 Time taken for tests: 3.246 seconds Complete requests: 5000 Failed requests: 0 Total transferred: 6665000 bytes HTML transferred: 5495000 bytes Requests per second: 1540.32 [#/sec] (mean) Time per request: 129.843 [ms] (mean) Time per request: 0.649 [ms] (mean, across all concurrent requests) Transfer rate: 2005.12 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 1.3 0 16 Processing: 31 87 12.8 94 124 Waiting: 0 87 13.7 94 124 Total: 31 87 12.8 94 124 Percentage of the requests served within a certain time (ms) 50% 94 66% 94 75% 94 80% 94 90% 99 95% 109 98% 109 99% 113 100% 124 (longest request)

Nginx https (with http2 enabled)

C:\Apache24\bin>abs -n 5000 -c 200 https://localhost:8200/abc/index.html?param=abc This is ApacheBench, Version 2.3 <$Revision: 1826891 $> Server Software: nginx/1.15.4 Server Hostname: localhost Server Port: 8200 SSL/TLS Protocol: TLSv1.2,ECDHE-RSA-AES256-GCM-SHA384,2048,256 TLS Server Name: localhost Document Path: /abc/index.html?param=abc Document Length: 1099 bytes Concurrency Level: 200 Time taken for tests: 108.985 seconds Complete requests: 5000 Failed requests: 0 Total transferred: 6780000 bytes HTML transferred: 5495000 bytes Requests per second: 45.88 [#/sec] (mean) Time per request: 4359.386 [ms] (mean) Time per request: 21.797 [ms] (mean, across all concurrent requests) Transfer rate: 60.75 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 16 4201 506.8 4251 4755 Processing: 0 32 12.6 31 88 Waiting: 0 32 12.6 31 88 Total: 62 4232 506.9 4283 4800 Percentage of the requests served within a certain time (ms) 50% 4283 66% 4342 75% 4413 80% 4439 90% 4484 95% 4547 98% 4694 99% 4727 100% 4800 (longest request)

Compared to Apache http (here CPU is around 90 to 100% utilized)

C:\Apache24\bin>ab -n 5000 -c 200 http://localhost:6200/abc/index.html?param=abc Server Software: Apache/2.4.33 Server Hostname: localhost Server Port: 6200 Document Path: /abc/index.html?param=abc Document Length: 1099 bytes Concurrency Level: 200 Time taken for tests: 1.781 seconds Complete requests: 5000 Failed requests: 0 Total transferred: 6810000 bytes HTML transferred: 5495000 bytes Requests per second: 2806.99 [#/sec] (mean) Time per request: 71.251 [ms] (mean) Time per request: 0.356 [ms] (mean, across all concurrent requests) Transfer rate: 3733.51 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 0 1.6 0 16 Processing: 16 69 16.0 63 125 Waiting: 0 57 16.0 63 125 Total: 16 69 16.0 63 125 Percentage of the requests served within a certain time (ms) 50% 63 66% 78 75% 78 80% 78 90% 94 95% 94 98% 94 99% 109 100% 125 (longest request)

And Apache https is as follows (http 1.1) & note that http 1.1 in nginx didn't improve its performance:

C:\Apache24\bin>abs -n 5000 -c 200 https://localhost:7200/abc/index.html?param=abc This is ApacheBench, Version 2.3 <$Revision: 1826891 $> Server Software: Apache/2.4.33 Server Hostname: localhost Server Port: 7200 SSL/TLS Protocol: TLSv1.2,ECDHE-RSA-AES256-GCM-SHA384,2048,256 TLS Server Name: localhost Document Path: /abc/index.html?param=abc Document Length: 1099 bytes Concurrency Level: 200 Time taken for tests: 8.747 seconds Complete requests: 5000 Failed requests: 0 Total transferred: 6810000 bytes HTML transferred: 5495000 bytes Requests per second: 571.60 [#/sec] (mean) Time per request: 349.894 [ms] (mean) Time per request: 1.749 [ms] (mean, across all concurrent requests) Transfer rate: 760.27 [Kbytes/sec] received Connection Times (ms) min mean[+/-sd] median max Connect: 0 198 42.7 188 391 Processing: 62 145 39.1 140 385 Waiting: 0 76 28.3 78 250 Total: 62 343 63.0 331 615 Percentage of the requests served within a certain time (ms) 50% 331 66% 369 75% 380 80% 389 90% 422 95% 465 98% 500 99% 536 100% 615 (longest request)

My nginx configuration:

worker_processes auto; events { worker_connections 1024; } http { include mime.types; default_type application/octet-stream; sendfile on; keepalive_timeout 65; server { listen 8100; server_name localhost; location / { root html; index index.html index.htm; } #error_page 404 /404.html; # redirect server error pages to the static page /50x.html # error_page 500 502 503 504 /50x.html; location = /50x.html { root html; } } server { listen 8200 ssl http2; server_name localhost; ssl_certificate C:/nginx-1.13.12/conf/server.crt; ssl_certificate_key C:/nginx-1.13.12/conf/server.key; ssl_session_cache shared:SSL:10m; ssl_session_timeout 10m; ssl_ciphers HIGH:!aNULL:!MD5; ssl_prefer_server_ciphers on; gzip on; gzip_comp_level 1; gzip_vary on; gzip_types text/css text/javascript text/xml text/plain text/x-component application/javascript application/json application/xml application/rss+xml font/truetype font/opentype application/vnd.ms-fontobject image/svg+xml; gzip_static on; location /ipo_reits/ { root html; index index.html index.htm; ## here we redirect to the homepage in case of nginx 404 try_files $uri $uri/ /ipo_reits/index.html; # error_page 404 =301 /; } location /api/ { proxy_pass https://localhost:7001/; } } }

Update

I have tested on Linux as advised by kworr & nginx performance was much better in https

Nginx http: 17000 req/sec & https around 1200 req/sec, apache has a similar performance ratio between https 1150 req/sec & http 13000 req/sec

Why is that huge difference between http & https in both web servers ?

Http is around 12 times faster than https though many references are assuring difference shall be negligible?

Improve this question
6
  • This can be a cipher difference too. Can you check what ciphers apache uses when serving data? Or do attach apache config. PS: I don't think nginx-on-windows is production ready, it's not suppported by the team. To get meaningful results please retry same test on some unix host. Or please do check how connections are accepted and what you can tweak in listen clause. Commented Oct 2, 2018 at 15:03
  • @kworr Thanks for your help, actually the apache benchmark tool shows in the output shown above that nginx & apache are using the same cipher SSL/TLS Protocol: TLSv1.2,ECDHE-RSA-AES256-GCM-SHA384,2048,256 I didn't know nginx is not prod ready, I will try to redo the test on linux & update the post Commented Oct 2, 2018 at 15:51
  • @kworr thanks a lot for your help, performance on linux on same machine was way higher, I have posted an answer showing that to help anyone who will face the same, thanks Commented Oct 2, 2018 at 18:30
  • Since the core question is still valid, as http even in linux is 12 times faster, I updated the question & removed my answer, I hope someone can explain to me why http is that much faster than https & if that is the normal case Commented Oct 2, 2018 at 19:55
  • https means your data will be encrypted and decrypted, that's a lot of work added. Commented Oct 2, 2018 at 21:02

1 Answer 1

Reset to default
4

HTTPS will be considerably slower than HTTP for this simply test case.

HTTPS means two things:

  1. You need an initial set up to agree the encryption parameters (the TLS handshake).
  2. You need compute power to encrypt each request before sending and decrypt each request after receiving.

Now point 2 is negligible these days with the correct set up and for most traffic. Large volumes of traffic (e.g. if you were delivering mostly video like Netflix or YouTube) might notice a difference but most sites won’t really unless on really antiquated hardware or really bad set up.

Point one however is interesting. TLSv1.2, that you are using, requires 2 full round trips to establish the HTTPS connection on top of the 1.5 round trips required for a TCP connection and the 1 round trip for the actual HTTP request. TLSv1.3 reduces this to one round trip (or even zero if re-establishing a connection) and that’s one of the many reasons people are getting excited by it and it is starting to be come available in implementations now.

Therefore, for now, I would expect HTTPS to be almost twice as slow as HTTP at the very least - ignoring the encryption processing cost itself completely and assuming the download of the web page is small, as you are comparing 4.5 round trips to 2.5 round trips.

However the calculations do have a cost. HTTPS uses public key asymmetric encryption based on the TLS certificate provided by the site. This is needed because you don’t have a pre-agreed secret key you can use with every website in the world (especially those ones by yet created!) so you cant use the faster symmetric encryption. Asymmetric encryption is notoriously slow, so to help counteract that, this is only used to establish an symmetric key which is then used for the rest of the encryption. However it means the initial handshake again will be considerably slower because it is doing that expensive public key encryption and decryption.

I’ve never measure the difference using ab to be honest but I’m not completely surprised by it being 10-13 times slower for these reasons compared to using HTTP which just requires establishing a TCP connection and sending the page back.

So why the heck are any websites using HTTPS and how come most people say the performance impact is not that much?!?! Well that is because very few websites load just one page. A website is usually made of a myriad of resources (75 on average) and only the first one for each domain is hit by that handshake delay.

Therefore your ab test is not representative of the real word (unless you have a VERY simple website). Instead you should use wget -p or similar to download both the web page and the dependencies. Then time that and only then can you see what the difference is. I would still expect a difference but not as much as you have seen with your simple ab test.

However even that is not representative as downloading is only one part. If it takes 1 second to download your site and 15 seconds to chug through all the JavaScript, advertising tracking...etc. then even a doubling of that 1 second download time isn’t going to make a material difference. So to be a truly accurate test you should measure both downloading and rendering to get a true cost. Tools like webpagetest can help with this.

You should also ensure your HTTPS set up is optimised to use faster ciphers, ensure TLS resumption is enabled (so any subsequent connections don’t suffer as much of a performance penalty) and other settings.

Ultimately we are moving to an HTTPS world for good reasons and while there is a performance impact for that initial set up, there are also other performance benefits like Brotli compression and HTTP/2 which are only available over HTTPS (for both idealogical and practical reasons). These can leave HTTP in the dust when it comes to performance though often sites showcases this (like that one of mine) are often based on artificial tests that are not really representative of most websites so most websites won’t see quite that much performance improvement!

Improve this answer

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.