2

I have set up nginx reverse proxy and added caching to it using nginx caching.

I can verify whether the requested copy is cached or not using added header.

add_header X-Cache-Status $upstream_cache_status;

i have set to cache page if its get accessed more than twice.

proxy_cache_min_uses 2;

Nginx caching is working as it should, i can see the X-Cache-Status: MISS twice and later X-Cache-Status: HIT if i try to access the webpage from any browser.

But if i try to check same with curl , if the page is not cached i am getting 

X-Cache-Status: MISS

and if the page is cached i am getting 

X-Cache-Status: HIT

But my issue is , i cant able to warm up cache using curl. i mean i cant make webserver make cache specific page by accessing page using curl even if i access it 10 times using curl.

why is that ?

So far i have tried .

curl -I https://example.com/page1 curl -v https://example.com/page1 curl --verbose https://example.com/page1 curl -svo /dev/null https://example.com/page1

using latest centos 7x64

latest Nginx

[root]# curl -V curl 7.29.0 (x86_64-redhat-linux-gnu) libcurl/7.29.0 NSS/3.19.1 Basic ECC zlib/1.2.7 libidn/1.28 libssh2/1.4.3 Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smtp smtps telnet tftp Features: AsynchDNS GSS-Negotiate IDN IPv6 Largefile NTLM NTLM_WB SSL libz

update1: here is my nginx config file

proxy_cache_path /usr/local/nginx/cache levels=1:2 keys_zone=mycache:10m max_size=10g use_temp_path=off; server { access_log /dev/null; listen my_ip:80; server_name example.com ; location / { access_log off; proxy_cache mycache; proxy_cache_min_uses 2; proxy_cache_valid 200 10m; proxy_cache_valid 404 1m; proxy_cache_use_stale error timeout invalid_header updating http_500 http_502 http_503 http_504; proxy_cache_lock on; proxy_pass http://box1.example.com/; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; #hide headers proxy_hide_header Cache-Control; #ignore header proxy_ignore_headers Cache-Control Expires; #add custom headers for users add_header X-Cache-Status $upstream_cache_status; expires 3h; add_header Cache-Control "public, max-age=10800, s-maxage=10800"; #add no mobile redirect cookie add_header Set-Cookie "__cf_mob_redir=0;Domain=.example.com;"; } }

update 2:
I am using cloudflare on my domain. I have tried disabling cloudflare service altogether and only using it as a DNS server . Tried again with curl and got same response.

It does not matter whether cloudflare is enabled or not curl cant make pages cache.

Update 3 :

Tried using different User-agnts but received same response.

curl -A "Mozilla/5.0 (Windows; U; Windows NT 5.1; de; rv:1.9.2.3) Gecko/20100401 Firefox/3.6.3" -svo /dev/null http://example.com/some-page
Improve this question
11
  • Please share your Nginx config, especially the part around caching. Commented Nov 17, 2016 at 4:28
  • You're using old http1 cache control headers, have a read of a tutorial I wrote about that photographerstechsupport.com/tutorials/… . This doesn't answer your question but is worth considering. Short version: only header required is "Cache-Control". Unfortunately I can't answer your curl question. If it's not causing a problem I'd probably let it go, but it'd be interesting to get an answer. Commented Nov 17, 2016 at 18:47
  • @Tim Thanks updated config file. and still the same. Commented Nov 23, 2016 at 13:49
  • Do you see cache misses from web browsers? Try Firefox and maybe Chrome with the plug "Live HTTP Headers" it's ideal for this. If web browsers work but curl doesn't then there's no actual problem, just curiosity. Commented Nov 23, 2016 at 18:22
  • @Tim yes, i can see MISS twice then HIT afterwards in browser. practically its working perfectly. but i would like to warm up cache from time to time. so i need curl to cache pages too. and i am using cloudflare on my website, i even tried disabling cloudflare and trying with curl, but still the same. Commented Nov 25, 2016 at 5:25

1 Answer 1

Reset to default
1

0) As I can see you check https:// site but configuration for http:// version. I guess it's not a problem in your case but better to check that you edit right configuration file.
1) You should check what you got from upstream curl -I http://box1.example.com/page1 and I guess that there will be Set-Cookie header. So you should either add Set-Cookie to proxy_ignore_headers or remove cookies from upstream's reply. Or use separate locations: one for pages where you could ignore cookies for caching and another for pages where you can't ignore cookies.

Improve this answer

You must log in to answer this question.