0

I am using Nginx as an SSL terminator in front of varnish. Backends are litespeed. I have two wordpress sites in which both of them uses HTTPS protocol. I have a Haproxy in front of my infrastructure which only passes SSL traffics to Nginx.

One of my websites is loading fast and when I check the status of cached items through firebug, everything get a 'HIT' like .js .css and the domain name: domain1.com.

When I check these items for the second wordpress site, .js and HTML file get 'MISS' the same as the domain name domain2.com.

The configuration of Nginx and Varnish and the backend is the same for both web sites. I also disabled all of its plugins and the ones I think affect the caching :WP Fastest Cache, 404 to 301,Broken Link Checker and some others.

The only difference is that the second web site has a robot which gathers news from other sites and put them on the sites(some kind of web crawler).

What causes the varnish not to cache cache everything on the second site?


Edit:
I added the varnish log for a requests which is not cached:

varnishlog -g request -q "ReqUrl ~ '/wp-content/themes/farda/js/jquery.min.js'" * << Request >> 149 - Begin req 148 rxreq - Timestamp Start: 1499626236.025652 0.000000 0.000000 - Timestamp Req: 1499626236.025652 0.000000 0.000000 - ReqStart 192.168.1.108 57470 - ReqMethod GET - ReqURL /wp-content/themes/farda/js/jquery.min.js - ReqProtocol HTTP/1.0 - ReqHeader X-Real-IP: 192.168.1.105 - ReqHeader X-Forwarded-For: 192.168.1.105 - ReqHeader X-Forwarded-Proto: https - ReqHeader X-Nginx: on - ReqHeader Host: mytourguide.ir - ReqHeader Connection: close - ReqHeader User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:51.0) Gecko/20100101 Firefox/51.0 - ReqHeader Accept: */* - ReqHeader Accept-Language: en-US,en;q=0.5 - ReqHeader Accept-Encoding: gzip, deflate, br - ReqHeader Referer: https://mytourguide.ir/ - ReqHeader Pragma: no-cache - ReqHeader Cache-Control: no-cache - ReqUnset X-Forwarded-For: 192.168.1.105 - ReqHeader X-Forwarded-For: 192.168.1.105, 192.168.1.108 - VCL_call RECV - ReqHeader Cookie: - ReqUnset Cookie: - ReqHeader Cookie: - ReqUnset Cookie: - ReqHeader Cookie: - ReqUnset Cookie: - ReqHeader Cookie: - ReqUnset Cookie: - ReqUnset Accept-Encoding: gzip, deflate, br - ReqHeader Accept-Encoding: gzip - ReqUnset X-Forwarded-For: 192.168.1.105, 192.168.1.108 - ReqHeader X-Forwarded-For: 192.168.1.108 - ReqUnset Accept-Language: en-US,en;q=0.5 - ReqUnset User-Agent: Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:51.0) Gecko/20100101 Firefox/51.0 - ReqHeader cookie: - ReqUnset cookie: - ReqHeader cookie: - VCL_return hash - VCL_call HASH - VCL_return lookup - Debug "XXXX MISS" - VCL_call MISS - VCL_return fetch - Link bereq 150 fetch - Timestamp Fetch: 1499626236.029726 0.004075 0.004075 - RespProtocol HTTP/1.1 - RespStatus 200 - RespReason OK - RespHeader Cache-Control: max-age=2592000, private - RespHeader Expires: Mon, 09 Jul 2018 18:50:18 GMT - RespHeader Last-Modified: Tue, 17 Jan 2017 14:46:21 GMT - RespHeader Content-Type: text/javascript; charset=UTF-8 - RespHeader Content-Length: 33342 - RespHeader Content-Encoding: gzip - RespHeader Vary: Accept-Encoding,User-Agent - RespHeader Date: Sun, 09 Jul 2017 18:50:18 GMT - RespHeader Accept-Ranges: bytes - RespHeader Server: LiteSpeed - RespHeader X-Varnish: 149 - RespHeader Age: 0 - RespHeader Via: 1.1 varnish-v4 - VCL_call DELIVER - RespHeader X-Cache: MISS - RespUnset X-Varnish: 149 - RespUnset Via: 1.1 varnish-v4 - RespHeader X-Configured-By: ServerSetup.ir - VCL_return deliver - Timestamp Process: 1499626236.029739 0.004087 0.000012 - Debug "RES_MODE 2" - RespHeader Connection: close - RespHeader Accept-Ranges: bytes - Timestamp Resp: 1499626236.030691 0.005039 0.000952 - Debug "XXX REF 2" - ReqAcct 445 0 445 448 33342 33790 - End ** << BeReq >> 150 -- Begin bereq 149 fetch -- Timestamp Start: 1499626236.025722 0.000000 0.000000 -- BereqMethod GET -- BereqURL /wp-content/themes/farda/js/jquery.min.js -- BereqProtocol HTTP/1.0 -- BereqHeader X-Real-IP: 192.168.1.105 -- BereqHeader X-Forwarded-Proto: https -- BereqHeader X-Nginx: on -- BereqHeader Host: mytourguide.ir -- BereqHeader Accept: */* -- BereqHeader Referer: https://mytourguide.ir/ -- BereqHeader Pragma: no-cache -- BereqHeader Accept-Encoding: gzip -- BereqHeader X-Forwarded-For: 192.168.1.108 -- BereqHeader cookie: -- BereqProtocol HTTP/1.1 -- BereqHeader X-Varnish: 150 -- VCL_call BACKEND_FETCH -- VCL_return fetch -- Backend 17 apache web1(192.168.1.11,,5050) -- Timestamp Bereq: 1499626236.025798 0.000076 0.000076 -- Timestamp Beresp: 1499626236.027971 0.002249 0.002173 -- BerespProtocol HTTP/1.1 -- BerespStatus 200 -- BerespReason OK -- BerespHeader Cache-Control: max-age=2592000, private -- BerespHeader Expires: Mon, 09 Jul 2018 18:50:18 GMT -- BerespHeader Last-Modified: Tue, 17 Jan 2017 14:46:21 GMT -- BerespHeader Content-Type: text/javascript; charset=UTF-8 -- BerespHeader Content-Length: 33342 -- BerespHeader Content-Encoding: gzip -- BerespHeader Vary: Accept-Encoding,User-Agent -- BerespHeader Date: Sun, 09 Jul 2017 18:50:18 GMT -- BerespHeader Accept-Ranges: bytes -- BerespHeader Server: LiteSpeed -- BerespHeader Connection: Keep-Alive -- TTL RFC 2592000 -1 -1 1499626236 1499626236 1499626218 1531162218 2592000 -- VCL_call BACKEND_RESPONSE -- TTL VCL 120 10 0 1499626236 -- VCL_return deliver -- Storage malloc Transient -- ObjProtocol HTTP/1.1 -- ObjStatus 200 -- ObjReason OK -- ObjHeader Cache-Control: max-age=2592000, private -- ObjHeader Expires: Mon, 09 Jul 2018 18:50:18 GMT -- ObjHeader Last-Modified: Tue, 17 Jan 2017 14:46:21 GMT -- ObjHeader Content-Type: text/javascript; charset=UTF-8 -- ObjHeader Content-Length: 33342 -- ObjHeader Content-Encoding: gzip -- ObjHeader Vary: Accept-Encoding,User-Agent -- ObjHeader Date: Sun, 09 Jul 2017 18:50:18 GMT -- ObjHeader Accept-Ranges: bytes -- ObjHeader Server: LiteSpeed -- Fetch_Body 3 length stream -- Gzip u F - 33342 95992 80 206366 266672 -- BackendReuse 17 web1(192.168.1.11,,5050) -- Timestamp BerespBody: 1499626236.030637 0.004915 0.002666 -- Length 33342 -- BereqAcct 291 0 291 375 33342 33717 -- End
Improve this question
3
  • If you can add the output of varnishlog -g request from a request you expect to be cached, but which isn't that'll make it much easier to figure out what's wrong. Commented Jul 9, 2017 at 15:04
  • I doubt anyone can tell you based on the information you've provided. You're probably going to need to edit your question to provide a few things, including a) curl showing response headers to each layer (not just Haproxy, also direct curls to the web server(s), b) configurations c) possibly logs. I suggest you look at the caching headers, and also you need to format your question better - I've fixed it up twice now. Commented Jul 9, 2017 at 18:54
  • Thank you. you are right. I added the varnish log for a requests. Commented Jul 9, 2017 at 18:56

1 Answer 1

Reset to default
1

The problem is with backend (LiteSpeed).

Note this line in your varnishlog:

-- ObjHeader Cache-Control: max-age=2592000, private

Varnish built-in VCL will not cache URLs which send Cache-Control: private. Thus it's always being not cached, and MISS-ed later on.

Proper solution: configure LiteSpeed to enable public cache as opposed to private cache.

Improve this answer
5
  • Thank you @Daniel V. I think you are right. But there is another problem. My litespeed license is for VPS with one core ,so I cannot enable public cache! because cache configuration is not accessible with this license. Should I change the web server to something else like nginx or apache? Or there is another way to enabling public cache in litespeed. Commented Jul 10, 2017 at 5:32
  • Another matter is that, I am using Litespeed for two wordpress sites. Why litespeed sends private for Cache-Control for one site and does not send anything for the other one?!! I think something more is changing the Cache-Control parameter in the backend, am I right? Commented Jul 10, 2017 at 5:49
  • @Sinai, this header could very well be sent by PHP (via Wordpress plugin for example), but I don't see any rationale in any plugin doing that in a way that it will affect entire website. So I think in most probability the difference is with LiteSpeed settings. I have little understanding about LiteSpeed settings Commented Jul 10, 2017 at 13:58
  • Nginx doesn't mess with with Cache-Control header unless you tell it too so it might be worth using it instead. Anyway, have you checked the possible ways to set public cache on the linked page in pargraph "Set cache policy at LSWS Server level"? You should be able to enable proper header with .htaccess <IfModule LiteSpeed> CacheLookup public on </IfModule> Commented Jul 10, 2017 at 13:58
  • Thank you @Daniel. The litespeed configurations for both web sites are the same because it is a shared hosting server which has several web sites. Yes I tested CachLookup but it did not work. It is because of the license. I switched to nginx and now everything is cached fine. I also use fastcgi_cache of the nginx too. But now I have other problems such as rewrite rules for a prestashop website in nginx!! I am struggling with that:D Commented Jul 11, 2017 at 8:02

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.