Multipart ranges in Nginx reverse proxy

Question

I am trying to setup Nginx as a reverse proxy. The upstream server is serving some media files.

Because of the large amount of requests for these files and also since these files are not going to change for at least a couple of weeks, I'm caching the upstream responses in Nginx and serving all subsequent requests from the cache.

proxy_cache_path /home/bandc/content levels=1:2 keys_zone=content_cache:10m max_size=10g inactive=15d use_temp_path=off; upstream mycdn { server myserver.dev; } location / { proxy_cache content_cache; proxy_pass http://mycdn; proxy_cache_methods GET HEAD; proxy_cache_valid 200 302 7d; proxy_cache_valid 404 10m; add_header x-cache $upstream_cache_status; .... } 

The clients can send Range requests for large media files. The upstream server does support range requests.

The problem is if I send a request with multiple byte ranges before sending any GET or single Range request (i.e the response hasn't been cached before this multiple byte ranges request), Nginx delivers the whole file with 200 OK instead of the requested ranges with 206 Partial Content. But once the content has been cached, all multipart range requests work flowlessly.

I looked around a bit and found this blog post:

How Does NGINX Handle Byte Range Requests? If the file is up‑to‑date in the cache, then NGINX honors a byte range request and serves only the specified bytes of the item to the client. If the file is not cached, or if it’s stale, NGINX downloads the entire file from the origin server. If the request is for a single byte range, NGINX sends that range to the client as soon as it is encountered in the download stream. If the request specifies multiple byte ranges within the same file, NGINX delivers the entire file to the client when the download completes.

Is there any way to ensure that if the file is not cached yet, multipart range should be served from upstream alone (without caching) and eventually from local cache after Nginx caches it when GET or a Range request with a single byte range is performed?

Answer 1

I think you should use the method Cache Lock mentioned in this guide. This will allow multiple byte ranges or complete file. There is an alternative method of Cache Slicing but since your content does change it would not be a good choice in this case.

proxy_cache_path /tmp/mycache keys_zone=mycache:10m; server { listen 80; proxy_cache mycache; slice 1m; proxy_cache_key $host$uri$is_args$args$slice_range; proxy_set_header Range $slice_range; proxy_http_version 1.1; proxy_cache_valid 200 206 1h; location / { proxy_pass http://origin:80; } 

}