How to force nginx to resolve DNS (of a dynamic hostname) everytime when doing proxy_pass?

Question

I am using nginx/0.7.68, running on CentOS, with the following configuration:

server { listen 80; server_name ***; index index.html index.htm index.php default.html default.htm default.php; location / { root /***; proxy_pass http://***:8888; index index.html index.htm; } # where *** is my variables 

The proxy_pass is to a DNS record whose IP changes frequently. Nginx caches the outdated IP address, resulting in a request to the wrong IP address.

How can I stop nginx from caching the IP address, when it is outdated?

Answer 1

It's an intriguing question and AFAIK that's not going to work well. You can try to use the upstream module and use the directives for failover to see if it works as a hack.

2018 edit: a lot of things changed. Check the answer by @ohaal to get real information about this.

Answer 2

Accepted answer didn't work for me on nginx/1.4.2.

Using a variable in proxy_pass forces re-resolution of the DNS names because NGINX treats variables differently to static configuration. From the NGINX proxy_pass documentation:

Parameter value can contain variables. In this case, if an address is specified as a domain name, the name is searched among the described server groups, and, if not found, is determined using a resolver.

For example:

server { ... resolver 127.0.0.1; set $backend "http://dynamic.example.com:80"; proxy_pass $backend; ... } 

Note: A resolver (i.e. the name server to use) MUST be available and configured for this to work (and entries inside a /etc/hosts file won't be used in a lookup).

By default, version 1.1.9 or later versions of NGINX cache answers using the TTL value of a response and an optional valid parameter allows the cache time to be overridden:

resolver 127.0.0.1 [::1]:5353 valid=30s; 

Before version 1.1.9, tuning of caching time was not possible, and nginx always cached answers for the duration of 5 minutes..

Answer 3

There is valuable information in gansbrest comment and ohaal answer.

But I think it's important to mention this official nginx article, posted in 2016, it clearly explains nginx behaviour on this matter and the possible solutions: https://www.nginx.com/blog/dns-service-discovery-nginx-plus/

We indeed have to "Set the Domain Name in a Variable" and use the resolver directive.

however, using a variable changes the rewrite behaviour. You may have to use the rewrite directive, it depends on your location and proxy_pass setup.

PS: would have post a comment but not enough points yet...

Answer 4

ohaal's answer takes most of us there, but there is a case where the DNS resolver does not live at 127.0.0.1 (eg when you're in a special containerized environment)

In that case, you may want to change the nginx conf to resolver ${DNS_SERVER};. Then, before you start nginx, run

export DNS_SERVER=$(cat /etc/resolv.conf |grep -i '^nameserver'|head -n1|cut -d ' ' -f2) envsubst '${DNS_SERVER}' < your_nginx.conf.template > your_nginx.conf 

Note, that you need the gettext package installed, as that provides the envsubst command.

Answer 5

As of nginx version 1.27.3 non-enterprise versions can also use resolve parameter on server inside upstream block to make domains re-resolve.

resolver 8.8.8.8 upstream http_backend { zone upstream_dynamic 64k; server custom.domain.tld:443 resolve; keepalive 16; } server { ... location /http/ { proxy_pass https://http_backend; proxy_http_version 1.1; proxy_set_header Connection ""; ... } } 

In order for this parameter to work, the resolver directive must be specified in the http block or in the corresponding upstream block.

Answer 6

I've hacked together a script to watch a conf.d folder upstreams for dns changes and reload nginx upon detection. It's a first pass, and surely can be improved (next pass, I'll use nginx -T to parse upstreams specifically. Same idea could be used for proxy_pass directives):

#!/bin/bash get_upstreams() { local files=$@ grep -hEo '(server\s+)[^:;]+' $files | cut -d' ' -f 2 } resolve_hosts() { local hosts=$@ for h in $hosts; do dig +short $h; done | sort -u } watch_dir=$1 [ -d $watch_dir ] || exit 2 upstreams=$(get_upstreams $watch_dir/*) ips=$(resolve_hosts $upstreams) if [ ! "$ips" ]; then echo "Found no resolvable hosts in $watch_dir files." fi host_hash=$(echo $ips | /usr/bin/sha512sum) echo $host_hash echo $ips while [ -d $watch_dir ]; do sleep 30 upstreams=$(get_upstreams $watch_dir/*) ips=$(resolve_hosts $upstreams) new_hash=$(echo $ips | /usr/bin/sha512sum) if [ "$host_hash" != "$new_hash" ]; then echo Detected an upstream address change. $ips echo Reloading nginx echo $new_hash echo $ips /sbin/service nginx reload host_hash=$new_hash fi done 

Answer 7

The most voted answer from @ohaal didn't work for me. I had to use a variant of the proposed solution:

resolver 127.0.0.11 ipv6=off valid=10s; location ~ ^/SomePath/(.*)$ { proxy_pass http://dynamic.example.com:80/SomePath/$1$is_args$args; } location ~ ^/Other/Path/(.*)$ { proxy_pass http://dynamic.example.com:80/Other/Path/$1$is_args$args; } 

Unlike the other solution, this works also of for urls like:

http://proxied:80/SomePath/bla http://proxied:80/SomePath/bla/bla?whatever=123 http://proxied:80/Other/Path/bla http://proxied:80/Other/Path/bla/bla 

See https://stackoverflow.com/a/8130872/948938