Set Nginx Upstream Using Header

Question

I've got an Nginx reverse proxy sitting in front of a cluster of microservices. When a new client connects I'd ideally like to use whatever upstream has the least number of connections. They'll receive a token in the response body that they'll pass in the header for all subsequent requests.

I would like the same upstream server to handle all future requests related to that token.

Using "ip_hash" won't work because I need to load balance requests from the same origin IP. (But w/ different header tokens) I can't hardcode a "map" because the microservices are transient. Each request already has to pass through an "auth_request" that is capable of exposing a variable with the proper upstream IP to connect to but I'm not sure how to proxy pass to a dynamic variable.

I'm already using Redis and Memcached for a number of other tasks and I could easily load a "key = header token", "value = upstream IP" into either. But once again I'm not sure how to use a cache value as a proxy upstream target. Most of the cache based solutions I've seen can only return the cache value as a client response, not use it within a "location" block for further routing.

I'm open to any ideas on how to pin a request to a specific upstream using a header value that doesn't rely on hardcoding a map or IP/URI hashing.

Thanks in advance, I've been stuck on this for over 5 hours. :(

Answer 1

i have same issue, i figure out just with "hash" method

upstream backend { # if you use Tengine (https://github.com/alibaba/tengine) # consistent_hash $http_customheadername; hash $http_customheadername; server 127.0.0.1:8080; server 127.0.0.1:8081; server 127.0.0.1:8082; } server { listen 80; server_name test2.com; default_type text/plain; location / { proxy_pass http://backend; } } server { listen 8080; server_name _; location / { echo "8080"; } } server { listen 8081; server_name _; location / { echo "8081"; } } server { listen 8082; server_name _; location / { echo "8082"; } } 

you can set you unique value to header.