DEV Community

Viacheslav Poturaev
Viacheslav Poturaev

Posted on

Using Nginx as a proxy to multiple Unix sockets

TL;DR Listening port may be a contended resource on a busy shared machine, unix sockets are virtually unlimited. Nginx can expose them with a single port and prefixed URLs.

In some situations you may want to run many (instances of) applications on a single machine. Each instance may need to provide internal information (e.g. Prometheus /metrics, profiling/debug handlers) over restricted HTTP.

When number of instances grows it becomes a burden to provision listening ports without conflicts. In contrast, using Unix sockets allows for more transparency (readable filenames) and scalability (easy to come up with unique name).

Here is a small demo program written in Go that would serve trivial HTTP service with Unix socket.

package main import ( "context" "flag" "io/fs" "log" "net" "net/http" "os" "os/signal" ) func main() { var socketPath string flag.StringVar(&socketPath, "socket", "./soc1", "Path to unix socket.") flag.Parse() if socketPath == "" { flag.Usage() return } listener, err := net.Listen("unix", socketPath) if err != nil { log.Println(err.Error()) return } // By default, unix socket would only be available to same user. // If we want access it from Nginx, we need to loosen permissions. err = os.Chmod(socketPath, fs.ModePerm) if err != nil { log.Println(err) return } httpServer := http.Server{ Handler: http.HandlerFunc(func(writer http.ResponseWriter, request *http.Request) { log.Println(request.URL.String()) if _, err := writer.Write([]byte(request.URL.String())); err != nil { log.Println(err.Error()) } }), } // Setting up graceful shutdown to clean up Unix socket. go func() { sigint := make(chan os.Signal, 1) signal.Notify(sigint, os.Interrupt) <-sigint if err := httpServer.Shutdown(context.Background()); err != nil { log.Printf("HTTP Server Shutdown Error: %v", err) } }() log.Printf("Service is listening on socket file %s", socketPath) err = httpServer.Serve(listener) if err != nil { log.Println(err.Error()) return } } 
Enter fullscreen mode Exit fullscreen mode

Now let's run a couple of instances in separate shells.

./soc -socket /home/ubuntu/soc1 
Enter fullscreen mode Exit fullscreen mode
./soc -socket /home/ubuntu/soc2 
Enter fullscreen mode Exit fullscreen mode

Here is a minimal Nginx config to serve those instances with URL prefixes. It would receive http://my-host/soc1/foo/bar, strip path prefix /soc1 and pass /foo/bar to soc1.

server { listen 80 default; location /soc1/ { proxy_pass http://soc1/; } location /soc2/ { proxy_pass http://soc2/; } } upstream soc1 { server unix:/home/ubuntu/soc1; } upstream soc2 { server unix:/home/ubuntu/soc2; } 
Enter fullscreen mode Exit fullscreen mode

Every Unix socket is defined as upstream and has /location statement in server.

It is also possible to use Unix sockets directly in /location, like in

 location /soc1/ { proxy_pass http://unix:/home/ubuntu/soc1; } 
Enter fullscreen mode Exit fullscreen mode

however it has an unwanted limitation that you can not add trailing / to proxy_pass. And this means that URL will be passed as is, e.g. soc1 will receive /soc1/foo instead of /foo.

To avoid such limitation we can use named upstream and add trailing / to proxy_pass.

 location /soc1/ { proxy_pass http://soc1/; # Mind trailing "/". } 
Enter fullscreen mode Exit fullscreen mode

Top comments (1)

Collapse
 
azcraft profile image
AzCraft

Thanks a lot, that was very useful to me. :D