DEV Community

Cover image for GRPC over unix socket Protocol
Gopher profile image Javad Rajabzadeh
Javad Rajabzadeh for Gopher

Posted on • Edited on

GRPC over unix socket Protocol

#career

GitHub logo ja7ad / grpc-unix-socket

GRPC over unix socket Protocol

GRPC over unix socket Protocol

Unix sockets, or Unix Domain Sockets, allow bidirectional data exchange between processes running on the same machine In order to transmit data between processes, the file system provides a reliable and efficient mechanism. The kernel is the only component involved in communication between processes. The processes communicate by reading and writing to the same socket file, which is managed by the kernel. Kernels handle communication details, such as synchronization, buffering, and error handling, and ensure that data is delivered reliably and correctly.

Advantage :

  • Fast communication
  • Efficient
  • Low overhead
  • Stability

Disadvantages :

  • Limited on same machine
sequenceDiagram Client->>+Server: Health Request (1.02187 µs per response) Server->>+Client: Serve message
Loading 
$ netstat -a -p --unix | grep grpc (Not all processes could be identified, non-owned process info will not be shown, you would have to be root to see it all.) unix 2 [ ACC ] STREAM
…
Enter fullscreen mode Exit fullscreen mode
View on GitHub

Unix sockets, or Unix Domain Sockets, allow bidirectional data exchange between processes running on the same machine.
In order to transmit data between processes, the file system provides a reliable and efficient mechanism. The kernel is the only component involved in communication between processes. The processes communicate by reading and writing to the same socket file, which is managed by the kernel. Kernels handle communication details, such as synchronization, buffering, and error handling, and ensure that data is delivered reliably and correctly.

Advantage :

  • Fast communication
  • Efficient
  • Low overhead
  • Stability

Disadvantages :

  • Limited on same machine

unix socket

$ netstat -a -p --unix | grep grpc (Not all processes could be identified, non-owned process info will not be shown, you would have to be root to see it all.) unix 2 [ ACC ] STREAM LISTENING 119977 98036/main /tmp/grpc.sock
Enter fullscreen mode Exit fullscreen mode

A TCP/IP socket is a mechanism for communicating between processes over a network.

tcp

package main import ( "google.golang.org/grpc" "google.golang.org/grpc/health" "google.golang.org/grpc/health/grpc_health_v1" "google.golang.org/grpc/reflection" "log" "net" "os" "os/signal" "syscall" ) const ( // unix socket PROTOCOL = "unix" SOCKET = "/tmp/grpc.sock" // tcp protocol PROTOCOL_TCP = "tcp" ADDR = "localhost:3000" ) func main() { ln, err := net.Listen(PROTOCOL, SOCKET) if err != nil { log.Fatal(err) } tcpLn, err := net.Listen(PROTOCOL_TCP, ADDR) if err != nil { log.Fatal(err) } c := make(chan os.Signal, 1) signal.Notify(c, os.Interrupt, syscall.SIGTERM) go func() { <-c os.Remove(SOCKET) os.Exit(1) }() srv := grpc.NewServer() grpc_health_v1.RegisterHealthServer(srv, health.NewServer()) reflection.Register(srv) go func() { log.Printf("grpc ran on tcp protocol %s", ADDR) log.Fatal(srv.Serve(tcpLn)) }() log.Printf("grpc ran on unix socket protocol %s", SOCKET) log.Fatal(srv.Serve(ln)) }
Enter fullscreen mode Exit fullscreen mode

Test Benchmark and profiling 

package main import ( "context" "google.golang.org/grpc" "google.golang.org/grpc/health/grpc_health_v1" "net" "os" "runtime/pprof" "testing" "time" ) func Benchmark_UNIX(b *testing.B) { f, err := os.Create("unix.prof") if err != nil { b.Fatal(err) } if err := pprof.StartCPUProfile(f); err != nil { b.Fatal(err) } defer pprof.StopCPUProfile() dialer := func(addr string, t time.Duration) (net.Conn, error) { return net.Dial(PROTOCOL, addr) } conn, err := grpc.Dial(SOCKET, grpc.WithInsecure(), grpc.WithDialer(dialer)) if err != nil { b.Fatal(err) } health := grpc_health_v1.NewHealthClient(conn) ctx := context.Background() for i := 0; i < b.N; i++ { _, err := health.Check(ctx, &grpc_health_v1.HealthCheckRequest{}) if err != nil { b.Fatal(err) } } } func Benchmark_TCP(b *testing.B) { f, err := os.Create("tcp.prof") if err != nil { b.Fatal(err) } if err := pprof.StartCPUProfile(f); err != nil { b.Fatal(err) } defer pprof.StopCPUProfile() conn, err := grpc.Dial(ADDR, grpc.WithInsecure()) if err != nil { b.Fatal(err) } health := grpc_health_v1.NewHealthClient(conn) ctx := context.Background() for i := 0; i < b.N; i++ { _, err := health.Check(ctx, &grpc_health_v1.HealthCheckRequest{}) if err != nil { b.Fatal(err) } } }
Enter fullscreen mode Exit fullscreen mode 
goos: linux goarch: amd64 pkg: github.com/Ja7ad/grpc-unix-socket/server cpu: Intel(R) Core(TM) i5-3570 CPU @ 3.40GHz Benchmark_UNIX Benchmark_UNIX-4 100000 102187 ns/op 4960 B/op 96 allocs/op
Enter fullscreen mode Exit fullscreen mode 
goos: linux goarch: amd64 pkg: github.com/Ja7ad/grpc-unix-socket/server cpu: Intel(R) Core(TM) i5-3570 CPU @ 3.40GHz Benchmark_TCP Benchmark_TCP-4 100000 127188 ns/op 4961 B/op 96 allocs/op
Enter fullscreen mode Exit fullscreen mode

Result of 100k benchmark with 20 time test as new/old (benchstat) 

name old time/op new time/op delta _UNIX-4 100µs ± 2% 100µs ± 2% ~ (p=0.668 n=17+20) _TCP-4 125µs ± 1% 124µs ± 1% -0.15% (p=0.031 n=20+20) name old alloc/op new alloc/op delta _UNIX-4 4.96kB ± 0% 4.96kB ± 0% ~ (p=0.454 n=20+20) _TCP-4 4.96kB ± 0% 4.96kB ± 0% ~ (all equal) name old allocs/op new allocs/op delta _UNIX-4 96.0 ± 0% 96.0 ± 0% ~ (all equal) _TCP-4 96.0 ± 0% 96.0 ± 0% ~ (all equal)
Enter fullscreen mode Exit fullscreen mode

UseCase

When you run many services on the same machine and run them in containers or hosts, you can use unix sockets for efficient and fast communication.

Containers need to share sock files via volumes, since unix sockets are over socket files.

docker volume create --name=socket docker run socket:/var/socket serviceA docker run socket:/var/socket serviceB
Enter fullscreen mode Exit fullscreen mode

How to test server/client?

Server :

  • run server (go1.19) : 
$ go run -mod vendor grpc.go
Enter fullscreen mode Exit fullscreen mode
  • benchmark command : 
$ go test -bench=. -benchtime=100000x -benchmem
Enter fullscreen mode Exit fullscreen mode

Top comments (0)