Gunicorn not creating socket on nginx

Question

I am trying to setup Django, Gunicorn and nginx.

I have configured Gunicorn in order to run, but it seems that I am experiencing a problem with nginx configuration to use Gunicorn.

Here is my configuration:

/etc/systemd/system/gunicorn.service

[Unit] Description=gunicorn daemon Requires=socket After=network.target [Service] PIDFile=/run/gunicorn/pid User=root Group=root RuntimeDirectory=gunicorn WorkingDirectory=/srv/myproject/current ExecStart=/srv/venvs/myenv/bin/gunicorn --pid /run/gunicorn/pid \ --bind unix:/run/gunicorn/socket myapp.wsgi:application ExecReload=/bin/kill -s HUP $MAINPID ExecStop=/bin/kill -s TERM $MAINPID PrivateTmp=true [Install] WantedBy=multi-user.target 

Gunicorn status looks like this

sudo systemctl status gunicorn

gunicorn.service - gunicorn daemon Loaded: loaded (/etc/systemd/system/gunicorn.service; enabled; vendor preset: enabled) Active: active (running) since Thu 2018-01-18 23:32:11 UTC; 3min 23s ago Process: 6347 ExecStop=/bin/kill -s TERM $MAINPID (code=exited, status=0/SUCCESS) Main PID: 6355 (gunicorn) Tasks: 2 Memory: 195.7M CPU: 1.426s CGroup: /system.slice/gunicorn.service ├─6355 /srv/venvs/myenv/bin/python3.6 /srv/venvs/myenv/bin/gunicorn --pid /run/gunicorn/pid --bind unix:/run/gunicorn/socket myapp.wsgi:application └─6360 /srv/venvs/myenv/bin/python3.6 /srv/venvs/myenv/bin/gunicorn --pid /run/gunicorn/pid --bind unix:/run/gunicorn/socket myapp.wsgi:application Jan 18 23:32:11 python-server systemd[1]: Stopped gunicorn daemon. Jan 18 23:32:11 python-server systemd[1]: Started gunicorn daemon. Jan 18 23:32:11 python-server gunicorn[6355]: [2018-01-18 23:32:11 +0000] [6355] [INFO] Starting gunicorn 19.7.1 Jan 18 23:32:11 python-server gunicorn[6355]: [2018-01-18 23:32:11 +0000] [6355] [INFO] Listening at: unix:/run/gunicorn/socket (6355) Jan 18 23:32:11 python-server gunicorn[6355]: [2018-01-18 23:32:11 +0000] [6355] [INFO] Using worker: sync Jan 18 23:32:11 python-server gunicorn[6355]: [2018-01-18 23:32:11 +0000] [6360] [INFO] Booting worker with pid: 6360 

My nginx configuration

/etc/nginx/sites-enabled/myserver-python

server { server_tokens off; listen 443 ssl; server_name myserver.com; keepalive_timeout 70; ssl_certificate /etc/ssl/certs/myserver.com.merged.crt; ssl_certificate_key /etc/ssl/private/myserver.com.key; ssl_protocols TLSv1 TLSv1.1 TLSv1.2; ssl_ciphers HIGH:!aNULL:!MD5; #access_log /var/log/nginx/access.myserver.log; access_log /var/log/nginx/access.myserver.log; #error_log /var/log/nginx/error.myserver.log; error_log /var/log/nginx/error.myserver.log; location / { include proxy_params; proxy_pass http://unix:/run/gunicorn/socket; } } 

On the path /run/gunicorn I can see only pid file. File socket has never created.

This is how the error log looks like

/var/log/nginx/error.myserver.log

2018/01/18 23:09:00 [crit] 5764#5764: *1 connect() to unix:/run/gunicorn/socket failed (2: No such file or directory) while connecting to upstream, client: 212.251.167.250, server: my-server.com, request: "GET / HTTP/1.1", upstream: "http://unix:/run/gunicorn/socket:/",, host: "myserver.com" 

Can anyone see what is wrong here? Why socket file is not made and pid file is made?

Answer 1

The socket that gunicorn actually opened, as shown in the journal entries, differs from the socket configured in your systemd unit.

The journal shows what gunicorn actually did:

Jan 18 23:08:49 python-server gunicorn[5858]: [2018-01-18 23:08:49 +0000] [5858] [INFO] Listening at: unix:/run/gunicorn/socket (5858) 

Note that it actually opened /run/gunicorn/socket. But your nginx configuration and systemd unit specify to open /run/gunicorn/gunicorn.socket, which is a different path.

My first thought is that you probably changed the path in the systemd unit, but haven't run systemctl daemon-reload. Any changes in your systemd units won't take effect until you run this (or reboot).

Therefore, I would have systemd reload and then try restarting gunicorn.

systemctl daemon-reload systemctl restart gunicorn.service 

Answer 2

Configuration was correct, but the problem was that gunicorn service wasn't properly restarted.

Since the gunicorn was installed as systemd, it should be restarted with systemctl restart gunicorn.

After that it worked fine.

Answer 3

Another reason a gunicorn socket file will not be created or allow read/write access is due to SELinux blocking access. SELinux is most common in RedHat / Fedora / CentOS flavors of Linux.

For instance, if you have SELinux running and the daemon creates a socket file outside of /run, /var/run, or a sub-directory below either of these, then on daemon restart the socket file will have the wrong file context. This would show up in the SELinux log (/var/log/audit/audit.log) as an error like:

type=AVC msg=audit(1569976379.346:40651): avc: denied { connectto } for pid=25861 comm="httpd" path="/home/username/gunicorn.sock" scontext=system_u:system_r:httpd_t:s0 tcontext=system_u:system_r:init_t:s0 tclass=unix_stream_socket permissive=0 

You might also have an issue with a web server talking to the socket if the "user" or "group" permissions used by the web server process don't have access to read/write to the socket file. This may mean changing the User=root or Group=root (shown in the OP) within the systemd daemon's "Unit" file.

systemctl edit --full gunicorn.service 

and then reloading the unit file and gunicorn

sudo systemctl daemon-reload sudo systemctl restart gunicorn && systemctl status gunicorn