Deploy Django with nginx and gunicorn

This article introduces the method of deploying Django with nginx and gunicorn on Linux (ubuntu20). I installed wordpress on LEMP on my original machine, see  Installing wordpress lemp on ubuntu 20  , the main page is Worpress and the Django application is also installed. Of course wordpress is not necessary.

This article does not introduce the establishment of the django project, but only introduces its deployment. Of course, you still need to verify the django application before deployment.

The reference original text is  How To Set Up Django with Postgres, Nginx, and Gunicorn on Ubuntu 20.04. The   original text introduces a lot and is hard to understand, but this article only focuses on the deployment of django.

Software installation and engineering preparation

Execute the following installation command:

sudo apt update
sudo apt install python3-pip python3-dev libpq-dev  nginx curl

There is a problem with libpq-dev on my host, so I won’t install it. Use the following command:

sudo apt install python3-pip python3-dev  nginx curl

To create a virtual space, install the software first:

sudo -H pip3 install --upgrade pip
sudo -H pip3 install virtualenv

Create a project directory:

mkdir ~/mysite
cd ~/mysite

Create a virtual space:

virtualenv mysiteenv

Activate virtual space:

source mysiteenv/bin/activate

The interface is like this:

leon@ubuntu-s-1vcpu-1gb-tor1-01:~$ source myproject/myprojectenv/bin/activate
(myprojectenv) leon@ubuntu-s-1vcpu-1gb-tor1-01:~$

This prompt line has a virtual environment name.

Install Django gunicorn in a virtual environment. Note that the installation uses pip instead of pip3:

pip install django gunicorn 

Build django and application

Setting up projects, settings, etc., I won't talk about it here. You can refer to: Python's web framework django's introductory tutorial 1

Make sure that the firewall opens port 8000:

sudo ufw allow 8000

Start the application:

~/mysite/manage.py runserver 0.0.0.0:8000

To verify, enter your application name in the browser, for example: http://127.0.0.1:8000/polls/

Or: http://138.197.144.170/covid/

Here you should see that your application is running normally.

Test the functionality of the Gunicorn service

First look at the contents of the project mysite directory:

leon@ubuntu-s-1vcpu-1gb-tor1-01:~/mysite/mysite$ ls
__init__.py  __pycache__  asgi.py  settings.py  urls.py  views.py  views.py.save  wsgi.py

We use the file wsgi.py

Guniorn’s service verification looks like this:

cd ~/mysite
gunicorn --bind 0.0.0.0:8000 mysite.wsgi
Use gunicorn to start django application service like this. It has the same effect as starting the django application above.

Enter your application name in the browser, for example: http://127.0.0.1:8000/polls/

Or: http://138.197.144.170/covid/

Here you should see that your application is also running normally.

Exit the virtual space with the following command:

deactivate

Create systemd socket and service files for Gunicorn

sudo nano /etc/systemd/system/gunicorn.socket

Here, we will create a [Unit] section to describe the socket, a [Socket] section to define the socket location, and create a [Install] section to ensure that the socket is created at the correct time:

[Unit]
Description=gunicorn socket

[Socket]
ListenStream=/run/gunicorn.sock

[Install]
WantedBy=sockets.target

Next, create and use sudo permissions to open Gunicorn's systemd service file in a text editor. The service file name should match the socket file name, but the extension is different:

sudo nano /etc/systemd/system/gunicorn.service

Starting from the [Unit] section, this section is used to specify metadata and relevance. We will describe our service here and tell the init system to start this service only after reaching the network goal. Because our service depends on the socket in the socket file, we need to include a Requires directive to indicate this relationship:

[Unit]
Description=gunicorn daemon
Requires=gunicorn.socket
After=network.target

Next, we will open the [Servicce] section. Specify the user and group to run here. Since the process requires ownership of all related files, the regular user account ownership of the process needs to be granted. We will give group ownership to the www-data group so that Nginx can easily communicate with Gunicorn.

Then, we will map out the working directory and specify the command to start the service. In this case, we must specify the full path of the Gunicorn executable file, which has been installed in our virtual environment. We will bind the process to the Unix socket created in the /run directory so that the process can communicate with Nginx. We log all data to standard output so that the journal processing can collect Gunicorn logs. We can also specify any optional Gunicorn adjustments here. For example, in this case, we specified 3 worker processes:

[Unit]
Description=gunicorn daemon
Requires=gunicorn.socket
After=network.target

[Service]
User=leon
Group=www-data
WorkingDirectory=/home/leon/mysite
ExecStart=/home/leon/mysite/mysiteenv/bin/gunicorn \
          --access-logfile - \
          --workers 3 \
          --bind unix:/run/gunicorn.sock \
          mysite.wsgi:application

Here we see user = leon which is the user name with root privileges

workingDirecctory = /home/leon/mysite This is the absolute directory name of the django project directory

ExecStart = /home/leon/mysite/mysiteenv/bin/gunicorn This is the absolute path of gunicorn installed in our virtual environment

mysite.wsgi:appliation, you can see that there is an application = get_wsgi_application() in wsgi.py in the above directory

--bind unix:/run/guniorn.sok corresponds to ListenStream in the gunicorn.socket file above.

 

 Finally, we will add a [Install] section. If the service needs to be enabled to start at boot, it will tell systemd what the service is linked to. We want the service to start when the regular multi-user system is up and running:

[Unit]
Description=gunicorn daemon
Requires=gunicorn.socket
After=network.target

[Service]
User=leon
Group=www-data
WorkingDirectory=/home/leon/myprojectdir
ExecStart=/home/leon/myprojectdir/myprojectenv/bin/gunicorn \
          --access-logfile - \
          --workers 3 \
          --bind unix:/run/gunicorn.sock \
          myproject.wsgi:application

[Install]
WantedBy=multi-user.target

In this way, our systemd service file is complete. Save and close.

Now, we can start and enable the Gunicorn socket. Later, when starting, a socket file will be created at /run/gunicorn.sock. After establishing a connection with the socket, systemd will automatically start gunicorn.service to handle it:

sudo systemctl start gunicorn.socket
sudo systemctl enable gunicorn.socket

Check the Gunicorn socket file

Check the status of the process to see if it can start successfully:

sudo systemctl status gunicorn.socket

The output should look like this:

● gunicorn.socket - gunicorn socket
     Loaded: loaded (/etc/systemd/system/gunicorn.socket; enabled; vendor preset: enabled)
     Active: active (listening) since Sat 2021-02-06 22:55:01 UTC; 43s ago
   Triggers: ● gunicorn.service
     Listen: /run/gunicorn.sock (Stream)
      Tasks: 0 (limit: 1137)
     Memory: 0B
     CGroup: /system.slice/gunicorn.socket

Feb 06 22:55:01 ubuntu-s-1vcpu-1gb-tor1-01 systemd[1]: Listening on gunicorn socket.

Next, check if the gunicorn.sock file exists in the /run directory:

file /run/gunicorn.sock

There should be output like this:

/run/gunicorn.sock: socket

If the systemctl status command indicates that an error has occurred, or the gunicorn.sock file cannot be found in the directory, it indicates that the Gunicorn socket cannot be created correctly. Enter the following to view the Gunicorn socket log:

sudo journalctl -u gunicorn.socket

Before proceeding, if there is an error, you need to check the /etc/systemd/system/gunicorn.socket file to solve all the problems.

Test socket activation

If only the gunicorn.socket unit is started, but the socket has not yet received any connection, then gunicorn.service will not be active. You can check by entering the following:

sudo systemctl status gunicorn

It should be the following output:

 gunicorn.service - gunicorn daemon
   Loaded: loaded (/etc/systemd/system/gunicorn.service; disabled; vendor preset: enabled)
   Active: inactive (dead)

In order to test the socket activation mechanism, you can send the connection to the socket via curl by typing the following:

curl --unix-socket /run/gunicorn.sock localhost

The HTML output should be received from the application in the terminal. This shows that Gunicorn has started and is able to serve the Django application. You can verify whether the Gunicorn service is running by typing the following command:

sudo systemctl status gunicorn

Should have output like this

gunicorn.service - gunicorn daemon
     Loaded: loaded (/etc/systemd/system/gunicorn.service; disabled; vendor preset: enabled)
     Active: active (running) since Fri 2020-06-26 18:52:21 UTC; 2s ago
TriggeredBy: ● gunicorn.socket
   Main PID: 22914 (gunicorn)
      Tasks: 4 (limit: 1137)
     Memory: 89.1M
     CGroup: /system.slice/gunicorn.service
             ├─22914 /home/sammy/myprojectdir/myprojectenv/bin/python /home/sammy/myprojectdir/myprojectenv/bin/gunicorn --access-logfile - --workers 3 --bind unix:/run/gunico>
             ├─22927 /home/sammy/myprojectdir/myprojectenv/bin/python /home/sammy/myprojectdir/myprojectenv/bin/gunicorn --access-logfile - --workers 3 --bind unix:/run/gunico>
             ├─22928 /home/sammy/myprojectdir/myprojectenv/bin/python /home/sammy/myprojectdir/myprojectenv/bin/gunicorn --access-logfile - --workers 3 --bind unix:/run/gunico>
             └─22929 /home/sammy/myprojectdir/myprojectenv/bin/python /home/sammy/myprojectdir/myprojectenv/bin/gunicorn --access-logfile - --workers 3 --bind unix:/run/gunico>

Jun 26 18:52:21 django-tutorial systemd[1]: Started gunicorn daemon.
Jun 26 18:52:21 django-tutorial gunicorn[22914]: [2020-06-26 18:52:21 +0000] [22914] [INFO] Starting gunicorn 20.0.4
Jun 26 18:52:21 django-tutorial gunicorn[22914]: [2020-06-26 18:52:21 +0000] [22914] [INFO] Listening at: unix:/run/gunicorn.sock (22914)
Jun 26 18:52:21 django-tutorial gunicorn[22914]: [2020-06-26 18:52:21 +0000] [22914] [INFO] Using worker: sync
Jun 26 18:52:21 django-tutorial gunicorn[22927]: [2020-06-26 18:52:21 +0000] [22927] [INFO] Booting worker with pid: 22927
Jun 26 18:52:21 django-tutorial gunicorn[22928]: [2020-06-26 18:52:21 +0000] [22928] [INFO] Booting worker with pid: 22928
Jun 26 18:52:21 django-tutorial gunicorn[22929]: [2020-06-26 18:52:21 +0000] [22929] [INFO] Booting worker with pid: 22929

If the output of curl or the output of systemctl status indicates a problem, please check the log for additional details:

sudo journalctl -u gunicorn

Check the problem in the /etc/systemd/system/gunicorn.service file. If you make changes to the /etc/systemd/system/gunicorn.service file, reload the daemon to re-read the service definition and restart the Gunicorn process by typing the following command:

sudo systemctl daemon-reload
sudo systemctl restart gunicorn

Before proceeding, make sure there is no problem here.

Configure Nginx proxy to pass to Gunicorn

Now that Gunicorn has been established, we need to configure Nginx to pass traffic to the process.

First, create and open a new server block in the Nginx sites-available directory (/etc/nginx/sites-available). I am just a domain name, so I directly modify the default file. My host originally ran the wordpress service, I just specifically designated the django service to provide 2 applications.

The purpose of this modification is that the services under the domain name /polls and /covid all point to the Gunicorn service of djiango, the others are managed by wordpress, and the rest of this part is not shown here.

The revised content is as follows:

server {
    ...

     location /polls/ {
     include proxy_params;
     proxy_pass http://unix:/run/gunicorn.sock;
     }

     location /covid/ {
     include proxy_params;
     proxy_pass http://unix:/run/gunicorn.sock;
     }
}

Check that nginx has no configuration errors:

sudo nginx -t

If there is nothing wrong, restart nginx to make the configuration take effect:

sudo systemctl restart nginx

Troubleshooting for Nginx and Gunicorn

If your application is not displayed in the last step, you need to troubleshoot the installation.

Nginx displays the default page instead of the Django application
If Nginx displays the default page instead of proxying to the application, it usually means that the server_name in the /etc/nginx/sites-available/myproject file needs to be adjusted to point to the server's IP address or domain name.

Nginx uses server_name to determine which server block to use to respond to the request. If you receive the default Nginx page, it means that Nginx cannot explicitly match the request to the server block, so it will fall back to the default block defined in /etc/nginx/sites-available/default.

The server_name in the server block of the project must be more specific than the server_name in the default server block to be selected.

Nginx displays the 502 Bad Gateway error, instead of the
502 error of the Django application, it means that Nginx cannot successfully proxy the request. Various configuration issues are indicated as 502 errors, so more information is needed for proper troubleshooting.

The main place to find more information is in the Nginx error log. Usually, this will tell you what caused the problem during the agent incident. Follow the Nginx error log by entering the following:

sudo tail -F /var/log/nginx/error.log

Now, make another request in the browser to generate a new error (try to refresh the page). You should receive a new error message and write it to the log. If you look at the message, it should help narrow the scope of the problem.

You may receive the following message:

connect() to unix:/run/gunicorn.sock failed (2: No such file or directory)

This indicates that Nginx cannot find the gunicorn.sock file in the given location. The proxy_pass location defined in the /etc/nginx/sites-available/myproject file should be compared with the actual location of the gunicorn.sock file generated by the gunicorn.socket system unit.

If the gunicorn.sock file is not found in the /run directory, it usually means that the systemd socket file cannot create it. Return to the section about checking the Gunicorn socket file to complete the troubleshooting steps for Gunicorn step by step.

connect() to unix:/run/gunicorn.sock failed (13: Permission denied)

This indicates that Nginx cannot connect to the Gunicorn socket due to permission issues. This can happen when the root user is used instead of the sudo user to perform the process. Although systemd can create Gunicorn socket files, Nginx cannot access it.

This can happen if any location between the root directory (/) gunicorn.sock file has limited permissions. By passing the absolute path of the socket file to the namei command, we can view the permissions and ownership values ​​of the socket file and each of its parent directories:

namei -l /run/gunicorn.sock

The output is similar to this:

f: /run/gunicorn.sock
drwxr-xr-x root root /
drwxr-xr-x root root run
srw-rw-rw- root root gunicorn.sock

The output shows the permissions of each directory component. By looking at permissions (first column), owner (second column), and group owner (third column), we can determine what type of access to the socket file is allowed.

In the above example, the socket file and each directory leading to the socket file have global read and execute permissions (the permissions of the directory end with rx instead of ---). The Nginx process should be able to successfully access the socket.

If any directory leading to the socket does not have global read and execute permissions, Nginx will not be able to access the socket without allowing global read and execute permissions or ensuring that group ownership is granted to Nginx to participate One of the groups.

I have not experienced some troubleshooting, but simply translated the original text, and it is not complete. If you encounter a problem, please refer to the original text, or leave a message, or find it online.

The introduction is over.