2020 latest nginx+gunicorn+supervisor deployment detailed strategy of production environment of project based on flash development

This strategy is based on the version of Ubuntu 1804, the Huawei cloud server used by the server, python 3 (python2 has completely stopped maintenance in 2020, so it is necessary to switch to Python 3). Welcome to add my QQ6398903, or QQ group to discuss related technologies

1, Prepare the modules required for the project

You need to export all package information used in the development environment
`` pip freeze > requirements.txt
Then upload the requirements.txt file to the server

2, Preparing a python virtual environment for production

After all, the production environment may also run multiple python projects, so you need to prepare the virtual environment of python
Ubuntu 1804 only has python3 HA by default. To install the virtual environment, you need to install venv first
``sudo apt install python3-venv
Create a directory to hold the virtual environment


mkdir -p /opt/python/
sudo python3 -m venv myprj     #And finally, the name of the project
. /opt/python/myprj/bin/activate    #Activate the virtual environment. Don't forget the name of the virtual environment in front of the command line after the activation of the previous point [.], indicating success
pip install --upgrade pip    #Upgrade Pip first
pip intall -r requirements.txt   # Install all python modules at once

3, Install gunicorn

gunicorn is a web server based on wsgi (Web Service Gateway Interface), which is unique to flask (uwsgi is generally used in django)

pip install gevent   #If you don't use gevent in your project, you need to install it before installing gunicorn
pip install gunicorn 
deactivate     #Exit virtual environment

You can start a program directly using the gunicorn command, for example
``gunicorn -w 4 -b 127.0.0.1:5000 main:app
[note] the above command needs to be run in the project directory when the virtual environment is activated. The project startup file is the instance of main.py and the variable name is the flash of app
Pay special attention to main.py


if __name__ == "__main__":
    app.run()

Otherwise, when gunicorn starts the project, an error will be reported
The following are common command descriptions:

-c CONFIG, --config=CONFIG
# Set the profile.
-b BIND, --bind=BIND
# Set the port to which the service needs to be bound. It is recommended to use HOST:PORT
-w WORKERS, --workers=WORKERS
# Set the number of worker processes. It is recommended to set 2-4 servers for each core.
-k MODULE
# Select the module to use for asynchronous work.

4, Configure the configuration information of gunicorn

gunicorn directly uses the. py file as the configuration file, so it supports the py syntax

from gevent import monkey
monkey.patch_all()
import multiprocessing

#debug = True
loglevel = 'debug'
bind = '127.0.0.1:6000' #Binding ports for communication with Nginx
pidfile = 'log/gunicorn.pid'
accesslog = 'log/access.log'
errorlog = 'log/debug.log'
#daemon = True
workers = multiprocessing.cpu_count() * 2 + 1
worker_class = 'gevent' #The default is blocking mode, preferably gevent mode

Save to conf/guni.conf in the project directory

In the above configuration file, there are two comments, one is notice

  1. debug=True is easy to understand. It can be turned on when it's just launched. It's convenient to find errors. When it's officially turned on, you need to log off
  2. If you want to use supervisor to manage, you need to log out
  3. You need to create a log folder in the project file, otherwise the error will not start, because our log file and pid need to be saved in the log directory

5, Start gunicorn through configuration file and start the Daemons

Change the previous order,
gunicorn -w 4 -b 127.0.0.1:5000 -c conf/guni.conf main:app if demo = true in the conf file, it will run in the background as a daemons or use nohup gunicorn - W4 - B 127.0.0.1:5000 - C conf / guni.conf Main: app
At this time, you can exit the python virtual environment
``deactivate - exits the virtual environment


6, Configure nginx

For the installation, configuration and use of nginx, please refer to my previous tutorial 2020 Ubuntu 1804 installation nginx latest stable version 1.16 detailed tutorial notes
Continue to supplement the previous documents
In the directory / etc/nginx/conf.d /, create a new myprj.conf file and fill in the following information

server {
        listen 80;
        root /data/python/myprj;
        server_name xxx.xx.xxx.xxx;
        location / {
            proxy_set_header x-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header Host $http_host;
            proxy_pass http://Localhost: 6000 /; ා the port number of the gunicorn binding
        }
        # To configure static files:
        location ~ ^\/static\/.*$ {
            root /data/python/myprj;
        }
}

The most common problem in configuring nginx is the configuration of static files. Once they are not configured properly, the pictures and css files cannot be accessed
The difference between using root directory and alias directory in Nginx path location configuration

  1. The directory specified by alias is accurate, that is, the files in the path directory accessed by location matching are directly found in the alias directory;
 location /static/ {
        alias  /var/www/static/;
    }
#Note: the directory specified by alias must be followed by "/", that is, / var/www/static / cannot be changed to / var/www/static "
  1. The directory specified by root is the upper level directory of the path directory accessed by location matching. This path directory must exist in the directory specified by root;
location /static/ {
        root  /var/www/;
    }
#Note: the / static / specified in the location must exist in the / var/www / directory specified by root.
#The access effect of the two configurations is the same. 
  1. Configuration habits
    In general, good habits in nginx configuration are:
    • Configure the root directory in location /
    • Configure the alias virtual directory in location /somepath /

  2. nginx load new configuration
nginx -t     #Be sure to check first to see the success message,
nginx -s reload

7, Install and configure supervisor

supervisor is a process management tool developed by python. It can not only manage Python projects, java projects and PHP PFM background startup programs

  1. install
    `` sudo apt install supervisor
  2. configuration file
    Go to / ect/supervisor/conf.d/ and create a new myprj.conf file
    ``vim myprj.conf
    Fill in the following


[program:aijiaoyu]
command=/opt/python/myprj./bin/gunicorn -c /data/python/myprj./env/gunc.py main:app
directory=/data/python/myprj
user=root
autorestart=true
startsecs=10
startretires=3
redirect_stderr=true
stdout_logfile_maxbytes=100MB
stdout_logfile_backups = 20
stdout_logfile=/var/log/supervisor/myprj.log

Simple description of the configuration file

program:myprj]   #Registered project name
directory=/data/python/myprj
command=/opt/python/myprj./bin/gunicorn -c /data/python/myprj./env/gunc.py main:app Program start command
autostart=true       ; stay supervisord It starts automatically when it starts
startsecs=10         ; If there is no abnormal exit after 10 seconds of starting, it means that the process is started normally. The default is 1 second
autorestart=true     ; Automatic restart after program exit,Optional values:[unexpected,true,false],Default is unexpected,Indicates that the process is not restarted until it is accidentally killed
startretries=3       ; Number of automatic retries for startup failure, default is 3
user=root          ; Which user is used to start the process? The default is root
priority=999         ; Process start priority, 999 by default, priority start with small value
redirect_stderr=true ; hold stderr Redirect to stdout,default false
stdout_logfile_maxbytes=100MB  ; stdout Log file size, default 50 MB
stdout_logfile_backups = 20   ; stdout Number of log file backups, default is 10
; stdout For log files, it should be noted that when the specified directory does not exist, it cannot be started normally, so you need to create the directory manually( supervisord Automatically create log file)
stdout_logfile=/var/log/supervisor/myprj.log
stopasgroup=false     ;Default is false,Whether to send to this process group when the process is killed stop Signals, including subprocesses
killasgroup=false     ;Default is false,Send to process group kill Signals, including subprocesses

Let supervisor update configuration

sudo supervisorctl update   #Update configuration
sudo supervisorctl reload   #restart
sudo supervisorctl status  #View all projects
sudo supervisorctl stop/start/restart  entry name

[note] it must not be started as a daemons, otherwise the supervisor will not detect the process, so it will be considered that the process failed to start

summary

Every time I read someone's blog, I feel incomplete, so I rewrite this tutorial. A completed process
If it's useful to you, please like it, or pay attention to it. Welcome to join the group to exchange technology

Reference documents:

  1. Differences and deployment between nginx+uwsgi and nginx+gunicorn
  2. gunicorn deploying the Flask service
  3. Using gunicorn to deploy the flask project

Tags: Python Nginx supervisor pip

Posted on Mon, 04 May 2020 06:09:38 -0400 by joejoejoe