项目部署,Django部署到线上

本篇导航:

前言

《Django开发简单Blog系统》系列中,我们已经完成了一个迷你Web项目。那么,怎么把这个项目发布到线上呢?怎样给它一个域名呢?

思路:nginx + uwsgi

  • 简单粗暴
  • uwsgi
  • nginx
  • supervisor

环境准备

 

服务器

阿里云服务器,centos7系统。

一、简单粗暴

python

升级python到3.6.1,统一线上和本地python环境。

1、下载python3.6.1源码
wget https://www.python.org/ftp/python/3.6.1/Python-3.6.1.tar.xz

2、解压源码

xz -d Python-3.6.1.tar.xz
tar -xvf Python-3.6.1.tar

3、编译源码

mkdir /usr/local/python3
cd Python-3.6.1
./configure --prefix=/usr/local/python3 --enable-optimizations
make && make install

如果编译失败,需要先更新编译环境:

gcc -v 
g++ -v

yum install gcc
yum install gcc-c++

注:我的环境版本为 gcc version 4.8.5 20150623 (Red Hat 4.8.5-11) (GCC)

4、替换python

cd /usr/bin
mv python python.bak
ln -s /usr/local/python3/bin/python3.6 /usr/bin/python
ll python*
python -V

5、解决遗留问题
所有python相关的应用,如果使用/usr/bin/python开头的脚本,替换为/usr/bin/python2.7。比如:

vim /usr/bin/yum
vim /usr/libexec/urlgrabber-ext-down

项目开发完毕,在部署之前需要再配置文件中将 ALLOWED_HOSTS配置设置为:当前服务器IP或*,如:

uwsgi

pip install uwsgi

编写测试:

# test.py
def application(env, start_response):
    start_response('200 OK', [('Content-Type','text/html')])
    return [b"Hello World"]

启动测试:
uwsgi --http :8001 --wsgi-file test.py

报错:uwsgi: command not found,看来我们需要把python3/bin加入到path。
vim /etc/profile,在文件最底部找到PATH,添加:

:/usr/local/python3/bin

使配置生效:source /etc/profile

访问 http://ip:8001
,即可看到Hello World 。

ALLOWED_HOSTS = ["*",]

nginx和mysql

参考《在CentOS7上配置PHP运行环境》,安装好了nginx和mysql。

示例源码:猛击下载

项目部署

然后将源码上传至服务器指定目录,如:/data/
,然后执行命令来运行:

代码准备

1、克隆项目到服务器
git clone https://github.com/voidking/djsite.git

2、安装django
pip install django

3、安装pymysql
pip install pymysql

注: 上传命令: scp
/Users/wupeiqi/PycharmProjects/oldboy-1.zip
root@192.168.10.33:/data/

数据库准备

1、创建数据库

# mysql -uroot -p
mysql> create database `djsite` default character set utf8 collate utf8_general_ci; 
mysql> exit;

2、修改djsite/djsite/settings.py中的数据库配置
vim djsite/djsite/settings.py

3、创建表结构

python manage.py makemigrations
python manage.py migrate

报错:

django.db.utils.InternalError: (1665, 'Cannot execute statement: impossible to write to binary log since BINLOG_FORMAT = STATEMENT and at least one table uses a storage engine limited to row-based logging. InnoDB is limited to row-logging when transaction isolation level is READ COMMITTED or READ UNCOMMITTED.')

修改mysql的binlog格式为混合模式:

# mysql -uroot -p
mysql> set global binlog_format=mixed;

删除数据库djsite中的所有表,然后再次执行:

python manage.py migrate

解压,进入目录并执行以下命令:

启动项目

python3 mange.py runserver 0.0.0.0:8000 

数据库问题

cd djsite
python manage.py runserver

报错:

File "/usr/local/python3/lib/python3.6/site-packages/django/db/backends/mysql/base.py", line 36, in <module>
    raise ImproperlyConfigured("mysqlclient 1.3.3 or newer is required; you have %s" % Database.__version__)
django.core.exceptions.ImproperlyConfigured: mysqlclient 1.3.3 or newer is required; you have 0.7.11.None

解决办法:

vim /usr/local/python3/lib/python3.6/site-packages/django/db/backends/mysql/base.py

进入vim命令模式,输入/version,按N查找下一个,找到:

if version < (1, 3, 3):
    raise ImproperlyConfigured("mysqlclient 1.3.3 or newer is required; you have %s" % Database.__version__)

注释掉它,问题解决。

 图片 1

url问题

cd djsite
python manage.py runserver

再次报错:

File "/root/djsite/djsite/urls.py", line 21, in <module>
    url(r'^blog/', include('blog.urls', namespace='blog')),
  File "/usr/local/python3/lib/python3.6/site-packages/django/urls/conf.py", line 39, in include
    'Specifying a namespace in include() without providing an app_name '
django.core.exceptions.ImproperlyConfigured: Specifying a namespace in include() without providing an app_name is not supported. Set the app_name attribute in the included module, or pass a 2-tuple containing the list of patterns and app_name instead.

解决办法:

vim /usr/local/python3/lib/python3.6/site-packages/django/urls/conf.py

找到:

if namespace and not app_name:
    raise ImproperlyConfigured(
        'Specifying a namespace in include() without providing an app_name '
        'is not supported. Set the app_name attribute in the included '
        'module, or pass a 2-tuple containing the list of patterns and '
        'app_name instead.',
    )

注释掉它,问题解决。


查看效果

cd djsite
python manage.py runserver

启动成功,在服务器上测试访问:
curl localhost:8000/blog/index

使用浏览器查看
http://ip:8000/blog/index
,却无法访问。这是因为在settings.py中,ALLOWED_HOSTS的配置为:

ALLOWED_HOSTS = []

官方文档说:

When DEBUG is True and ALLOWED_HOSTS is empty, the host is validated
against [‘localhost’, ‘127.0.0.1’, ‘[::1]’].

修改ALLOWED_HOSTS的配置为:

ALLOWED_HOSTS = ['*']

然后启动命令改为:python manage.py runserver 0.0.0.0:8000,此时即可在浏览器看到部署好的项目。

如果还是不能访问,尝试先关闭防火墙:systemctl stop firewalld

 

二、uwsgi

Django框架运行依赖wsgi(本质提供socket服务端),众多模块实现了wsgi规范,而django框架中默认使用wsigiref模块来实现,他由于性能比较低,所以用于本地开发和测试,而线上部署时需要使用uwsgi来代替。

1、在服务器上安装uwsgi

pip3 install uwsgi

2、单文件使用uwsgi

在服务器上编写一个Python文件:

def application(env, start_response):
    start_response('200 OK', [('Content-Type','text/html')])
    return [b"Hello World"]

在服务器上执行命令启动Web服务器:

uwsgi --http :9001 --wsgi-file app.py
# 或
uwsgi --http :9002 --wsgi-file foobar.py --master --processes 4

访问查看即可:

图片 2

3、 django程序使用uwsgi

将开发好的django程序拷贝到服务器目录【同示例一】,即:/data/oldboy
,执行命令启动:

uwsgi --http :9005 --chdir /data/oldboy/ --wsgi-file oldboy/wsgi.py --master --processes 4

 

1. 创建配置文件 oldboy.ini

[uwsgi]
http = 0.0.0.0:9005
chdir = /data/oldboy/
wsgi-file = oldboy/wsgi.py
processes = 4
static-map = /static=/data/oldboy/allstatic

2. 根据配置文件启动uwsgi

uwsigi --ini  oldboy.ini

配置文件的形式启动uwsgi

PS:
如果有虚拟环境可以添加上虚拟环境路径配置: virtualenv ``= /env````/oldboy``_venv

此时访问时,会出现找不到静态文件的错误。

想要uwsgi处理静态文件,需要先将django的静态文件收集到制定目录,然后再设置对应关系。

  • 收集django静态文件

    • 在django的配置文件中添加:STATIC_ROOT =
      os.path.join(BASE_DIR,”allstatic”)
    • 执行 python3 manage.py collectstatic
      命令,至此django项目所有相关静态文件都会收集到制定目录。
  • 设置uwsgi静态文件对应关系

    • uwsgi –http :9005 –chdir /data/oldboy/ –wsgi-file
      oldboy/wsgi.py –master –processes 4 –static-map
      /static=/data/oldboy/allstatic

访问地址,即可看到你的女朋友们了….


nginx配置

1、首先,在万网上配置域名解析,添加A记录,解析到阿里云服务器IP。假设解析好的域名为django.voidking.com。

2、在nginx的vhost中,添加django.voidking.com.conf,内容为:

server {
    listen 80;
    server_name django.voidking.com;
    charset utf-8;
    location /{
        proxy_set_header   Host             $host;
        proxy_set_header   X-Real-IP        $remote_addr;
        proxy_set_header  X-Forwarded-For  $proxy_add_x_forwarded_for;
        client_max_body_size       1024m;
        client_body_buffer_size    128k;
        client_body_temp_path      data/client_body_temp;
        proxy_connect_timeout      90;
        proxy_send_timeout         90;
        proxy_read_timeout         90;
        proxy_buffer_size          4k;
        proxy_buffers              4 32k;
        proxy_busy_buffers_size    64k;
        proxy_temp_file_write_size 64k;
        proxy_temp_path            data/proxy_temp;

        proxy_pass http://127.0.0.1:8000;
    }
}

3、重启nginx,./nginx -s reload

4、测试访问
服务器:curl django.voidking.com/blog/index
本地浏览器:http://django.voidking.com/blog/index

至此,django项目已经部署成功,没有用到uwsgi。如果给django添加守护进程,那么我们的部署就接近完美了。那么,uwsgi又能干什么呢,我们继续研究。

 

三、nginx

利用Nginx做反向代理、负载均衡以及处理静态文件。

1、安装Nginx

yum install nginx

2、配置nginx

图片 3图片 4

user root;
worker_processes 4;

error_log /var/log/nginx/error.log;
pid /var/run/nginx.pid;

events {
    worker_connections  1024;
}


http {
    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile            on;
    tcp_nopush          on;
    tcp_nodelay         on;
    keepalive_timeout   65;

    include             /etc/nginx/mime.types;
    default_type        application/octet-stream;

    upstream django {
        server 127.0.0.1:8001; 
        # server 192.1.123.1; 
    }
    server {
        listen      80;

        charset     utf-8;

        # max upload size
        client_max_body_size 75M;

        location /static {
            alias  /data/oldboy/allstatic; 
        }

        location / {
            uwsgi_pass  django;
            include     uwsgi_params;
        }
    }
}

示例配置文件/etc/nginx/nginx.conf

确保nginx配置文件目录下有uwsgi_params文件(默认)

3、配置uwsgi

为了确保让所有请求均通过80端口来访问网站,将uwsgi的配置文件修改为:

[uwsgi]
socket = 127.0.0.1:9005
chdir = /data/oldboy/
wsgi-file = oldboy/wsgi.py
processes = 4
logto = /tmp/oldboy.log

4、启动uwsgi和nginx

uwsgi --ini /data/oldboy/oldboy.ini &

/etc/init.d/nginx start

uwsgi

 

四、supervisor

supervisor是一个对进程管理的软件,可以帮助我们启动uwsgi并维护(uwsgi进程关闭时,自动将其启动起来)。

1、安装

yum install supervisor

2、配置 vim
/etc/supervisor.conf

图片 5图片 6

[supervisord]
http_port=/var/tmp/supervisor.sock ; (default is to run a UNIX domain socket server)
;http_port=127.0.0.1:9001  ; (alternately, ip_address:port specifies AF_INET)
;sockchmod=0700              ; AF_UNIX socketmode (AF_INET ignore, default 0700)
;sockchown=nobody.nogroup     ; AF_UNIX socket uid.gid owner (AF_INET ignores)
;umask=022                   ; (process file creation umask;default 022)
logfile=/var/log/supervisor/supervisord.log ; (main log file;default $CWD/supervisord.log)
logfile_maxbytes=50MB       ; (max main logfile bytes b4 rotation;default 50MB)
logfile_backups=10          ; (num of main logfile rotation backups;default 10)
loglevel=info               ; (logging level;default info; others: debug,warn)
pidfile=/var/run/supervisord.pid ; (supervisord pidfile;default supervisord.pid)
nodaemon=false              ; (start in foreground if true;default false)
minfds=1024                 ; (min. avail startup file descriptors;default 1024)
minprocs=200                ; (min. avail process descriptors;default 200)

;nocleanup=true              ; (don't clean up tempfiles at start;default false)
;http_username=user          ; (default is no username (open system))
;http_password=123           ; (default is no password (open system))
;childlogdir=/tmp            ; ('AUTO' child log dir, default $TEMP)
;user=chrism                 ; (default is current user, required if root)
;directory=/tmp              ; (default is not to cd during start)
;environment=KEY=value       ; (key value pairs to add to environment)

[supervisorctl]
serverurl=unix:///var/tmp/supervisor.sock ; use a unix:// URL  for a unix socket
;serverurl=http://127.0.0.1:9001 ; use an http:// url to specify an inet socket
;username=chris              ; should be same as http_username if set
;password=123                ; should be same as http_password if set
;prompt=mysupervisor         ; cmd line prompt (default "supervisor")

; The below sample program section shows all possible program subsection values,
; create one or more 'real' program: sections to be able to control them under
; supervisor.

;[program:theprogramname]
;command=/bin/cat            ; the program (relative uses PATH, can take args)
;priority=999                ; the relative start priority (default 999)
;autostart=true              ; start at supervisord start (default: true)
;autorestart=true            ; retstart at unexpected quit (default: true)
;startsecs=10                ; number of secs prog must stay running (def. 10)
;startretries=3              ; max # of serial start failures (default 3)
;exitcodes=0,2               ; 'expected' exit codes for process (default 0,2)
;stopsignal=QUIT             ; signal used to kill process (default TERM)
;stopwaitsecs=10             ; max num secs to wait before SIGKILL (default 10)
;user=chrism                 ; setuid to this UNIX account to run the program
;log_stdout=true             ; if true, log program stdout (default true)
;log_stderr=true             ; if true, log program stderr (def false)
;logfile=/var/log/cat.log    ; child log path, use NONE for none; default AUTO
;logfile_maxbytes=1MB        ; max # logfile bytes b4 rotation (default 50MB)
;logfile_backups=10          ; # of logfile backups (default 10)



[program:oldboy]
command=/usr/local/bin/uwsgi /data/oldboy/oldboy.ini ;命令
priority=999                ; 优先级(越小越优先)
autostart=true              ; supervisord启动时,该程序也启动
autorestart=true            ; 异常退出时,自动启动
startsecs=10                ; 启动后持续10s后未发生异常,才表示启动成功
startretries=3              ; 异常后,自动重启次数
exitcodes=0,2               ; exit异常抛出的是0、2时才认为是异常
stopsignal=QUIT             ; 杀进程的信号
stopwaitsecs=10             ; 向进程发出stopsignal后等待OS向supervisord返回SIGCHILD 的时间。若超时则supervisord将使用SIGKILL杀进程 
user=chrism                 ; 设置启动该程序的用户
log_stdout=true             ; 如果为True,则记录程序日志
log_stderr=false            ; 如果为True,则记录程序错误日志
logfile=/var/log/cat.log    ; 程序日志路径
logfile_maxbytes=1MB        ; 日志文件最大大小
logfile_backups=10          ; 日志文件最大数量

配置详细

3、启动

supervisord /etc/supervisor.conf
或
/etc/init.d/supervisor start

 

一般启动

1、编写wsgi.py文件
编写django_wsgi.py文件,将其放在与文件manage.py同一个目录下。

#!/usr/bin/env python
# coding: utf-8

import os,django
from django.core.handlers.wsgi import WSGIHandler

os.environ.setdefault("DJANGO_SETTINGS_MODULE", "djsite.settings")
django.setup()
application = WSGIHandler()

2、启动项目
uwsgi --http :8000 --chdir ~/djsite/ --module django_wsgi

3、查看启动结果
lsof -i :8000ps aux | grep uwsgi

4、测试访问
http://ip:8000/blog/index
此时,页面是没有样式的,也就是说静态资源加载失败。

5、配置静态资源
uwsgi --http :8000 --chdir ~/djsite/ --module django_wsgi --static-map=/static=static
此时,页面样式就正常了。

高级启动

1、新建uwsgi.ini,与manage.py在同一级目录。

[uwsgi]
http = :8000
chdir = /root/djsite/
wsgi-file = django_wsgi.py
static-map = '/static=static'

2、启动uwsgi
uwsgi uwsgi.ini

3、测试访问
http://ip:8000/blog/index

supervisor

关闭shell后,uwsgi服务就很快关闭了。为了让它后台运行,需要让它变成守护进程。

安装supervisor

pip install supervisor

报错,因为supervisor不支持python3:
Supervisor requires Python 2.4 or later but does not work on any version
of Python 3. You are using version 3.6.1 (default, Dec 6 2017,
12:03:59)
[GCC 4.8.5 20150623 (Red Hat 4.8.5-11)]. Please install using a
supported version.
Command “python setup.py egg_info” failed with error code 1 in
/tmp/pip-build-y9wv4fmm/supervisor/

安装pyenv

为了使用supervisor,我们需要python2.7的环境。而多版本python的管理,推荐使用pyenv。

1、安装pyenv套装
curl -L https://raw.githubusercontent.com/pyenv/pyenv-installer/master/bin/pyenv-installer | bash
内容除了包含 pyenv 以外,还包含如下插件:

  • pyenv-doctor
  • pyenv-installer
  • pyenv-update
  • pyenv-virtualenv
  • pyenv-which-ext

2、路径添加
vim ~/.bash_profile,添加:

export PATH="/root/.pyenv/bin:$PATH"
eval "$(pyenv init -)"
eval "$(pyenv virtualenv-init -)"

3、使配置立即生效
source ~/.bash_profile

4、查看安装情况
pyenv -v

5、常用命令

  • 查看可安装的python版本列表:pyenv install -l
  • 安装指定版本的python:pyenv install 2.7.3
  • 查看已安装的python:pyenv versions
  • 查看当前设为默认的python版本:pyenv version

安装python2.7环境

1、配置pyenv下载源为本地目录(可选操作,不做的话下载速度会很慢)

mkdir /root/python/ && cd /root/python/

# 设置变量
export PYTHON_BUILD_CACHE_PATH=/root/python

# 设置变量
export PYTHON_BUILD_MIRROR_URL=/root/python

# 查看变量设置
env | grep PYTHON_BUILD_MIRROR_URL

wget https://www.python.org/ftp/python/2.7.13/Python-2.7.13.tar.xz

2、安装python2.7.13
pyenv install 2.7.13

3、改变全局版本
pyenv global 2.7.13python -V

附:改变回原版本
pyenv global systempython -V

4、刷新数据库
python rehash

安装虚拟环境

1、新建supervisor虚拟环境
pyenv virtualenv 2.7.13 supervisor

2、激活虚拟环境
source /root/.pyenv/versions/2.7.13/envs/supervisor/bin/activate supervisor
或者source activate supervisor

3、安装supervisor
yum install supervisor
pip install supervisor

4、生成配置文件

mkdir -p /etc/supervisor/
echo_supervisord_conf > /etc/supervisord.conf

5、修改配置文件
vim /etc/supervisord.conf,添加:

[include]
files = /etc/supervisor/*.conf

6、运行
/root/.pyenv/versions/2.7.13/envs/supervisor/bin/supervisord -c /etc/supervisord.conf

7、编辑supervisord.service
vi /usr/lib/systemd/system/supervisord.service,修改为:

[Unit]
Description=Process Monitoring and Control Daemon
After=rc-local.service nss-user-lookup.target

[Service]
Type=forking
ExecStart=/root/.pyenv/versions/2.7.13/envs/supervisor/bin/supervisord -c /etc/supervisord.conf  
ExecReload=/root/.pyenv/versions/2.7.13/envs/supervisor/bin/supervisorctl reload       
ExecStop=/root/.pyenv/versions/2.7.13/envs/supervisor/bin/supervisorctl shutdown      

[Install]
WantedBy=multi-user.target

8、重启supervisor

ps aux | grep supervisord
systemctl stop supervisord
systemctl start supervisord

9、开机启动
systemctl enable supervisord

守护uwsgi

1、在/etc/supervisor中新建djsite.conf文件:

[program:djsite]
command=/usr/local/python3/bin/uwsgi --http :8000 --chdir /root/djsite/ --module django_wsgi --static-map=/static=static
directory=/root/djsite/
startsecs=0
stopwaitsecs=0
autostart=true
autorestart=true

2、重启supervisor

systemctl stop supervisord
systemctl start supervisord

附:重启djsite命令

supervisorctl -c /etc/supervisord.conf restart djsite

3、测试访问
http://ip:8000/blog/index
页面显示正常,至此守护进程配置成功。

4、退出supervisor环境
source deactivate,守护进程并没有受到影响。

nginx+uwsgi

以上,我们的djsite项目已经通过uwsgi方式启动起来,并且可以保持后台运行。nginx配置不改变的情况下,我们可以正常访问
http://django.voidking.com/blog/index
。此时,nginx作为反向代理,和uwsgi间通过http交互。

接下来,就配置下nginx和uwsgi通过socket结合的方式。原理:用户发送http请求到nginx,nginx通过socket把请求交给uwsgi,uwsgi拿到django的处理结果,通过socket返还给nginx,nginx通过http返回结果给用户。

1、因为nginx和uwsgi通过socket方式交互,我们需要修改uwsgi.ini的配置为:

[uwsgi]
socket = :8000
chdir = /root/djsite/
wsgi-file = django_wsgi.py
static-map = '/static=static'
master = true
processes = 2
enable-threads = true
# daemonize = /root/djsite/uwsgi.log

2、/etc/supervisor/djsite.conf,修改为

[program:djsite]command=/usr/local/python3/bin/uwsgi uwsgi.ini
directory=/root/djsite/
startsecs=0
stopwaitsecs=0
autostart=true
autorestart=true

3、重启supervisor
systemctl stop supervisord
systemctl start supervisord

4、修改nginx配置djsite.voidking.com.conf:

server {
    listen      80;
    server_name djsite.voidking.com;
    charset     utf-8;

    location / {
        uwsgi_pass     127.0.0.1:8000;
        include        uwsgi_params;
    }

    location /static {
        alias /root/djsite/static;
    }
}

5、重启nginx
./nginx -s reload

6、测试访问
此时,访问
http://ip:8000/blog/index
失败,访问
http://django.voidking.com/blog/index
正常。因为8000端口不再提供http服务,而是一个和nginx连接的socket。

7、static
请问,此时的静态资源,是通过uwsgi获取的?还是通过nginx直接获取的?做一个测试即可,修改uwsgi为:

[uwsgi]
socket = :8000
chdir = /root/djsite/
wsgi-file = django_wsgi.py
# static-map = '/static=static'
master = true
processes = 2
enable-threads = true
# daemonize = /root/djsite/uwsgi.log

此时,uwsgi不再提供静态资源。重启supervisor,页面样式正常,可见,静态资源是通过nginx获取的。之所以可以获取到,是因为我们之前在djsite/settings.py中配置了:

STATICFILES_DIRS = (
    os.path.join(BASE_DIR, "static"),
)

小结

至此,django部署完毕,我们实现了三种部署方法:

  • nginx + django(http方式)
  • nginx + uwsgi(http方式)
  • nginx + uwsgi(socket方式)

在此过程中,解决了一些奇怪的bug,学习了升级python的方法,学习了使用pyenv安装多版本python的方法(类似的还有anaconda),学习了给django或者uwsgi添加守护进程的方法,收获颇丰。

书签

Python
Web部署方式总结

Python网络框架——Web服务器

Django在生产环境中的部署

Django
部署(Nginx)

使用Supervisor管理SpiderKeeper和Scrapyd

相关文章