apache2 wsgi socket rotaion (by quqi99)

作者：张华 发表于：2023-05-22

版权声明：可以任意转载，转载时请务必以超链接形式标明文章原始出处和作者信息及本版权声明

问题

for lp bug https://bugs.launchpad.net/ubuntu/+source/mod-wsgi/+bug/1863232

reproducer

lxc launch ubuntu:focal focal
lxc exec focal bash
su - ubuntu
sudo apt update

sudo apt-get install apache2 libapache2-mod-wsgi -y
sudo sed -i 's/^KeepAlive Off/KeepAlive On/g' /etc/apache2/apache2.conf
sudo sed -i '/^KeepAliveTimeout/ s/ .*/ 15/' /etc/apache2/apache2.conf
cat <<EOF | sudo tee /var/www/html/hello-world.py
def application(environ, start_response):
    status = '200 OK'
    output = b'Hello World!\n'
    response_headers = [('Content-type', 'text/plain'),
                        ('Content-Length', str(len(output)))]
    start_response(status, response_headers)
    return [output]
EOF
cat <<EOF | sudo tee /etc/apache2/conf-available/wsgi.conf
WSGIScriptAlias /hello-world /var/www/html/hello-world.py
WSGIDaemonProcess 127.0.0.1 processes=2 threads=16 display-name=%{GROUP}
WSGIProcessGroup 127.0.0.1
EOF
sudo a2enconf wsgi
curl 127.0.0.1/hello-world

用mpm_event + WSGISocketRotation=on不会有问题

题外话，Apache服务器一共有三种稳定的MPM（Multi-Processing Module，多进程处理模块）模式：

• prefork 中没有线程的概念，是多进程模型，一个进程处理一个连接；稳定；响应快。其缺点是在连接数比较大时就非常消耗内存
• worker 是多进程多线程模型，一个进程有多个线程，每个线程处理一个连接。与prefork相比，worker模式更节省系统的内存资源。不过，需要注意worker模式下的Apache与php等程序模块的兼容性
• event 是worker模式的变种，它把服务进程从连接中分离出来,在开启KeepAlive场合下相对worker模式能够承受的了更高的并发负载。event模式不能很好的支持https的访问（HTTP认证相关的问题）

用默认的mpm_event （sudo a2dismod mpm_worker && sudo a2enmod mpm_event && sudo systemctl restart apache2）, 即使WSGISocketRotation=On也不会有问题。

sudo a2dismod mpm_worker
sudo a2enmod mpm_event
sudo systemctl restart apache2

$ ls -1 /var/run/apache2/wsgi.*.sock
/var/run/apache2/wsgi.4570.0.1.sock
$ sudo systemctl reload apache2.service 
$ ls -1 /var/run/apache2/wsgi.*.sock
/var/run/apache2/wsgi.4570.1.1.sock

#send two HTTP requests in the same connection (keep-alive used)
cat <<EOF >http-request
GET /hello-world HTTP/1.1
Host: 127.0.0.1
Connection: keep-alive

EOF

$ (cat http-request; sleep 1; cat http-request; sleep 9) | telnet 127.0.0.1 80 2>&1 | while read line; do echo "$(date +'%T') == $line"; done
07:12:03 == Trying 127.0.0.1...
07:12:03 == Connected to 127.0.0.1.
07:12:03 == Escape character is '^]'.
07:12:03 == HTTP/1.1 200 OK
07:12:03 == Date: Mon, 22 May 2023 07:12:03 GMT
07:12:03 == Server: Apache/2.4.41 (Ubuntu)
07:12:03 == Content-Length: 13
07:12:03 == Vary: Accept-Encoding
07:12:03 == Keep-Alive: timeout=15, max=100
07:12:03 == Connection: Keep-Alive
07:12:03 == Content-Type: text/plain
07:12:03 == 
07:12:03 == Hello World!
07:12:04 == HTTP/1.1 200 OK
07:12:04 == Date: Mon, 22 May 2023 07:12:04 GMT
07:12:04 == Server: Apache/2.4.41 (Ubuntu)
07:12:04 == Content-Length: 13
07:12:04 == Vary: Accept-Encoding
07:12:04 == Keep-Alive: timeout=15, max=99
07:12:04 == Connection: Keep-Alive
07:12:04 == Content-Type: text/plain
07:12:04 == 
07:12:04 == Hello World!
07:12:13 == Connection closed by foreign host.

用mpm_worker + WSGISocketRotation=on在bionic时会有问题

用mpm_worker + WSGISocketRotation=on时在bionic时会有问题，focal不会有问题

sudo a2dismod mpm_event
sudo a2enmod mpm_worker
apache2ctl -M |grep mpm
sudo systemctl restart apache2
(cat http-request; sleep 1; cat http-request; sleep 9) | telnet 127.0.0.1 80 2>&1 | while read line; do echo "$(date +'%T') == $line"; done

$ rmadison mod-wsgi |grep -E 'bionic|focal'
 mod-wsgi | 4.5.17-1               | bionic          | source
 mod-wsgi | 4.5.17-1ubuntu1.1      | bionic-security | source
 mod-wsgi | 4.5.17-1ubuntu1.1      | bionic-updates  | source
 mod-wsgi | 4.6.8-1ubuntu3         | focal           | source
 mod-wsgi | 4.6.8-1ubuntu3.1       | focal-security  | source
 mod-wsgi | 4.6.8-1ubuntu3.1       | focal-updates   | source
 
 hua@t440p:/bak/work/apache2/mod_wsgi$ git tag --contains 13169f2a0610d7451fae92a414e8e20b91e348c9 |head -n2
4.6.0
4.6.1

使用WSGISocketRotation=off时也不会有问题

echo 'WSGISocketRotation off' |sudo tee -a /etc/apache2/conf-available/wsgi.conf
$ sudo systemctl reload apache2.service 
$ ls -1 /var/run/apache2/wsgi.*.sock
/var/run/apache2/wsgi.5187.u33.1.sock
$ sudo systemctl reload apache2.service 
$ ls -1 /var/run/apache2/wsgi.*.sock
/var/run/apache2/wsgi.5187.u33.1.sock

(cat http-request; sleep 1; cat http-request; sleep 9) | telnet 127.0.0.1 80 2>&1 | while read line; do echo "$(date +'%T') == $line"; done

直接测8754端口

为什么客户用mpm_worker时还出现这个问题呢？考虑到KeepAliveTimeout=5, 所以直接在客户环境的nova-cloud-controller/0中测试8754端口(nova-api)

cat <<EOF >http-request
GET / HTTP/1.1
Host: 127.0.0.1
Connection: keep-alive

EOF
(cat http-request; sleep 1) | telnet 127.0.0.1 8754
(cat http-request; sleep 6; cat http-request; sleep 9) | telnet 127.0.0.1 8754 2>&1 | while read line; do echo "$(date +'%T') == $line"; done
for i in {1..100}; do echo $i; date; (cat http-request; sleep 1; cat http-request; sleep 9) | telnet 127.0.0.1 8754 2>&1 | while read line; do echo "$(date +'%T') == $line"; done >>output.txt; done