Nginx study notes 3-Nginx log variables and common modules

1. HTTP request

yum -y install curl

Client---------request----->Server

Server-------response---->client

request contains request line, request header, request data

response contains status line, message header, response body

Use linux to send http requests:

curl www.baidu.com

If you don't have curl, you need to install curl first

yum -y install curl

[root@VM_69_65_centos ~]# curl www.baidu.com
<!DOCTYPE html>
<!--STATUS OK--><html> <head><meta http-equiv=content-type content=text/html;charset=utf-8><meta http-equiv=X-UA-Compatible content=IE=Edge><meta content=always name=referrer><link rel=stylesheet type=text/css href=http://s1.bdstatic.com/r/www/cache/bdorz/baidu.min.css><title>百度一下，你就知道</title></head> <body link=#0000cc> <div id=wrapper> <div id=head> <div class=head_wrapper> <div class=s_form> <div class=s_form_wrapper> <div id=lg> <img hidefocus=true src=//www.baidu.com/img/bd_logo1.png width=270 height=129> </div> <form id=form name=f action=//www.baidu.com/s class=fm> <input type=hidden name=bdorz_come value=1> <input type=hidden name=ie value=utf-8> <input type=hidden name=f value=8> <input type=hidden name=rsv_bp value=1> <input type=hidden name=rsv_idx value=1> <input type=hidden name=tn value=baidu><span class="bg s_ipt_wr"><input id=kw name=wd class=s_ipt value maxlength=255 autocomplete=off autofocus></span><span class="bg s_btn_wr"><input type=submit id=su value=百度一下 class="bg s_btn"></span> </form> </div> </div> <div id=u1> <a href=http://news.baidu.com name=tj_trnews class=mnav>新闻</a> <a href=http://www.hao123.com name=tj_trhao123 class=mnav>hao123</a> <a href=http://map.baidu.com name=tj_trmap class=mnav>地图</a> <a href=http://v.baidu.com name=tj_trvideo class=mnav>视频</a> <a href=http://tieba.baidu.com name=tj_trtieba class=mnav>贴吧</a> <noscript> <a href=http://www.baidu.com/bdorz/login.gif?login&tpl=mn&u=http%3A%2F%2Fwww.baidu.com%2f%3fbdorz_come%3d1 name=tj_login class=lb>登录</a> </noscript> <script>document.write('<a href="http://www.baidu.com/bdorz/login.gif?login&tpl=mn&u='+ encodeURIComponent(window.location.href+ (window.location.search === "" ? "?" : "&")+ "bdorz_come=1")+ '" name="tj_login" class="lb">登录</a>');</script> <a href=//www.baidu.com/more/ name=tj_briicon class=bri style="display: block;">更多产品</a> </div> </div> </div> <div id=ftCon> <div id=ftConw> <p id=lh> <a href=http://home.baidu.com>关于百度</a> <a href=http://ir.baidu.com>About Baidu</a> </p> <p id=cp>&copy;2017 Baidu <a href=http://www.baidu.com/duty/>使用百度前必读</a>  <a href=http://jianyi.baidu.com/ class=cp-feedback>意见反馈</a> 京ICP证030173号  <img src=//www.baidu.com/img/gs.gif> </p> </div> </div> </div> </body> </html>

In this way, the request can only see the request body, and we need to use it if we want to see more

curl -v  www.baidu.com

[root@VM_69_65_centos ~]# curl -v  www.baidu.com
* About to connect() to www.baidu.com port 80 (#0)
*  Trying 180.149.131.98...
* Connected to www.baidu.com (180.149.131.98) port 80 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.29.0
> Host: www.baidu.com
> Accept: */*
> 
< HTTP/1.1 200 OK
< Server: bfe/1.0.8.18
< Date: Tue, 16 Jan 2018 06:11:06 GMT
< Content-Type: text/html
< Content-Length: 2381
< Last-Modified: Mon, 23 Jan 2017 13:27:36 GMT
< Connection: Keep-Alive
< ETag: "588604c8-94d"
< Cache-Control: private, no-cache, no-store, proxy-revalidate, no-transform
< Pragma: no-cache
< Set-Cookie: BDORZ=27315; max-age=86400; domain=.baidu.com; path=/
< Accept-Ranges: bytes
< 
<!DOCTYPE html>
<!--STATUS OK--><html> <head><meta http-equiv=content-type content=text/html;charset=utf-8><meta http-equiv=X-UA-Compatible content=IE=Edge><meta content=always name=referrer><link rel=stylesheet type=text/css href=http://s1.bdstatic.com/r/www/cache/bdorz/baidu.min.css><title>百度一下，你就知道</title></head> <body link=#0000cc> <div id=wrapper> <div id=head> <div class=head_wrapper> <div class=s_form> <div class=s_form_wrapper> <div id=lg> <img hidefocus=true src=//www.baidu.com/img/bd_logo1.png width=270 height=129> </div> <form id=form name=f action=//www.baidu.com/s class=fm> <input type=hidden name=bdorz_come value=1> <input type=hidden name=ie value=utf-8> <input type=hidden name=f value=8> <input type=hidden name=rsv_bp value=1> <input type=hidden name=rsv_idx value=1> <input type=hidden name=tn value=baidu><span class="bg s_ipt_wr"><input id=kw name=wd class=s_ipt value maxlength=255 autocomplete=off autofocus></span><span class="bg s_btn_wr"><input type=submit id=su value=百度一下 class="bg s_btn"></span> </form> </div> </div> <div id=u1> <a href=http://news.baidu.com name=tj_trnews class=mnav>新闻</a> <a href=http://www.hao123.com name=tj_trhao123 class=mnav>hao123</a> <a href=http://map.baidu.com name=tj_trmap class=mnav>地图</a> <a href=http://v.baidu.com name=tj_trvideo class=mnav>视频</a> <a href=http://tieba.baidu.com name=tj_trtieba class=mnav>贴吧</a> <noscript> <a href=http://www.baidu.com/bdorz/login.gif?login&tpl=mn&u=http%3A%2F%2Fwww.baidu.com%2f%3fbdorz_come%3d1 name=tj_login class=lb>登录</a> </noscript> <script>document.write('<a href="http://www.baidu.com/bdorz/login.gif?login&tpl=mn&u='+ encodeURIComponent(window.location.href+ (window.location.search === "" ? "?" : "&")+ "bdorz_come=1")+ '" name="tj_login" class="lb">登录</a>');</script> <a href=//www.baidu.com/more/ name=tj_briicon class=bri style="display: block;">更多产品</a> </div> </div> </div> <div id=ftCon> <div id=ftConw> <p id=lh> <a href=http://home.baidu.com>关于百度</a> <a href=http://ir.baidu.com>About Baidu</a> </p> <p id=cp>&copy;2017 Baidu <a href=http://www.baidu.com/duty/>使用百度前必读</a>  <a href=http://jianyi.baidu.com/ class=cp-feedback>意见反馈</a> 京ICP证030173号  <img src=//www.baidu.com/img/gs.gif> </p> </div> </div> </div> </body> </html>
* Connection #0 to host www.baidu.com left intact

映射到空设备上

[root@VM_69_65_centos ~]# curl -v  www.baidu.com > /dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* About to connect() to www.baidu.com port 80 (#0)
*   Trying 61.135.169.125...
* Connected to www.baidu.com (61.135.169.125) port 80 (#0)
> GET / HTTP/1.1                  //请求行 GET为请求方法 后面是请求的协议
> User-Agent: curl/7.29.0         //head的内容
> Host: www.baidu.com		  //报文信息
> Accept: */*
> 
< HTTP/1.1 200 OK 		//200状态码 OK报文
< Server: bfe/1.0.8.18
< Date: Tue, 16 Jan 2018 06:12:02 GMT
< Content-Type: text/html

//响应的head的信息
< Content-Length: 2381
< Last-Modified: Mon, 23 Jan 2017 13:27:29 GMT
< Connection: Keep-Alive
< ETag: "588604c1-94d"
< Cache-Control: private, no-cache, no-store, proxy-revalidate, no-transform
< Pragma: no-cache
< Set-Cookie: BDORZ=27315; max-age=86400; domain=.baidu.com; path=/
< Accept-Ranges: bytes
< 
{ [data not shown]
100  2381  100  2381    0     0   134k      0 --:--:-- --:--:-- --:--:--  136k
* Connection #0 to host www.baidu.com left intact
[root@VM_69_65_centos ~]#

2. Nginx log type

Include: error.log access_log

error.log records the error status of processing http requests and the error status of Nginx's own services.

access_log records the access status of each Nginx request

There are two main instructions related to the nginx server log, one is log_format, which is used to set the log format, and the other is access_log, which is used to specify the storage path, format and cache size of the log file. Generally, the log configuration in the nginx configuration file (/ usr/local/nginx/conf/nginx.conf).

nginx's log_format has many optional parameters to indicate the active status of the server. The default is:
log_format access '$remote_addr - $remote_user [$time_local] "$request" ' '$status $body_bytes_sent "$http_referer" ' '" $http_user_agent" "$http_x_forwarded_for"';

The configuration syntax of log_format:

log_format

Syntax:log_format name[escape=default|json] string ...;

Default:logformat combined"...";

Context:http

Let's check the log configuration of Nginx

Path: /etc/nginx/nginx.conf

user  nginx;
worker_processes  1;

error_log  /var/log/nginx/error.log warn;   //配置nginx错误日志的地方 warn是错误日志的级别(warn以上的错误记录)
pid        /var/run/nginx.pid;


events {
    worker_connections  1024;
}


http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';
    //$后面代表Nginx的变量
    access_log  /var/log/nginx/access.log  main;  //access_log的路径 main表示以main的格式来标识日志

    sendfile        on;
    #tcp_nopush     on;

    keepalive_timeout  65;

    #gzip  on;

    include /etc/nginx/conf.d/*.conf;
}

以下为error.log

2018/01/15 20:50:13 [error] 30536#30536: *1 open() "/usr/share/nginx/html/favicon.ico" failed (2: No such file or directory), client: 36.106.176.206, server: localhost, request: "GET /favicon.ico HTTP/1.1", host: "211.159.168.30"
2018/01/16 00:06:45 [error] 30536#30536: *10 open() "/usr/share/nginx/html/hndUnblock.cgi" failed (2: No such file or directory), client: 189.62.120.171, server: localhost, request: "GET /hndUnblock.cgi HTTP/1.1", host: "211.159.168.30"
2018/01/16 00:06:48 [error] 30536#30536: *11 open() "/usr/share/nginx/html/tmUnblock.cgi" failed (2: No such file or directory), client: 189.62.120.171, server: localhost, request: "GET /tmUnblock.cgi HTTP/1.1", host: "211.159.168.30"
2018/01/16 00:06:51 [error] 30536#30536: *12 open() "/usr/share/nginx/html/moo" failed (2: No such file or directory), client: 189.62.120.171, server: localhost, request: "GET /moo HTTP/1.1", host: "211.159.168.30"
2018/01/16 00:06:58 [error] 30536#30536: *14 open() "/usr/share/nginx/html/getcfg.php" failed (2: No such file or directory), client: 189.62.120.171, server: localhost, request: "POST /getcfg.php HTTP/1.1", host: "211.159.168.30"
2018/01/16 00:07:05 [error] 30536#30536: *15 open() "/usr/share/nginx/html/getcfg.php" failed (2: No such file or directory), client: 189.62.120.171, server: localhost, request: "POST /getcfg.php HTTP/1.1", host: "211.159.168.30"
2018/01/16 00:32:05 [error] 30536#30536: *16 open() "/usr/share/nginx/html/azenv.php" failed (2: No such file or directory), client: 95.213.187.190, server: localhost, request: "POST http://check.best-proxies.ru/azenv.php?auth=151603392515&a=PSCN&i=3550455838&p=80 HTTP/1.1", host: "check.best-proxies.ru", referrer: "http://best-proxies.ru/"
2018/01/16 02:18:28 [error] 30536#30536: *19 open() "/usr/share/nginx/html/echo.php" failed (2: No such file or directory), client: 139.162.88.63, server: localhost, request: "GET http://clientapi.ipip.net/echo.php?info=1234567890 HTTP/1.1", host: "clientapi.ipip.net"
2018/01/16 06:07:15 [error] 30536#30536: *24 open() "/usr/share/nginx/html/404/search_children.js" failed (2: No such file or directory), client: 120.132.3.65, server: localhost, request: "GET http://www.qq.com/404/search_children.js HTTP/1.1", host: "www.qq.com"
2018/01/16 06:38:59 [error] 30536#30536: *25 open() "/usr/share/nginx/html/webconfig.ini" failed (2: No such file or directory), client: 205.209.159.44, server: localhost, request: "GET /webconfig.ini HTTP/1.1", host: "211.159.168.30"
2018/01/16 08:20:52 [error] 30536#30536: *26 open() "/usr/share/nginx/html/azenv.php" failed (2: No such file or directory), client: 95.213.187.189, server: localhost, request: "POST http://check.best-proxies.ru/azenv.php?auth=151606205281&a=PSCN&i=3550455838&p=80 HTTP/1.1", host: "check.best-proxies.ru", referrer: "http://best-proxies.ru/"
2018/01/16 10:51:12 [error] 30536#30536: *32 open() "/usr/share/nginx/html/index.action" failed (2: No such file or directory), client: 58.218.201.54, server: localhost, request: "GET /index.action HTTP/1.1", host: "211.159.168.30"
2018/01/16 11:57:11 [error] 30536#30536: *33 open() "/usr/share/nginx/html/manager/html" failed (2: No such file or directory), client: 218.93.201.199, server: localhost, request: "GET /manager/html HTTP/1.1", host: "211.159.168.30:80"
2018/01/16 13:30:24 [error] 30536#30536: *34 open() "/usr/share/nginx/html/forum.php" failed (2: No such file or directory), client: 106.120.160.119, server: localhost, request: "GET /forum.php?mod=forumdisplay&fid=2 HTTP/1.1", host: "211.159.168.30", referrer: "http://211.159.168.30/forum.php?mod=forumdisplay&fid=2"
2018/01/16 13:30:56 [error] 30536#30536: *36 open() "/usr/share/nginx/html/forum.php" failed (2: No such file or directory), client: 220.181.132.198, server: localhost, request: "GET /forum.php?mod=forumdisplay&fid=2 HTTP/1.1", host: "211.159.168.30"
2018/01/16 13:30:56 [error] 30536#30536: *37 open() "/usr/share/nginx/html/forum.php" failed (2: No such file or directory), client: 171.13.14.145, server: localhost, request: "GET /forum.php?mod=forumdisplay&fid=2 HTTP/1.1", host: "211.159.168.30"
2018/01/16 13:31:10 [error] 30536#30536: *38 open() "/usr/share/nginx/html/forum.php" failed (2: No such file or directory), client: 182.118.20.158, server: localhost, request: "GET /forum.php?mod=forumdisplay&fid=2 HTTP/1.1", host: "211.159.168.30"
2018/01/16 13:31:18 [error] 30536#30536: *39 open() "/usr/share/nginx/html/forum.php" failed (2: No such file or directory), client: 182.118.20.145, server: localhost, request: "GET /forum.php?mod=forumdisplay&fid=2 HTTP/1.1", host: "211.159.168.30"
2018/01/16 13:31:23 [error] 30536#30536: *40 open() "/usr/share/nginx/html/forum.php" failed (2: No such file or directory), client: 182.118.20.149, server: localhost, request: "GET /forum.php?mod=forumdisplay&fid=2 HTTP/1.1", host: "211.159.168.30"
2018/01/16 14:40:23 [error] 30536#30536: *41 open() "/usr/share/nginx/html/home.php" failed (2: No such file or directory), client: 106.120.161.66, server: localhost, request: "GET /home.php HTTP/1.1", host: "211.159.168.30", referrer: "http://211.159.168.30/home.php"
2018/01/16 14:40:46 [error] 30536#30536: *42 open() "/usr/share/nginx/html/home.php" failed (2: No such file or directory), client: 101.199.108.53, server: localhost, request: "GET /home.php HTTP/1.1", host: "211.159.168.30"
2018/01/16 14:41:26 [error] 30536#30536: *43 open() "/usr/share/nginx/html/home.php" failed (2: No such file or directory), client: 171.13.14.145, server: localhost, request: "GET /home.php HTTP/1.1", host: "211.159.168.30"
2018/01/16 14:42:00 [error] 30536#30536: *44 open() "/usr/share/nginx/html/home.php" failed (2: No such file or directory), client: 171.13.14.132, server: localhost, request: "GET /home.php HTTP/1.1", host: "211.159.168.30"
2018/01/16 14:42:00 [error] 30536#30536: *45 open() "/usr/share/nginx/html/home.php" failed (2: No such file or directory), client: 171.13.14.151, server: localhost, request: "GET /home.php HTTP/1.1", host: "211.159.168.30"

以下为access.log


36.106.176.206 - - [15/Jan/2018:20:50:13 +0800] "GET /favicon.ico HTTP/1.1" 404 571 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/55.0.2883.87 Safari/537.36" "-"
36.106.176.206 - - [15/Jan/2018:20:50:14 +0800] "GET / HTTP/1.1" 200 555 "-" "Mozilla/5.0 (Windows NT 10.0; WOW64; Trident/7.0; rv:11.0) like Gecko" "-"
106.120.161.66 - - [15/Jan/2018:20:51:04 +0800] "GET / HTTP/1.1" 200 555 "http://211.159.168.30/" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; Tablet PC 2.0)" "-"
220.181.132.195 - - [15/Jan/2018:20:51:32 +0800] "GET / HTTP/1.1" 200 555 "-" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36" "-"
187.106.44.13 - - [15/Jan/2018:20:54:48 +0800] "t3 12.2.1" 400 173 "-" "-" "-"
189.62.120.171 - - [16/Jan/2018:00:06:45 +0800] "GET /hndUnblock.cgi HTTP/1.1" 404 169 "-" "Wget(linux)" "-"
189.62.120.171 - - [16/Jan/2018:00:06:48 +0800] "GET /tmUnblock.cgi HTTP/1.1" 404 169 "-" "Wget(linux)" "-"
189.62.120.171 - - [16/Jan/2018:00:06:51 +0800] "GET /moo HTTP/1.1" 404 169 "-" "Wget(linux)" "-"
189.62.120.171 - - [16/Jan/2018:00:06:54 +0800] "GET / HTTP/1.1" 200 555 "-" "Wget(linux)" "-"
189.62.120.171 - - [16/Jan/2018:00:06:58 +0800] "POST /getcfg.php HTTP/1.1" 404 169 "-" "Wget(linux)" "-"
189.62.120.171 - - [16/Jan/2018:00:07:05 +0800] "POST /getcfg.php HTTP/1.1" 404 169 "-" "Wget(linux)" "-"
95.213.187.190 - - [16/Jan/2018:00:32:05 +0800] "POST http://check.best-proxies.ru/azenv.php?auth=151603392515&a=PSCN&i=3550455838&p=80 HTTP/1.1" 404 571 "http://best-proxies.ru/" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0)" "-"
205.209.159.44 - - [16/Jan/2018:00:39:29 +0800] "GET / HTTP/1.1" 200 555 "-" "Mozilla/5.0 (compatible; Nmap Scripting Engine; https://nmap.org/book/nse.html)" "-"
47.93.112.232 - - [16/Jan/2018:01:29:16 +0800] "GET / HTTP/1.1" 200 555 "-" "-" "-"
139.162.88.63 - - [16/Jan/2018:02:18:28 +0800] "GET http://clientapi.ipip.net/echo.php?info=1234567890 HTTP/1.1" 404 169 "-" "Go-http-client/1.1" "-"
104.236.182.189 - - [16/Jan/2018:03:02:48 +0800] "GET / HTTP/1.1" 200 555 "-" "Mozilla/5.0 zgrab/0.x" "-"
120.132.3.65 - - [16/Jan/2018:06:07:15 +0800] "GET http://www.qq.com/404/search_children.js HTTP/1.1" 404 571 "-" "Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.114 Safari/537.36" "-"
120.132.3.65 - - [16/Jan/2018:06:07:15 +0800] "\x04\x01\x00PpTi4\x00" 400 173 "-" "-" "-"
120.132.3.65 - - [16/Jan/2018:06:07:15 +0800] "\x05\x01\x00" 400 173 "-" "-" "-"
205.209.159.44 - - [16/Jan/2018:06:38:59 +0800] "GET /webconfig.ini HTTP/1.1" 404 169 "-" "Mozilla/5.0 (compatible; Nmap Scripting Engine; https://nmap.org/book/nse.html)" "-"
95.213.187.189 - - [16/Jan/2018:08:20:52 +0800] "POST http://check.best-proxies.ru/azenv.php?auth=151606205281&a=PSCN&i=3550455838&p=80 HTTP/1.1" 404 571 "http://best-proxies.ru/" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0)" "-"
47.93.90.175 - - [16/Jan/2018:09:34:09 +0800] "GET / HTTP/1.1" 200 555 "-" "-" "-"
123.59.146.153 - - [16/Jan/2018:10:15:57 +0800] "\x16\x03\x01\x01\x22\x01\x00\x01\x1E\x03\x03\xDA\xF2\x07\x92\x89\xD5\x16\xD0\xA8\x03\xA8\xEA\xDE\x95\xF7\x90\xDF\x98\x11\xB2\x01\xB5v\x0F\x13be\xAF^\xD3\xC56\x00\x00\x88\xC00\xC0,\xC0(\xC0$\xC0\x14\xC0" 400 173 "-" "-" "-"
123.59.146.153 - - [16/Jan/2018:10:16:05 +0800] "USER test +iw test :Test Wuz Here" 400 173 "-" "-" "-"
123.59.146.153 - - [16/Jan/2018:10:16:05 +0800] "GET / HTTP/1.1" 200 555 "-" "Mozilla/5.0 (Windows NT 6.1; WOW64; rv:52.0) Gecko/20100101 Firefox/52.0" "-"
58.218.201.54 - - [16/Jan/2018:10:51:12 +0800] "GET / HTTP/1.1" 200 555 "-" "User-Agent:Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.2; .NET CLR 1.0.3705" "-"
58.218.201.54 - - [16/Jan/2018:10:51:12 +0800] "GET /index.action HTTP/1.1" 404 571 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_3) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/56.0.2924.87 Safari/537.36" "-"
218.93.201.199 - - [16/Jan/2018:11:57:11 +0800] "GET /manager/html HTTP/1.1" 404 571 "-" "Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.2; WOW64; Trident/6.0)" "-"
106.120.160.119 - - [16/Jan/2018:13:30:24 +0800] "GET /forum.php?mod=forumdisplay&fid=2 HTTP/1.1" 404 571 "http://211.159.168.30/forum.php?mod=forumdisplay&fid=2" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; Tablet PC 2.0)" "-"
220.181.132.180 - - [16/Jan/2018:13:30:41 +0800] "GET / HTTP/1.1" 200 555 "http://211.159.168.30/" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36" "-"
220.181.132.198 - - [16/Jan/2018:13:30:56 +0800] "GET /forum.php?mod=forumdisplay&fid=2 HTTP/1.1" 404 571 "-" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36" "-"
171.13.14.145 - - [16/Jan/2018:13:30:56 +0800] "GET /forum.php?mod=forumdisplay&fid=2 HTTP/1.1" 404 571 "-" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36" "unknown"
182.118.20.158 - - [16/Jan/2018:13:31:10 +0800] "GET /forum.php?mod=forumdisplay&fid=2 HTTP/1.1" 404 571 "-" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36" "unknown"
182.118.20.145 - - [16/Jan/2018:13:31:18 +0800] "GET /forum.php?mod=forumdisplay&fid=2 HTTP/1.1" 404 571 "-" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36" "unknown"
182.118.20.149 - - [16/Jan/2018:13:31:23 +0800] "GET /forum.php?mod=forumdisplay&fid=2 HTTP/1.1" 404 571 "-" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36" "unknown"
106.120.161.66 - - [16/Jan/2018:14:40:23 +0800] "GET /home.php HTTP/1.1" 404 571 "http://211.159.168.30/home.php" "Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR 2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; Tablet PC 2.0)" "-"
101.199.108.53 - - [16/Jan/2018:14:40:46 +0800] "GET /home.php HTTP/1.1" 404 571 "-" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36" "-"
171.13.14.145 - - [16/Jan/2018:14:41:26 +0800] "GET /home.php HTTP/1.1" 404 571 "-" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36" "unknown"
171.13.14.132 - - [16/Jan/2018:14:42:00 +0800] "GET /home.php HTTP/1.1" 404 571 "-" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36" "unknown"
171.13.14.151 - - [16/Jan/2018:14:42:00 +0800] "GET /home.php HTTP/1.1" 404 571 "-" "Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/36.0.1985.125 Safari/537.36" "unknown"
95.76.223.173 - - [16/Jan/2018:15:09:35 +0800] "GET / HTTP/1.0" 200 555 "-" "Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0)" "-"

3. Nginx variables

Http request variable

arg_PARAMETER (request parameter), http_HEADER (request header), sent_http_HEADER (response header returned by the server)

for example:

[root@VM_69_65_centos ~]# curl -v www.baidu.com >/dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0* About to connect() to www.baidu.com port 80 (#0)
*   Trying 220.181.112.244...
* Connected to www.baidu.com (220.181.112.244) port 80 (#0)
> GET / HTTP/1.1
> User-Agent: curl/7.29.0 //这里证明request使用的是curl，用的是7.29.0的版本进行的请求

如果想要让日志记录User-Agent我们需要修改nginx.conf（/etc/nginx/nginx.conf）配置文件

http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;


    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

//修改上面这一段，下面为修改后的结果

修改时需要注意每一个字母都要小写，而且-要改为下划线_

log_format  main  '$http_user_agent' '$remote_addr - $remote_user [$time_local] "$request" '           '$status $body_bytes_sent "$http_referer" '
          '"$http_user_agent" "$http_x_forwarded_for"';


    access_log  /var/log/nginx/access.log  main;


    sendfile        on;
    #tcp_nopush     on;


    keepalive_timeout  65;


    #gzip  on;


    include /etc/nginx/conf.d/*.conf;
}
> Host: www.baidu.com
> Accept: */*
> 
< HTTP/1.1 200 OK
< Server: bfe/1.0.8.18
< Date: Tue, 16 Jan 2018 07:38:26 GMT
< Content-Type: text/html
< Content-Length: 2381
< Last-Modified: Mon, 23 Jan 2017 13:27:36 GMT
< Connection: Keep-Alive
< ETag: "588604c8-94d"
< Cache-Control: private, no-cache, no-store, proxy-revalidate, no-transform
< Pragma: no-cache
< Set-Cookie: BDORZ=27315; max-age=86400; domain=.baidu.com; path=/
< Accept-Ranges: bytes
< 
{ [data not shown]
100  2381  100  2381    0     0   119k      0 --:--:-- --:--:-- --:--:--  122k
* Connection #0 to host www.baidu.com left intact
[root@VM_69_65_centos ~]# 


    
    
     
     
     
     
      
      


    
    
     
     
之后我们使用

[root@VM_69_65_centos ~]# nginx -t -c /etc/nginx/nginx.conf
查看日志是否正确
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful之后我们再使用curl请本机127.0.0.1然后我们查看access.log

127.0.0.1 - - [16/Jan/2018:15:52:02 +0800] "GET / HTTP/1.1" 200 555 "-" "curl/7.29.0" "-"
127.0.0.1 - - [16/Jan/2018:15:52:09 +0800] "GET / HTTP/1.1" 200 555 "-" "curl/7.29.0" "-"
127.0.0.1 - - [16/Jan/2018:15:52:10 +0800] "GET / HTTP/1.1" 200 555 "-" "curl/7.29.0" "-"
127.0.0.1 - - [16/Jan/2018:15:52:10 +0800] "GET / HTTP/1.1" 200 555 "-" "curl/7.29.0" "-"

Then we found that the configuration took effect

Built-in variables - Nginx built-in

View the official website of nginx

http://nginx.org/en/docs/

http://nginx.org/en/docs/syslog.html

http://nginx.org/en/docs/http/ngx_http_log_module.html#access_log

Then let's take a look at what are the default log_formats of nginx

  log_format  main  '$http_user_agent' '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

parameter	illustrate	example
$remote_addr	client address	211.28.65.253
$remote_user	It is useless if the client user name does not enable the authentication module	--
$time_local	Access time and time zone	18/Jul/2012:17:00:01 +0800
$request	Request URI and HTTP protocol	"GET /article-10000.html HTTP/1.1"
$http_host	Request address, that is, the address you enter in the browser (IP or domain name)	192.168.100.100
$status	HTTP request status	200
$upstream_status	upstream status	200
$body_bytes_sent	The content size of the file sent to the client	1547
$http_referer	The source of the url jump (which is the upper level page, it can be used as an anti-leech link)	https://www.baidu.com/
$http_user_agent	User terminal browser and other information (you can record what the client uses to access, such as IE, curl)	"Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; SV1; GTB7.0; .NET4.0C;
$ssl_protocol	SSL protocol version	TLSv1
$ssl_cipher	Algorithms in Exchange Data	RC4-SHA
$upstream_addr	The address of the upstream in the background, that is, the address of the host that actually provides the service	10.10.10.100:80
$request_time	total time for the entire request	0.205
$upstream_response_time	During the request process, upstream response time	0.002

Custom variable - define it yourself

talk about it later.

Four, Nginx module explanation

Nginx official module

Nginx Chinese documentation: http://www.nginx.cn/doc/

[root@VM_69_65_centos ~]# nginx -V

查看nginx编译信息
nginx version: nginx/1.12.2
built by gcc 4.8.5 20150623 (Red Hat 4.8.5-16) (GCC) 
built with OpenSSL 1.0.2k-fips  26 Jan 2017
TLS SNI support enabled
configure arguments: --prefix=/etc/nginx --sbin-path=/usr/sbin/nginx --modules-path=/usr/lib64/nginx/modules --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log --http-log-path=/var/log/nginx/access.log --pid-path=/var/run/nginx.pid --lock-path=/var/run/nginx.lock --http-client-body-temp-path=/var/cache/nginx/client_temp --http-proxy-temp-path=/var/cache/nginx/proxy_temp --http-fastcgi-temp-path=/var/cache/nginx/fastcgi_temp --http-uwsgi-temp-path=/var/cache/nginx/uwsgi_temp --http-scgi-temp-path=/var/cache/nginx/scgi_temp --user=nginx --group=nginx --with-compat --with-file-aio --with-threads --with-http_addition_module --with-http_auth_request_module --with-http_dav_module --with-http_flv_module --with-http_gunzip_module --with-http_gzip_static_module --with-http_mp4_module --with-http_random_index_module --with-http_realip_module --with-http_secure_link_module --with-http_slice_module --with-http_ssl_module --with-http_stub_status_module --with-http_sub_module --with-http_v2_module --with-mail --with-mail_ssl_module --with-stream --with-stream_realip_module --with-stream_ssl_module --with-stream_ssl_preread_module --with-cc-opt='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector-strong --param=ssp-buffer-size=4 -grecord-gcc-switches -m64 -mtune=generic -fPIC' --with-ld-opt='-Wl,-z,relro -Wl,-z,now -pie'

--with- h ttp_stub_status_module Nginx client status

Configuration:

Syntax:stub_status;

Default: - (no default)

Context: server, location (configure based on server or location)

Enter /etc/nginx/nginx.d/default.conf (default configuration) In fact, the access path is configured here, and we put this module into the access path.

server {
    listen       80;
    server_name  localhost;

    #charset koi8-r;
    #access_log  /var/log/nginx/host.access.log  main;

    location / {
        root   /usr/share/nginx/html;
        index  index.html index.htm;
    }

   location /zjlstatus{ //这里的是自定义名
   stub_status;
}



    #error_page  404              /404.html;

    # redirect server error pages to the static page /50x.html
    #
    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   /usr/share/nginx/html;
    }

    # proxy the PHP scripts to Apache listening on 127.0.0.1:80
    #
    #location ~ \.php$ {
    #    proxy_pass   http://127.0.0.1;
    #}

    # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
    #
    #location ~ \.php$ {
    #    root           html;
    #    fastcgi_pass   127.0.0.1:9000;
    #    fastcgi_index  index.php;
    #    fastcgi_param  SCRIPT_FILENAME  /scripts$fastcgi_script_name;
    #    include        fastcgi_params;
    #}

    # deny access to .htaccess files, if Apache's document root
    # concurs with nginx's one
    #
    #location ~ /\.ht {
    #    deny  all;
    #}
}

Then we verify the correctness of nginx.conf

nginx -tc /etc/nginx/nginx.conf

Then we reload the service

nginx -s reload -c /etc/nginx/nginx.conf

After that we open our server path

http://211.159.XXX.XX/zjlstatus

Active connections: 5    //nginx当前活跃的连接数
server accepts handled requests 
 62 62 51  //第一个表示nginx握手的总的次数，第二个标识nginx所处理的连接数， 最后一个标识请求数
Reading: 0 Writing: 1 Waiting: 4 //第一个标识读的个数，第二个表示写的数目，第三个标识等待的数目

-with-http_random_index_module select a random home page

Syntax:random_index on|off;

Default: random_index off; (default off)

Context: local (can only be configured under local)

Continue to modify default.conf

we will modify

 location / {
        root   /usr/share/nginx/html;
        index  index.html index.htm;
    }

modified to

 location / {
        root   /opt/app/code;
        #index  index.html index.htm; #以后还是用正经注释吧。这里先把主页注释掉
	#之后添加随机主页
	random_index on;
    }

nginx -tc /etc/nginx/nginx.conf

systemctl reload rsyslog.service

systemctl reload nginx

nginx -s reload -c /etc/nginx/nginx.conf

Three homepages (html) should be added to the directory for testing

We enter this directory and set it as the home directory (~)

/opt/app/code/
pwd

Finally, when we visit our homepage, we will find that the homepage is random.

It should be noted here that the random home page cannot be a file starting with . (files starting with . are considered hidden files in linux)

--with-http_sub_module HTTP content substitution

To list some common uses:

Syntax: sub_filter string (string before replacement) replacement (string after replacement);

Default:-

Context:http,server,location

Syntax: sub_filter_last_modified on|off (check whether the content of the server has changed, and judge whether there is an update)

Default: sub_filter_last_modified off;

Context:http,server,location

Syntax:sub_filter_once on|off (on only matches the first one, OFF matches all specified content)

Default: sub_filter_once on;

Context:http,server,location

 location / {
        root   /opt/app/code;
        index  index.html index.htm;
		
	sub_filter '要替换的'  '替换成什么';

sub_filter_last_modified on;#检测更新

sub_filter_once off;#全局替换

Nginx request limit

Connection frequency limit - limit_conn_ module

Request frequency limit - limit_req_ module

Request and connection of HTTP protocol

HTTP1.0TCP cannot be multiplexed

HTTP1.1 sequential TCP multiplexing

HTTP2.0 multiplexing TCP multiplexing

HTTP requests are established on the basis of a TCP connection

A TCP request generates at least one HTTP request

1. Connection limit

Syntax:limit_conn_zone key zone=name:size;

Default:-

Context:http

Syntax:limit_conn zone number;

Default:-

Context:http,server,location

2. Request Limitation

Syntax:limit_req_zone key zone=name:size rate=rate;

Default:-

Context:http

Syntax:limit_req zone=name [burst=number] [nodelay];

Default:-

Context:http,server,location

limit_conn_zone $binary_remote_addr zone=conn_zone:1m;
limit_req_zone $binary_remtoe_addr zone=req_zone:1m rate=1r/s; #一个ip地址的客户端1秒允许发送一个请求 1MB
server {
    listen       80;
    server_name  localhost;

    #charset koi8-r;
    #access_log  /var/log/nginx/host.access.log  main;

    location / {
        root   /opt/app/code;
        index  index.html index.htm;
		#random_index on;
	#limit_conn conn_zone 1;
	#limit_req zone=req_zone burst=3 nodelay;
	#limit_req zone=req_zone burst=3;
	#limit_req zone=req_zone;
    }
	location /zjlstatus{
		stub_status;
	}	

    #error_page  404              /404.html;

    # redirect server error pages to the static page /50x.html
    #
    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   /usr/share/nginx/html;
    }

    # proxy the PHP scripts to Apache listening on 127.0.0.1:80
    #
    #location ~ \.php$ {
    #    proxy_pass   http://127.0.0.1;
    #}

    # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
    #
    #location ~ \.php$ {
    #    root           html;
    #    fastcgi_pass   127.0.0.1:9000;
    #    fastcgi_index  index.php;
    #    fastcgi_param  SCRIPT_FILENAME  /scripts$fastcgi_script_name;
    #    include        fastcgi_params;
    #}

    # deny access to .htaccess files, if Apache's document root
    # concurs with nginx's one
    #
    #location ~ /\.ht {
    #    deny  all;
    #}
}

Nginx access control

IP-based access control - http_access_module

allow

Syntax: allow address (IP address) | CIDR (network segment) | unix (scoket access) | all (all);

Default:-

Context:http,server,location,limit_except

not allowed

Syntax: deny address (IP address) | CIDR (network segment) | unix (scoket access) | all (all);

Default:-

Context:http,server,location,limit_except

User based trust login - http_auth_basic_module

default.conf

server {
    listen       80;
    server_name  localhost;

    #charset koi8-r;
    #access_log  /var/log/nginx/host.access.log  main;

 location / {
        root   /opt/app/code;
        #index  admin.html;
    }
	
location ~^/admin.html {
        root   /opt/app/code;
	deny   36.106.4.102;
	deny   36.106.4.103;
	deny   36.106.4.104;
	allow all;
    }

    #error_page  404              /404.html;

    # redirect server error pages to the static page /50x.html
    #
    error_page   500 502 503 504  /50x.html;
    location = /50x.html {
        root   /usr/share/nginx/html;
    }

    # proxy the PHP scripts to Apache listening on 127.0.0.1:80
    #
    #location ~ \.php$ {
    #    proxy_pass   http://127.0.0.1;
    #}

    # pass the PHP scripts to FastCGI server listening on 127.0.0.1:9000
    #
    #location ~ \.php$ {
    #    root           html;
    #    fastcgi_pass   127.0.0.1:9000;
    #    fastcgi_index  index.php;
    #    fastcgi_param  SCRIPT_FILENAME  /scripts$fastcgi_script_name;
    #    include        fastcgi_params;
    #}

    # deny access to .htaccess files, if Apache's document root
    # concurs with nginx's one
    #
    #location ~ /\.ht {
    #    deny  all;
    #}
}

nginx -t -c /etc/nginx/nginx.conf
nginx -s reload -c /etc/nginx/nginx.conf

http_access_module has limitations

http_x_forwarded_for

http_x_forwarder_for =Client IP, Proxy(1) IP ,Proxy(2) IP,...

How to work around the limitations?

Method 1. Use other HTTP header information to control access, such as HTTP_X_FORWARD_FOR

Method 2. Combining the geo module for

Method 3: Transfer through HTTP custom variables

http_auth_basic_module based on user trust login

Syntax: auth_basic string (indicating open, and display this string on the front end)|off;

Default: auth_basic off;

Context:http,server,location,limit_except

Syntax:auth_basic_user_file file (file, file configuration file, used to store account password information);

Default:--

Context:http,server,location,limit_except

First generate a file that stores user passwords:

[root@VM_69_65_centos nginx]# htpasswd -c ./auth_conf jeson
New password: 
Re-type new password: 
Adding password for user jeson

生成的auth_conf文件中的内容：jeson:$apr1$XzHJd5JC$ueCc/zNNgHaU8FOQPgyee/

limitation:

User information depends on the file method

Operation management machinery, inefficiency

3 solutions

Nginx combines LUA to realize university verification

Nginx and LDAP get through, use nginx-auth-ldap module

third party module

Not introduced yet.