使用elk收集nginx日志grafana展示

文档仅收集nginx的日志来grafana展示，如果要收集ingress-nginx的日志来grafana展示见 https://www.putianhui.cn/posts/60dba0725b3e 这个文档中的安装ingress-nginx配置

安装es681

我这里用docker安装的es681，这里先安装docker

1	$ curl https://www.putianhui.cn/package/script/install_docker.sh \|bash

创建es数据和logs目录并启动

1
2
3

$ mkdir -p /data/es-data/{data,logs}
$ cd /data/es-data/ && chmod 777 data && chmod 777 logs
$ docker run -d --name es681 -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" -v /data/es-data/data:/usr/share/elasticsearch/data -v /data/es-data/logs:/usr/share/elasticsearch/logs elasticsearch:6.8.1

安装logstash

下载logstash的rpm包,注意版本和es版本一致

1	$ wget https://artifacts.elastic.co/downloads/logstash/logstash-6.8.1.rpm

通过rpm安装

# 安装openjdk1.8
$ yum install java-1.8.0-openjdk

# 安装logstash
$ rpm -ivh logstash-6.8.1.rpm

配置nginx日志格式

$ vim /etc/nginx/nginx.conf
log_format main   '{"@timestamp":"$time_iso8601",'
                        '"@source":"$server_addr",'
                        '"hostname":"$hostname",'
                        '"ip":"$remote_addr",'
                        '"client":"$remote_addr",'
                        '"request_method":"$request_method",'
                        '"scheme":"$scheme",'
                        '"domain":"$server_name",'
                        '"referer":"$http_referer",'
                        '"request":"$request_uri",'
                        '"args":"$args",'
                        '"size":$body_bytes_sent,'
                        '"status": $status,'
                        '"responsetime":$request_time,'
                        '"upstreamtime":"$upstream_response_time",'
                        '"upstreamaddr":"$upstream_addr",'
                        '"http_user_agent":"$http_user_agent",'
                        '"https":"$https"'
                        '}';

nginx虚拟站点修改日志存储目录并应用日志格式

$ vim /etc/nginx/conf.d/test.conf
server {
    server_name test.putianhui.cn;
    listen 80;
    access_log  /var/log/nginx/test.out  main;

    location / {
    root /data/test;
    index index.html;
    }
}

$ nginx -t
$ nginx -s reload

$ ls /var/log/nginx/
access.log  error.log  test.out # test.out为新生成的虚拟站点日志

配置logstash收集nginx日志目录下的*.out日志

$ cd /etc/logstash/

$ vim conf.d/nginx_access.conf
input {
    file {
        path => [ "/var/log/nginx/*.out" ]
        ignore_older => 0
    codec => json
    }
}

filter {
    mutate {
        convert => [ "status","integer" ]
        convert => [ "size","integer" ]
        convert => [ "upstreatime","float" ]
        remove_field => "message"
    }
    geoip {
        source => "ip"
    }
}

output {
    elasticsearch {
        hosts => "192.168.66.73:9200"		# 这里写es的信息
        index => "logstash-nginx-access-%{+YYYY.MM.dd}"
    }
}

启动logstash

1	$ nohup /usr/share/logstash/bin/logstash -f /etc/logstash/conf.d/nginx_access.conf > ./logstash.log 2>&1 &

当有用户访问我们的虚拟站点查看es是否有索引创建并有数据

安装grafana

$ cd /data
$ wget https://dl.grafana.com/enterprise/release/grafana-enterprise-7.5.12.linux-amd64.tar.gz
$ tar -zxvf grafana-enterprise-7.5.12.linux-amd64.tar.gz

# 启动grafana
$ nohup /data/grafana-7.5.12/bin/grafana-server > /data/grafana-7.5.12/grafana.log  2>&1 &

安装grafana的插件

# 安装grafana的插件
$ cd grafana-7.5.12
$ grafana-cli plugins install grafana-worldmap-panel
$ bin/grafana-cli plugins install grafana-piechart-panel 1.3.8

由于默认安装的插件放到了/var/lib/grafana/plugins目录下，我们grafana是二进制安装的，需要把对应的插件移动到我们二进制grafana的plugin目录下