用ELK可视化suricata监控下的流量

安装elasticsearch注意

添加非root用户

我们可能涉及到创建一个非root用户
sudo useradd es
sudo passwd es
然后输入这个用户的密码
然后对/usr/local/elasticsearch进行修改用户
chown -R es:es elasticsearch
前面的es表示将elasticsearch权限给es用户，后面的es表示将用户es移动到group es
为什么是这个文件，因为我们把tar.gz解压到了/usr/local，过程如下
tar -zxvf ..... -C /usr/local即可

下面是安装elasticsearch成功运行后的信息

✅ Elasticsearch security features have been automatically configured!
✅ Authentication is enabled and cluster connections are encrypted.

ℹ️  Password for the elastic user (reset with `bin/elasticsearch-reset-password -u elastic`):
  -cI=lIuJr2XhOQFgjpSd

ℹ️  HTTP CA certificate SHA-256 fingerprint:
  2493cecae8862777b29a8280cc1bf83272382319b3713c2d47080e34a212d3da

ℹ️  Configure Kibana to use this cluster:
• Run Kibana and click the configuration link in the terminal when Kibana starts.
• Copy the following enrollment token and paste it into Kibana in your browser (valid for the next 30 minutes):
  eyJ2ZXIiOiI4LjYuMCIsImFkciI6WyIxOTIuMTY4LjMuNzg6OTIwMCJdLCJmZ3IiOiIyNDkzY2VjYWU4ODYyNzc3YjI5YTgyODBjYzFiZjgzMjcyMzgyMzE5YjM3MTNjMmQ0NzA4MGUzNGEyMTJkM2RhIiwia2V5IjoicGlZaTNvVUJodXd2SHdVVjZyOW06WTY4Z0V4b0tRYWlCSEpKb2NjNFp2USJ9

ℹ️ Configure other nodes to join this cluster:
• Copy the following enrollment token and start new Elasticsearch nodes with `bin/elasticsearch --enrollment-token <token>` (valid for the next 30 minutes):
  eyJ2ZXIiOiI4LjYuMCIsImFkciI6WyIxOTIuMTY4LjMuNzg6OTIwMCJdLCJmZ3IiOiIyNDkzY2VjYWU4ODYyNzc3YjI5YTgyODBjYzFiZjgzMjcyMzgyMzE5YjM3MTNjMmQ0NzA4MGUzNGEyMTJkM2RhIiwia2V5IjoicHlZaTNvVUJodXd2SHdVVjZyOW86eHBBTnc0a09SeWFsSHpXX1plWURZQSJ9

  If you're running in Docker, copy the enrollment token and run:
  `docker run -e "ENROLLMENT_TOKEN=<token>" docker.elastic.co/elasticsearch/elasticsearch:8.6.0`

下面是具体实现可视化的大概部署（整体流程）：

安装 Logstash：
curl -L -O https://artifacts.elastic.co/downloads/logstash/logstash-6.2.0.deb
sudo dpkg -i logstash-6.2.0.deb

sudo touch /etc/logstash/conf.d/logstash.conf
vim /etc/logstash/conf.d/logstash.conf

input {
file {
    path => ["/var/log/suricata/eve.json"]
    codec =>  "json"
    type => "SuricataIDPS"
}

}

filter {
if [type] == "SuricataIDPS" {
    date {
    match => [ "timestamp", "ISO8601" ]
    }
    ruby {
    code => "
        if event.get('[event_type]') == 'fileinfo'
        event.set('[fileinfo][type]', event.get('[fileinfo][magic]').to_s.split(',')[0])
        end
    "
    }

    ruby{
    code => "
        if event.get('[event_type]') == 'alert'
        sp = event.get('[alert][signature]').to_s.split(' group ')
        if (sp.length == 2) and /\A\d+\z/.match(sp[1])
            event.set('[alert][signature]', sp[0])
        end
        end
        "
    }
}

if [src_ip]  {
    geoip {
    source => "src_ip"
    target => "geoip"
    #database => "/opt/logstash/vendor/geoip/GeoLiteCity.dat"
    add_field => [ "[geoip][coordinates]", "%{[geoip][longitude]}" ]
    add_field => [ "[geoip][coordinates]", "%{[geoip][latitude]}"  ]
    }
    mutate {
    convert => [ "[geoip][coordinates]", "float" ]
    }
    if ![geoip.ip] {
    if [dest_ip]  {
        geoip {
        source => "dest_ip"
        target => "geoip"
        #database => "/opt/logstash/vendor/geoip/GeoLiteCity.dat"
        add_field => [ "[geoip][coordinates]", "%{[geoip][longitude]}" ]
        add_field => [ "[geoip][coordinates]", "%{[geoip][latitude]}"  ]
        }
        mutate {
        convert => [ "[geoip][coordinates]", "float" ]
        }
    }
    }
}
}

output {
elasticsearch {
    hosts => ["localhost:9200"]
}
}

sudo chmod 775 /var/log/suricata/eve.json

sudo systemctl start logstash

安装Kibana
kibana-7.13.0-linux-aarch64.tar.gz
curl -L -O https://artifacts.elastic.co/downloads/kibana/kibana-7.2.0-linux-x86_64.tar.gz
tar -xzvf kibana-7.2.0-linux-x86_64.tar.gz
cd kibana-7.2.0-linux-x86_64/
./bin/kibana

访问http://localhost:5601


#挂载kibana服务
#sudo systemctl daemon-reload
#sudo systemctl enable kibana.service
#启动和停止
#sudo systemctl start kibana.service
#sudo systemctl stop kibana.service
总步骤：
systemctl start elasticsearch
sudo systemctl start logstash
cd kibana-7.2.0-linux-x86_64/
./bin/kibana

其中注意elasticsearch的conf配置文件中server.host等的修改，并且ssl安全选项一律选择false，我记得有好几个。
kibana的配置文件，同样注意serverhost和serverport的选取，这个含义是我们将kibana部署在哪个服务器上，也就是可以通过nginx端口转发转发到serverhost;其次注意修改elasticsearch地址，注意和elasticsearch配置文件要相符。

filebeat安装

filebeat的安装，和filebeat enable module suricata(好像是这个，自己查一下)
然后filebeat要在配置文件中填写kibana部分和elasticsearch部分！然后就可以开始了
./filebeat -e setup

这是一个简单的回忆，详见filebeat官网。

回忆包括了大部分要注意的事项。

如果用docker部署，需要注意的步骤

vim /etc/default/elasticsearch
# Elasticsearch Java path
#ES_JAVA_HOME=

# Elasticsearch configuration directory
#ES_PATH_CONF=/etc/elasticsearch

./elasticsearch-keystore create
./elasticsearch-setup-passwords interactive#这是docker启动前的步骤。

启动之后会有如下显示。
✅ Elasticsearch security features have been automatically configured!
✅ Authentication is enabled and cluster connections are encrypted.

ℹ️  Password for the elastic user (reset with `bin/elasticsearch-reset-password -u elastic`):
  GydWhzxn7_DSKcyibo6r

ℹ️  HTTP CA certificate SHA-256 fingerprint:
  7a754432b17957ae2e3752aa8207974689f1595a78b97518d1651cf36f0a3443

ℹ️  Configure Kibana to use this cluster:
• Run Kibana and click the configuration link in the terminal when Kibana starts.
• Copy the following enrollment token and paste it into Kibana in your browser (valid for the next 30 minutes):
  eyJ2ZXIiOiI4LjYuMiIsImFkciI6WyIxNzIuMTcuMC4yOjkyMDAiXSwiZmdyIjoiN2E3NTQ0MzJiMTc5NTdhZTJlMzc1MmFhODIwNzk3NDY4OWYxNTk1YTc4Yjk3NTE4ZDE2NTFjZjM2ZjBhMzQ0MyIsImtleSI6IkJUYVB0SWNCLVNiQ0JzOEN3dGJIOjZUNTlRN1FfU2xhenlvbExSclc3LWcifQ==

ℹ️  Configure other nodes to join this cluster:
• On this node:
  ⁃ Create an enrollment token with `bin/elasticsearch-create-enrollment-token -s node`.
  ⁃ Uncomment the transport.host setting at the end of config/elasticsearch.yml.
  ⁃ Restart Elasticsearch.
• On other nodes:
  ⁃ Start Elasticsearch with `bin/elasticsearch --enrollment-token <token>`, using the enrollment token that you generated.