filebeat安装命令（for dockerfile rpm）

RUN set -eux; \
    \
    rpm --import https://packages.elastic.co/GPG-KEY-elasticsearch; \
    { \
        echo '#!/usr/bin/env bash'; \
        echo '[elastic-7.x]'; \
        echo 'name=Elastic repository for 7.x packages'; \
        echo 'baseurl=https://artifacts.elastic.co/packages/7.x/yum'; \
        echo 'gpgcheck=1'; \
        echo 'gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch'; \
        echo 'enabled=1'; \
        echo 'autorefresh=1'; \
        echo 'type=rpm-md'; \
    } > /etc/yum.repos.d/elastic7.repo; \
    yum install -y filebeat; \
    chkconfig --add filebeat; \

也可以下载rpm离线安装，注意缓存问题：详见《我的dockerfile构建笔记》

报错：or switch to the OSS distribution of filebeat，解决方法为安装Apache版（OSS版）filebeat

下载地址：https://www.elastic.co/cn/downloads/past-releases/filebeat-oss-7-4-2

安装

curl -L -O url
sudo rpm -vi filebeat-6.8.5-x86_64.rpm

启动：

/usr/share/filebeat/bin/filebeat -e -c /etc/filebeat/filebeat.yml

输出到Logstash
output.logstash:
  hosts: ["localhost:5044"]

输出到es：
output.elasticsearch:
  hosts: ["https://localhost:9200"]
  username: "filebeat_internal"
  password: "YOUR_PASSWORD"

输出到File
output.file:
  paths:
    - "/var/log/app.log"
    - /var/log/wifi.log
  filename: filebeat
 
输出到控制台
output.console:
  pretty: true

filebeat.inputs:
- type: log
  paths:
    - "/var/log/app.log"
    - /var/log/wifi.log

#支持正则 排除匹配的文件

  exclude_files: [access.log]

合并多行为一行（例如Java的异常堆栈信息）

  multiline:
    pattern: '^[201|202]'
    negate: true
    match: after

在source中增加自定义字段

  fields:
    app_id: '{$HOSTNAME}'

另一个合并方式：

  multiline:
    pattern: '^\t'
    negate: false
    match: after

自定义filebeat推送到es的索引（index）名称

参见官方文档：

https://www.elastic.co/guide/en/beats/filebeat/7.4/elasticsearch-output.html

The default is "filebeat-%{[agent.version]}-%{+yyyy.MM.dd}" (for example, "filebeat-7.4.2-2019-11-27"). If you change this setting, you also need to configure the setup.template.name and setup.template.pattern options.

示例

output.elasticsearch:
  hosts: ["http://localhost:9200"]
  index: "java-%{[agent.hostname]}-%{[agent.version]}-%{+yyyy.MM.dd}"
  
setup.template.name: "filebeat"
setup.template.pattern: "java-*"

Filebeat优化实践：

https://my.oschina.net/u/2612999/blog/1518876

以sidecar方式部署filebeat，参见：

https://www.cnblogs.com/WisWang/p/9092185.html

https://blog.51cto.com/ylw6006/2107307