openstack
/
kolla
Code Issues Proposed changes

Fluentd: add fluent-plugin-concat:1.0.0 gem 

Actual Fluentd config doesn't manage multiline log pattern like:

2017-08-16 21:41:03.097 7 WARNING oslo_config.cfg [req-34ea3f23-f780-4059-b911-2788ea4af7b2 - - - - -] Option "firewall_driver" from group "DEFAULT" is deprecated for removal (
nova-network is deprecated, as are any related configuration options.
). Its value may be silently ignored in the future.

This log will generate 1 Fluentd records for each line (3 records),
this is really a wrong behaviour, and made Kibana interface unusable.

This patch add fluent-plugin-concat in order to concatenate multiline
log separated in multiple events.

Change-Id: I83e583605b49b7c421109d662de1291f38bd5c07
Closes-Bug: #1711399
This commit is contained in:
Bertrand Lallau 2017-08-17 18:20:04 +02:00
parent 8110b55d60
commit 83baad304d
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
docker/fluentd/Dockerfile.j2
View File

@ -30,7 +30,7 @@ LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" build-date="{{ build
# https://github.com/fluent/fluent-plugin-grok-parser
RUN ulimit -n 65536 \
&& sed -i -e "s/USER=td-agent/USER=root/" -e "s/GROUP=td-agent/GROUP=root/" /etc/init.d/td-agent \
&& td-agent-gem install fluent-plugin-parser fluent-plugin-kubernetes_metadata_filter fluent-plugin-elasticsearch fluent-plugin-grep fluent-plugin-grok-parser:0.3.1 fluent-plugin-rewrite-tag-filter \
&& td-agent-gem install fluent-plugin-concat:1.0.0 fluent-plugin-parser fluent-plugin-kubernetes_metadata_filter fluent-plugin-elasticsearch fluent-plugin-grep fluent-plugin-grok-parser:0.3.1 fluent-plugin-rewrite-tag-filter \
&& rm -f /etc/td-agent/td-agent.conf
COPY fluentd_sudoers /etc/sudoers.d/kolla_fluentd_sudoers