Add elasticsearch6 and kibana6 images

Elasticsearch and Kibana only support CentOS 8 from version 6.x, but in
Train Kolla uses ELK 5.x.

This change adds new elasticsearch6 and kibana6 images based on ELK 6.x
only for CentOS 7 and 8. This enables two things:

1. provide elasticsearch and kibana for CentOS 8 in Train
2. provide a migration path from CentOS 7 to 8 that does not involve a
   simultaneous major upgrade, by first upgrading CentOS 7 to ELK 6.x.

The new images are based on the upgrade to 6.x in Ussuri in
I8b52b2630a72393bf4b9c7095fc67db6c36c27fa.

This includes a switch to start using oss tagged binaries. These
binaries contain only features that are available under the Apache 2.0
license.

Related: blueprint elasticsearch-kibana-version-upgrade

Co-authored-by: Dincer Celik <hello@dincercelik.com>
Change-Id: I8b52b2630a72393bf4b9c7095fc67db6c36c27fa

docker/elasticsearch6/Dockerfile.j2

@@ -0,0 +1,33 @@
+FROM {{ namespace }}/{{ image_prefix }}base:{{ tag }}
+LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" build-date="{{ build_date }}"
+
+{% block elasticsearch_header %}{% endblock %}
+
+{% import "macros.j2" as macros with context %}
+
+{{ macros.configure_user(name='elasticsearch', shell='/bin/bash', homedir='/usr/share/elasticsearch') }}
+
+{% set elasticsearch_packages = [
+    'java-11-openjdk-headless',
+    'elasticsearch-oss',
+] %}
+
+ENV JAVA_HOME /usr/lib/jvm/jre-11-openjdk/
+ENV ES_SKIP_SET_KERNEL_PARAMETERS true
+
+COPY elasticsearch.repo /etc/yum.repos.d/
+
+{{ macros.install_packages(elasticsearch_packages | customizable("packages")) }}
+COPY extend_start.sh /usr/local/bin/kolla_extend_start
+
+# NOTE: By default the shell of the elasticsearch user is /bin/false. We have to
+# change the shell to /bin/bash to be able to run elasticsearch as a non-root user.
+#
+# https://discuss.elastic.co/t/running-as-non-root-user-service-wrapper-has-changed/7863
+
+RUN chmod 755 /usr/local/bin/kolla_extend_start
+
+{% block elasticsearch_footer %}{% endblock %}
+{% block footer %}{% endblock %}
+
+USER elasticsearch

docker/elasticsearch6/elasticsearch.repo

@@ -0,0 +1,6 @@
+[elasticsearch-kibana-logstash-6.x]
+name=ELK repository for 6.x packages
+baseurl=https://artifacts.elastic.co/packages/oss-6.x/yum
+gpgcheck=1
+gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
+enabled=1

docker/elasticsearch6/extend_start.sh

@@ -0,0 +1,8 @@
+#!/bin/bash
+
+if [[ ! -d "/var/log/kolla/elasticsearch" ]]; then
+    mkdir -p /var/log/kolla/elasticsearch
+fi
+if [[ $(stat -c %a /var/log/kolla/elasticsearch) != "755" ]]; then
+    chmod 755 /var/log/kolla/elasticsearch
+fi

docker/kibana6/Dockerfile.j2

@@ -0,0 +1,22 @@
+FROM {{ namespace }}/{{ image_prefix }}base:{{ tag }}
+LABEL maintainer="{{ maintainer }}" name="{{ image_name }}" build-date="{{ build_date }}"
+
+{% block kibana_header %}{% endblock %}
+
+{% import "macros.j2" as macros with context %}
+
+{{ macros.configure_user(name='kibana') }}
+
+{% set kibana_packages = ['kibana-oss'] %}
+
+COPY elasticsearch.repo /etc/yum.repos.d/
+
+{{ macros.install_packages(kibana_packages | customizable("packages")) }}
+
+COPY extend_start.sh /usr/local/bin/kolla_extend_start
+RUN chmod 755 /usr/local/bin/kolla_extend_start
+
+{% block kibana_footer %}{% endblock %}
+{% block footer %}{% endblock %}
+
+USER kibana

docker/kibana6/elasticsearch.repo

@@ -0,0 +1,6 @@
+[elasticsearch-kibana-logstash-6.x]
+name=ELK repository for 6.x packages
+baseurl=https://artifacts.elastic.co/packages/oss-6.x/yum
+gpgcheck=1
+gpgkey=https://artifacts.elastic.co/GPG-KEY-elasticsearch
+enabled=1

docker/kibana6/extend_start.sh

@@ -0,0 +1,10 @@
+#!/bin/bash
+
+KIBANA_LOG_DIR="/var/log/kolla/kibana"
+
+if [[ ! -d "${KIBANA_LOG_DIR}" ]]; then
+    mkdir -p "${KIBANA_LOG_DIR}"
+fi
+if [[ $(stat -c %U:%G "${KIBANA_LOG_DIR}") != "kibana:kolla" ]]; then
+    chown kibana:kolla "${KIBANA_LOG_DIR}"
+fi

kolla/image/build.py

@@ -172,6 +172,8 @@ UNBUILDABLE_IMAGES = {
         "bifrost-base",  # tries to install 'mysql-server' which is not in
                          # Debian 'buster'
         "cyborg-base",
+        "elasticsearch6",   # Only required for CentOS 8 migration.
+        "kibana6",          # Only required for CentOS 8 migration.
         "monasca-grafana",  # FIXME(hrw): some ssl issues to fix
         "mongodb",
         "opendaylight",  # no binary package
@@ -182,6 +184,8 @@ UNBUILDABLE_IMAGES = {
     },
     'ubuntu': {
         "cyborg-base",
+        "elasticsearch6",   # Only required for CentOS 8 migration.
+        "kibana6",          # Only required for CentOS 8 migration.
         "qdrouterd",  # There is no qdrouterd package for ubuntu bionic
     },
 

releasenotes/notes/elk-6-573a9f2b4af4444a.yaml

@@ -0,0 +1,6 @@
+---
+features:
+  - |
+    Adds new ``elasticsearch6`` and ``kibana6`` images for CentOS 7 and 8 only.
+    These images are used to provide compatibility between versions in CentOS 7
+    and 8 images. These images will only be available in the Train release.