Add build process and osh-infra images

This patch adds image building process for OSH.

Each root folder contains a 'kind' of images to build. Each folder will
have its own parent job, inheriting from a 'openstack-helm-images-base'
job. The folder 'parent job' is used for building distribution
variants (centos_7, ubuntu_xenial, opensuse_15).

Folders containing Dockerfiles (a different Dockerfiles per
distribution) will use the docker building play (docker-build.yml).

Dockerfiles come from openstack-helm-infra repository,
with the following changes:
* The deprecated "MAINTAINER" directive has been replaced with
  LABEL maintainer=<maintaineremail>
* If Dockerfiles' default arguments were mismatching the
  documentation or Makefiles, they were adapted (see libvirt).
* Relative paths to the openstack-helm-infra repo have been removed.

Each folder is given a convenience build script (build.sh) which
can be used to build a specific image manually.

This script's content gets included in the documentation to avoid
duplications of content.

This patch focuses on each image separately.

Another patch will be added to deal with a repo-wide image building
process, whether for CI purposes, or for developer's purposes,
with the help of a Makefile.

Note: Kubeadm-aio image was not imported in this process due to
circular dependencies: Image building currently requires the
helm charts of openstack-helm-infra.

Change-Id: I2d01e3f2c34d6dd7db4a1c3897dc4b994bf02623
This commit is contained in:
Jean-Philippe Evrard 2018-11-05 15:02:13 +01:00
parent 2803e7c16a
commit 8468a18ae3
17 changed files with 478 additions and 0 deletions

View File

@ -5,8 +5,65 @@ Welcome to OpenStack-Helm-Images's documentation!
This repository is in charge of the image building for
openstack-helm repositories.
Please check the documentation of each section for the
relevant build instructions.
By default, these images are built on a Ubuntu 18.04 LTS
node.
Setup a build node
==================
Here are the instructions to setup a build node with
Ubuntu 18.04 LTS:
::
apt update
apt install -y docker.io git
Modifying the build with environment
====================================
Unless explicitly written, all the `build.sh`
convenience scripts allow to pass arguments to the
docker build process: The `build.sh` scripts have a
environment variable (`extra_build_args`), which can
be used to pass arbitrary data.
Next to the extra arguments, you can modify the
`build.sh` behavior by setting the following
environment variables:
::
VERSION
DISTRO
REGISTRY_URI=${REGISTRY_URI:-"openstackhelm/"}
`VERSION` is the expected tag version of the image,
and defaults to `latest`
`DISTRO` is used if you want to build an image with
a different Dockerfile, for example with another
distribution. `Dockerfile.${DISTRO}` must match
an existing filename.
`REGISTRY_URI` is part of the image name, representing
the location of the image, used in the image tagging
process. For example `REGISTRY_URI` could be
`docker.io/openstackhelm/`. In that case, the full
name and tag of the `vbmc` image would be:
::
docker.io/openstackhelm/vbmc:latest
Please check each section of the documentation for
an overview of the build process for each container.
.. toctree::
:maxdepth: 2
:caption: Contents:
libvirt
mariadb
vbmc
loci

20
doc/source/libvirt.rst Normal file
View File

@ -0,0 +1,20 @@
=======================
libvirt container image
=======================
This container builds a small image with Libvirt for use with OpenStack-Helm.
Manual build for Ubuntu Xenial
==============================
Here are the instructions for building Xenial image:
.. literalinclude:: ../../libvirt/build.sh
:lines: 7-13
:language: shell
Alternatively, this step can be performed by running the script directly:
.. code-block:: shell
./libvirt/build.sh

21
doc/source/mariadb.rst Normal file
View File

@ -0,0 +1,21 @@
=======================
MariaDB container image
=======================
This image is based on upstream MariaDB image, with extra Kubernetes
libraries to work with OpenStack-Helm
Manual build for Ubuntu Xenial
==============================
Here are the instructions for building Xenial image:
.. literalinclude:: ../../mariadb/build.sh
:lines: 7-12
:language: shell
Alternatively, this step can be performed by running the script directly:
.. code-block:: shell
./mariadb/build.sh

21
doc/source/vbmc.rst Normal file
View File

@ -0,0 +1,21 @@
====================
vBMC container image
====================
This container builds a small image with kubectl and some other
utilities for use in both the ironic checks and development.
Manual build for CentOS 7
=========================
Here are the instructions for building CentOS 7 vBMC image:
.. literalinclude:: ../../vbmc/build.sh
:lines: 7-12
:language: shell
Alternatively, this step can be performed by running the script directly:
.. code-block:: shell
./vbmc/build.sh

View File

@ -0,0 +1,41 @@
FROM docker.io/ubuntu:xenial
LABEL maintainer="pete.birley@att.com"
ARG LIBVIRT_VERSION="1.3.1-1ubuntu10.24"
ARG CEPH_RELEASE=luminous
ARG PROJECT=nova
ARG UID=42424
ARG GID=42424
ADD https://download.ceph.com/keys/release.asc /etc/apt/ceph-release.asc
RUN set -ex ;\
export DEBIAN_FRONTEND=noninteractive ;\
apt-key add /etc/apt/ceph-release.asc ;\
rm -f /etc/apt/ceph-release.asc ;\
echo "deb http://download.ceph.com/debian-${CEPH_RELEASE}/ xenial main" | tee /etc/apt/sources.list.d/ceph.list ;\
apt-get update ;\
apt-get upgrade -y ;\
apt-get install --no-install-recommends -y \
ceph-common \
cgroup-tools \
dmidecode \
ebtables \
iproute2 \
libvirt-bin=${LIBVIRT_VERSION} \
pm-utils \
qemu \
qemu-block-extra \
qemu-efi \
openvswitch-switch ;\
groupadd -g ${GID} ${PROJECT} ;\
useradd -u ${UID} -g ${PROJECT} -M -d /var/lib/${PROJECT} -s /usr/sbin/nologin -c "${PROJECT} user" ${PROJECT} ;\
mkdir -p /etc/${PROJECT} /var/log/${PROJECT} /var/lib/${PROJECT} /var/cache/${PROJECT} ;\
chown ${PROJECT}:${PROJECT} /etc/${PROJECT} /var/log/${PROJECT} /var/lib/${PROJECT} /var/cache/${PROJECT} ;\
usermod -a -G kvm ${PROJECT} ;\
apt-get clean -y ;\
rm -rf \
/var/cache/debconf/* \
/var/lib/apt/lists/* \
/var/log/* \
/tmp/* \
/var/tmp/*

15
libvirt/build.sh Executable file
View File

@ -0,0 +1,15 @@
#!/bin/bash
SCRIPT=`realpath $0`
SCRIPT_DIR=`dirname ${SCRIPT}`
## Only build from main folder
cd ${SCRIPT_DIR}/..
IMAGE="libvirt"
LIBVIRT_VERSION=${LIBVIRT_VERSION:-"1.3.1-1ubuntu10.24"}
VERSION=${VERSION:-latest}
DISTRO=${DISTRO:-ubuntu_xenial}
REGISTRY_URI=${REGISTRY_URI:-"openstackhelm/"}
EXTRA_TAG_INFO=${EXTRA_TAB_INFO:-"-${LIBVIRT_VERSION}"}
docker build -f ${IMAGE}/Dockerfile.${DISTRO} --network=host -t ${REGISTRY_URI}${IMAGE}:${VERSION}-${DISTRO}${EXTRA_TAG_INFO} --build-arg LIBVIRT_VERSION="${LIBVIRT_VERSION}" ${extra_build_args} ${IMAGE}
cd -

View File

@ -0,0 +1,22 @@
FROM docker.io/mariadb@sha256:d4cf9fbdf33a2940ca35c653bf2b702cbaed0bff87ade8c3e3ee9eab81b38b27
#FROM docker.io/mariadb:10.2.18
RUN set -ex ;\
apt-get update ;\
apt-get upgrade -y ;\
apt-get install -y --no-install-recommends \
python-pip ;\
pip --no-cache-dir install --upgrade pip ;\
hash -r ;\
pip --no-cache-dir install --upgrade setuptools ;\
pip --no-cache-dir install --upgrade \
configparser \
iso8601 \
kubernetes ;\
apt-get clean -y ;\
rm -rf \
/var/cache/debconf/* \
/var/lib/apt/lists/* \
/var/log/* \
/tmp/* \
/var/tmp/*

14
mariadb/build.sh Executable file
View File

@ -0,0 +1,14 @@
#!/bin/bash
SCRIPT=`realpath $0`
SCRIPT_DIR=`dirname ${SCRIPT}`
## Only build from main folder
cd ${SCRIPT_DIR}/..
IMAGE="mariadb"
VERSION=${VERSION:-latest}
DISTRO=${DISTRO:-ubuntu_xenial}
REGISTRY_URI=${REGISTRY_URI:-"openstackhelm/"}
EXTRA_TAG_INFO=${EXTRA_TAG_INFO:-""}
docker build -f ${IMAGE}/Dockerfile.${DISTRO} --network=host -t ${REGISTRY_URI}${IMAGE}:${VERSION}-${DISTRO}${EXTRA_TAG_INFO} ${extra_build_args} ${IMAGE}
cd -

43
vbmc/Dockerfile.centos_7 Normal file
View File

@ -0,0 +1,43 @@
FROM centos:7
LABEL maintainer="pete.birley@att.com"
ARG PROJECT=nova
ARG UID=42424
ARG GID=42424
RUN set -ex ;\
yum -y upgrade ;\
yum -y install \
epel-release \
centos-release-openstack-newton \
centos-release-qemu-ev ;\
yum -y install \
ceph-common \
git \
libcgroup-tools \
libguestfs \
libvirt \
libvirt-daemon \
libvirt-daemon-config-nwfilter \
libvirt-daemon-driver-lxc \
libvirt-daemon-driver-nwfilter \
libvirt-devel \
openvswitch \
python-devel \
qemu-kvm ;\
yum -y group install \
"Development Tools" ;\
yum clean all ;\
rm -rf /var/cache/yum ;\
curl https://bootstrap.pypa.io/get-pip.py -o /tmp/get-pip.py ;\
python /tmp/get-pip.py ;\
rm -f /tmp/get-pip.py ;\
TMP_DIR=$(mktemp -d) ;\
git clone https://github.com/openstack/virtualbmc ${TMP_DIR} ;\
pip install -U ${TMP_DIR} ;\
rm -rf ${TMP_DIR} ;\
groupadd -g ${GID} ${PROJECT} ;\
useradd -u ${UID} -g ${PROJECT} -M -d /var/lib/${PROJECT} -s /usr/sbin/nologin -c "${PROJECT} user" ${PROJECT} ;\
mkdir -p /etc/${PROJECT} /var/log/${PROJECT} /var/lib/${PROJECT} /var/cache/${PROJECT} ;\
chown ${PROJECT}:${PROJECT} /etc/${PROJECT} /var/log/${PROJECT} /var/lib/${PROJECT} /var/cache/${PROJECT} ;\
usermod -a -G qemu ${PROJECT}

14
vbmc/build.sh Executable file
View File

@ -0,0 +1,14 @@
#!/bin/bash
SCRIPT=`realpath $0`
SCRIPT_DIR=`dirname ${SCRIPT}`
## Only build from main folder
cd ${SCRIPT_DIR}/..
IMAGE="vbmc"
VERSION=${VERSION:-latest}
DISTRO=${DISTRO:-"centos_7"}
REGISTRY_URI=${REGISTRY_URI:-"openstackhelm/"}
EXTRA_TAG_INFO=${EXTRA_TAG_INFO:-""}
docker build -f ${IMAGE}/Dockerfile.${DISTRO} --network=host -t ${REGISTRY_URI}${IMAGE}:${VERSION}-${DISTRO}${EXTRA_TAG_INFO} ${extra_build_args} ${IMAGE}
cd -

20
zuul.d/base-jobs.yaml Normal file
View File

@ -0,0 +1,20 @@
- job:
name: openstack-helm-images-base
parent: base
abstract: true
description: |
This job is building a docker image for
OpenStack-Helm usage.
Image specific tests can be added by
running a post script on the relevant
jobs.
irrelevant-files:
- ^\.git.*
- ^.*\.(example|md|rst)$
- ^doc/.*
- ^releasenotes/.*
timeout: 3600
pre-run:
- zuul.d/playbooks/pre-run.yml
run: zuul.d/playbooks/docker-build.yml
nodeset: ubuntu-bionic

44
zuul.d/libvirt.yaml Normal file
View File

@ -0,0 +1,44 @@
---
# Copyright 2018, SUSE LINUX GmbH.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
- project:
check:
jobs:
- openstack-helm-images-libvirt-ubuntu_xenial
gate:
jobs:
- openstack-helm-images-libvirt-ubuntu_xenial
#experimental:
# jobs:
# - openstack-helm-infra-five-ubuntu
- job:
name: openstack-helm-images-libvirt
parent: openstack-helm-images-base
abstract: true
files:
- ^libvirt/.*
vars:
image_path: libvirt
- job:
name: openstack-helm-images-libvirt-ubuntu_xenial
parent: openstack-helm-images-libvirt
files:
- ^libvirt/build.sh
- ^libvirt/Dockerfile.ubuntu_xenial$
- ^zuul.d/libvirt.yaml
vars:
distro: "ubuntu_xenial"

44
zuul.d/mariadb.yaml Normal file
View File

@ -0,0 +1,44 @@
---
# Copyright 2018, SUSE LINUX GmbH.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
- project:
check:
jobs:
- openstack-helm-images-mariadb-ubuntu_xenial
gate:
jobs:
- openstack-helm-images-mariadb-ubuntu_xenial
#experimental:
# jobs:
# - openstack-helm-infra-five-ubuntu
- job:
name: openstack-helm-images-mariadb
parent: openstack-helm-images-base
abstract: true
files:
- ^mariadb/.*
vars:
image_path: mariadb
- job:
name: openstack-helm-images-mariadb-ubuntu_xenial
parent: openstack-helm-images-mariadb
files:
- ^mariadb/build.sh
- ^mariadb/Dockerfile.ubuntu_xenial$
- ^zuul.d/mariadb.yaml
vars:
distro: "ubuntu_xenial"

View File

@ -0,0 +1,16 @@
---
# This play will only get consumed in osh-images.
# For depends-on to osh-images and osh changes triggering a rebuild of
# osh-infra or osh, this play won't be used, as we won't know what to build, and
# therefore will build everything for a 'standard' distribution.
# No need to change zuul.project.src_dir to static paths then.
- hosts: all[0]
tasks:
- name: Build image
docker_image:
buildargs: "{{ buildargs | default(omit) }}"
state: present
path: "{{ zuul.project.src_dir }}/{{ image_path }}"
dockerfile: "Dockerfile.{{ distro }}"
name: "openstackhelm/{{ image_path }}"
tag: "{{ version | default('latest') }}-{{ distro }}"

View File

@ -0,0 +1,31 @@
---
- hosts: all[0]
pre_tasks:
- name: Create docker folder
file:
path: /var/lib/docker
state: directory
become: True
- name: Mount tmpfs to build faster
mount:
path: /var/lib/docker/
src: tmpfs
fstype: tmpfs
opts: size=25g
state: mounted
become: True
roles:
- role: install-docker
post_tasks:
- name: Install extra docker libraries for ansible
become: True
package:
name:
- python3-docker
- python-docker
state: present
- name: Ensure docker service is started
become: true
service:
name: docker.service
state: started

55
zuul.d/vbmc.yaml Normal file
View File

@ -0,0 +1,55 @@
---
# Copyright 2018, SUSE LINUX GmbH.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
- project:
check:
jobs:
# We should add a job here that tests the vbmc image
# Alternatively, we can add a deploy using vbmc, and reuse
# the deploy's functional testing
- openstack-helm-images-vbmc-centos_7
gate:
jobs:
- openstack-helm-images-vbmc-centos_7
- job:
name: openstack-helm-images-vbmc
parent: openstack-helm-images-base
abstract: true
files:
- ^vbmc/.*
vars:
image_path: vbmc
- job:
name: openstack-helm-images-vbmc-centos_7
parent: openstack-helm-images-vbmc
files:
- ^vbmc/build.sh
- ^vbmc/Dockerfile.centos_7$
- ^zuul.d/vbmc.yaml
vars:
distro: "centos_7"
# This is an example of how to re-use the jobs for
# multi-distro. The image would get built with a
# different tag, based on distro argument.
#- job:
# name: openstack-helm-images-vbmc-ubuntu_bionic
# parent: openstack-helm-images-vbmc
# files:
# - ^vbmc/Dockerfile.ubuntu_bionic$
# vars:
# distro: "ubuntu_bionic"