Add build process and osh-infra images
This patch adds image building process for OSH. Each root folder contains a 'kind' of images to build. Each folder will have its own parent job, inheriting from a 'openstack-helm-images-base' job. The folder 'parent job' is used for building distribution variants (centos_7, ubuntu_xenial, opensuse_15). Folders containing Dockerfiles (a different Dockerfiles per distribution) will use the docker building play (docker-build.yml). Dockerfiles come from openstack-helm-infra repository, with the following changes: * The deprecated "MAINTAINER" directive has been replaced with LABEL maintainer=<maintaineremail> * If Dockerfiles' default arguments were mismatching the documentation or Makefiles, they were adapted (see libvirt). * Relative paths to the openstack-helm-infra repo have been removed. Each folder is given a convenience build script (build.sh) which can be used to build a specific image manually. This script's content gets included in the documentation to avoid duplications of content. This patch focuses on each image separately. Another patch will be added to deal with a repo-wide image building process, whether for CI purposes, or for developer's purposes, with the help of a Makefile. Note: Kubeadm-aio image was not imported in this process due to circular dependencies: Image building currently requires the helm charts of openstack-helm-infra. Change-Id: I2d01e3f2c34d6dd7db4a1c3897dc4b994bf02623
This commit is contained in:
parent
2803e7c16a
commit
8468a18ae3
|
@ -5,8 +5,65 @@ Welcome to OpenStack-Helm-Images's documentation!
|
|||
This repository is in charge of the image building for
|
||||
openstack-helm repositories.
|
||||
|
||||
Please check the documentation of each section for the
|
||||
relevant build instructions.
|
||||
|
||||
By default, these images are built on a Ubuntu 18.04 LTS
|
||||
node.
|
||||
|
||||
Setup a build node
|
||||
==================
|
||||
|
||||
Here are the instructions to setup a build node with
|
||||
Ubuntu 18.04 LTS:
|
||||
::
|
||||
|
||||
apt update
|
||||
apt install -y docker.io git
|
||||
|
||||
Modifying the build with environment
|
||||
====================================
|
||||
|
||||
Unless explicitly written, all the `build.sh`
|
||||
convenience scripts allow to pass arguments to the
|
||||
docker build process: The `build.sh` scripts have a
|
||||
environment variable (`extra_build_args`), which can
|
||||
be used to pass arbitrary data.
|
||||
|
||||
Next to the extra arguments, you can modify the
|
||||
`build.sh` behavior by setting the following
|
||||
environment variables:
|
||||
::
|
||||
|
||||
VERSION
|
||||
DISTRO
|
||||
REGISTRY_URI=${REGISTRY_URI:-"openstackhelm/"}
|
||||
|
||||
`VERSION` is the expected tag version of the image,
|
||||
and defaults to `latest`
|
||||
|
||||
`DISTRO` is used if you want to build an image with
|
||||
a different Dockerfile, for example with another
|
||||
distribution. `Dockerfile.${DISTRO}` must match
|
||||
an existing filename.
|
||||
|
||||
`REGISTRY_URI` is part of the image name, representing
|
||||
the location of the image, used in the image tagging
|
||||
process. For example `REGISTRY_URI` could be
|
||||
`docker.io/openstackhelm/`. In that case, the full
|
||||
name and tag of the `vbmc` image would be:
|
||||
::
|
||||
|
||||
docker.io/openstackhelm/vbmc:latest
|
||||
|
||||
Please check each section of the documentation for
|
||||
an overview of the build process for each container.
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 2
|
||||
:caption: Contents:
|
||||
|
||||
libvirt
|
||||
mariadb
|
||||
vbmc
|
||||
loci
|
||||
|
|
|
@ -0,0 +1,20 @@
|
|||
=======================
|
||||
libvirt container image
|
||||
=======================
|
||||
|
||||
This container builds a small image with Libvirt for use with OpenStack-Helm.
|
||||
|
||||
Manual build for Ubuntu Xenial
|
||||
==============================
|
||||
|
||||
Here are the instructions for building Xenial image:
|
||||
|
||||
.. literalinclude:: ../../libvirt/build.sh
|
||||
:lines: 7-13
|
||||
:language: shell
|
||||
|
||||
Alternatively, this step can be performed by running the script directly:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
./libvirt/build.sh
|
|
@ -0,0 +1,21 @@
|
|||
=======================
|
||||
MariaDB container image
|
||||
=======================
|
||||
|
||||
This image is based on upstream MariaDB image, with extra Kubernetes
|
||||
libraries to work with OpenStack-Helm
|
||||
|
||||
Manual build for Ubuntu Xenial
|
||||
==============================
|
||||
|
||||
Here are the instructions for building Xenial image:
|
||||
|
||||
.. literalinclude:: ../../mariadb/build.sh
|
||||
:lines: 7-12
|
||||
:language: shell
|
||||
|
||||
Alternatively, this step can be performed by running the script directly:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
./mariadb/build.sh
|
|
@ -0,0 +1,21 @@
|
|||
====================
|
||||
vBMC container image
|
||||
====================
|
||||
|
||||
This container builds a small image with kubectl and some other
|
||||
utilities for use in both the ironic checks and development.
|
||||
|
||||
Manual build for CentOS 7
|
||||
=========================
|
||||
|
||||
Here are the instructions for building CentOS 7 vBMC image:
|
||||
|
||||
.. literalinclude:: ../../vbmc/build.sh
|
||||
:lines: 7-12
|
||||
:language: shell
|
||||
|
||||
Alternatively, this step can be performed by running the script directly:
|
||||
|
||||
.. code-block:: shell
|
||||
|
||||
./vbmc/build.sh
|
|
@ -0,0 +1,41 @@
|
|||
FROM docker.io/ubuntu:xenial
|
||||
LABEL maintainer="pete.birley@att.com"
|
||||
|
||||
ARG LIBVIRT_VERSION="1.3.1-1ubuntu10.24"
|
||||
ARG CEPH_RELEASE=luminous
|
||||
ARG PROJECT=nova
|
||||
ARG UID=42424
|
||||
ARG GID=42424
|
||||
|
||||
ADD https://download.ceph.com/keys/release.asc /etc/apt/ceph-release.asc
|
||||
RUN set -ex ;\
|
||||
export DEBIAN_FRONTEND=noninteractive ;\
|
||||
apt-key add /etc/apt/ceph-release.asc ;\
|
||||
rm -f /etc/apt/ceph-release.asc ;\
|
||||
echo "deb http://download.ceph.com/debian-${CEPH_RELEASE}/ xenial main" | tee /etc/apt/sources.list.d/ceph.list ;\
|
||||
apt-get update ;\
|
||||
apt-get upgrade -y ;\
|
||||
apt-get install --no-install-recommends -y \
|
||||
ceph-common \
|
||||
cgroup-tools \
|
||||
dmidecode \
|
||||
ebtables \
|
||||
iproute2 \
|
||||
libvirt-bin=${LIBVIRT_VERSION} \
|
||||
pm-utils \
|
||||
qemu \
|
||||
qemu-block-extra \
|
||||
qemu-efi \
|
||||
openvswitch-switch ;\
|
||||
groupadd -g ${GID} ${PROJECT} ;\
|
||||
useradd -u ${UID} -g ${PROJECT} -M -d /var/lib/${PROJECT} -s /usr/sbin/nologin -c "${PROJECT} user" ${PROJECT} ;\
|
||||
mkdir -p /etc/${PROJECT} /var/log/${PROJECT} /var/lib/${PROJECT} /var/cache/${PROJECT} ;\
|
||||
chown ${PROJECT}:${PROJECT} /etc/${PROJECT} /var/log/${PROJECT} /var/lib/${PROJECT} /var/cache/${PROJECT} ;\
|
||||
usermod -a -G kvm ${PROJECT} ;\
|
||||
apt-get clean -y ;\
|
||||
rm -rf \
|
||||
/var/cache/debconf/* \
|
||||
/var/lib/apt/lists/* \
|
||||
/var/log/* \
|
||||
/tmp/* \
|
||||
/var/tmp/*
|
|
@ -0,0 +1,15 @@
|
|||
#!/bin/bash
|
||||
SCRIPT=`realpath $0`
|
||||
SCRIPT_DIR=`dirname ${SCRIPT}`
|
||||
## Only build from main folder
|
||||
cd ${SCRIPT_DIR}/..
|
||||
|
||||
IMAGE="libvirt"
|
||||
LIBVIRT_VERSION=${LIBVIRT_VERSION:-"1.3.1-1ubuntu10.24"}
|
||||
VERSION=${VERSION:-latest}
|
||||
DISTRO=${DISTRO:-ubuntu_xenial}
|
||||
REGISTRY_URI=${REGISTRY_URI:-"openstackhelm/"}
|
||||
EXTRA_TAG_INFO=${EXTRA_TAB_INFO:-"-${LIBVIRT_VERSION}"}
|
||||
docker build -f ${IMAGE}/Dockerfile.${DISTRO} --network=host -t ${REGISTRY_URI}${IMAGE}:${VERSION}-${DISTRO}${EXTRA_TAG_INFO} --build-arg LIBVIRT_VERSION="${LIBVIRT_VERSION}" ${extra_build_args} ${IMAGE}
|
||||
|
||||
cd -
|
|
@ -0,0 +1,22 @@
|
|||
FROM docker.io/mariadb@sha256:d4cf9fbdf33a2940ca35c653bf2b702cbaed0bff87ade8c3e3ee9eab81b38b27
|
||||
#FROM docker.io/mariadb:10.2.18
|
||||
|
||||
RUN set -ex ;\
|
||||
apt-get update ;\
|
||||
apt-get upgrade -y ;\
|
||||
apt-get install -y --no-install-recommends \
|
||||
python-pip ;\
|
||||
pip --no-cache-dir install --upgrade pip ;\
|
||||
hash -r ;\
|
||||
pip --no-cache-dir install --upgrade setuptools ;\
|
||||
pip --no-cache-dir install --upgrade \
|
||||
configparser \
|
||||
iso8601 \
|
||||
kubernetes ;\
|
||||
apt-get clean -y ;\
|
||||
rm -rf \
|
||||
/var/cache/debconf/* \
|
||||
/var/lib/apt/lists/* \
|
||||
/var/log/* \
|
||||
/tmp/* \
|
||||
/var/tmp/*
|
|
@ -0,0 +1,14 @@
|
|||
#!/bin/bash
|
||||
SCRIPT=`realpath $0`
|
||||
SCRIPT_DIR=`dirname ${SCRIPT}`
|
||||
## Only build from main folder
|
||||
cd ${SCRIPT_DIR}/..
|
||||
|
||||
IMAGE="mariadb"
|
||||
VERSION=${VERSION:-latest}
|
||||
DISTRO=${DISTRO:-ubuntu_xenial}
|
||||
REGISTRY_URI=${REGISTRY_URI:-"openstackhelm/"}
|
||||
EXTRA_TAG_INFO=${EXTRA_TAG_INFO:-""}
|
||||
docker build -f ${IMAGE}/Dockerfile.${DISTRO} --network=host -t ${REGISTRY_URI}${IMAGE}:${VERSION}-${DISTRO}${EXTRA_TAG_INFO} ${extra_build_args} ${IMAGE}
|
||||
|
||||
cd -
|
|
@ -0,0 +1,43 @@
|
|||
FROM centos:7
|
||||
LABEL maintainer="pete.birley@att.com"
|
||||
|
||||
ARG PROJECT=nova
|
||||
ARG UID=42424
|
||||
ARG GID=42424
|
||||
|
||||
RUN set -ex ;\
|
||||
yum -y upgrade ;\
|
||||
yum -y install \
|
||||
epel-release \
|
||||
centos-release-openstack-newton \
|
||||
centos-release-qemu-ev ;\
|
||||
yum -y install \
|
||||
ceph-common \
|
||||
git \
|
||||
libcgroup-tools \
|
||||
libguestfs \
|
||||
libvirt \
|
||||
libvirt-daemon \
|
||||
libvirt-daemon-config-nwfilter \
|
||||
libvirt-daemon-driver-lxc \
|
||||
libvirt-daemon-driver-nwfilter \
|
||||
libvirt-devel \
|
||||
openvswitch \
|
||||
python-devel \
|
||||
qemu-kvm ;\
|
||||
yum -y group install \
|
||||
"Development Tools" ;\
|
||||
yum clean all ;\
|
||||
rm -rf /var/cache/yum ;\
|
||||
curl https://bootstrap.pypa.io/get-pip.py -o /tmp/get-pip.py ;\
|
||||
python /tmp/get-pip.py ;\
|
||||
rm -f /tmp/get-pip.py ;\
|
||||
TMP_DIR=$(mktemp -d) ;\
|
||||
git clone https://github.com/openstack/virtualbmc ${TMP_DIR} ;\
|
||||
pip install -U ${TMP_DIR} ;\
|
||||
rm -rf ${TMP_DIR} ;\
|
||||
groupadd -g ${GID} ${PROJECT} ;\
|
||||
useradd -u ${UID} -g ${PROJECT} -M -d /var/lib/${PROJECT} -s /usr/sbin/nologin -c "${PROJECT} user" ${PROJECT} ;\
|
||||
mkdir -p /etc/${PROJECT} /var/log/${PROJECT} /var/lib/${PROJECT} /var/cache/${PROJECT} ;\
|
||||
chown ${PROJECT}:${PROJECT} /etc/${PROJECT} /var/log/${PROJECT} /var/lib/${PROJECT} /var/cache/${PROJECT} ;\
|
||||
usermod -a -G qemu ${PROJECT}
|
|
@ -0,0 +1,14 @@
|
|||
#!/bin/bash
|
||||
SCRIPT=`realpath $0`
|
||||
SCRIPT_DIR=`dirname ${SCRIPT}`
|
||||
## Only build from main folder
|
||||
cd ${SCRIPT_DIR}/..
|
||||
|
||||
IMAGE="vbmc"
|
||||
VERSION=${VERSION:-latest}
|
||||
DISTRO=${DISTRO:-"centos_7"}
|
||||
REGISTRY_URI=${REGISTRY_URI:-"openstackhelm/"}
|
||||
EXTRA_TAG_INFO=${EXTRA_TAG_INFO:-""}
|
||||
docker build -f ${IMAGE}/Dockerfile.${DISTRO} --network=host -t ${REGISTRY_URI}${IMAGE}:${VERSION}-${DISTRO}${EXTRA_TAG_INFO} ${extra_build_args} ${IMAGE}
|
||||
|
||||
cd -
|
|
@ -0,0 +1,20 @@
|
|||
- job:
|
||||
name: openstack-helm-images-base
|
||||
parent: base
|
||||
abstract: true
|
||||
description: |
|
||||
This job is building a docker image for
|
||||
OpenStack-Helm usage.
|
||||
Image specific tests can be added by
|
||||
running a post script on the relevant
|
||||
jobs.
|
||||
irrelevant-files:
|
||||
- ^\.git.*
|
||||
- ^.*\.(example|md|rst)$
|
||||
- ^doc/.*
|
||||
- ^releasenotes/.*
|
||||
timeout: 3600
|
||||
pre-run:
|
||||
- zuul.d/playbooks/pre-run.yml
|
||||
run: zuul.d/playbooks/docker-build.yml
|
||||
nodeset: ubuntu-bionic
|
|
@ -0,0 +1,44 @@
|
|||
---
|
||||
# Copyright 2018, SUSE LINUX GmbH.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
- project:
|
||||
check:
|
||||
jobs:
|
||||
- openstack-helm-images-libvirt-ubuntu_xenial
|
||||
gate:
|
||||
jobs:
|
||||
- openstack-helm-images-libvirt-ubuntu_xenial
|
||||
#experimental:
|
||||
# jobs:
|
||||
# - openstack-helm-infra-five-ubuntu
|
||||
|
||||
- job:
|
||||
name: openstack-helm-images-libvirt
|
||||
parent: openstack-helm-images-base
|
||||
abstract: true
|
||||
files:
|
||||
- ^libvirt/.*
|
||||
vars:
|
||||
image_path: libvirt
|
||||
|
||||
- job:
|
||||
name: openstack-helm-images-libvirt-ubuntu_xenial
|
||||
parent: openstack-helm-images-libvirt
|
||||
files:
|
||||
- ^libvirt/build.sh
|
||||
- ^libvirt/Dockerfile.ubuntu_xenial$
|
||||
- ^zuul.d/libvirt.yaml
|
||||
vars:
|
||||
distro: "ubuntu_xenial"
|
|
@ -0,0 +1,44 @@
|
|||
---
|
||||
# Copyright 2018, SUSE LINUX GmbH.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
- project:
|
||||
check:
|
||||
jobs:
|
||||
- openstack-helm-images-mariadb-ubuntu_xenial
|
||||
gate:
|
||||
jobs:
|
||||
- openstack-helm-images-mariadb-ubuntu_xenial
|
||||
#experimental:
|
||||
# jobs:
|
||||
# - openstack-helm-infra-five-ubuntu
|
||||
|
||||
- job:
|
||||
name: openstack-helm-images-mariadb
|
||||
parent: openstack-helm-images-base
|
||||
abstract: true
|
||||
files:
|
||||
- ^mariadb/.*
|
||||
vars:
|
||||
image_path: mariadb
|
||||
|
||||
- job:
|
||||
name: openstack-helm-images-mariadb-ubuntu_xenial
|
||||
parent: openstack-helm-images-mariadb
|
||||
files:
|
||||
- ^mariadb/build.sh
|
||||
- ^mariadb/Dockerfile.ubuntu_xenial$
|
||||
- ^zuul.d/mariadb.yaml
|
||||
vars:
|
||||
distro: "ubuntu_xenial"
|
|
@ -0,0 +1,16 @@
|
|||
---
|
||||
# This play will only get consumed in osh-images.
|
||||
# For depends-on to osh-images and osh changes triggering a rebuild of
|
||||
# osh-infra or osh, this play won't be used, as we won't know what to build, and
|
||||
# therefore will build everything for a 'standard' distribution.
|
||||
# No need to change zuul.project.src_dir to static paths then.
|
||||
- hosts: all[0]
|
||||
tasks:
|
||||
- name: Build image
|
||||
docker_image:
|
||||
buildargs: "{{ buildargs | default(omit) }}"
|
||||
state: present
|
||||
path: "{{ zuul.project.src_dir }}/{{ image_path }}"
|
||||
dockerfile: "Dockerfile.{{ distro }}"
|
||||
name: "openstackhelm/{{ image_path }}"
|
||||
tag: "{{ version | default('latest') }}-{{ distro }}"
|
|
@ -0,0 +1,31 @@
|
|||
---
|
||||
- hosts: all[0]
|
||||
pre_tasks:
|
||||
- name: Create docker folder
|
||||
file:
|
||||
path: /var/lib/docker
|
||||
state: directory
|
||||
become: True
|
||||
- name: Mount tmpfs to build faster
|
||||
mount:
|
||||
path: /var/lib/docker/
|
||||
src: tmpfs
|
||||
fstype: tmpfs
|
||||
opts: size=25g
|
||||
state: mounted
|
||||
become: True
|
||||
roles:
|
||||
- role: install-docker
|
||||
post_tasks:
|
||||
- name: Install extra docker libraries for ansible
|
||||
become: True
|
||||
package:
|
||||
name:
|
||||
- python3-docker
|
||||
- python-docker
|
||||
state: present
|
||||
- name: Ensure docker service is started
|
||||
become: true
|
||||
service:
|
||||
name: docker.service
|
||||
state: started
|
|
@ -0,0 +1,55 @@
|
|||
---
|
||||
# Copyright 2018, SUSE LINUX GmbH.
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License");
|
||||
# you may not use this file except in compliance with the License.
|
||||
# You may obtain a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS,
|
||||
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
|
||||
# See the License for the specific language governing permissions and
|
||||
# limitations under the License.
|
||||
|
||||
- project:
|
||||
check:
|
||||
jobs:
|
||||
# We should add a job here that tests the vbmc image
|
||||
# Alternatively, we can add a deploy using vbmc, and reuse
|
||||
# the deploy's functional testing
|
||||
- openstack-helm-images-vbmc-centos_7
|
||||
gate:
|
||||
jobs:
|
||||
- openstack-helm-images-vbmc-centos_7
|
||||
|
||||
- job:
|
||||
name: openstack-helm-images-vbmc
|
||||
parent: openstack-helm-images-base
|
||||
abstract: true
|
||||
files:
|
||||
- ^vbmc/.*
|
||||
vars:
|
||||
image_path: vbmc
|
||||
|
||||
- job:
|
||||
name: openstack-helm-images-vbmc-centos_7
|
||||
parent: openstack-helm-images-vbmc
|
||||
files:
|
||||
- ^vbmc/build.sh
|
||||
- ^vbmc/Dockerfile.centos_7$
|
||||
- ^zuul.d/vbmc.yaml
|
||||
vars:
|
||||
distro: "centos_7"
|
||||
|
||||
# This is an example of how to re-use the jobs for
|
||||
# multi-distro. The image would get built with a
|
||||
# different tag, based on distro argument.
|
||||
#- job:
|
||||
# name: openstack-helm-images-vbmc-ubuntu_bionic
|
||||
# parent: openstack-helm-images-vbmc
|
||||
# files:
|
||||
# - ^vbmc/Dockerfile.ubuntu_bionic$
|
||||
# vars:
|
||||
# distro: "ubuntu_bionic"
|
Loading…
Reference in New Issue