Resync repo content from fuel-web/docs
The docs content are synced from fuel-web/docs before final migration. Implements: blueprints.launchpad.net/fuel/+spec/fuel-docs-migration Change-Id: If9a03c29036a212568d573069488d4197913a226
This commit is contained in:
parent
b661bb7965
commit
2a0ed8484c
|
@ -11,10 +11,7 @@ This repository contains a set of GNU Make build scripts.
|
|||
Quick start
|
||||
-----------
|
||||
|
||||
1. You must use one of the following distributions to build Fuel components or the build process may fail. Note that build only works for x64 platforms.
|
||||
|
||||
* Ubuntu 12.04
|
||||
* Ubuntu 14.04
|
||||
1. You must use Ubuntu 14.04 distribution to build Fuel components or the build process may fail. Note that build only works for x64 platforms.
|
||||
|
||||
2. Check whether you have git installed in
|
||||
your system. To do that, use the following command:
|
||||
|
@ -85,9 +82,10 @@ Build system structure
|
|||
Fuel consists of several components such as web interface,
|
||||
puppet modules, orchestration components, testing components.
|
||||
Source code of all those components is split into multiple git
|
||||
repositories like:
|
||||
repositories like, for example:
|
||||
|
||||
- https://github.com/openstack/fuel-web
|
||||
- https://github.com/openstack/fuel-ui
|
||||
- https://github.com/openstack/fuel-astute
|
||||
- https://github.com/openstack/fuel-ostf
|
||||
- https://github.com/openstack/fuel-library
|
||||
|
@ -130,10 +128,6 @@ pieces of Fuel build system:
|
|||
ones which are to be copied on Fuel ISO even if Internet
|
||||
connection is down.
|
||||
|
||||
* **puppet** - contains the code used
|
||||
to pack Fuel puppet modules into a tarball that is afterwards
|
||||
put on Fuel ISO.
|
||||
|
||||
* **packages** - contains DEB and RPM
|
||||
specs as well as make code for building those packages,
|
||||
included in Fuel DEB and RPM mirrors.
|
||||
|
@ -141,30 +135,11 @@ pieces of Fuel build system:
|
|||
* **bootstrap** - contains a make script intended
|
||||
to build CentOS-based miniroot image (a.k.a initrd or initramfs).
|
||||
|
||||
* **image** - contains **make** scripts for building CentOS
|
||||
and Ubuntu images using the Fuel mirrors, built
|
||||
from the scripts in the *mirror* directory. The images
|
||||
are alternative to using the standard anaconda and debian installers.
|
||||
|
||||
* **docker** - contains the make scripts to
|
||||
build docker containers, deployed on the Fuel Master node.
|
||||
|
||||
* **upgrade** - contains make scripts for building Fuel upgrade tarball.
|
||||
|
||||
* **iso** - contains **make** scripts for building Fuel ISO file.
|
||||
|
||||
**Fuel-main** also contains a set of directories which are not directly
|
||||
related to Fuel build processes:
|
||||
|
||||
* **virtualbox** - contains a set of shell scripts
|
||||
which allow one to deploy Fuel demo lab easily using VirtualBox.
|
||||
|
||||
* **utils** - contains a set of utilities used for
|
||||
maintaining Fuel components.
|
||||
|
||||
* **fuelweb_test** and **fuelweb_ui_test** - contain
|
||||
the code of Fuel system tests.
|
||||
|
||||
|
||||
.. _build-targets:
|
||||
|
||||
|
@ -187,11 +162,6 @@ Build targets
|
|||
* **iso** - used for building Fuel ISO. If build succeeds,
|
||||
ISO is put into build/artifacts folder.
|
||||
|
||||
* **img** - used for building Fuel flash stick image,
|
||||
binary copied to a flash stick. That
|
||||
stick is then used as a bootable device and
|
||||
contains Fuel ISO as well as some auxiliary boot files.
|
||||
|
||||
* **clean** - removes build directory.
|
||||
|
||||
* **deep_clean** - removes build directory and local mirror.
|
||||
|
@ -233,20 +203,6 @@ They are defined in **config.mk** file:
|
|||
just ISO name.
|
||||
By default, it is **$(ARTS_DIR)/$(ISO_NAME).iso**.
|
||||
|
||||
* **UPGRADE_TARBALL_NAME** - defines the name of upgrade tarball.
|
||||
By default, it is **$(UPGRADE_TARBALL_NAME).tar**.
|
||||
|
||||
* **UPGRADE_TARBALL_PATH** - used to define full upgrade tarball path.
|
||||
By default, it is **$(ARTS_DIR)/$(UPGRADE_TARBALL_NAME).tar**.
|
||||
|
||||
* **VBOX_SCRIPTS_NAME** - defines the name of the archive with
|
||||
VirtualBox scripts.
|
||||
By default, it is placed into **$(VBOX_SCRIPTS_NAME).zip**.
|
||||
|
||||
* **VBOX_SCRIPTS_PATH** - defines full path for
|
||||
VirtualBox scripts archive.
|
||||
By default, it is **$(ARTS_DIR)/$(VBOX_SCRIPTS_NAME).zip**
|
||||
|
||||
* Fuel ISO contains some default settings for the
|
||||
Fuel Master node. These settings can be customized
|
||||
during Fuel Master node installation.
|
||||
|
@ -270,8 +226,6 @@ They are defined in **config.mk** file:
|
|||
Other options
|
||||
-------------
|
||||
|
||||
* **BUILD_OPENSTACK_PACKAGES** - list of Openstack packages to be rebuilt from source.
|
||||
|
||||
* **[repo]_REPO** - remote source code repo.
|
||||
URL or git repository can be specified for each of the Fuel components.
|
||||
(FUELLIB, NAILGUN, ASTUTE, OSTF).
|
||||
|
@ -310,19 +264,4 @@ Other options
|
|||
For example,
|
||||
*qemu2,http://osci-obs.vm.mirantis.net:82/centos-fuel-5.1-stable-15943/centos/ libvirt,http://osci-obs.vm.mirantis.net:82/centos-fuel-5.1-stable-17019/centos/*.
|
||||
|
||||
* **EXTRA_DEP_REPOS** - extra repos with DEB packages.
|
||||
Each repo must consist of an url,
|
||||
distro and section parts.
|
||||
Repos must be separated by bar:
|
||||
<first_repo_path>|<second_repo_path>
|
||||
For example,
|
||||
*http://fuel-repository.mirantis.com/repos/ubuntu-fuel-5.1-stable-15955/ubuntu/|http://fuel-repository.mirantis.com/repos/ubuntu-fuel-5.1-stable-15953/ubuntu/*.
|
||||
|
||||
** **FEATURE_GROUPS** - options for the ISO.
|
||||
Combination of the following:
|
||||
|
||||
* mirantis (use mirantis logos and logic)
|
||||
|
||||
* experimental (allow experimental features on Fuel web UI)
|
||||
|
||||
Note that if you want to add more packages to the Fuel Master node, you should update the **requirements-rpm.txt** and the **requirements-deb.txt** files.
|
||||
Note that if you want to add more packages to the Fuel Master node, you should update the **requirements-rpm.txt** file.
|
||||
|
|
|
@ -24,8 +24,8 @@ Fuel architecture, and the provisioning and deployment
|
|||
process:
|
||||
|
||||
* `Fuel architecture on the OpenStack wiki <https://wiki.openstack.org/wiki/Fuel#Fuel_architecture>`_
|
||||
* :doc:`Architecture section of Fuel documentation </devdocs/develop/architecture>`
|
||||
* :doc:`Visual of provisioning tasks </devdocs/develop/sequence>`
|
||||
* :doc:`Architecture section of Fuel documentation <architecture>`
|
||||
* :doc:`Visual of provisioning tasks <sequence>`
|
||||
|
||||
Adding Zabbix Role
|
||||
------------------
|
||||
|
@ -52,7 +52,7 @@ Additions to Fuel-Web for Zabbix role
|
|||
|
||||
In fuel-web, the `Support for Zabbix
|
||||
<https://review.openstack.org/#/c/84408/>`_ commit added the
|
||||
additional role to :doc:`Nailgun </devdocs/develop/nailgun>`. The
|
||||
additional role to :doc:`Nailgun <nailgun>`. The
|
||||
reader is urged to review this commit closely as a good
|
||||
example of where specific additions fit. In order to
|
||||
include this as an option in the Fuel deployment process,
|
||||
|
|
|
@ -124,8 +124,7 @@ Let's assume that the Fuel Master node has been deployed:
|
|||
::
|
||||
|
||||
tar -xzvf dd-src.tar.gz
|
||||
cd dd-src
|
||||
find . | cpio --quiet -o -H newc | gzip -9 > /tmp/initrd_update.img
|
||||
find dd-src/ | cpio --quiet -o -H newc | gzip -9 > /tmp/initrd_update.img
|
||||
|
||||
#. Copy into the TFTP (PXE) bootstrap folder:
|
||||
|
||||
|
@ -224,8 +223,7 @@ To change initramfs image, follow these steps:
|
|||
|
||||
::
|
||||
|
||||
cd /tmp/initrd-orig/initramfs
|
||||
find . | cpio --quiet -o -H newc | gzip -9 > /tmp/initramfs.img.new
|
||||
find /tmp/initrd-orig/initramfs | cpio --quiet -o -H newc | gzip -9 > /tmp/initramfs.img.new
|
||||
|
||||
#. Clean up. Remove */tmp/initrd-orig* temporary folder:
|
||||
|
||||
|
@ -272,584 +270,4 @@ follow these steps:
|
|||
cobbler sync
|
||||
|
||||
|
||||
.. _chroot:
|
||||
|
||||
Creating Ubuntu chroot on the Fuel Master node
|
||||
----------------------------------------------
|
||||
|
||||
.. note:: There is an alternative way of creating a ``chroot`` folder on the
|
||||
Fuel Master node. You can download prebuilt `VM images`_ for Ubuntu and
|
||||
run it with your favorite hypervisor. You can also use an IBP Ubuntu image
|
||||
which is built to your Fuel Master node.
|
||||
|
||||
.. _`VM images`: http://uec-images.ubuntu.com/trusty/current
|
||||
|
||||
This section describes how to create a chroot with Ubuntu on the Fuel Master
|
||||
node and provides the implementation script.
|
||||
|
||||
Creating a ``chroot`` folder on Ubuntu can be useful for:
|
||||
|
||||
* Rebuilding kernel modules for Ubuntu
|
||||
* Creating DKMS DEB packages from sources
|
||||
* Building kernel modules binaries for a given kernel version with DKMS
|
||||
|
||||
The script below creates ``chroot`` on the Fuel Master node using a prebuilt
|
||||
Ubuntu cloud image **trusty-server-cloudimg-amd64-root.tar.gz** that is
|
||||
downloaded from the `VM images`_ site. The name of the image and the link
|
||||
are kept in the ``UBUNTU_IMAGE`` and ``PREBUILT_IMAGE_LINK`` variables
|
||||
respectively.
|
||||
|
||||
.. note:: Before you copy and run the script, modify the ``UBUNTU_IMAGE``,
|
||||
``PREBUILT_IMAGE_LINK``, ``DISTRO_RELEASE``, ``KERNEL_FLAVOR``, or
|
||||
``MIRROR_DISTRO`` variables if required.
|
||||
|
||||
The script completes the following steps:
|
||||
|
||||
#. Creates ``chroot`` in the ``/tmp`` folder with the *ubuntu-chroot.XXXXX*
|
||||
template name (where *XXXXX* is substituted with digits and characters,
|
||||
for example, ``/tmp/ubuntu-chroot.Yusk8G``).
|
||||
#. Mounts the ``/proc`` filesystem and creates a ``/dev`` folder with links to
|
||||
``/proc`` into the ``chroot`` folder.
|
||||
#. Prepares a configuration for the ``apt`` package manager.
|
||||
#. Downloads and installs an additional set of packages, listed in the
|
||||
``UBUNTU_PKGS`` variable, to ``chroot``. The packages are required to build
|
||||
DKMS and deal with the DEB packages. These packages are: ``linux-headers``,
|
||||
``dkms``, ``build-essential``, and ``debhelper``.
|
||||
|
||||
.. note:: The Fuel Master node should have access to the Internet to download
|
||||
a required DEB package from the Ubuntu repository.
|
||||
|
||||
Unmount the ``chroot/proc`` file system and delete ``chroot``
|
||||
when you do not need it anymore.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
#!/bin/bash
|
||||
|
||||
# Define the kernel flavor and path to the link to a prebuild image.
|
||||
[ -z "$KERNEL_FLAVOR" ] && KERNEL_FLAVOR="-generic-lts-trusty"
|
||||
[ -z "$DISTRO_RELEASE" ] && DISTRO_RELEASE="trusty"
|
||||
[ -z "$UBUNTU_IMAGE" ] && UBUNTU_IMAGE="trusty-server-cloudimg-amd64-root.tar.gz"
|
||||
[ -z "$PREBUILT_IMAGE_LINK" ] && \
|
||||
PREBUILT_IMAGE_LINK="http://uec-images.ubuntu.com/${DISTRO_RELEASE}/current"
|
||||
|
||||
UBUNTU_PKGS="linux-headers${KERNEL_FLAVOR} linux-firmware dkms build-essential debhelper"
|
||||
|
||||
# Create a temporary directory (ubuntu-chroot) using the command:
|
||||
# [ -z "$root_dir" ] &&
|
||||
root_dir=$(mktemp -d --tmpdir ubuntu-chroot.XXXXX)
|
||||
chmod 755 ${root_dir}
|
||||
|
||||
# Download a prebuilt image and un-tar it.
|
||||
# Check if it has been downloaded already.
|
||||
if [ ! -e "$UBUNTU_IMAGE" ]; then
|
||||
# download
|
||||
wget ${PREBUILT_IMAGE_LINK}/${UBUNTU_IMAGE}
|
||||
fi
|
||||
tar -xzvf "${UBUNTU_IMAGE}" -C ${root_dir}
|
||||
|
||||
# Install required packages and resolve dependencies.
|
||||
chroot $root_dir env \
|
||||
LC_ALL=C \
|
||||
DEBIAN_FRONTEND=noninteractive \
|
||||
DEBCONF_NONINTERACTIVE_SEEN=true \
|
||||
TMPDIR=/tmp \
|
||||
TMP=/tmp \
|
||||
PATH=$PATH:/sbin:/bin \
|
||||
apt-get update
|
||||
|
||||
chroot $root_dir env \
|
||||
LC_ALL=C \
|
||||
DEBIAN_FRONTEND=noninteractive \
|
||||
DEBCONF_NONINTERACTIVE_SEEN=true \
|
||||
TMPDIR=/tmp \
|
||||
TMP=/tmp \
|
||||
PATH=$PATH:/sbin:/bin \
|
||||
apt-get install --force-yes --yes $UBUNTU_PKGS
|
||||
|
||||
echo "Don't forget to delete $root_dir at the end"
|
||||
|
||||
|
||||
Adding DKMS kernel modules into bootstrap (Ubuntu)
|
||||
--------------------------------------------------
|
||||
|
||||
The key strength of `Dynamic Kernel Module Support (DKMS) <https://help.ubuntu.com//community/DKMS>`_
|
||||
is the ability to rebuild the required kernel module for a different version of
|
||||
kernels. But there is a drawback of installing DKMS kernel modules into
|
||||
bootstrap. DKMS builds a module during installation, that queries the
|
||||
installation of additional packages like ``linux-headers`` and a tool-chain
|
||||
building. It unnecessarily oversizes the bootstrap. The DKMS package actually
|
||||
should be installed into an IBP (image-based provisioning) image, which will
|
||||
be deployed on nodes and be re-built during the kernel updates.
|
||||
|
||||
.. note::
|
||||
You can add kernel modules on bootstrap by making the
|
||||
kernel module binaries in a form of a DEB package and by installing the
|
||||
package on bootstrap like other packages.
|
||||
|
||||
DKMS provides an ability to build a DEB package and a disk driver archive
|
||||
on the fly from sources.
|
||||
|
||||
Ubuntu packages can be built on the Fuel Master node in ``chroot`` with Ubuntu
|
||||
deployed in ``chroot``. For details, see :ref:`chroot`.
|
||||
|
||||
**To create a DKMS package in the ``.deb`` format:**
|
||||
|
||||
#. Copy the required module sources to a folder with the corresponding name
|
||||
located in ``/usr/src`` of ``chroot``.
|
||||
#. Create a ``dkms.conf`` configuration file in the ``/usr/src`` directory.
|
||||
#. Optimize the ``dkms.conf`` file as described in the
|
||||
:ref:`dkms_example` section.
|
||||
|
||||
.. note::
|
||||
If you already have a DKMS package built with sources and want to simply
|
||||
export the kernel module binaries to DEB format, install the existing
|
||||
DKMS package into the ``chroot`` folder (and skip the
|
||||
:ref:`Creating DKMS <create_dkms>` chapter).
|
||||
|
||||
.. _create_dkms:
|
||||
|
||||
Creating a DKMS package from sources
|
||||
++++++++++++++++++++++++++++++++++++
|
||||
|
||||
Before creating a ``DKMS`` package from sources, verify that you have
|
||||
completed the following steps:
|
||||
|
||||
#. Create the :ref:`chroot folder <chroot>`.
|
||||
#. Install the following packages to the ``chroot`` folder: ``DKMS``,
|
||||
``build-essential``, and ``debhelper``.
|
||||
|
||||
Once you complete the steps above, create a DKMS package from sources:
|
||||
|
||||
#. Create a folder for a required kernel module in the *<module name>-<version>*
|
||||
format in the ``/usr/src`` directory located in ``chroot``.
|
||||
For example, if the module name is i40e and module version is 1.3.47,
|
||||
create a ``/usr/src/i40e-1.3.47`` folder in ``chroot``.
|
||||
|
||||
#. Copy the sources into the created folder.
|
||||
|
||||
#. Create and modify a ``dkms.conf`` file in the
|
||||
``<chroot folder>/usr/src/<module>-<version>/`` directory.
|
||||
|
||||
Example of a minimal dkms.conf file
|
||||
***********************************
|
||||
|
||||
Below is an example of a minimal
|
||||
`dkms.conf <http://linux.dell.com/dkms/dkms-for-developers.pdf>`_ file:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
PACKAGE_NAME="$module_name-dkms"
|
||||
PACKAGE_VERSION="$module_version"
|
||||
BUILT_MODULE_NAME="$module_name"
|
||||
DEST_MODULE_LOCATION="/updates"
|
||||
|
||||
The parameters in the minimal ``dkms.conf`` file are obligatory but not
|
||||
sufficient to build a module. Therefore, proceed with adding additional
|
||||
parameters to the ``dkms.conf`` file to make it operational. See the
|
||||
:ref:`dkms_example` section for details.
|
||||
|
||||
.. _dkms_example:
|
||||
|
||||
Example of an improved dkms.conf file
|
||||
*************************************
|
||||
|
||||
To make your ``dkms.conf`` file operational, add and configure the following
|
||||
fields: ``MAKE``, ``CLEAN``, and ``BUILD_MODULE_LOCATION``. There are also internal
|
||||
variables in DKMS that you can use in ``dkms.conf``, for example,
|
||||
``$kernelver``. For details, see `DKMS Manual page <http://linux.dell.com/dkms/manpage.html>`_.
|
||||
|
||||
The table below lists the fields that we use in our example to optimize the
|
||||
``dkms.conf`` file:
|
||||
|
||||
======================= ===================================================
|
||||
PACKAGE_NAME The DKMS package name.
|
||||
PACKAGE_VERSION The DKMS package version.
|
||||
BUILT_MODULE_NAME The binary kernel module name to be installed.
|
||||
DEST_MODULE_LOCATION The install location of the binary kernel module.
|
||||
MAKE The :command:`make` command to build the kernel
|
||||
module bounded to the kernel version, sources, and
|
||||
so on.
|
||||
BUILD_KERNEL The kernel version for which the module should be
|
||||
build. Use an internal variable ``$kernelver`` here.
|
||||
CLEAN The ``clean`` directive to clean up after the module
|
||||
build.
|
||||
BUILT_MODULE_LOCATION The location of the sources in the DKMS tree.
|
||||
REMAKE_INITRD Whether the ``initrd`` will be rebuilt or not when
|
||||
the module is installed.
|
||||
======================= ===================================================
|
||||
|
||||
For the i40e module that is used in our example, the following configuration
|
||||
is applied:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
PACKAGE_NAME="i40e-dkms"
|
||||
PACKAGE_VERSION="1.3.47"
|
||||
BUILT_MODULE_NAME="i40e"
|
||||
DEST_MODULE_LOCATION="/updates"
|
||||
MAKE="make -C src/ KERNELDIR=/lib/modules/\${kernelver}/build"
|
||||
BUILD_KERNEL="\${kernelver}"
|
||||
CLEAN="make -C src/ clean"
|
||||
BUILT_MODULE_LOCATION="src/"
|
||||
REMAKE_INITRD="yes"
|
||||
|
||||
.. note::
|
||||
The path that is set in the configuration file is bound to the DKMS tree.
|
||||
For example, ``DEST_MODULE_LOCATION="/updates"`` actually means
|
||||
``/lib/modules/$kernelver/updates``.
|
||||
|
||||
We recommend that you install new modules in the ``/updates`` directory for a
|
||||
`safe update <http://www.linuxvox.com/2009/10/update-kernel-modules-the-smart-and-safe-way>`_
|
||||
of the kernel modules.
|
||||
|
||||
Exporting DKMS package and kernel binaries
|
||||
******************************************
|
||||
|
||||
When ``dkms.conf`` is ready, you can build the binaries in ``chroot`` and
|
||||
export the ``DKMS`` package with kernel module binaries to the ``.deb`` format.
|
||||
|
||||
Use the DKMS commands to add and build a DKMS module for a particular kernel
|
||||
version.
|
||||
|
||||
When the build is done, run the following commands to create a DEB package
|
||||
and a disk-driver ``.tar`` archive in ``chroot``:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
mkdeb
|
||||
mkdriverdisk
|
||||
|
||||
See details in the bash script below.
|
||||
|
||||
The script builds a DKMS package in ``chroot``. The output is a disk-driver
|
||||
archive containing the module binaries built against the kernel installed in
|
||||
the ``chroot`` .
|
||||
|
||||
The second produced package is a DKMS module. The output is placed into the
|
||||
``/tmp/dkms-deb`` folder:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ ls /tmp/dkms-deb/
|
||||
i40e-1.3.47-ubuntu-dd.tar i40e-dkms_1.3.47_all.deb
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
The script requires following parameters to be provided:
|
||||
$1 - ``chroot`` folder with Ubuntu has been deployed
|
||||
$2 - module name
|
||||
$3 - module version
|
||||
$4 - path to the folder where is sources of the kernel module
|
||||
|
||||
.. warning::
|
||||
The script unmounts the ``/proc`` file system from ``chroot`` and
|
||||
finally deletes ``chroot`` made by the first script. Run the script with
|
||||
the root privileges.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
#!/bin/bash
|
||||
# Check passed parameters, expectations are following:
|
||||
# $1 - chroot folder with Ubuntu has been deployed
|
||||
# $2 - module name
|
||||
# $3 - module version
|
||||
# $4 - path to the folder where is sources of the kernel module
|
||||
|
||||
if [ $# != 4 ] ;
|
||||
then
|
||||
echo "ERR: Passed wrong number of parameters, the expectation are following"
|
||||
echo " $1 - chroot folder with Ubuntu has been deployed"
|
||||
echo " $2 - module name"
|
||||
echo " $3 - module version"
|
||||
echo " $4 - path to the folder where is sources of the module"
|
||||
echo "$0 <chroot_dir> <module-name> <module-version> <path-to-src>"
|
||||
exit 1;
|
||||
else
|
||||
root_dir=$1 # chroot folder
|
||||
module_name=$2
|
||||
module_version=$3
|
||||
module_src_dir=$4
|
||||
fi
|
||||
if [ ! -d "$root_dir" ] || [ ! -d "$module_src_dir" ] ;
|
||||
then
|
||||
echo "ERR: The $root_dir or $module_src_dir was not found";
|
||||
exit 1;
|
||||
fi
|
||||
|
||||
output_dir="/tmp/dkms-deb"
|
||||
|
||||
# Create the folder ${root_dir}/usr/src/${module-name}-${module-version}
|
||||
mkdir -p "${root_dir}/usr/src/${module_name}-${module_version}"
|
||||
chmod 755 "${root_dir}/usr/src/${module_name}-${module_version}"
|
||||
|
||||
# Copy sources into the folder
|
||||
cp -R "$module_src_dir"/* \
|
||||
${root_dir}/usr/src/${module_name}-${module_version}
|
||||
|
||||
# Create the dkms.conf package
|
||||
cat > "${root_dir}/usr/src/${module_name}-${module_version}/dkms.conf" <<-EOF
|
||||
MAKE="make -C src/ KERNELDIR=/lib/modules/\${kernelver}/build"
|
||||
BUILD_KERNEL="\${kernelver}"
|
||||
CLEAN="make -C src/ clean"
|
||||
BUILT_MODULE_NAME="$module_name"
|
||||
BUILT_MODULE_LOCATION="src/"
|
||||
DEST_MODULE_LOCATION="/updates"
|
||||
PACKAGE_NAME="$module_name-dkms"
|
||||
PACKAGE_VERSION="$module_version"
|
||||
REMAKE_INITRD="yes"
|
||||
EOF
|
||||
|
||||
# Deduce the kernel version
|
||||
KERNELDIR=$(ls -d ${root_dir}/lib/modules/*)
|
||||
kv="${KERNELDIR##*/}"
|
||||
|
||||
# Build the binaries by DKMS
|
||||
# Add the dkms
|
||||
chroot $root_dir env \
|
||||
LC_ALL=C \
|
||||
DEBIAN_FRONTEND=noninteractive \
|
||||
DEBCONF_NONINTERACTIVE_SEEN=true \
|
||||
TMPDIR=/tmp \
|
||||
TMP=/tmp \
|
||||
PATH=$PATH:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
|
||||
BUILD_KERNEL=${kv} \
|
||||
dkms add -m "${module_name}"/"${module_version}" -k ${kv}
|
||||
|
||||
# Build the kernel module by dkms
|
||||
chroot $root_dir env \
|
||||
LC_ALL=C \
|
||||
DEBIAN_FRONTEND=noninteractive \
|
||||
DEBCONF_NONINTERACTIVE_SEEN=true \
|
||||
TMPDIR=/tmp \
|
||||
TMP=/tmp \
|
||||
PATH=$PATH:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
|
||||
BUILD_KERNEL=${kv} \
|
||||
dkms build -m "${module_name}"/"${module_version}" -k ${kv}
|
||||
|
||||
# Create the deb-dkms package
|
||||
chroot $root_dir env \
|
||||
LC_ALL=C \
|
||||
DEBIAN_FRONTEND=noninteractive \
|
||||
DEBCONF_NONINTERACTIVE_SEEN=true \
|
||||
TMPDIR=/tmp \
|
||||
TMP=/tmp \
|
||||
PATH=$PATH:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
|
||||
BUILD_KERNEL=${kernelver} \
|
||||
dkms mkdeb -m "${module_name}"/"${module_version}" -k ${kv}
|
||||
|
||||
# Create the disk-driver archive with
|
||||
# module binaries in deb package ready to install on bootstrap
|
||||
chroot $root_dir env \
|
||||
LC_ALL=C \
|
||||
DEBIAN_FRONTEND=noninteractive \
|
||||
DEBCONF_NONINTERACTIVE_SEEN=true \
|
||||
TMPDIR=/tmp \
|
||||
TMP=/tmp \
|
||||
BUILD_KERNEL=${kv} \
|
||||
PATH=$PATH:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
|
||||
dkms mkdriverdisk -m "${module_name}"/"${module_version}" \
|
||||
-k ${kv} -d ubuntu --media tar
|
||||
|
||||
# Create /tmp/dkms-deb folder and copy the created deb file into it
|
||||
if [ ! -d "${output_dir}" ];
|
||||
then
|
||||
mkdir -p ${output_dir}
|
||||
fi
|
||||
# Copy the built deb dkms package into the folder
|
||||
# and driver disk tar archive.i
|
||||
# The archive contains the binary module as a deb package for given kernel version
|
||||
#
|
||||
cp ${root_dir}/var/lib/dkms/${module_name}/${module_version}/deb/*.deb ${output_dir}
|
||||
cp ${root_dir}/var/lib/dkms/${module_name}/${module_version}/driver_disk/*.tar ${output_dir}
|
||||
|
||||
# Don't forget to umount ${root_dir}/proc and remove ${root_dir}
|
||||
umount ${root_dir}/proc
|
||||
rm -Rf ${root_dir}
|
||||
|
||||
Extracting kernel module binaries
|
||||
*********************************
|
||||
|
||||
The ``/tmp/dkms-deb`` folder contains a built DKMS DEB package. You can
|
||||
install it into IBP. The ``DEB`` package with the kernel module binaries
|
||||
built for a given kernel version is archived in the disk-driver archive.
|
||||
|
||||
Unpack the ``.tar`` file and copy the ``.deb`` file into the repository.
|
||||
For example, if the archive is ``i40e-1.3.47-ubuntu-dd.tar`` and the i40e
|
||||
module was built for kernel 3.13.0-77-generic, the output should be the
|
||||
following:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
tar -xvf i40e-1.3.47-ubuntu-dd.tar
|
||||
./
|
||||
./ubuntu-drivers/
|
||||
./ubuntu-drivers/3.13.0/
|
||||
./ubuntu-drivers/3.13.0/i40e_1.3.47-3.13.0-77-generic_x86_64.deb
|
||||
...
|
||||
|
||||
The ``i40e_1.3.47-3.13.0-77-generic_x86_64.deb`` package contains the kernel
|
||||
module binaries for kernel 3.13.0-77-generic that you install on the
|
||||
bootstrap with the kernel.
|
||||
|
||||
.. warning::
|
||||
Updating the new kernel for Ubuntu requires rebuilding the DKMS package
|
||||
against a new kernel in order to get the module binaries package.
|
||||
|
||||
Known Issues
|
||||
************
|
||||
|
||||
Not all the kernel module sources can be compiled by DKMS.
|
||||
|
||||
DKMS builds the given drivers sources against different kernels versions.
|
||||
The ABI (kernel functions) may be changed among different kernels, and
|
||||
the compilation of a module can potentially fail when calling
|
||||
non-existing of expired functions.
|
||||
|
||||
The example below shows an attempt to build a module taken from one kernel
|
||||
version against the other kernel version:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
# dkms build -m be2net/10.4u
|
||||
|
||||
Kernel preparation unnecessary for this kernel. Skipping...
|
||||
|
||||
Building module:
|
||||
make clean
|
||||
make: *** No rule to make target `clean'. Stop.
|
||||
(bad exit status: 2)
|
||||
{ make KERNELRELEASE=3.13.0-77-generic -C /lib/modules/3.13.0-77-generic/build SUBDIRS=/var/lib/dkms/be2net/10.4u/build modules; } >> /var/lib/dkms/be2net/10.4u/build/make.log 2>&1
|
||||
(bad exit status: 2)
|
||||
ERROR (dkms apport): binary package for be2net: 10.4u not found
|
||||
Error! Bad return status for module build on kernel: 3.13.0-77-generic (x86_64)
|
||||
Consult /var/lib/dkms/be2net/10.4u/build/make.log for more information.
|
||||
|
||||
The ``make.log`` file contains errors that some functions or structures
|
||||
have not been declared or declared implicitly:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
# cat /var/lib/dkms/be2net/10.4u/build/make.log
|
||||
DKMS make.log for be2net-10.4u for kernel 3.13.0-77-generic (x86_64)
|
||||
|
||||
make: Entering directory `/usr/src/linux-headers-3.13.0-77-generic'
|
||||
CC [M] /var/lib/dkms/be2net/10.4u/build/be_main.o
|
||||
/var/lib/dkms/be2net/10.4u/build/be_main.c: In function ‘be_mac_addr_set’:
|
||||
/var/lib/dkms/be2net/10.4u/build/be_main.c:315:2: error: implicit declaration of function ‘ether_addr_copy’ [-Werror=implicit-function-declaration]
|
||||
ether_addr_copy(netdev->dev_addr, addr->sa_data);
|
||||
^
|
||||
/var/lib/dkms/be2net/10.4u/build/be_main.c: In function ‘be_get_tx_vlan_tag’:
|
||||
/var/lib/dkms/be2net/10.4u/build/be_main.c:727:2: error: implicit declaration of function ‘skb_vlan_tag_get’ [-Werror=implicit-function-declaration]
|
||||
vlan_tag = skb_vlan_tag_get(skb);
|
||||
^
|
||||
/var/lib/dkms/be2net/10.4u/build/be_main.c: In function ‘be_get_wrb_params_from_skb’:
|
||||
/var/lib/dkms/be2net/10.4u/build/be_main.c:789:2: error: implicit declaration of function ‘skb_vlan_tag_present’ [-Werror=implicit-function-declaration]
|
||||
if (skb_vlan_tag_present(skb)) {
|
||||
^
|
||||
/var/lib/dkms/be2net/10.4u/build/be_main.c: In function ‘be_insert_vlan_in_pkt’:
|
||||
/var/lib/dkms/be2net/10.4u/build/be_main.c:1001:3: error: implicit declaration of function ‘vlan_insert_tag_set_proto’ [-Werror=implicit-function-declaration]
|
||||
skb = vlan_insert_tag_set_proto(skb, htons(ETH_P_8021Q),
|
||||
^
|
||||
/var/lib/dkms/be2net/10.4u/build/be_main.c:1001:7: warning: assignment makes pointer from integer without a cast [enabled by default]
|
||||
skb = vlan_insert_tag_set_proto(skb, htons(ETH_P_8021Q),
|
||||
^
|
||||
/var/lib/dkms/be2net/10.4u/build/be_main.c:1011:7: warning: assignment makes pointer from integer without a cast [enabled by default]
|
||||
skb = vlan_insert_tag_set_proto(skb, htons(ETH_P_8021Q),
|
||||
^
|
||||
/var/lib/dkms/be2net/10.4u/build/be_main.c: In function ‘be_xmit_workarounds’:
|
||||
/var/lib/dkms/be2net/10.4u/build/be_main.c:1132:3: error: implicit declaration of function ‘skb_put_padto’ [-Werror=implicit-function-declaration]
|
||||
if (skb_put_padto(skb, 36))
|
||||
^
|
||||
/var/lib/dkms/be2net/10.4u/build/be_main.c: In function ‘be_xmit’:
|
||||
/var/lib/dkms/be2net/10.4u/build/be_main.c:1299:19: error: ‘struct sk_buff’ has no member named ‘xmit_more’
|
||||
bool flush = !skb->xmit_more;
|
||||
|
||||
To make the kernel module sources compatible with different kernels, the
|
||||
sources should contain the wrappers, which are re-declaring changed functions
|
||||
depending on the kernel version. This work should be done by driver developers.
|
||||
|
||||
The example below shows the ``compat.h`` file wrapper:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
/*
|
||||
* This file is part of the Linux NIC driver for Emulex networking products.
|
||||
*
|
||||
* Copyright (C) 2005-2015 Emulex. All rights reserved.
|
||||
*
|
||||
* EMULEX and SLI are trademarks of Emulex.
|
||||
* www.emulex.com
|
||||
* linux-drivers@emulex.com
|
||||
*
|
||||
* This program is free software; you can redistribute it and/or modify
|
||||
* it under the terms of version 2 of the GNU General Public License as
|
||||
* published by the Free Software Foundation.
|
||||
*
|
||||
* This program is distributed in the hope that it will be useful.
|
||||
* ALL EXPRESS OR IMPLIED CONDITIONS, REPRESENTATIONS AND WARRANTIES,
|
||||
* INCLUDING ANY IMPLIED WARRANTY OF MERCHANTABILITY, FITNESS FOR A
|
||||
* PARTICULAR PURPOSE, OR NON-INFRINGEMENT, ARE DISCLAIMED, EXCEPT TO THE
|
||||
* EXTENT THAT SUCH DISCLAIMERS ARE HELD TO BE LEGALLY INVALID.
|
||||
* See the GNU General Public License for more details, a copy of which
|
||||
* can be found in the file COPYING included with this package
|
||||
*/
|
||||
|
||||
#ifndef BE_COMPAT_H
|
||||
#define BE_COMPAT_H
|
||||
|
||||
#ifdef RHEL_RELEASE_CODE
|
||||
#define RHEL
|
||||
#endif
|
||||
|
||||
#ifndef RHEL_RELEASE_CODE
|
||||
#define RHEL_RELEASE_CODE 0
|
||||
#endif
|
||||
|
||||
#ifndef RHEL_RELEASE_VERSION
|
||||
#define RHEL_RELEASE_VERSION(a,b) (((a) << 8) + (b))
|
||||
#endif
|
||||
|
||||
#ifndef NETIF_F_HW_VLAN_CTAG_DEFINED
|
||||
#define NETIF_F_HW_VLAN_CTAG_TX NETIF_F_HW_VLAN_TX
|
||||
#define NETIF_F_HW_VLAN_CTAG_RX NETIF_F_HW_VLAN_RX
|
||||
#define NETIF_F_HW_VLAN_CTAG_FILTER NETIF_F_HW_VLAN_FILTER
|
||||
#endif
|
||||
|
||||
/*************************** NAPI backport ********************************/
|
||||
#if LINUX_VERSION_CODE < KERNEL_VERSION(2, 6, 27)
|
||||
|
||||
/* RHEL 5.4+ has a half baked napi_struct implementation.
|
||||
* Bypass it and use simulated NAPI using multiple netdev structs
|
||||
*/
|
||||
#ifdef RHEL
|
||||
typedef struct napi_struct rhel_napi;
|
||||
#endif
|
||||
|
||||
#define netif_napi_add netif_napi_add_compat
|
||||
#define netif_napi_del netif_napi_del_compat
|
||||
#define napi_gro_frags(napi) napi_gro_frags((rhel_napi*) napi)
|
||||
#define napi_get_frags(napi) napi_get_frags((rhel_napi*) napi)
|
||||
#define vlan_gro_frags(napi, g, v) vlan_gro_frags((rhel_napi*) napi, g, v);
|
||||
#define napi_schedule(napi) netif_rx_schedule((napi)->dev)
|
||||
#define napi_enable(napi) netif_poll_enable((napi)->dev)
|
||||
#define napi_disable(napi) netif_poll_disable((napi)->dev)
|
||||
#define napi_complete(napi) napi_gro_flush((rhel_napi *)napi); \
|
||||
netif_rx_complete(napi->dev)
|
||||
#define napi_schedule_prep(napi) netif_rx_schedule_prep((napi)->dev)
|
||||
#define __napi_schedule(napi) __netif_rx_schedule((napi)->dev)
|
||||
|
||||
#define napi_struct napi_struct_compat
|
||||
|
||||
struct napi_struct_compat {
|
||||
#ifdef RHEL
|
||||
rhel_napi napi; /* must be the first member */
|
||||
#endif
|
||||
struct net_device *dev;
|
||||
int (*poll) (struct napi_struct *napi, int budget);
|
||||
};
|
||||
|
||||
extern void netif_napi_del_compat(struct napi_struct *napi);
|
||||
extern void netif_napi_add_compat(struct net_device *, struct napi_struct *,
|
||||
int (*poll) (struct napi_struct *, int), int);
|
||||
#endif /*********************** NAPI backport *****************************/
|
||||
|
|
|
@ -35,6 +35,7 @@ the Fuel components:
|
|||
apt-get install git
|
||||
git clone https://github.com/openstack/fuel-main
|
||||
git clone https://github.com/openstack/fuel-web
|
||||
git clone https://github.com/openstack/fuel-ui
|
||||
git clone https://github.com/openstack/fuel-agent
|
||||
git clone https://github.com/openstack/fuel-astute
|
||||
git clone https://github.com/openstack/fuel-ostf
|
||||
|
@ -73,11 +74,13 @@ tag of Fuel:
|
|||
# Repos and versions
|
||||
FUELLIB_COMMIT?=tags/5.0
|
||||
NAILGUN_COMMIT?=tags/5.0
|
||||
FUEL_UI_COMMIT?=tags/5.0
|
||||
ASTUTE_COMMIT?=tags/5.0
|
||||
OSTF_COMMIT?=tags/5.0
|
||||
|
||||
FUELLIB_REPO?=https://github.com/openstack/fuel-library.git
|
||||
NAILGUN_REPO?=https://github.com/openstack/fuel-web.git
|
||||
FUEL_UI_REPO?=https://github.com/openstack/fuel-ui.git
|
||||
ASTUTE_REPO?=https://github.com/openstack/fuel-astute.git
|
||||
OSTF_REPO?=https://github.com/openstack/fuel-ostf.git
|
||||
|
||||
|
@ -118,7 +121,6 @@ your Fuel ISO build environment on Ubuntu 14.04:
|
|||
sudo gem install bundler -v 1.2.1
|
||||
sudo gem install builder
|
||||
sudo pip install xmlbuilder jinja2
|
||||
sudo npm install -g gulp
|
||||
|
||||
#. If you haven't already done so, get the source code::
|
||||
|
||||
|
@ -224,12 +226,25 @@ those services.
|
|||
source ~/.rvm/scripts/rvm
|
||||
rvm install 2.1
|
||||
rvm use 2.1
|
||||
git clone https://github.com/nulayer/raemon.git
|
||||
cd raemon
|
||||
git checkout b78eaae57c8e836b8018386dd96527b8d9971acc
|
||||
gem build raemon.gemspec
|
||||
gem install raemon-0.3.0.gem
|
||||
cd ..
|
||||
rm -Rf raemon
|
||||
|
||||
#. Install or update dependencies and run unit tests::
|
||||
|
||||
cd fuel-astute
|
||||
./run_tests.sh
|
||||
|
||||
#. (optional) Run Astute MCollective integration test (you'll need to
|
||||
have MCollective server running for this to work)::
|
||||
|
||||
cd fuel-astute
|
||||
bundle exec rspec spec/integration/mcollective_spec.rb
|
||||
|
||||
Running Fuel Puppet Modules Unit Tests
|
||||
--------------------------------------
|
||||
|
||||
|
@ -286,7 +301,7 @@ Now you can create the virtual environment and activate it.
|
|||
::
|
||||
|
||||
virtualenv fuel-web-venv
|
||||
. virtualenv/bin/activate
|
||||
. fuel-web-venv/bin/activate
|
||||
|
||||
And then install the dependencies.
|
||||
::
|
||||
|
|
|
@ -40,6 +40,8 @@ structure includes the following attributes::
|
|||
regex:
|
||||
source: "^[A-z0-9]+$"
|
||||
error: "Invalid data"
|
||||
min: 1
|
||||
max: 3
|
||||
|
||||
* *label* is a setting title that is displayed on UI
|
||||
* *weight* defines the order in which this setting is displayed in its group.
|
||||
|
@ -55,6 +57,8 @@ structure includes the following attributes::
|
|||
* *select* - drop-down list
|
||||
* *hidden* - invisible input
|
||||
* *file* - file contents input
|
||||
* *text_list* - multiple sigle line text inputs
|
||||
* *textarea_list* - multiple multiline text inputs
|
||||
|
||||
* *regex* section is applicable for settings of "text" type. "regex.source"
|
||||
is used when validating with a regular expression. "regex.error" contains
|
||||
|
@ -65,6 +69,10 @@ structure includes the following attributes::
|
|||
* *values* list is needed for settings of "radio" or "select" type to declare
|
||||
its possible values. Options from "values" list also support dependencies
|
||||
and conflcits declaration.
|
||||
* *min* is actual for settings of "text_list" or "textarea_list" type
|
||||
to declare a minimum input list length for the setting
|
||||
* *max* is actual for settings of "text_list" or "textarea_list" type
|
||||
to declare a maximum input list length for the setting
|
||||
|
||||
.. _restrictions:
|
||||
|
||||
|
|
|
@ -8,24 +8,8 @@ For information on how to get source code see :ref:`getting-source`.
|
|||
Preparing Development Environment
|
||||
---------------------------------
|
||||
|
||||
.. warning:: Nailgun requires Python 2.6 with development files. Please check
|
||||
installed Python version using ``python --version``. If the version check
|
||||
does not match, please install Python 2.6 as described in step 1.
|
||||
If Python version matches, skip step 1.
|
||||
|
||||
Please note that these instructions were tested on Ubuntu 12.04.4/14.04.1 (64 bit)
|
||||
that contains Python 2.7 and requires downgrade to Python 2.6.
|
||||
PPA instructions listed below install Python 2.6 but do not remove Python 2.7.
|
||||
The default Python version remains 2.7.
|
||||
You will have to specify Python version when you create virtual environment (see 8.1.1.1. step 5).
|
||||
|
||||
|
||||
#. Install Python 2.6 using
|
||||
`PPA <https://launchpad.net/~fkrull/+archive/ubuntu/deadsnakes>`::
|
||||
|
||||
sudo add-apt-repository --yes ppa:fkrull/deadsnakes
|
||||
sudo apt-get update
|
||||
sudo apt-get install --yes python2.6 python2.6-dev
|
||||
.. warning:: Nailgun requires Python 2.7. Please check
|
||||
installed Python version using ``python --version``.
|
||||
|
||||
#. Nailgun can be found in fuel-web/nailgun
|
||||
|
||||
|
@ -59,17 +43,12 @@ Preparing Development Environment
|
|||
sudo apt-get install --yes python-dev python-pip
|
||||
|
||||
#. Install virtualenv. This step increases flexibility
|
||||
when dealing with environment settings and package installation.
|
||||
This step is obligatory if you need to downgrade to Python version 2.6
|
||||
(that Nailgun depends on)::
|
||||
when dealing with environment settings and package installation::
|
||||
|
||||
sudo pip install virtualenv virtualenvwrapper
|
||||
. /usr/local/bin/virtualenvwrapper.sh # you can save this to .bashrc
|
||||
whereis python2.6 # Prints the intall path of python 2.6, let's say /usr/bin/python2.6.
|
||||
# Copy the output and paste it in the command below for option -p
|
||||
mkvirtualenv fuel -p /usr/bin/python2.6 # you can use any name instead of 'fuel'
|
||||
mkvirtualenv fuel # you can use any name instead of 'fuel'
|
||||
workon fuel # command selects the particular environment
|
||||
python --version # verify that default Python version inside virtual environment is 2.6
|
||||
|
||||
#. Install Python dependencies. This section assumes that you use virtual environment.
|
||||
Otherwise, you must install all packages globally.
|
||||
|
@ -80,24 +59,21 @@ Preparing Development Environment
|
|||
cd fuel-web
|
||||
pip install --allow-all-external -r nailgun/test-requirements.txt
|
||||
|
||||
#. Install Nailgun in the developers mode by running the command below in the
|
||||
`nailgun` folder. Thanks to that, Nailgun extensions will be discovered::
|
||||
|
||||
python setup.py develop
|
||||
|
||||
Or if you are using pip::
|
||||
|
||||
pip install -e .
|
||||
|
||||
#. Create required folder for log files::
|
||||
|
||||
sudo mkdir /var/log/nailgun
|
||||
sudo chown -R `whoami`.`whoami` /var/log/nailgun
|
||||
sudo chmod -R a+w /var/log/nailgun
|
||||
|
||||
#. Install NodeJS and JS dependencies::
|
||||
|
||||
sudo apt-get remove --yes nodejs nodejs-legacy
|
||||
sudo apt-get install --yes software-properties-common
|
||||
sudo add-apt-repository --yes ppa:chris-lea/node.js
|
||||
sudo apt-get update
|
||||
sudo apt-get install --yes nodejs
|
||||
sudo npm install -g gulp
|
||||
sudo chown -R `whoami`.`whoami` ~/.npm
|
||||
cd nailgun
|
||||
npm install
|
||||
|
||||
Setup for Nailgun Unit Tests
|
||||
----------------------------
|
||||
|
||||
|
@ -111,14 +87,10 @@ Setup for Nailgun Unit Tests
|
|||
workon fuel #activate virtual environment created in the previous section
|
||||
pip install tox
|
||||
|
||||
#. Run the Nailgun backend unit tests::
|
||||
#. Run the Nailgun backend unit tests and flake8 test::
|
||||
|
||||
sudo apt-get install puppet-common #install missing package required by tasklib tests
|
||||
./run_tests.sh --no-webui
|
||||
|
||||
#. Run the Nailgun flake8 test::
|
||||
|
||||
./run_tests.sh --flake8
|
||||
./run_tests.sh
|
||||
|
||||
#. You can also run the same tests by hand, using tox itself::
|
||||
|
||||
|
@ -160,28 +132,6 @@ For example:
|
|||
performance tests.
|
||||
|
||||
|
||||
|
||||
Setup for Web UI Tests
|
||||
----------------------
|
||||
|
||||
#. UI tests use Selenium server, so you need to install Java Runtime
|
||||
Environment (JRE) 1.6 or newer version.
|
||||
|
||||
#. You also need to install Firefox - it is used as the default browser for
|
||||
tests.
|
||||
|
||||
#. Run full Web UI test suite (this will wipe your Nailgun database in
|
||||
PostgreSQL)::
|
||||
|
||||
cd fuel-web
|
||||
./run_tests.sh --webui
|
||||
|
||||
By default Firefox browser is used. You can specify the browser using
|
||||
BROWSER environment variable::
|
||||
|
||||
BROWSER=chrome ./run_tests.sh --webui
|
||||
|
||||
|
||||
.. _running-parallel-tests-py:
|
||||
|
||||
Running parallel tests with py.test
|
||||
|
@ -253,52 +203,9 @@ Running Nailgun in Fake Mode
|
|||
|
||||
python manage.py run -p 8000 --fake-tasks-amqp | egrep --line-buffered -v '^$|HTTP' >> /var/log/nailgun.log 2>&1 &
|
||||
|
||||
#. If you plan to use Fuel UI:
|
||||
|
||||
* Update JS dependencies::
|
||||
|
||||
npm install
|
||||
|
||||
* If you don't plan to modify Fuel UI, you may want just to build static
|
||||
version which is served by nailgun::
|
||||
|
||||
gulp build
|
||||
|
||||
Please note that after pulling updates from fuel-web repo you may need to
|
||||
run this command again.
|
||||
|
||||
To specify custom output directory location use
|
||||
`static-dir` option::
|
||||
|
||||
gulp build --static-dir=static_compressed
|
||||
|
||||
To speed up build process you may also want to disable uglification and
|
||||
source maps generation::
|
||||
|
||||
gulp build --no-uglify --no-sourcemaps
|
||||
|
||||
* If you plan to modify Fuel UI, there is more convenient option --
|
||||
a development server. It watches for file changes and automatically
|
||||
rebuilds changed modules (significantly faster than full rebuild)
|
||||
and triggers page refresh in browsers::
|
||||
|
||||
gulp dev-server
|
||||
|
||||
By default it runs on port 8080 and assumes that nailgun runs on
|
||||
port 8000. You can override this by using the following options::
|
||||
|
||||
gulp dev-server --dev-server-host=127.0.0.1 --dev-server-port=8080 --nailgun-host=127.0.0.1 --nailgun-port=8000
|
||||
|
||||
If you don't want to use a development server but would like to recompile
|
||||
the bundle on any change, use::
|
||||
|
||||
gulp build --watch
|
||||
|
||||
If automatic rebuild on change doesn't work, most likely you need to
|
||||
increase the limit of inotify watches::
|
||||
|
||||
echo 100000 | sudo tee /proc/sys/fs/inotify/max_user_watches
|
||||
|
||||
Nailgun in fake mode is usually used for Fuel UI development and Fuel UI
|
||||
functional tests. For more information, please check out README file in
|
||||
the fuel-ui repo.
|
||||
|
||||
Note: Diagnostic Snapshot is not available in a Fake mode.
|
||||
|
||||
|
|
|
@ -76,7 +76,7 @@ The test docstrings are another important piece and you should always stick to t
|
|||
|
||||
- test duration - an estimate of how much a test will take
|
||||
|
||||
deployment tags (optional) - gives information about what kind of environment the test will be run, possible values are CENTOS, Ubuntu, RHEL nova_network, Heat, Murano, Sahara)
|
||||
deployment tags (optional) - gives information about what kind of environment the test will be run, possible values are CENTOS, Ubuntu, RHEL nova_network, Heat, Sahara)
|
||||
|
||||
Here's a test example which confirms the above explanations:
|
||||
|
||||
|
@ -145,7 +145,7 @@ How to execute my tests?
|
|||
Simplest way is to install Fuel, and OSTF will be installed as part of it.
|
||||
- install virtualbox
|
||||
- build Fuel ISO: :ref:`building-fuel-iso`
|
||||
- use `virtualbox scripts to run an ISO <https://github.com/openstack/fuel-main/tree/master/virtualbox>`_
|
||||
- use `virtualbox scripts to run an ISO <https://github.com/openstack/fuel-virtualbox/tree/master/>`_
|
||||
- once the installation is finished, go to Fuel UI (usually it's 10.20.0.2:8000) and create a new cluster with necessary configuration
|
||||
- execute::
|
||||
|
||||
|
|
|
@ -16,7 +16,8 @@ and many other customizations with a few lines of code in system tests.
|
|||
|
||||
After 6.0 release, fuel-devops was divided into 2.5.x and 2.9.x versions. Two
|
||||
separate versions of fuel-devops provide backward compatibility for system
|
||||
tests which have been refactored since the last major release. `How to migrate`_
|
||||
tests which have been refactored since the last major release. Look here
|
||||
`how to migrate`_ from older devops.
|
||||
|
||||
For sources please refer to
|
||||
`fuel-devops repository on github <https://github.com/openstack/fuel-devops>`_.
|
||||
|
@ -26,73 +27,85 @@ For sources please refer to
|
|||
Installation
|
||||
-------------
|
||||
|
||||
The installation procedure can be implemented via PyPI in Python virtual environment
|
||||
(suppose you are using *Ubuntu 12.04* or *Ubuntu 14.04*):
|
||||
The installation procedure can be implemented via PyPI in Python virtual
|
||||
environment (suppose you are using *Ubuntu 12.04* or *Ubuntu 14.04*):
|
||||
|
||||
Before using it, please install the following required dependencies:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo apt-get install git \
|
||||
postgresql \
|
||||
postgresql-server-dev-all \
|
||||
sudo apt-get install --yes \
|
||||
git \
|
||||
libyaml-dev \
|
||||
libffi-dev \
|
||||
python-dev \
|
||||
python-libvirt \
|
||||
python-pip \
|
||||
qemu-kvm \
|
||||
qemu-utils \
|
||||
qemu \
|
||||
libvirt-bin \
|
||||
libvirt-dev \
|
||||
ubuntu-vm-builder \
|
||||
bridge-utils
|
||||
vlan \
|
||||
bridge-utils \
|
||||
genisoimage
|
||||
|
||||
sudo apt-get update && sudo apt-get upgrade -y
|
||||
|
||||
.. _DevOpsPyPIvenv:
|
||||
|
||||
Devops installation in `virtualenv <http://virtualenv.readthedocs.org/en/latest/>`_
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
Devops installation in `virtualenv <http://virtualenv.readthedocs.org/en/latest/virtualenv.html>`_
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
1. Install packages needed for building python eggs
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo apt-get install python-virtualenv libpq-dev libgmp-dev
|
||||
sudo apt-get install --yes python-virtualenv libpq-dev libgmp-dev pkg-config
|
||||
|
||||
2. In case you are using *Ubuntu 12.04* let's update pip and virtualenv, otherwise you can skip this step
|
||||
2. In case you are using *Ubuntu 12.04* let's update pip and virtualenv,
|
||||
otherwise you can skip this step
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo pip install pip virtualenv --upgrade
|
||||
hash -r
|
||||
|
||||
4. Create virtualenv for the *devops* project
|
||||
3. In oder to store the path where your Python virtualenv will be located
|
||||
create your working directory and use the following environment variable. If
|
||||
it is not specified, it will use the current working directory:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
virtualenv --system-site-packages <path>/fuel-devops-venv
|
||||
export WORKING_DIR=$HOME/working_dir
|
||||
mkdir $HOME/working_dir
|
||||
|
||||
.. note:: If you want to use different devops versions in the same time, you can create several different folders for each version, and then activate required virtual environment for each case.
|
||||
For example: ::
|
||||
virtualenv --system-site-packages <path>/fuel-devops-venv # For fuel-devops 2.5.x
|
||||
virtualenv --system-site-packages <path>/fuel-devops-venv-2.9 # For fuel-devops 2.9.x
|
||||
4. Create virtualenv for the *devops* project (e.g. ``fuel-devops-venv``).
|
||||
Note: the related directory will be used for the ``VENV_PATH`` variable:
|
||||
|
||||
<path> represents the path where your Python virtualenv will be located. (e.g. ~/venv). If it is not specified, it will use the current working directory.
|
||||
.. code-block:: bash
|
||||
|
||||
cd $WORKING_DIR
|
||||
sudo apt-get install --yes python-virtualenv
|
||||
virtualenv --no-site-packages fuel-devops-venv
|
||||
|
||||
.. note:: If you want to use different devops versions in the same time, you
|
||||
can create several different folders for each version, and then activate the
|
||||
required virtual environment for each case.
|
||||
|
||||
For example::
|
||||
|
||||
virtualenv --no-site-packages fuel-devops-venv # For fuel-devops 2.5.x
|
||||
virtualenv --no-site-packages fuel-devops-venv-2.9 # For fuel-devops 2.9.x
|
||||
|
||||
5. Activate virtualenv and install *devops* package using PyPI.
|
||||
In order to indentify the latest available versions you would like to install,
|
||||
visit `fuel-devops <https://github.com/openstack/fuel-devops/tags>`_ repo. For
|
||||
Fuel 6.0 and earlier, take the latest fuel-devops 2.5.x (e.g.
|
||||
fuel-devops.git@2.5.6). For Fuel 6.1 and later, use 2.9.x or newer (e.g.
|
||||
fuel-devops.git@2.9.11):
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
source <path>/fuel-devops-venv/bin/activate
|
||||
pip install git+https://github.com/openstack/fuel-devops.git@<version> --upgrade
|
||||
|
||||
where <version> is the specific version of fuel-devops you would like to
|
||||
install. For Fuel 6.0 and earlier, take the latest fuel-devops 2.5.x. For Fuel
|
||||
6.1 and later, use 2.9.x or newer. See more information on the latest available
|
||||
versions in `fuel-devops <https://github.com/openstack/fuel-devops/tags>`_
|
||||
repo.
|
||||
. fuel-devops-venv/bin/activate
|
||||
pip install git+https://github.com/openstack/fuel-devops.git@2.9.11 --upgrade
|
||||
|
||||
setup.py in fuel-devops repository does everything required.
|
||||
|
||||
|
@ -129,29 +142,78 @@ Create libvirt's pool
|
|||
Permissions to run KVM VMs with libvirt with current user
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
Give current user permissions to use libvirt (Do not forget to log out and log back in!)
|
||||
Give current user permissions to use libvirt: do not forget to log out and log
|
||||
back in.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo usermod $(whoami) -a -G libvirtd,sudo
|
||||
|
||||
Configuring Postgresql database
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
Configuring database
|
||||
~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
You can configure PostgreSQL database or as an alternative SQLite.
|
||||
|
||||
Configuring PostgreSQL
|
||||
+++++++++++++++++++++++
|
||||
|
||||
Install postgresql package:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo apt-get install --yes postgresql
|
||||
|
||||
Set local peers to be trusted by default, create user and db and load fixtures.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
pg_version=$(dpkg-query --show --showformat='${version;3}' postgresql)
|
||||
pg_createcluster $pg_version main --start
|
||||
sudo sed -ir 's/peer/trust/' /etc/postgresql/9.*/main/pg_hba.conf
|
||||
sudo service postgresql restart
|
||||
sudo -u postgres createuser -P <user> # see default <user> and <db> below
|
||||
sudo -u postgres createdb <db> -O <user>
|
||||
|
||||
* in **2.9.x version**, default <user> and <db> are **fuel_devops**
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo -u postgres createuser -P fuel_devops
|
||||
sudo -u postgres psql -c "CREATE ROLE fuel_devops WITH LOGIN PASSWORD 'fuel_devops'"
|
||||
sudo -u postgres createdb fuel_devops -O fuel_devops
|
||||
|
||||
* in **2.5.x version**, default <user> and <db> are **devops**
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo -u postgres createuser -P devops
|
||||
sudo -u postgres psql -c "CREATE ROLE devops WITH LOGIN PASSWORD 'devops'"
|
||||
sudo -u postgres createdb devops -O devops
|
||||
|
||||
Configuring SQLite3 database
|
||||
+++++++++++++++++++++++++++++
|
||||
|
||||
Install SQLite3 library:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo apt-get install --yes libsqlite3-0
|
||||
|
||||
Export the path to the SQLite3 database as the database name:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
export DEVOPS_DB_NAME=$WORKING_DIR/fuel-devops
|
||||
export DEVOPS_DB_ENGINE="django.db.backends.sqlite3
|
||||
|
||||
Configuring Django
|
||||
~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
After the database setup, we can install the django tables and data:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
django-admin.py syncdb --settings=devops.settings
|
||||
django-admin.py migrate devops --settings=devops.settings
|
||||
|
||||
* in 2.5.x version, default <user> and <db> are **devops**
|
||||
* in 2.9.x version, default <user> and <db> are **fuel_devops**
|
||||
|
||||
.. note:: Depending on your Linux distribution,
|
||||
`django-admin <http://django-admin-tools.readthedocs.org>`_ may refer
|
||||
to system-wide django installed from package. If this happens you could get
|
||||
|
@ -165,11 +227,13 @@ Set local peers to be trusted by default, create user and db and load fixtures.
|
|||
[Optional] Enabling `Nested Paging <http://en.wikipedia.org/wiki/Second_Level_Address_Translation>`_
|
||||
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
|
||||
|
||||
This option is enabled by default in the KVM kernel module
|
||||
The following section covers only Intel platform. This option is enabled by
|
||||
default in the KVM kernel module. If the file ``qemu-system-x86.conf`` does not
|
||||
exist, you have to create it.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ cat /etc/modprobe.d/qemu-system-x86.conf
|
||||
cat /etc/modprobe.d/qemu-system-x86.conf
|
||||
options kvm_intel nested=1
|
||||
|
||||
In order to be sure that this feature is enabled on your system,
|
||||
|
@ -177,6 +241,8 @@ please run:
|
|||
|
||||
.. code-block:: bash
|
||||
|
||||
sudo apt-get install --yes cpu-checker
|
||||
sudo modprobe kvm_intel
|
||||
sudo kvm-ok && cat /sys/module/kvm_intel/parameters/nested
|
||||
|
||||
The result should be:
|
||||
|
@ -191,24 +257,56 @@ The result should be:
|
|||
Environment creation via Devops + Fuel_QA or Fuel_main
|
||||
-------------------------------------------------------
|
||||
|
||||
Depending on the Fuel release, you may need a different repository. In case of
|
||||
6.0 or earlier, please use *fuel-main* repository. For 6.1 and later, the
|
||||
*fuel-qa* is required.
|
||||
Depending on the Fuel release, you may need a different repository.
|
||||
|
||||
1. Clone GIT repository
|
||||
|
||||
For 6.1 and later, the *fuel-qa* is required:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
git clone https://github.com/openstack/fuel-qa # fuel-main for 6.0 and earlier
|
||||
git clone https://github.com/openstack/fuel-qa
|
||||
cd fuel-qa/
|
||||
|
||||
2. Install requirements
|
||||
.. note:: It is recommended to use the stable branch related to the ISO version.
|
||||
For instance, with FUEL v7.0 ISO:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
git clone https://github.com/openstack/fuel-qa -b stable/7.0
|
||||
|
||||
In case of 6.0 or earlier, please use *fuel-main* repository:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
source <path>/fuel-devops-venv/bin/activate
|
||||
git clone https://github.com/openstack/fuel-main -b stable/6.0
|
||||
cd fuel-main/
|
||||
|
||||
|
||||
2. Install requirements (follow :ref:`DevOpsPyPIvenv` section for the
|
||||
WORKING_DIR variable)
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
. $WORKING_DIR/fuel-devops-venv/bin/activate
|
||||
pip install -r ./fuelweb_test/requirements.txt --upgrade
|
||||
|
||||
.. note:: A certain version of fuel-devops is specified in the
|
||||
./fuelweb_test/requirements.txt , so it will overwrite the already installed
|
||||
fuel-devops. For example, for fuel-master branch stable/6.0, there is:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
git+git://github.com/stackforge/fuel-devops.git@2.5.6
|
||||
|
||||
It is recommended to install the django tables and data after installing
|
||||
fuel-qa requiremets:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
django-admin.py syncdb --settings=devops.settings
|
||||
django-admin.py migrate devops --settings=devops.settings
|
||||
|
||||
3. Check :ref:`DevOpsConf` section
|
||||
|
||||
4. Prepare environment
|
||||
|
@ -217,35 +315,40 @@ Download Fuel ISO from
|
|||
`Nightly builds <https://ci.fuel-infra.org/view/ISO/>`_
|
||||
or build it yourself (please, refer to :ref:`building-fuel-iso`)
|
||||
|
||||
Next, you need to define several variables for the future environment
|
||||
Next, you need to define several variables for the future environment:
|
||||
* the path where is located your iso (e.g. $WORKING_DIR/fuel-community-7.0.iso)
|
||||
* the number of nodes instantiated for the environment (e.g. 5)
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
export ISO_PATH=<path_to_iso>
|
||||
export NODES_COUNT=<number_nodes>
|
||||
export ISO_PATH=$WORKING_DIR/fuel-community-7.0.iso
|
||||
export NODES_COUNT=5
|
||||
|
||||
Optionally you can specify the name of your test environment (it will
|
||||
be used as a prefix for the domains and networks names created by
|
||||
libvirt, defaults is =fuel_system_test=)
|
||||
libvirt, defaults is ``fuel_system_test``).
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
export ENV_NAME=<name_of_env>
|
||||
export ENV_NAME=fuel_system_test
|
||||
export VENV_PATH=$WORKING_DIR/fuel-devops-venv
|
||||
|
||||
.. code-block:: bash
|
||||
If you want to use separated files for snapshots you need to set env variable
|
||||
and use the following required versions:
|
||||
|
||||
export VENV_PATH=<path>/fuel-devops-venv
|
||||
* fuel-devops >= 2.9.17
|
||||
* libvirtd >= 1.2.12
|
||||
|
||||
If you want to use separated files for snapshots you need to use libvirtd in version >= 1.2.12
|
||||
and set env variable. This change will switch snapshots created by libvirt from internal
|
||||
to external mode.
|
||||
This change will switch snapshots created by libvirt from internal to external
|
||||
mode.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
export SNAPSHOTS_EXTERNAL=true
|
||||
|
||||
.. note:: External snapshots by default uses ~/.devops/snap directory to store memory dumps.
|
||||
If you want to use other directory you can set SNAPSHOTS_EXTERNAL_DIR variable.
|
||||
.. note:: External snapshots by default uses ~/.devops/snap directory to store
|
||||
memory dumps. If you want to use other directory you can set
|
||||
SNAPSHOTS_EXTERNAL_DIR variable.
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
|
@ -269,13 +372,17 @@ For more information about how tests work, read the usage information
|
|||
|
||||
./utils/jenkins/system_tests.sh -h
|
||||
|
||||
Important notes for Sahara and Murano tests
|
||||
--------------------------------------------
|
||||
Important notes for Sahara tests
|
||||
--------------------------------
|
||||
* It is not recommended to start tests without KVM.
|
||||
* For the best performance Put Sahara image
|
||||
`savanna-0.3-vanilla-1.2.1-ubuntu-13.04.qcow2 <http://sahara-files.mirantis.com/savanna-0.3-vanilla-1.2.1-ubuntu-13.04.qcow2>`_
|
||||
(md5: 9ab37ec9a13bb005639331c4275a308d) in /tmp/ before start, otherwise
|
||||
(If Internet access is available) the image will download automatically.
|
||||
|
||||
Important notes for Murano tests
|
||||
--------------------------------
|
||||
* Murano is deprecated in Fuel 9.0.
|
||||
* Put Murano image `ubuntu-murano-agent.qcow2 <http://sahara-files.mirantis.com/ubuntu-murano-agent.qcow2>`_
|
||||
(md5: b0a0fdc0b4a8833f79701eb25e6807a3) in /tmp before start.
|
||||
* Running Murano tests on instances without an Internet connection will fail.
|
||||
|
@ -311,7 +418,8 @@ To migrate from older devops, follow these steps:
|
|||
You must remove system-wide fuel-devops and switch to separate venvs with
|
||||
different versions of fuel-devops, for Fuel 6.0.x (and older) and 6.1 release.
|
||||
|
||||
Repositories 'fuel-main' and 'fuel-qa', that contain system tests, must use different Python virtual environments, for example:
|
||||
Repositories 'fuel-main' and 'fuel-qa', that contain system tests, must use
|
||||
different Python virtual environments, for example:
|
||||
|
||||
* ~/venv-nailgun-tests - used for 6.0.x and older releases. Contains version 2.5.x of fuel-devops
|
||||
* ~/venv-nailgun-tests-2.9 - used for 6.1 and above. Contains version 2.9.x of fuel-devops
|
||||
|
@ -324,7 +432,8 @@ By default, the network pool is configured as follows:
|
|||
* 10.108.0.0/16 for devops 2.5.x
|
||||
* 10.109.0.0/16 for 2.9.x
|
||||
|
||||
Please check other settings in *devops.settings*, especially the connection settings to the database.
|
||||
Please check other settings in *devops.settings*, especially the connection
|
||||
settings to the database.
|
||||
|
||||
Before using devops in Python venv, you need to `install system dependencies`_
|
||||
|
||||
|
@ -339,7 +448,7 @@ To update fuel-devops, you can use the following examples:
|
|||
echo "Python virtual env exist"
|
||||
else
|
||||
rm -rf ~/venv-nailgun-tests
|
||||
virtualenv --system-site-packages ~/venv-nailgun-tests
|
||||
virtualenv --no-site-packages ~/venv-nailgun-tests
|
||||
fi
|
||||
source ~/venv-nailgun-tests/bin/activate
|
||||
pip install -r https://raw.githubusercontent.com/openstack/fuel-main/master/fuelweb_test/requirements.txt --upgrade
|
||||
|
@ -352,7 +461,7 @@ To update fuel-devops, you can use the following examples:
|
|||
echo "Python virtual env exist"
|
||||
else
|
||||
rm -rf ~/venv-nailgun-tests-2.9
|
||||
virtualenv --system-site-packages ~/venv-nailgun-tests-2.9
|
||||
virtualenv --no-site-packages ~/venv-nailgun-tests-2.9
|
||||
fi
|
||||
source ~/venv-nailgun-tests-2.9/bin/activate
|
||||
pip install -r https://raw.githubusercontent.com/openstack/fuel-qa/master/fuelweb_test/requirements.txt --upgrade
|
||||
|
@ -369,13 +478,45 @@ To upgrade 6.1 jobs, follow these steps:
|
|||
|
||||
* make a separate Python venv, for example in ~/venv-nailgun-tests-2.9
|
||||
* install `requirements <https://github.com/openstack/fuel-qa/blob/master/fuelweb_test/requirements.txt>`_ of system tests
|
||||
* if you are using system tests on CI, please configure your CI to use new Python venv, or export path to the new Python venv in the variable VENV_PATH:
|
||||
export VENV_PATH=<path>/fuel-devops-venv-2.9
|
||||
* if you are using system tests on CI, please configure your CI to use new
|
||||
Python venv, or export path to the new Python venv in the variable
|
||||
``VENV_PATH`` (follow :ref:`DevOpsPyPIvenv` section for the WORKING_DIR
|
||||
variable):
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
export VENV_PATH=$WORKING_DIR/fuel-devops-venv-2.9
|
||||
|
||||
|
||||
Known issues
|
||||
------------
|
||||
|
||||
* Some versions of libvirt contain a bug that breaks QEMU virtual machine
|
||||
XML. You can see this when tests crush with a *libvirt: QEMU Driver error:
|
||||
unsupported configuration: host doesn't support invariant TSC*. See: `Bug 1133155 <https://bugzilla.redhat.com/show_bug.cgi?id=1133155>`_.
|
||||
unsupported configuration: host doesn't support invariant TSC*. See:
|
||||
`Bug 1133155 <https://bugzilla.redhat.com/show_bug.cgi?id=1133155>`_.
|
||||
|
||||
Workaround: upgrade libvirt to the latest version.
|
||||
|
||||
* If the same version of fuel-devops is used with several different databases
|
||||
(for example, with multiple sqlite3 databases, or with a separated database for
|
||||
each devops in different python virtual environments), there will be a
|
||||
collision between Libvirt bridge names and interfaces.
|
||||
|
||||
Workaround: use the same database for the same version of the fuel-devops.
|
||||
|
||||
- for **2.9.x**, export the following env variables:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
export DEVOPS_DB_NAME=fuel_devops
|
||||
export DEVOPS_DB_USER=fuel_devops
|
||||
export DEVOPS_DB_PASSWORD=fuel_devops
|
||||
|
||||
- for **2.5.x**, edit the dict for variable ``DATABASES``:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
vim $WORKING_DIR/fuel-devops-venv/lib/python2.7/site-packages/devops/settings.py
|
||||
|
||||
|
||||
|
|
|
@ -0,0 +1,91 @@
|
|||
Seed server
|
||||
===========
|
||||
|
||||
Seed server serves the files, such as ISO and disk images that were uploaded
|
||||
from other servers or clients. Seed can host the content by the rsync, http,
|
||||
or torrent protocol, depending on the Hiera's role configuration.
|
||||
|
||||
Deployment
|
||||
----------
|
||||
|
||||
Before you deploy Seed server, verify that you have completed the following tasks:
|
||||
|
||||
#. Deploy the Puppet Master
|
||||
#. Verify that the DNS solution works.
|
||||
#. On the Puppet Master, create ``seed``, a dedicated Hiera role from which all
|
||||
necessary services as opentracker will install.
|
||||
|
||||
.. note::
|
||||
For torrent download, do not include the ``fuel_project::apps::mirror``
|
||||
|
||||
#. On the Jenkins Master, verify that the ``seed`` Hiera role exists:
|
||||
|
||||
.. code-block:: ini
|
||||
|
||||
---
|
||||
classes:
|
||||
- 'fuel_project::common'
|
||||
- 'fuel_project::apps::seed'
|
||||
- 'fuel_project::apps::firewall'
|
||||
- 'opentracker'
|
||||
|
||||
fuel_project::apps::seed::vhost_acl_allow:
|
||||
- 10.0.0.2/32 # IP's slave example on which ISO is build
|
||||
|
||||
fuel_project::apps::seed::service_fqdn: 'seed.test.local'
|
||||
|
||||
fuel_project::apps::seed::seed_cleanup_dirs:
|
||||
- dir: '/var/www/seed/fuelweb-iso'
|
||||
ttl: 11
|
||||
pattern: 'fuel-*'
|
||||
|
||||
fuel_project::apps::firewall::rules:
|
||||
'1000 - allow ssh connections from 0.0.0.0/0':
|
||||
source: 0.0.0.0/0
|
||||
dport: 22
|
||||
proto: tcp
|
||||
action: accept
|
||||
|
||||
'1000 - allow data upload connections from temp build1.test.local':
|
||||
source: 10.0.0.2/32
|
||||
dport: 17333
|
||||
proto: tcp
|
||||
action: accept
|
||||
|
||||
'1000 - allow zabbix-agent connections from 10.0.0.200/32':
|
||||
source: 10.0.0.200/32
|
||||
dport: 10050
|
||||
proto: tcp
|
||||
action: accept
|
||||
|
||||
'1000 - allow torrent traffic within 10.0.0.0/8 network':
|
||||
source: 10.0.0.0/8
|
||||
dport: 8080
|
||||
proto: tcp
|
||||
action: accept
|
||||
|
||||
To deploy Seed server, complete the following steps:
|
||||
|
||||
#. Install base Ubuntu 14.04 with SSH service and set appropriate FQDN.
|
||||
|
||||
#. Install puppet agent package:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
apt-get update; apt-get install -y puppet
|
||||
|
||||
#. Enable puppet agent:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
puppet agent --enable
|
||||
|
||||
#. Run the deployment of the ``seed`` role:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
FACTER_ROLE=seed FACTER_LOCATION=us1 puppet agent -tvd \
|
||||
--server puppet-master.test.local --waitforcert 60
|
||||
|
||||
The last action requests the client's certificate. To continue the puppet run,
|
||||
the certificate should be signed from the Puppet Master.
|
|
@ -1,310 +0,0 @@
|
|||
.. _third-party-testing:
|
||||
|
||||
Third Party Testing
|
||||
===================
|
||||
|
||||
Overview
|
||||
--------
|
||||
|
||||
Gerrit has an event stream which can be subscribed to, using this it
|
||||
is possible to test commits against testing systems beyond those
|
||||
supplied by Fuel's Jenkins setup. It is also possible for these
|
||||
systems to feed information back into Gerrit. What's more, they
|
||||
can leave non-gating votes on Gerrit review requests.
|
||||
|
||||
There are several examples of systems that read the Gerrit event stream
|
||||
and run their own tests on the commits
|
||||
`on this page <https://wiki.openstack.org/wiki/ThirdPartySystems>`_.
|
||||
For each patch set the third party system tests, the system adds a comment
|
||||
in Gerrit with a summary of the test result and links to the test artifacts.
|
||||
|
||||
Requirements
|
||||
------------
|
||||
|
||||
* Until a third party testing system operates in a stable fashion, third
|
||||
party tests can comment on patches but not vote on them.
|
||||
|
||||
* A system can also be set up to only do '+1' reviews and leave all the
|
||||
'-1's to be manually confirmed.
|
||||
|
||||
* A third-party system may only leave one comment per patch set
|
||||
(unless it is retriggered).
|
||||
|
||||
* The maintainers are responsible for re-triggering tests when their third
|
||||
party testing system breaks.
|
||||
|
||||
* Support recheck to request re-running a test.
|
||||
|
||||
* Support the following syntaxes: ``recheck``.
|
||||
* Recheck means recheck everything. A single recheck comment should
|
||||
re-trigger all testing systems.
|
||||
|
||||
* Publish contact information for the maintainers.
|
||||
|
||||
* All accounts must be previously set by posting launchpad bug to add your
|
||||
system.
|
||||
* Maintainers are encouraged to be in IRC regularly to make it
|
||||
faster to contact them.
|
||||
|
||||
* Include a public link to all test artifacts to make debugging failed tests
|
||||
easier (using a dns name over a hardcoded ip is recommended).
|
||||
This should include:
|
||||
|
||||
* Environment details
|
||||
|
||||
* This must include a utc timestamp of the test run
|
||||
* Test configuration
|
||||
|
||||
* Skipped tests
|
||||
* logs should include a trace of the commands used
|
||||
* OpenStack logs
|
||||
* Tempest logs (including ``testr_results.html.gz``)
|
||||
|
||||
* logs must be browsable; logs requiring download, installation or login
|
||||
to access are not acceptable
|
||||
|
||||
.. note:: All test artifacts must be retained for one month.
|
||||
|
||||
Reading the Event Stream
|
||||
------------------------
|
||||
|
||||
It is possible to use ssh to connect to ``review.fuel-infra.org`` on port 29418
|
||||
with your ssh key if you have a normal reviewer account in Gerrit.
|
||||
|
||||
This will give you a real-time JSON stream of events happening inside Gerrit.
|
||||
For example:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ ssh -p 29418 USERNAME@review.fuel-infra.org gerrit stream-events
|
||||
|
||||
Will give a stream with an output like this (line breaks and
|
||||
indentation added in this document for readability, the real JSON will
|
||||
be all one line per event):
|
||||
|
||||
.. code-block:: javascript
|
||||
|
||||
{"type":"comment-added","change":
|
||||
{"project":"openstack/keystone","branch":"stable/essex","topic":"bug/969088","id":"I18ae38af62b4c2b2423e20e436611fc30f844ae1","number":"7385","subject":"Make import_nova_auth only create roles which don\u0027t already exist","owner":
|
||||
{"name":"Chuck Short","email":"chuck.short@canonical.com","username":"zulcss"},"url":"https://review.fuel-infra.org/7385"},
|
||||
"patchSet":
|
||||
{"number":"1","revision":"aff45d69a73033241531f5e3542a8d1782ddd859","ref":"refs/changes/85/7385/1","uploader":
|
||||
{"name":"Chuck Short","email":"chuck.short@canonical.com","username":"zulcss"},
|
||||
"createdOn":1337002189},
|
||||
"author":
|
||||
{"name":"Mark McLoughlin","email":"markmc@redhat.com","username":"markmc"},
|
||||
"approvals":
|
||||
[{"type":"CRVW","description":"Code Review","value":"2"},{"type":"APRV","description":"Approved","value":"0"}],
|
||||
"comment":"Hmm, I actually thought this was in Essex already.\n\nIt\u0027s a pretty annoying little issue for folks migrating for nova auth. Fix is small and pretty safe. Good choice for backporting"}
|
||||
|
||||
For most purposes you will want to trigger on ``patchset-created`` for when a
|
||||
new patchset has been uploaded.
|
||||
|
||||
Further documentation on how to use the events stream can be found in `Gerrit's stream event documentation page <http://gerrit-documentation.googlecode.com/svn/Documentation/2.3/cmd-stream-events.html>`_.
|
||||
|
||||
Posting Result To Gerrit
|
||||
------------------------
|
||||
|
||||
External testing systems can give non-gating votes to Gerrit by means
|
||||
of a -1/+1 verify vote. Comments should also be provided to explain what kind
|
||||
of test failed. We do also ask that the comments contain public links to the
|
||||
failure so that the developer can see what caused the failure.
|
||||
|
||||
An example of how to post this is as follows:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
$ ssh -p 29418 USERNAME@review.fuel-infra.org gerrit review -m '"Test failed on MegaTestSystem <http://megatestsystem.org/tests/1234>"' --verified=-1 c0ff33
|
||||
|
||||
In this example ``c0ff33`` is the commit ID for the review. You can
|
||||
set the verified to either `-1` or `+1` depending on whether or not it
|
||||
passed the tests.
|
||||
|
||||
Further documentation on the `review` command in Gerrit can be found in the `Gerrit review documentation page <http://gerrit-documentation.googlecode.com/svn/Documentation/2.3/cmd-review.html>`_.
|
||||
|
||||
We do suggest cautious testing of these systems and have a development Gerrit
|
||||
setup to test on if required. In SmokeStack's case all failures are manually
|
||||
reviewed before getting pushed to OpenStack, while this may not scale it is
|
||||
advisable during the initial testing of the setup.
|
||||
|
||||
There are several triggers that gerrit will match to alter the
|
||||
formatting of comments. The raw regular expressions can be seen in
|
||||
`gerrit.pp <https://git.openstack.org/cgit/openstack-infra/system-config/tree/modules/openstack_project/manifests/gerrit.pp>`_.
|
||||
For example, to have your test results formatted in the same manner as
|
||||
the upstream Jenkins results, use a template for each result matching::
|
||||
|
||||
* test-name-no-spaces http://link.to/result : [SUCCESS|FAILURE] some comment about the test
|
||||
|
||||
.. _request-account-label:
|
||||
|
||||
Creating a Service Account
|
||||
--------------------------
|
||||
|
||||
In order to post comments as a Third Party CI System and eventually verify
|
||||
your build status on Gerrit patches, you will need a dedicated Gerrit
|
||||
CI account. You will need to create this account in our OpenID provider
|
||||
`Launchpad <https://launchpad.net>`_. You may already have an existing
|
||||
personal account in Launchpad, but you should create a new and entirely
|
||||
separate account for this purpose.
|
||||
|
||||
Once you have created this account with the OpenID provider you can log
|
||||
into Gerrit with that new account as you would with your normal user
|
||||
account. Once logged in you will need to do several things:
|
||||
|
||||
1. Set an SSH username at https://review.fuel-infra.org/#/settings/ if
|
||||
it isn't already set. This is the username your CI system will use to
|
||||
SSH to Gerrit in order to read the event stream.
|
||||
|
||||
2. Set the account's fullname at https://review.fuel-infra.org/#/settings/contact
|
||||
This name should follow a few rules in order to make it clear in Gerrit
|
||||
comments what this CI system exists to test. The name should have three
|
||||
pieces ``Organization`` ``Product/technology`` ``CI designator``. The
|
||||
organization value should be your company name or other organization
|
||||
affiliation. Product/technology should describe the product or technology
|
||||
you are testing in conjunction with OpenStack. This should be the name of
|
||||
a component which cannot be tested in the official OpenStack
|
||||
infrastructure (requires particular physical hardware, proprietary
|
||||
software, some hypervisor feature not available in public clouds,
|
||||
et cetera). Note this should not be the name of an OpenStack project but
|
||||
rather the thing you are testing with OpenStack projects. And finally
|
||||
the CI designator is used to denote this is a CI system so that automatic
|
||||
Gerrit comment parsers can filter these comments out. This value should
|
||||
be ``CI`` for most CI systems but can be ``Bot`` if you are not
|
||||
performing continuous integration. An example of a proper name would be
|
||||
something like ``IBM DB2 CI``.
|
||||
|
||||
3. Add the SSH public key you will be using to the Gerrit account at
|
||||
https://review.fuel-infra.org/#/settings/ssh-keys You can generate an
|
||||
ssh key using ``ssh-keygen``. You want to give Gerrit the contents of
|
||||
the generated id_rsa.pub file.
|
||||
|
||||
Once you have done this you will have everything you need to comment on
|
||||
Gerrit changes from our CI system but you will not be able to vote +/-1
|
||||
Verified on changes. To get voting rights you will need to get the release
|
||||
group of the project you are testing to add you to their project specific
|
||||
<project>-ci group. Please contact the project in question when you are
|
||||
ready to start voting and they can add you to this group.
|
||||
|
||||
|
||||
|
||||
The Jenkins Gerrit Trigger Plugin Way
|
||||
-------------------------------------
|
||||
|
||||
There is a Gerrit Trigger plugin for Jenkins which automates all of the
|
||||
processes described in this document. So if your testing system is Jenkins
|
||||
based you can use it to simplify things. You will still need an account to do
|
||||
this as described in the :ref:`request-account-label` section above.
|
||||
|
||||
The Gerrit Trigger plugin for Jenkins can be found on `the Jenkins
|
||||
repository`_. You can install it using the Advanced tab in the
|
||||
Jenkins Plugin Manager.
|
||||
|
||||
.. _the Jenkins repository: http://repo.jenkins-ci.org/repo/com/sonyericsson/hudson/plugins/gerrit/gerrit-trigger/
|
||||
|
||||
Once installed Jenkins will have a new `Gerrit Trigger` option in the `Manage
|
||||
Jenkins` menu. This should be given the following options::
|
||||
|
||||
Hostname: review.fuel-infra.org
|
||||
Frontend URL: https://review.fuel-infra.org/
|
||||
SSH Port: 29418
|
||||
Username: (the Gerrit user)
|
||||
SSH Key File: (path to the user SSH key)
|
||||
|
||||
Verify
|
||||
------
|
||||
Started: 0
|
||||
Successful: 1
|
||||
Failed: -1
|
||||
Unstable: 0
|
||||
|
||||
Code Review
|
||||
-----------
|
||||
Started: 0
|
||||
Successful: 0
|
||||
Failed: 0
|
||||
Unstable: 0
|
||||
|
||||
(under Advanced Button):
|
||||
|
||||
Stated: (blank)
|
||||
Successful: gerrit approve <CHANGE>,<PATCHSET> --message 'Build Successful <BUILDS_STATS>' --verified <VERIFIED> --code-review <CODE_REVIEW>
|
||||
Failed: gerrit approve <CHANGE>,<PATCHSET> --message 'Build Failed <BUILDS_STATS>' --verified <VERIFIED> --code-review <CODE_REVIEW>
|
||||
Unstable: gerrit approve <CHANGE>,<PATCHSET> --message 'Build Unstable <BUILDS_STATS>' --verified <VERIFIED> --code-review <CODE_REVIEW>
|
||||
|
||||
Note that it is useful to include something in the messages about what testing
|
||||
system is supplying these messages.
|
||||
|
||||
When creating jobs in Jenkins you will have the option to add triggers. You
|
||||
should configure as follows::
|
||||
|
||||
Trigger on Patchset Uploaded: ticked
|
||||
(the rest unticked)
|
||||
|
||||
Type: Plain
|
||||
Pattern: openstack/project-name (where project-name is the name of the project)
|
||||
Branches:
|
||||
Type: Path
|
||||
Pattern: **
|
||||
|
||||
This job will now automatically trigger when a new patchset is
|
||||
uploaded and will report the results to Gerrit automatically.
|
||||
|
||||
Testing your CI setup
|
||||
---------------------
|
||||
|
||||
You can use the ``fuel-external/test`` project to test your external CI
|
||||
infrastructure with OpenStack's Gerrit. By using the sandbox project you
|
||||
can test your CI system without affecting regular OpenStack reviews.
|
||||
|
||||
Once you confirm your CI system works as you expect, change your
|
||||
configuration of the gerrit trigger plugin or zuul to subscribe to gerrit
|
||||
events from your target project.
|
||||
|
||||
Permissions on your Third Party System
|
||||
--------------------------------------
|
||||
|
||||
When you create your CI account it will have no special permissions.
|
||||
This means it can comment on changes but generally not vote +/-1
|
||||
Verified on any changes. The exception to this is on the
|
||||
``fuel-external/test`` project. Any account is able to vote +/-1
|
||||
Verified on that account and it provides a way to test your CI's voting
|
||||
abilities before you vote on other projects.
|
||||
|
||||
.. _openstack-dev/ci-sandbox: https://review.fuel-infra.org/[ADDME]
|
||||
|
||||
The Fuel Infrastructure team disables mis-behaving third-party ci
|
||||
accounts at its discretion. This documentation endeavours to outline specific
|
||||
circumstances that may lead to an account being disabled. There have been
|
||||
times when third-party ci systems behave in ways we didn't envision
|
||||
and therefore were unable to document prior to the event. If your
|
||||
third-party ci system has been disabled, please don't hesitate to contact
|
||||
devops team.
|
||||
|
||||
In order to get your Third Pary CI account to have voting permissions on
|
||||
repos in gerrit in addition to ``fuel-external/test`` you have a greater
|
||||
chance of success if you follow these steps:
|
||||
|
||||
* Set up your system and test it according to "Testing your CI setup" outlined
|
||||
above (this will create a history of activity associated with your account
|
||||
which will be evaluated when you apply for voting permissions).
|
||||
|
||||
* Post comments, that adhere to the "Requirements" listed above, that
|
||||
demonstrate the format for your system communication to the repos
|
||||
you want your system to test.
|
||||
|
||||
* Once your Third Party Account has a history on gerrit so that others
|
||||
can evaluate your format for comments, and the stability of your
|
||||
voting pattern (in the sandbox repo):
|
||||
|
||||
* send an email to the fuel-devops mailing list nominating your
|
||||
system for voting permissions
|
||||
|
||||
* fuel-devops@mirantis.com
|
||||
|
||||
* present your account history
|
||||
* address any questions and concerns with your system
|
||||
|
||||
* If the members of the program you want voting permissions from agree
|
||||
your system should be able to vote, the release group for that program
|
||||
or project can add you to the <project>-ci group specific to that
|
||||
program/project.
|
|
@ -0,0 +1,58 @@
|
|||
Zabbix server
|
||||
=============
|
||||
|
||||
Zabbix is an open source tool designed for servers, services, and network
|
||||
monitoring.
|
||||
|
||||
Deployment
|
||||
----------
|
||||
|
||||
Zabbix can be deployed on a virtual machine with at least 1 GB RAM and 1 CPU.
|
||||
The HW requirements vary and depend on the potential Zabbix's server load and
|
||||
place of the database engine, for example, MySQL, regardless of whether it is
|
||||
located on the same host or on a dedicated one.
|
||||
|
||||
The puppet-manifests repository contains an example ``zbxserver`` role
|
||||
that can be used as a starting point for Zabbix server deployment.
|
||||
|
||||
To deploy Zabbix, perform the following tasks:
|
||||
|
||||
#. Install base Ubuntu 14.04 with SSH service and set appropriate FQDN.
|
||||
|
||||
#. Install puppet agent package:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
apt-get update; apt-get install -y puppet
|
||||
|
||||
#. Enable puppet agent:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
puppet agent --enable
|
||||
|
||||
#. Run the Jenkins Master deployment:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
FACTER_ROLE=zbxserver FACTER_LOCATION=us1 puppet agent -tvd \
|
||||
--server puppet-master.test.local --waitforcert 60
|
||||
|
||||
The last action requests the client's certificate. To continue the puppet run,
|
||||
the certificate should be signed from the Puppet Master.
|
||||
|
||||
Import templates
|
||||
----------------
|
||||
|
||||
You can import templates and related items, such as triggers, that comes from
|
||||
Fuel's Zabbix production server. To do this, complete the following tasks:
|
||||
|
||||
#. Clone the ``tools/zabbix-maintenance`` repository:
|
||||
|
||||
.. code-block:: console
|
||||
|
||||
git clone https://review.fuel-infra.org/tools/zabbix-maintenance
|
||||
|
||||
#. In the Zabbix server web UI, navigate to :guilabel:`Configuration ->
|
||||
Templates`. Import the required template file using the :guilabel:`Import`
|
||||
button.
|
Loading…
Reference in New Issue