Update unit test container setup and instructions
Update the TESTING.rst file to reflect that just installing and starting zookeeperd is no longer sufficient now that we require TLS and auth for the connection, as well as running database servers. Suggest the container-based setup script instead. Also improve that script to allow it to be invoked as a normal user, with root command escalation tool choice (e.g. "sudo") supplied through a ROOTCMD environment variable, so that things created inside the git worktree like the CA don't end up root-owned. Related, the tox-docker plugin previously suggested in the document is no longer a viable option for the same reasons, so clean up the plumbing for it in tox.ini as well. Change-Id: Iac32799425a5bd4b1bdbf56f34a2310241ac4499
This commit is contained in:
parent
68684d519e
commit
90d7dcb1ed
32
TESTING.rst
32
TESTING.rst
|
@ -1,34 +1,30 @@
|
||||||
===========================
|
============
|
||||||
Testing Your OpenStack Code
|
Testing Zuul
|
||||||
===========================
|
============
|
||||||
------------
|
------------
|
||||||
A Quickstart
|
A Quickstart
|
||||||
------------
|
------------
|
||||||
|
|
||||||
This is designed to be enough information for you to run your first tests.
|
This is designed to be enough information for you to run your first tests on
|
||||||
Detailed information on testing can be found here: https://wiki.openstack.org/wiki/Testing
|
an Ubuntu 20.04 (or later) host.
|
||||||
|
|
||||||
*Install pip*::
|
*Install pip*::
|
||||||
|
|
||||||
[apt-get | yum] install python-pip
|
sudo apt-get install python3-pip
|
||||||
|
|
||||||
More information on pip here: http://www.pip-installer.org/en/latest/
|
More information on pip here: http://www.pip-installer.org/en/latest/
|
||||||
|
|
||||||
*Use pip to install tox and tox-docker*::
|
*Use pip to install tox*::
|
||||||
|
|
||||||
pip install tox tox-docker
|
pip install tox
|
||||||
|
|
||||||
As of zuul v3, a running zookeeper is required to execute tests. Using the
|
A running zookeeper is required to execute tests, but it also needs to be
|
||||||
``-docker`` suffixed commands will ensure this is started automatically by tox,
|
configured for TLS and a certificate authority set up to handle socket
|
||||||
but if you do not wish to use this:
|
authentication. Because of these complexities, it's recommended to use a
|
||||||
|
helper script to set up these dependencies, as well as a database servers::
|
||||||
|
|
||||||
*Install zookeeper*::
|
sudo apt-get install docker-compose # or podman-compose if preferred
|
||||||
|
ROOTCMD=sudo tools/test-setup-docker.sh
|
||||||
[apt-get | yum] install zookeeperd
|
|
||||||
|
|
||||||
*Start zookeeper*::
|
|
||||||
|
|
||||||
service zookeeper start
|
|
||||||
|
|
||||||
.. note:: Installing and bulding javascript is not required, but tests that
|
.. note:: Installing and bulding javascript is not required, but tests that
|
||||||
depend on the javascript assets having been built will be skipped
|
depend on the javascript assets having been built will be skipped
|
||||||
|
|
|
@ -1,6 +1,14 @@
|
||||||
#!/bin/bash
|
#!/bin/bash
|
||||||
|
|
||||||
set -eu
|
# This runs ZooKeeper and databases in docker containers, which are
|
||||||
|
# required for tests.
|
||||||
|
|
||||||
|
# This setup needs to be run as a user that can run docker or podman, or by
|
||||||
|
# setting $ROOTCMD to a user substitution tool like "sudo" in the calling
|
||||||
|
# environment.
|
||||||
|
|
||||||
|
set -xeu
|
||||||
|
ROOTCMD=${ROOTCMD:-}
|
||||||
|
|
||||||
cd $(dirname $0)
|
cd $(dirname $0)
|
||||||
SCRIPT_DIR="$(pwd)"
|
SCRIPT_DIR="$(pwd)"
|
||||||
|
@ -29,9 +37,9 @@ fi
|
||||||
MYSQL="${DOCKER} exec zuul-test-mysql mysql -u root -pinsecure_worker"
|
MYSQL="${DOCKER} exec zuul-test-mysql mysql -u root -pinsecure_worker"
|
||||||
|
|
||||||
if [ "${COMPOSE}" == "docker-compose" ]; then
|
if [ "${COMPOSE}" == "docker-compose" ]; then
|
||||||
docker-compose rm -sf
|
${ROOTCMD} docker-compose rm -sf
|
||||||
else
|
else
|
||||||
podman-compose down
|
${ROOTCMD} podman-compose down
|
||||||
fi
|
fi
|
||||||
|
|
||||||
CA_DIR=$SCRIPT_DIR/ca
|
CA_DIR=$SCRIPT_DIR/ca
|
||||||
|
@ -39,15 +47,14 @@ CA_DIR=$SCRIPT_DIR/ca
|
||||||
mkdir -p $CA_DIR
|
mkdir -p $CA_DIR
|
||||||
$SCRIPT_DIR/zk-ca.sh $CA_DIR zuul-test-zookeeper
|
$SCRIPT_DIR/zk-ca.sh $CA_DIR zuul-test-zookeeper
|
||||||
|
|
||||||
export USER_ID=$(id -u)
|
${ROOTCMD} USER_ID=$(id -u) ${COMPOSE} up -d
|
||||||
${COMPOSE} up -d
|
|
||||||
|
|
||||||
echo "Waiting for mysql"
|
echo "Waiting for mysql"
|
||||||
timeout 30 bash -c "until ${MYSQL} -e 'show databases'; do sleep 0.5; done"
|
timeout 30 bash -c "until ${ROOTCMD} ${MYSQL} -e 'show databases'; do sleep 0.5; done"
|
||||||
echo
|
echo
|
||||||
|
|
||||||
echo "Setting up permissions for zuul tests"
|
echo "Setting up permissions for zuul tests"
|
||||||
${MYSQL} -e "GRANT ALL PRIVILEGES ON *.* TO 'openstack_citest'@'%' identified by 'openstack_citest' WITH GRANT OPTION;"
|
${ROOTCMD} ${MYSQL} -e "GRANT ALL PRIVILEGES ON *.* TO 'openstack_citest'@'%' identified by 'openstack_citest' WITH GRANT OPTION;"
|
||||||
${MYSQL} -u openstack_citest -popenstack_citest -e "SET default_storage_engine=MYISAM; DROP DATABASE IF EXISTS openstack_citest; CREATE DATABASE openstack_citest CHARACTER SET utf8;"
|
${ROOTCMD} ${MYSQL} -u openstack_citest -popenstack_citest -e "SET default_storage_engine=MYISAM; DROP DATABASE IF EXISTS openstack_citest; CREATE DATABASE openstack_citest CHARACTER SET utf8;"
|
||||||
|
|
||||||
echo "Finished"
|
echo "Finished"
|
||||||
|
|
9
tox.ini
9
tox.ini
|
@ -1,11 +1,8 @@
|
||||||
[tox]
|
[tox]
|
||||||
minversion = 3.2
|
minversion = 3.2
|
||||||
skipsdist = True
|
skipsdist = True
|
||||||
envlist = linters,py3{-docker}
|
envlist = linters,py3
|
||||||
ignore_basepython_conflict = True
|
ignore_basepython_conflict = True
|
||||||
# TODO(stephenfin): It would be good to set 'requires = tox-docker', but doing
|
|
||||||
# so borks the tools/pip.sh wrapper we're using here and probably isn't what
|
|
||||||
# we want in the gate :(
|
|
||||||
|
|
||||||
[testenv]
|
[testenv]
|
||||||
basepython = python3
|
basepython = python3
|
||||||
|
@ -19,7 +16,6 @@ setenv =
|
||||||
SQLALCHEMY_WARN_20={env:SQLALCHEMY_WARN_20:1}
|
SQLALCHEMY_WARN_20={env:SQLALCHEMY_WARN_20:1}
|
||||||
PYTHONWARNINGS=always::DeprecationWarning:zuul.driver.sql.sqlconnection,always::DeprecationWarning:tests.base,always::DeprecationWarning:tests.unit.test_database,always::DeprecationWarning:zuul.driver.sql.alembic.env,always::DeprecationWarning:zuul.driver.sql.alembic.script
|
PYTHONWARNINGS=always::DeprecationWarning:zuul.driver.sql.sqlconnection,always::DeprecationWarning:tests.base,always::DeprecationWarning:tests.unit.test_database,always::DeprecationWarning:zuul.driver.sql.alembic.env,always::DeprecationWarning:zuul.driver.sql.alembic.script
|
||||||
passenv =
|
passenv =
|
||||||
DOCKER_*
|
|
||||||
NODEPOOL_ZK_HOST
|
NODEPOOL_ZK_HOST
|
||||||
OS_LOG_CAPTURE
|
OS_LOG_CAPTURE
|
||||||
OS_LOG_DEFAULTS
|
OS_LOG_DEFAULTS
|
||||||
|
@ -40,8 +36,6 @@ whitelist_externals = bash
|
||||||
deps =
|
deps =
|
||||||
-r{toxinidir}/requirements.txt
|
-r{toxinidir}/requirements.txt
|
||||||
-r{toxinidir}/test-requirements.txt
|
-r{toxinidir}/test-requirements.txt
|
||||||
docker =
|
|
||||||
docker: zookeeper:3.4
|
|
||||||
commands =
|
commands =
|
||||||
bash -c 'stestr run --slowest --concurrency=`python -c "import multiprocessing; print(max(int(multiprocessing.cpu_count()-1),1))"` {posargs}'
|
bash -c 'stestr run --slowest --concurrency=`python -c "import multiprocessing; print(max(int(multiprocessing.cpu_count()-1),1))"` {posargs}'
|
||||||
|
|
||||||
|
@ -96,7 +90,6 @@ deps =
|
||||||
|
|
||||||
[testenv:remote]
|
[testenv:remote]
|
||||||
passenv =
|
passenv =
|
||||||
DOCKER_*
|
|
||||||
NODEPOOL_ZK_HOST
|
NODEPOOL_ZK_HOST
|
||||||
OS_LOG_CAPTURE
|
OS_LOG_CAPTURE
|
||||||
OS_LOG_DEFAULTS
|
OS_LOG_DEFAULTS
|
||||||
|
|
Loading…
Reference in New Issue