Migrate legacy jobs to project repository
Brings the legacy functional test jobs into the project repository, rewritten for zuul3. The base functional test class has been refactored to use clouds.yaml, and the functional-swift test has been refactored to use keystoneauth1. The devstack hooks have been removed as they are not necessary with zuul3. Needed-By: I2a5ff2ee29e2fb8a730406f9f0e6a450b96c85b8 Needed-By: I96600383c072e6d2926f5ff5b6a51057e53d35fd Change-Id: Iecea4375d1c4832b020b542ca4188b02444325ca
This commit is contained in:
parent
82a26a90e2
commit
33e6b01642
|
@ -0,0 +1,52 @@
|
|||
- job:
|
||||
name: glance_store-dsvm-functional-base
|
||||
parent: devstack-tox-functional
|
||||
description: |
|
||||
Base job for devstack-based functional tests for glance_store
|
||||
|
||||
Can only be used directly if a 'functional' testenv is defined
|
||||
in tox.ini (which currently is not the case).
|
||||
required-projects:
|
||||
- openstack/glance_store
|
||||
timeout: 4200
|
||||
vars:
|
||||
devstack_localrc:
|
||||
LIBS_FROM_GIT: glance_store
|
||||
devstack_services:
|
||||
# turn off ceilometer
|
||||
ceilometer-acentral: false
|
||||
ceilometer-acompute: false
|
||||
ceilometer-alarm-evaluator: false
|
||||
ceilometer-alarm-notifier: false
|
||||
ceilometer-anotification: false
|
||||
ceilometer-api: false
|
||||
ceilometer-collector: false
|
||||
# Hardcode glance_store path so the job can be run on glance patches
|
||||
zuul_work_dir: src/git.openstack.org/openstack/glance_store
|
||||
|
||||
- job:
|
||||
name: glance_store-dsvm-functional-filesystem
|
||||
parent: glance_store-dsvm-functional-base
|
||||
vars:
|
||||
tox_envlist: functional-filesystem
|
||||
|
||||
- job:
|
||||
name: glance_store-dsvm-functional-swift
|
||||
parent: glance_store-dsvm-functional-base
|
||||
required-projects:
|
||||
- openstack/swift
|
||||
vars:
|
||||
tox_envlist: functional-swift
|
||||
devstack_services:
|
||||
s-account: true
|
||||
s-container: true
|
||||
s-object: true
|
||||
s-proxy: true
|
||||
devstack_localrc:
|
||||
ENABLE_IDENTITY_V2: True
|
||||
|
||||
- project:
|
||||
experimental:
|
||||
jobs:
|
||||
- glance_store-dsvm-functional-filesystem
|
||||
- glance_store-dsvm-functional-swift
|
|
@ -1,9 +0,0 @@
|
|||
[tests]
|
||||
stores = file,swift
|
||||
|
||||
[admin]
|
||||
user = admin:admin
|
||||
key = secretadmin
|
||||
auth_version = 2
|
||||
auth_address = http://localhost:35357/v2.0
|
||||
region = RegionOne
|
|
@ -0,0 +1,78 @@
|
|||
===============================
|
||||
glance_store functional testing
|
||||
===============================
|
||||
|
||||
Writing functional tests for glance_store
|
||||
-----------------------------------------
|
||||
|
||||
The functional tests verify glance_store against a "live" backend. The tests
|
||||
are isolated so that a development environment doesn't have to all the backends
|
||||
available, just the particular backend whose driver the developer is working
|
||||
on.
|
||||
|
||||
To add tests for a driver:
|
||||
|
||||
1. Create a new module in ``glance_store/tests/functional`` with the driver
|
||||
name.
|
||||
|
||||
2. Create a submodule ``test_functional_{driver-name}`` containing a class
|
||||
that inherits from ``glance_store.tests.functional.BaseFunctionalTests``.
|
||||
The actual tests are in the ``BaseFunctionalTests`` class. The test
|
||||
classes for each driver do any extra setup/teardown necessary for that
|
||||
particular driver. (The idea is that all the backends should be able to
|
||||
pass the same tests.)
|
||||
|
||||
3. Add a testenv to ``tox.ini`` named ``functional-{driver-name}`` so
|
||||
that tox can run the tests for your driver. (Use the other functional
|
||||
testenvs as examples.)
|
||||
|
||||
4. If your driver is well-supported by devstack, it shouldn't be too hard
|
||||
to set up a gate job for the functional tests in ``.zuul.yaml``. (Use
|
||||
the other jobs defined in that file as examples.)
|
||||
|
||||
|
||||
Configuration
|
||||
-------------
|
||||
|
||||
The functional tests have been designed to work well with devstack so that
|
||||
we can run them in the gate. Thus the tests expect to find a yaml file
|
||||
containing valid credentials just like the ``clouds.yaml`` file created by
|
||||
devstack in the ``/etc/openstack`` directory. The test code knows where
|
||||
to find it, so if you're using devstack, you should be all set.
|
||||
|
||||
If you are not using devstack you should create a yaml file with the following
|
||||
format::
|
||||
|
||||
clouds:
|
||||
devstack-admin:
|
||||
auth:
|
||||
auth_url: https://172.16.132.143/identity
|
||||
password: example
|
||||
project_domain_id: default
|
||||
project_name: admin
|
||||
user_domain_id: default
|
||||
username: admin
|
||||
identity_api_version: '3'
|
||||
region_name: RegionOne
|
||||
volume_api_version: '2'
|
||||
|
||||
The clouds.yaml format allows for a set of credentials to be defined for each
|
||||
named cloud. By default, the tests will use the credentials for the cloud
|
||||
named **devstack-admin** (that's the cloud shown in the example above). You
|
||||
can change which cloud is read from ``clouds.yaml`` by exporting the
|
||||
environment variable ``OS_TEST_GLANCE_STORE_FUNC_TEST_CLOUD`` set to the name
|
||||
of the cloud you want used.
|
||||
|
||||
Where to put clouds.yaml
|
||||
------------------------
|
||||
|
||||
The tests will look for a file named ``clouds.yaml`` in the
|
||||
following locations (in this order, first found wins):
|
||||
|
||||
* current directory
|
||||
* ~/.config/openstack
|
||||
* /etc/openstack
|
||||
|
||||
You may also set the environment variable ``OS_CLIENT_CONFIG_FILE``
|
||||
to the absolute pathname of a file and that location will be
|
||||
inserted at the front of the search list.
|
|
@ -13,13 +13,11 @@
|
|||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
try:
|
||||
import configparser as ConfigParser
|
||||
except ImportError:
|
||||
from six.moves import configparser as ConfigParser
|
||||
from io import BytesIO
|
||||
from os import environ
|
||||
|
||||
import glance_store
|
||||
import os_client_config
|
||||
from oslo_config import cfg
|
||||
import testtools
|
||||
|
||||
|
@ -37,18 +35,33 @@ class Base(testtools.TestCase):
|
|||
def __init__(self, driver_name, *args, **kwargs):
|
||||
super(Base, self).__init__(*args, **kwargs)
|
||||
self.driver_name = driver_name
|
||||
self.config = ConfigParser.RawConfigParser()
|
||||
self.config.read('functional_testing.conf')
|
||||
|
||||
# check whether a particular cloud should be used
|
||||
cloud = environ.get('OS_TEST_GLANCE_STORE_FUNC_TEST_CLOUD',
|
||||
'devstack-admin')
|
||||
creds = os_client_config.OpenStackConfig().get_one_cloud(
|
||||
cloud=cloud)
|
||||
auth = creds.get_auth_args()
|
||||
self.username = auth["username"]
|
||||
self.password = auth["password"]
|
||||
self.project_name = auth["project_name"]
|
||||
self.user_domain_id = auth["user_domain_id"]
|
||||
self.project_domain_id = auth["project_domain_id"]
|
||||
self.keystone_version = creds.get_api_version('identity')
|
||||
self.cinder_version = creds.get_api_version('volume')
|
||||
self.region_name = creds.get_region_name()
|
||||
# auth_url in devstack clouds.yaml is unversioned
|
||||
if auth["auth_url"].endswith('/v3'):
|
||||
self.auth_url = auth["auth_url"]
|
||||
else:
|
||||
self.auth_url = '{}/v3'.format(auth["auth_url"])
|
||||
|
||||
# finally, load the configuration options
|
||||
glance_store.register_opts(CONF)
|
||||
|
||||
def setUp(self):
|
||||
super(Base, self).setUp()
|
||||
|
||||
stores = self.config.get('tests', 'stores').split(',')
|
||||
if self.driver_name not in stores:
|
||||
self.skipTest('Not running %s store tests' % self.driver_name)
|
||||
|
||||
CONF.set_override('stores', [self.driver_name], group='glance_store')
|
||||
CONF.set_override('default_store',
|
||||
self.driver_name,
|
||||
|
|
|
@ -1,33 +0,0 @@
|
|||
#!/bin/bash
|
||||
#
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
# This script is executed inside gate_hook function in devstack gate.
|
||||
|
||||
# NOTE(NiallBunting) The store to test is passed in here from the
|
||||
# project config.
|
||||
GLANCE_STORE_DRIVER=${1:-swift}
|
||||
|
||||
ENABLED_SERVICES+=",key,glance"
|
||||
|
||||
case $GLANCE_STORE_DRIVER in
|
||||
swift)
|
||||
ENABLED_SERVICES+=",s-proxy,s-account,s-container,s-object,"
|
||||
;;
|
||||
esac
|
||||
|
||||
export GLANCE_STORE_DRIVER
|
||||
|
||||
export ENABLED_SERVICES
|
||||
|
||||
$BASE/new/devstack-gate/devstack-vm-gate.sh
|
|
@ -1,79 +0,0 @@
|
|||
#!/bin/bash -xe
|
||||
|
||||
# Licensed under the Apache License, Version 2.0 (the "License"); you may
|
||||
# not use this file except in compliance with the License. You may obtain
|
||||
# a copy of the License at
|
||||
#
|
||||
# http://www.apache.org/licenses/LICENSE-2.0
|
||||
#
|
||||
# Unless required by applicable law or agreed to in writing, software
|
||||
# distributed under the License is distributed on an "AS IS" BASIS, WITHOUT
|
||||
# WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the
|
||||
# License for the specific language governing permissions and limitations
|
||||
# under the License.
|
||||
|
||||
# This script is executed inside post_test_hook function in devstack gate.
|
||||
|
||||
set -xe
|
||||
|
||||
export GLANCE_STORE_DIR="$BASE/new/glance_store"
|
||||
SCRIPTS_DIR="/usr/os-testr-env/bin/"
|
||||
GLANCE_STORE_DRIVER=${1:-swift}
|
||||
|
||||
function generate_test_logs {
|
||||
local path="$1"
|
||||
# Compress all $path/*.txt files and move the directories holding those
|
||||
# files to /opt/stack/logs. Files with .log suffix have their
|
||||
# suffix changed to .txt (so browsers will know to open the compressed
|
||||
# files and not download them).
|
||||
if [ -d "$path" ]
|
||||
then
|
||||
sudo find $path -iname "*.log" -type f -exec mv {} {}.txt \; -exec gzip -9 {}.txt \;
|
||||
sudo mv $path/* /opt/stack/logs/
|
||||
fi
|
||||
}
|
||||
|
||||
function generate_testr_results {
|
||||
if [ -f .testrepository/0 ]; then
|
||||
# Give job user rights to access tox logs
|
||||
sudo -H -u "$owner" chmod o+rw .
|
||||
sudo -H -u "$owner" chmod o+rw -R .testrepository
|
||||
|
||||
if [[ -f ".testrepository/0" ]] ; then
|
||||
"subunit-1to2" < .testrepository/0 > ./testrepository.subunit
|
||||
$SCRIPTS_DIR/subunit2html ./testrepository.subunit testr_results.html
|
||||
gzip -9 ./testrepository.subunit
|
||||
gzip -9 ./testr_results.html
|
||||
sudo mv ./*.gz /opt/stack/logs/
|
||||
fi
|
||||
|
||||
fi
|
||||
}
|
||||
|
||||
owner=jenkins
|
||||
|
||||
# Get admin credentials
|
||||
cd $BASE/new/devstack
|
||||
source openrc admin admin
|
||||
|
||||
# Go to the glance_store dir
|
||||
cd $GLANCE_STORE_DIR
|
||||
|
||||
sudo chown -R $owner:stack $GLANCE_STORE_DIR
|
||||
|
||||
sudo cp $GLANCE_STORE_DIR/functional_testing.conf.sample $GLANCE_STORE_DIR/functional_testing.conf
|
||||
|
||||
# Set admin creds
|
||||
iniset $GLANCE_STORE_DIR/functional_testing.conf admin key $ADMIN_PASSWORD
|
||||
|
||||
# Run tests
|
||||
echo "Running glance_store functional test suite"
|
||||
set +e
|
||||
# Preserve env for OS_ credentials
|
||||
sudo -E -H -u jenkins tox -e functional-$GLANCE_STORE_DRIVER
|
||||
EXIT_CODE=$?
|
||||
set -e
|
||||
|
||||
# Collect and parse result
|
||||
generate_testr_results
|
||||
exit $EXIT_CODE
|
|
@ -18,6 +18,8 @@ import logging
|
|||
import random
|
||||
import time
|
||||
|
||||
from keystoneauth1.identity import v3
|
||||
from keystoneauth1 import session
|
||||
from oslo_config import cfg
|
||||
import swiftclient
|
||||
|
||||
|
@ -33,27 +35,20 @@ class TestSwift(base.BaseFunctionalTests):
|
|||
def __init__(self, *args, **kwargs):
|
||||
super(TestSwift, self).__init__('swift', *args, **kwargs)
|
||||
|
||||
self.auth = self.config.get('admin', 'auth_address')
|
||||
user = self.config.get('admin', 'user')
|
||||
self.key = self.config.get('admin', 'key')
|
||||
self.region = self.config.get('admin', 'region')
|
||||
|
||||
self.tenant, self.username = user.split(':')
|
||||
|
||||
CONF.set_override('swift_store_user',
|
||||
user,
|
||||
'{1}:{0}'.format(self.username, self.project_name),
|
||||
group='glance_store')
|
||||
CONF.set_override('swift_store_auth_address',
|
||||
self.auth,
|
||||
self.auth_url,
|
||||
group='glance_store')
|
||||
CONF.set_override('swift_store_auth_version',
|
||||
self.keystone_version,
|
||||
group='glance_store')
|
||||
CONF.set_override('swift_store_key',
|
||||
self.key,
|
||||
group='glance_store')
|
||||
CONF.set_override('swift_store_create_container_on_put',
|
||||
True,
|
||||
self.password,
|
||||
group='glance_store')
|
||||
CONF.set_override('swift_store_region',
|
||||
self.region,
|
||||
self.region_name,
|
||||
group='glance_store')
|
||||
CONF.set_override('swift_store_create_container_on_put',
|
||||
True,
|
||||
|
@ -70,14 +65,18 @@ class TestSwift(base.BaseFunctionalTests):
|
|||
super(TestSwift, self).setUp()
|
||||
|
||||
def tearDown(self):
|
||||
auth = v3.Password(auth_url=self.auth_url,
|
||||
username=self.username,
|
||||
password=self.password,
|
||||
project_name=self.project_name,
|
||||
user_domain_id=self.user_domain_id,
|
||||
project_domain_id=self.project_domain_id)
|
||||
sess = session.Session(auth=auth)
|
||||
swift = swiftclient.client.Connection(session=sess)
|
||||
|
||||
for x in range(1, 4):
|
||||
time.sleep(x)
|
||||
try:
|
||||
swift = swiftclient.client.Connection(auth_version='2',
|
||||
user=self.username,
|
||||
key=self.key,
|
||||
tenant_name=self.tenant,
|
||||
authurl=self.auth)
|
||||
_, objects = swift.get_container(self.container)
|
||||
for obj in objects:
|
||||
swift.delete_object(self.container, obj.get('name'))
|
||||
|
|
6
tox.ini
6
tox.ini
|
@ -45,15 +45,17 @@ commands = python setup.py testr --coverage --testr-args='^(?!.*test.*coverage).
|
|||
[testenv:venv]
|
||||
commands = {posargs}
|
||||
|
||||
# See glance_store/tests/functional/README.rst for information on writing or
|
||||
# running functional tests.
|
||||
[testenv:functional-swift]
|
||||
sitepackages = True
|
||||
setenv = OS_TEST_PATH=./glance_store/tests/functional/swift
|
||||
commands = python setup.py testr --slowest --testr-args='glance_store.tests.functional.swift'
|
||||
commands = ostestr --slowest --testr-args='glance_store.tests.functional.swift'
|
||||
|
||||
[testenv:functional-filesystem]
|
||||
sitepackages = True
|
||||
setenv = OS_TEST_PATH=./glance_store/tests/functional/filesystem
|
||||
commands = python setup.py testr --slowest --testr-args='glance_store.tests.functional.filesystem'
|
||||
commands = ostestr --slowest --testr-args='glance_store.tests.functional.filesystem'
|
||||
|
||||
[flake8]
|
||||
# TODO(dmllr): Analyze or fix the warnings blacklisted below
|
||||
|
|
Loading…
Reference in New Issue