170 lines
9.1 KiB
Markdown
170 lines
9.1 KiB
Markdown
validate-tempest
|
|
================
|
|
|
|
Ansible role for running tempest tests on undercloud or overcloud.
|
|
|
|
Requirements
|
|
------------
|
|
|
|
This Ansible role allows to run tempest tests against installed undercloud or overcloud from undercloud.
|
|
It can use upstream sources with virtualenv, RDO packages and kolla container image for tempest.
|
|
|
|
Role Variables
|
|
--------------
|
|
|
|
* `tempest_undercloud`: false/true - whether to run tempest on installed undercloud (default: false)
|
|
* `tempest_overcloud`: false/true - whether to run tempest on installed overcloud (default: true)
|
|
* `tempest_format`: venv/packages/container - Which tempest installation to use - either install python virtual environment
|
|
with installed there python modules from requirements file, or to use installed with RDO RPM packages
|
|
or to use Kolla provided tempest container image (default: packages)
|
|
* `test_white_regex` - tests regular expression for tempest run, i.e. smoke or tempest.api.object_storage|keystone_tempest_plugin.
|
|
* `run_tempest`: false/true - to run tempest or not (default: false)
|
|
* `post_tempest`: false/true - to run post processing of tempest results after tempest run
|
|
* `tempest_config`: false/true - whether to prepare the script which configures and runs tempest or not (default: true)
|
|
* `tempest_whitelist`: list - list of tests you want to be executed. set `skip_file_src`
|
|
to empty if you don't want to run with blacklist_file option (default: [])
|
|
* `skip_file_src`: path to skip tests file, set it empty if running without skip list: `skip_file_src=''`
|
|
* `tempest_workers`: int - how many parallel workers to run (default is half the number of cores/threads)
|
|
* `tempest_until_failure`: false/true - default is false, repeat the run again and again until failure occurs
|
|
* `tempest_exit_on_failure`: true/false - whether to exit from role with tempest exit code (default: true)
|
|
* `tempestmail_config`: config.yaml - name of config file for tempestmail script
|
|
* `tempestmail_log_server`: <string> - Server where the logs are saved
|
|
* `tempest_track_resources`: true/false - whether to save the state of resources after tempest run (default: true)
|
|
* `tempest_log_dir`: <directory path> - The directory path where tempest log file is stored (default: /var/log/tempest)
|
|
* `tempest_log_file`: <file name> - The name of tempest log file (default: tempest.log)
|
|
* `check_tempest_bugs`: true/false - Will check every bugzilla and launchpad bug in the yaml skip file
|
|
* `tempest_plugins`: list - List of openstack services tempest plugins to be
|
|
installed
|
|
* `tempest_plugins_git`: list - (optional) List of URLs for openstack tempest plugins you want to have installed.
|
|
(Required) `tempest_format`: venv
|
|
For example:
|
|
```
|
|
tempest_format: venv
|
|
tempest_plugins_git: [https://git.openstack.org/openstack/tempest-tripleo-ui,
|
|
https://git.openstack.org/openstack/keystone-tempest-plugin,
|
|
https://git.openstack.org/openstack/cinder-tempest-plugin]
|
|
```
|
|
* `tempest_extra_config`: dict - A dict of tempest configuration which needs to be overridden in tempest.conf,
|
|
It should be like section.key: value.
|
|
* `tempest_conf_removal`: dict - A dict of tempest configuration which will be
|
|
removed from tempest.conf file.
|
|
Format: section.key: value
|
|
* `public_physical_network`: <string> The name of the border physical network (default: datacentre).
|
|
* `tempest_container_registry`: <string> The name of the container registry to use (default: docker.io/tripleomaster)
|
|
* `tempest_container_namespace`: <string> The name of tempest container image to use (default: centos-binary-tempest)
|
|
* `tempest_container_tag`: <string> The tag of the tempest container image to use (default: current-tripleo)
|
|
* `tempest_dir`: <string> The path to tempest workspace directory (default: /home/stack/tempest)
|
|
* `tempest_data`: <string> The path to keep tempest related files used for running tempest (default: /home/stack)
|
|
* `test_black_regex`: <list> A set of tempest tests to skip (default: [])
|
|
* `tempest_version_dict`: <dict> A dict with release name as key and tempest tag version for that release as value
|
|
* `tempest_version`: <string> The tempest version to use for running tempest
|
|
* `tempest_conf_version_dict`: <dict> A dict with release name as key and python-tempestconf tag version for that release
|
|
as value
|
|
* `tempest_test_image_version`: <float> The version of the cloud image used by tempest for testing
|
|
* `tempest_test_image_name`: <string> The name of the cloud image used by tempest for testing
|
|
* `tempest_test_image_path`: <Image full path/URL> The full path or url of the cloud image used by tempest for testing
|
|
* `floating_ip_cidr`: the network CIDR to be used for a public floating IP network during testing
|
|
* `public_net_name`, `public_net_type` : the name and type of the public neutron network, used for floating IPs during
|
|
the tempest testing
|
|
* `public_net_pool_start`, `public_net_pool_end`, `public_net_gateway`,
|
|
`public_physical_network`, `public_segmentation_id` -- parameters used to create the public floating IP network
|
|
* `tempest_deployer_input_file`: <file path> The path to tripleo deployer input file which contains the pre configured
|
|
configuration for the deployed cloud using TripleO.
|
|
* `tempest_os_cloud`: <string> String name of the cloud to export as OS_CLOUD when using clouds.yaml rather than stackrc
|
|
* `tempest_use_headless_chrome`: true/false - whether the headless chrome docker container should be pulled, and run
|
|
* `tempest_headless_chrome_port`: <int> - the entrypoint into the headless chrome container
|
|
|
|
|
|
Skip tests file
|
|
---------------
|
|
|
|
Tempest test can be skipped if they are in a skip file, using the whole test
|
|
name or a regular expression.
|
|
Since the intention is to use the list of skip tests in other parts of
|
|
quickstart-extras, it's easier to have this list of skip tests in yaml format
|
|
where you can set the skip test, reason and launchpad or bugzilla. This make
|
|
easier later to check whether the test still valid or not to be skipped.
|
|
Skip files are under roles/validate-tempest/vars/tempest_skip_VERSION.yaml.
|
|
In these files you can add a regex, together with a reason, a bugzilla
|
|
or launchpad bug and target host (undercloud/overcloud )related to that
|
|
particular test. If target host is set `undercloud:true`, it will be skipped
|
|
on the undercloud.Launchpad and/or bugzilla re optional, however you must
|
|
specify a reason why you are skipping the test.
|
|
|
|
Example of skip file:
|
|
|
|
---
|
|
known_failures:
|
|
- test: 'tempest.scenario.test_volume_boot_pattern'
|
|
reason: 'rdo-manager tempest smoke test failing on "floating ip pool not found"'
|
|
bz: 'https://bugzilla.redhat.com/show_bug.cgi?id=1272289'
|
|
- test: 'tempest.api.volume.test_volumes_get'
|
|
reason: 'rdo-manager tempest smoke test failing on "floating ip pool not found"'
|
|
- test: 'tempest.api.identity.*v3'
|
|
reason: 'Tempest Identify tests failed on V3 api on OSP-D puddle'
|
|
bz: 'https://bugzilla.redhat.com/show_bug.cgi?id=1266947'
|
|
- test: 'tempest.api.image.v2.test_images_metadefs_namespace_properties.MetadataNamespacePropertiesTest.test_basic_meta_def_namespace_property'
|
|
reason: 'glance is not calling glance-manage db_load_metadefs'
|
|
lp: 'https://bugs.launchpad.net/tripleo/+bug/1664995'
|
|
# Skip list for undercloud
|
|
- test: 'tempest.api.image.v2.test_images_metadefs_namespace_properties.MetadataNamespacePropertiesTest.test_basic_meta_def_namespace_property'
|
|
reason: 'glance is not calling glance-manage db_load_metadefs'
|
|
lp: 'https://bugs.launchpad.net/tripleo/+bug/1664995'
|
|
undercloud: true
|
|
|
|
|
|
Real time bug check
|
|
-------------------
|
|
|
|
If check_tempest_bugs is set to true, a script will be called and will check
|
|
in real time, all tests who has a bugzilla or a launchpad bug. This will
|
|
generate a new skip file, removing all the bugs that were already closed but
|
|
wasn't updated in the yaml skip file yet.
|
|
|
|
|
|
|
|
Using validate-tempest role with tripleo-quickstart
|
|
---------------------------------------------------
|
|
|
|
To run tempest with tripleo-quickstart, run the following command:
|
|
|
|
$WORKSPACE/tripleo-quickstart
|
|
|
|
bash quickstart.sh \
|
|
--bootstrap \
|
|
--tags all \
|
|
--config $WORKSPACE/config/general_config/$CONFIG.yml \
|
|
--working-dir $WORKSPACE/ \
|
|
--no-clone \
|
|
--release master-tripleo-ci \
|
|
--extra-vars test_ping=False \
|
|
--extra-vars run_tempest=True \
|
|
$VIRTHOST
|
|
|
|
Note: by using --extra-vars, we can change the validate-tempest role variables.
|
|
|
|
Dependencies
|
|
------------
|
|
|
|
No dependencies
|
|
|
|
Example Playbook
|
|
----------------
|
|
|
|
---
|
|
- name: Run tempest
|
|
hosts: undercloud
|
|
gather_facts: false
|
|
roles:
|
|
- validate-tempest
|
|
|
|
License
|
|
-------
|
|
|
|
Apache 2.0
|
|
|
|
Author Information
|
|
------------------
|
|
|
|
RDO-CI Team
|