openstack
/
tripleo-quickstart-extras
Code Issues Proposed changes

Improved validate-tempest role README documentation 

* Covered description of all role variables
* Added instruction on how to run tempest using TQ

Change-Id: If307541353410d653c5aa2e969d9f78b7d4b7838
This commit is contained in:
Chandan Kumar 2018-04-27 10:49:36 +05:30
parent 5e5a0eb462
commit 2d7455b7e5
1 changed files with 61 additions and 18 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

79
roles/validate-tempest/README.md
View File

@ -1,35 +1,35 @@
validate-tempest
=========
================
Run tempest tests on undercloud or overcloud.
Ansible role for running tempest tests on undercloud or overcloud.
Requirements
------------
This Ansible role allows to run tempest tests against installed undercloud or overcloud.
It can use both upstream sources with virtualenv and RDO packages for tempest.
This Ansible role allows to run tempest tests against installed undercloud or overcloud from undercloud.
It can use upstream sources with virtualenv, RDO packages and kolla container image for tempest.
Role Variables
--------------
* `tempest_undercloud`: false/true - whether to run tempest on installed undercloud
* `tempest_overcloud`: false/true - whether to run tempest on installed overcloud
* `tempest_format`: venv/packages - Which tempest installation to use - either install python virtual environment
* `tempest_undercloud`: false/true - whether to run tempest on installed undercloud (default: false)
* `tempest_overcloud`: false/true - whether to run tempest on installed overcloud (default: true)
* `tempest_format`: venv/packages/container - Which tempest installation to use - either install python virtual environment
with installed there python modules from requirements file, or to use installed with RDO RPM packages
* `tempest_log_file` - name of log file for tempest run
or to use Kolla provided tempest container image (default: packages)
* `tempest_log_file` - name of log file for tempest run (default: tempest_output.log)
* `test_white_regex` - tests regular expression for tempest run, i.e. smoke or tempest.api.object_storage|keystone_tempest_plugin.
`test_regex` is deprecated and switch to `test_white_regex`, will be removed by end of Rocky release.
* `run_tempest`: false/true - to run tempest or not
* `tempest_config`: false/true - whether to prepare the script which configures and runs tempest or not
* `run_tempest`: false/true - to run tempest or not (default: false)
* `post_tempest`: false/true - to run post processing of tempest results after tempest run
* `tempest_config`: false/true - whether to prepare the script which configures and runs tempest or not (default: true)
* `tempest_whitelist`: list - list of tests you want to be executed. set `skip_file_src`
to empty if you don't want to run with blacklist_file option.
Default value is an empty list
to empty if you don't want to run with blacklist_file option (default: [])
* `skip_file_src`: path to skip tests file, set it empty if running without skip list: `skip_file_src=''`
* `tempest_workers`: int - how many parallel workers to run (default is half the number of cores/threads)
* `tempest_until_failure`: false/true - default is false, repeat the run again and again until failure occurs
* `tempest_exit_on_failure`: true/false - whether to exit from role with tempest exit code (default: true)
* `tempestmail_config`: config.yaml - name of config file for tempestmail script
* `tempestmail_log_server`: string - Server where the logs are saved
* `tempestmail_log_server`: <string> - Server where the logs are saved
* `tempest_track_resources`: true/false - whether to save the state of resources after tempest run (default: true)
* `check_tempest_bugs`: true/false - Will check every bugzilla and launchpad bug in the yaml skip file
* `tempest_plugins`: list - List of openstack services tempest plugins to be
@ -45,6 +45,21 @@ Role Variables
* `tempest_container_tag`: <string> The tag of the tempest container image to use (default: current-tripleo)
* `tempest_dir`: <string> The path to tempest workspace directory (default: /home/stack/tempest)
* `test_black_regex`: <list> A set of tempest tests to skip (default: [])
* `tempest_version_dict`: <dict> A dict with release name as key and tempest tag version for that release as value
* `tempest_version`: <string> The tempest version to use for running tempest
* `tempest_conf_version_dict`: <dict> A dict with release name as key and python-tempestconf tag version for that release
as value
* `tempest_test_image_version`: <float> The version of the cloud image used by tempest for testing
* `tempest_test_image_name`: <string> The name of the cloud image used by tempest for testing
* `tempest_test_image_path`: <Image full path/URL> The full path or url of the cloud image used by tempest for testing
* `floating_ip_cidr`: the network CIDR to be used for a public floating IP network during testing
* `public_net_name`, `public_net_type` : the name and type of the public neutron network, used for floating IPs during
the tempest testing
* `public_net_pool_start`, `public_net_pool_end`, `public_net_gateway`,
`public_physical_network`, `public_segmentation_id` -- parameters used to create the public floating IP network
* `tempest_deployer_input_file`: <file path> The path to tripleo deployer input file which contains the pre configured
configuration for the deployed cloud using TripleO.
Skip tests file
---------------
@ -56,10 +71,11 @@ quickstart-extras, it's easier to have this list of skip tests in yaml format
where you can set the skip test, reason and launchpad or bugzilla. This make
easier later to check whether the test still valid or not to be skipped.
Skip files are under roles/validate-tempest/vars/tempest_skip_VERSION.yaml.
In these files you can add a regex, together with a reason and a bugzilla
or launchpad bug related to that particular test.
Launchpad and/or bugzilla re optional, however you must specify a reason why
you are skipping the test.
In these files you can add a regex, together with a reason, a bugzilla
or launchpad bug and target host (undercloud/overcloud )related to that
particular test. If target host is set `undercloud:true`, it will be skipped
on the undercloud.Launchpad and/or bugzilla re optional, however you must
specify a reason why you are skipping the test.
Example of skip file:
@ -76,6 +92,11 @@ Example of skip file:
- test: 'tempest.api.image.v2.test_images_metadefs_namespace_properties.MetadataNamespacePropertiesTest.test_basic_meta_def_namespace_property'
reason: 'glance is not calling glance-manage db_load_metadefs'
lp: 'https://bugs.launchpad.net/tripleo/+bug/1664995'
# Skip list for undercloud
- test: 'tempest.api.image.v2.test_images_metadefs_namespace_properties.MetadataNamespacePropertiesTest.test_basic_meta_def_namespace_property'
reason: 'glance is not calling glance-manage db_load_metadefs'
lp: 'https://bugs.launchpad.net/tripleo/+bug/1664995'
undercloud: true
Real time bug check
@ -86,6 +107,28 @@ in real time, all tests who has a bugzilla or a launchpad bug. This will
generate a new skip file, removing all the bugs that were already closed but
wasn't updated in the yaml skip file yet.
Using validate-tempest role with tripleo-quickstart
---------------------------------------------------
To run tempest with tripleo-quickstart, run the following command:
$WORKSPACE/tripleo-quickstart
bash quickstart.sh \
--bootstrap \
--tags all \
--config $WORKSPACE/config/general_config/$CONFIG.yml \
--working-dir $WORKSPACE/ \
--no-clone \
--release master-tripleo-ci \
--extra-vars test_ping=False \
--extra-vars run_tempest=True \
$VIRTHOST
Note: by using --extra-vars, we can change the validate-tempest role variables.
Dependencies
------------