openinfra
/
interop
Code Issues Proposed changes

Updated docs with fixes and additional info 

Fixed the command to load the list of tests. Removed trailing
whitespace. Added more information about running tests and
addressed flagged failures.

Change-Id: Ifb8590f1da931f07cf6ffba1082c51cfac07143b
This commit is contained in:
Chris Hoge 2015-04-24 09:51:00 -07:00
parent ad6c04828a
commit 7117b237fa
1 changed files with 11 additions and 3 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
2015.04/procedure.rst
View File

@ -10,13 +10,13 @@ Tempest can be run standalone, or under a test runner such as refstack-client
or rally. If only testing against Defcore capabilities, you can use the
--load-list argument and a file containing a list of the Defcore tests. If
run with the refstack-client, test output will be parsed to list only
passing tests in a json formatted file.
passing tests in a json formatted file.
The test names of the capabilities are derived from a recent release of
Tempest, from the time of capabilitiy approval. Keep in mind that Tempest
is under active development, and tests may move. If you're not seeing
full coverage, please consider reverting back to a Tempest sha that more
closely matches the capability release date. Please contact Chris Hoge
closely matches the capability release date. Please contact Chris Hoge
<chris@openstack.org> for assistance if needed.
It's important to run a recent version of Tempest, as major bugs related to
@ -58,17 +58,25 @@ or CentOS 7 have been verified) with administrator privileges.
directory:
``cd .tempest``
``./run_tempest -C <your_tempest.conf> -- --load-tests 2015.04.required.txt``
``./run_tempest -C <your_tempest.conf> -- --load-list 2015.04.required.txt``
* Review the test results, and when you're satisfied, send them to
chris@openstack.org. The results are stored in a json file in the directory
``.tempest/.testrepository``
* Every effort should be made to pass all of the required tests, but you
will want to compare any lists of failed tests to the list of flagged tests.
https://github.com/openstack/defcore/blob/master/2015.03/2015.04.required.txt
Known Issues and Recommendations
--------------------------------
Currently after failures modes Tempest does not clean up test resources. We
strongly recommend that you run Tempest against a test OpenStack cloud
rather than a production cloud. Successful tests against test deployments that
are functionally equivalent to production clouds is acceptable for current
capabilities assessment.
You may find it useful to run Swift tests as a seperate run, using the
``accounts.yaml`` framework to specify users with Swift-specific roles.