diff --git a/README.rst b/README.rst index 431be7c3f0..1b2e474e87 100644 --- a/README.rst +++ b/README.rst @@ -37,57 +37,61 @@ Tempest Design Principles that we strive to live by. Quickstart ---------- -To run Tempest, you first need to create a configuration file that -will tell Tempest where to find the various OpenStack services and -other testing behavior switches. +To run Tempest, you first need to create a configuration file that will tell +Tempest where to find the various OpenStack services and other testing behavior +switches. Where the configuration file lives and how you interact with it +depends on how you'll be running Tempest. There are 2 methods of using Tempest. +The first, which is a newer and recommended workflow treats Tempest as a system +installed program. The second older method is to run Tempest assuming your +working dir is the actually Tempest source repo, and there are a number of +assumptions related to that. For this section we'll only cover the newer method +as it is simpler, and quicker to work with. -The easiest way to create a configuration file is to generate a sample -in the ``etc/`` directory :: +#. You first need to install Tempest this is done with pip, after you check out + the Tempest repo you simply run something like:: - $> cd $TEMPEST_ROOT_DIR - $> oslo-config-generator --config-file \ - tools/config/config-generator.tempest.conf \ - --output-file etc/tempest.conf + $ pip install tempest -After that, open up the ``etc/tempest.conf`` file and edit the -configuration variables to match valid data in your environment. -This includes your Keystone endpoint, a valid user and credentials, -and reference data to be used in testing. + This can be done within a venv, but the assumption for this guide is that + the Tempest cli entry point will be in your shell's PATH. -.. note:: +#. Installing Tempest will create a /etc/tempest dir which will contain the + sample config file packaged with Tempest. The contents of /etc/tempest will + be copied to all local working dirs, so if there is any common configuration + you'd like to be shared between anyone setting up local Tempest working dirs + it's recommended that you copy or rename tempest.conf.sample to tempest.conf + and make those changes to that file in /etc/tempest - If you have a running devstack environment, Tempest will be - automatically configured and placed in ``/opt/stack/tempest``. It - will have a configuration file already set up to work with your - devstack installation. +#. Setup a local working Tempest dir. This is done using the tempest init + command:: -Tempest is not tied to any single test runner, but `testr`_ is the most commonly -used tool. Also, the nosetests test runner is **not** recommended to run Tempest. + tempest init cloud-01 -After setting up your configuration file, you can execute the set of Tempest -tests by using ``testr`` :: + works the same as:: - $> testr run --parallel + mkdir cloud-01 && cd cloud-01 && tempest init -.. _testr: http://testrepository.readthedocs.org/en/latest/MANUAL.html + This will create a new directory for running a single Tempest configuration. + If you'd like to run Tempest against multiple OpenStack deployments the idea + is that you'll create a new working directory for each to maintain separate + configuration files and local artifact storage for each. -To run one single test serially :: +#. Then cd into the newly created working dir and also modify the local + config files located in the etc/ subdir created by the ``tempest init`` + command. Tempest is expecting a tempest.conf file in etc/ so if only a + sample exists you must rename or copy it to tempest.conf before making + any changes to it otherwise Tempest will not know how to load it. - $> testr run tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON.test_reboot_non_existent_server +#. Once the configuration is done you're now ready to run Tempest. This can + be done with testr directly or any `testr`_ based test runner, like + `ostestr`_. For example, from the working dir running:: -Alternatively, you can use the run_tempest.sh script which will create a venv -and run the tests or use tox to do the same. Tox also contains several existing -job configurations. For example:: + $ ostestr --regex '(?!.*\[.*\bslow\b.*\])(^tempest\.(api|scenario|thirdparty))' - $> tox -efull - -which will run the same set of tests as the OpenStack gate. (it's exactly how -the gate invokes Tempest) Or:: - - $> tox -esmoke - -to run the tests tagged as smoke. + will run the same set of tests as the default gate jobs. +.. _testr: https://testrepository.readthedocs.org/en/latest/MANUAL.html +.. _ostestr: http://docs.openstack.org/developer/os-testr/ Configuration ------------- @@ -146,3 +150,62 @@ work on Python 3.4. However, because large parts of Tempest are self verifying there might be uncaught issues running on Python 3.4. So until there is a gating job which does a full Tempest run using Python 3.4 there isn't any guarantee that running Tempest under Python 3.4 is bug free. + +Legacy run method +----------------- + +The legacy method of running Tempest is to just treat the Tempest source code +as a python unittest repository and run directly from the source repo. When +running in this way you still start with a Tempest config file and the steps +are basically the same except that it expects you know where the Tempest code +lives on your system and requires a bit more manual interaction to get Tempest +running. For example, when running Tempest this way things like a lock file +directory do not get generated automatically and the burden is on the user to +create and configure that. + +To start you need to create a configuration file. The easiest way to create a +configuration file is to generate a sample in the ``etc/`` directory :: + + $> cd $TEMPEST_ROOT_DIR + $> oslo-config-generator --config-file \ + tools/config/config-generator.tempest.conf \ + --output-file etc/tempest.conf + +After that, open up the ``etc/tempest.conf`` file and edit the +configuration variables to match valid data in your environment. +This includes your Keystone endpoint, a valid user and credentials, +and reference data to be used in testing. + +.. note:: + + If you have a running devstack environment, Tempest will be + automatically configured and placed in ``/opt/stack/tempest``. It + will have a configuration file already set up to work with your + devstack installation. + +Tempest is not tied to any single test runner, but `testr`_ is the most commonly +used tool. Also, the nosetests test runner is **not** recommended to run Tempest. + +After setting up your configuration file, you can execute the set of Tempest +tests by using ``testr`` :: + + $> testr run --parallel + +.. _testr: http://testrepository.readthedocs.org/en/latest/MANUAL.html + +To run one single test serially :: + + $> testr run tempest.api.compute.servers.test_servers_negative.ServersNegativeTestJSON.test_reboot_non_existent_server + +Alternatively, you can use the run_tempest.sh script which will create a venv +and run the tests or use tox to do the same. Tox also contains several existing +job configurations. For example:: + + $> tox -efull + +which will run the same set of tests as the OpenStack gate. (it's exactly how +the gate invokes Tempest) Or:: + + $> tox -esmoke + +to run the tests tagged as smoke.