In order to use it, in any FS, set tempest_format to venv
and then set:
tempest_plugins_venv:
- '<complete upstream openstack tempest plugin git url>'
Change-Id: I95a4449ca3976a914faf1ba5f3f4c0e756a6fefc
* The file getting generated is tempest.log under /home/zuul/
tempest.log not tempest-output.log that's why it is not able
to indexed in logstash.
* And tempest_log_file var is used twice in validate-tempest role
and tempest.log is used at each place which also leads that
tempest_output.log was never found in ci logs.
Related-Bug:#1802971
Change-Id: I9bb9f8bdd0a17d2a1481356caaf186ed6348f6ba
Makes those files conformant with current linting rules and avoids
linting errors when we need to toch them again.
Previous doing "pre-commit run -a" uncovered these errors, now is no
longer reporting any other errors.
Change-Id: Ie4cf229c8f11c2b55b323eac23c89483b26d3781
* https://review.openstack.org/#/c/600517 introduces undercloud_container_cli
to choose docker/podman container runtime. This changes introduces to use
both docker/podman for running tempest tests.
* Tempest user does not have permission to create tempest workspace let's
create the tempest workspace and own it with the tempest user.
* Introducing tempest_data directory to keep tempest related data
which can be used by tempestconf and tempest run to tests
* Cleaned up stuff which is not necessary for tempest container.
Depends-On: https://review.openstack.org/#/c/614664/
Change-Id: I1b19b2580d1d4c9844e9221f18b23655b36b5a23
This is a better way to install and run stackviz, also removing some
tasks that's no longer necessary:
* Remove stackviz static files from the repo, since it's already in the
tarball
* Remove stackviz cloning, and using tarball
* Install stackviz directly from pip module
* Removing tempest-status.yml since we don't use to anything
* Move tempest-results.yml to run before stackviz, so we can take
advantage of the testrepository.subunit file to generate stackviz data
* Use testrepository.subunit to generate subunit2html report instead
of stestr/testr stream
Closes-Bug: 1779562
Change-Id: I918e5401b4f46e947bbaca99a48c2b21bce9a4bd
Since the standalone deployment uses clouds.yaml, we need the validate
tempest role to allow for configuration using OS_CLOUD rather than the
traditional {stack,overcloud}rc files.
Change-Id: Ied55638618b718ee15a2ef5bc2f89cef232b5b7e
Related-Blueprint: all-in-one
* In CI, when tempest container is used, It is good to dump the
logs in /var/log/tempest so that it would be easier to verify
tempest container is used.
* Introduced tempest_log_dir and tempest_log_file var to the
validate-tempest-playbook
Depends-On: https://review.openstack.org/#/c/566364/
Change-Id: Ifd72d902c623b3941a6c714e36b7cf104109a570
* https://review.openstack.org/551441 introduced test_black_regex
in order to skip test directly from featureset files. In order
to avoid confusion between test_regex and test_black_regex,
we are renaming test_regex to test_white_regex.
* Added deprecation message for the same.
Change-Id: I7e6240117c9fbab0e51d6248d74180c7a91dc9e2
* Introduced new workflow for validate-tempest role
- Remove previously created tempest artifacts
- Install tempest
- Create tempest workspace
- Create tempest resources
- Generate tempest configurations
- Run tempest
- Removed unused ironic undercloud fake services
- Removing public network from undercloud as there is no public network
- Added tempest_dir var to give path to tempest workspace
- use {{ working_dir }}/tempest_git for clonning stuff
- Added test_black_regex to skip a series of tests
- Fixed permission issue for tempest_dir to generate subunit files
- Install stestr subunit-filters and subunit for generating subunit files
- Added tempest_skip_master containing skip list for undercloud
- Introduced tempest_conf_version for installing particular version of
tempestconf from git
- Enable set -e to find the exit status of command running inside
container
Change-Id: I021432e222fae0e57a13ca4859ba77e990ac6c67
Tempestmail is being used in other CI that don't work like the
openstack upstream ci, and logs are stored somewhere else.
This patch allows the possibility to pass --log-url option
to tempestmail script override the default log path.
Change-Id: Iab9b448551d1a2ed59b47f91d5a9bcc204e0a703
There are leftovers of tripleo-tempest role name in files,
which causes errors in quickstart installation, fix all of them
to be validate-tempest.
Change-Id: I29f6d9f9ed0dd357dc642f7c47b47f7712d73661
tempest_workers used `ansible_processor_count` which is in fact the
number of physical CPUs - rarely more than two nowadays - when it should
in fact use number of CPU cores/threads which is `ansible_processor_vcpus`.
Change-Id: I7aa0dd29df5130e15bf6c1cf97774744e2f46921
This patch gives the option to pass parameters to tempest to
remove certain keys while generating tempest.conf by passing
it as tempest_conf_removal.
The tempest_conf_removal is applied in the final stage and
as such overrides even if values were prevously set.
dvr is default value in tempest_conf_removal - neutron extension.
python-tempestconf gets all available neutron extensions, even if
they are not enabled in a deployment. dvr is not generally configured
so to avoid failed tests, it's removed from tempest.conf
Change-Id: I708866aa3218b2e3c25e9b47672237bc1838d760
* tempest_extra_config will contains a dict with section.option as
a key followed by values like that
tempest_extra_config:
identity.admin_v2_api: True
* The above defined values will be overriden in tempest.conf with
the help of discover-tempest-config tool
* With this flag we can have more control over tempest.conf while
using it in featureset files.
Change-Id: If787aaf2de0f5a243aceb510f91915db32dc48c1
Since scenarios will start to run tempest, some of them requires
the installation of openstack services tempest plugin. Before, all
the plugins was being installed by default, however the tests being
executed had some conflicts and was decided to not install all at
once, but install only what is required. This patch enables this
behavior.
Change-Id: I9600024b1bd998a15d3aa2629212bdd6591c37d6
Due to scenario jobs starting to execute a set of tempest tests, it's
good to have an option to add these tests in a white list instead
of care about the complexity that a long regex line can bring.
Change-Id: Id1ba7af48173460ef41d9b4df3db723811dc67da
* --isolated, and --failing options does not work
with ostestr as these options are used with testr. Since ostestr
is used as a universal test runner for running tempest across the
board. If the above options got enabled by mistake it will break
the tempest run. So removing it.
* --load-list is replaced by whitelist-files and is taken care
by this review: https://review.openstack.org/#/c/485252/
Change-Id: I9745fdcf592c4dc5bf3cbaf11e9e6b11e209ea77
This patch adds a script that will check all the bugs in the
yaml skip list file which has a bugzilla or launchpad attached
and verify if the bug still open or closed. If closed, it will
remove from the skip list.
This also can generate a report showing what is still open and
what was already closed, as well as generate the skiplist in
tempest/testr format.
Change-Id: Iadd508184c350d074600316ad15f5274df87f01b
The workflow of
- configure_tempest is only defined twice and used in an important
condition but it is not really used anywhere; the variable which is
around is tempest_config. Use tempest_config properly, remove
then configure_tempest;
- execute validate-tempest also when tempest_config is true,
not only when run_tempest is true;
- execute the post-tempest task only if run_tempest is true;
no need to execute it if tempest was simply configured.
Change-Id: I84dce8ff4ebb1df2163b1de6279b35e564efea5b
* Run tempest cleanup --init-saved-state before running the tempest
tests to save resources before tempest execution.
* Run tempest cleanup --dry-run to see what new resources got created
after tempest run.
* use tempest_track_resources: true in defaults/main.yml to track
tempest resources before and after tempest run
Change-Id: I9787950996055bb2ca6876b5f42048be55028675
Earlier we are using complex regex '\[.*\b{{ test_regex }}\b.*\]'
to run test in validate-tempest role which appears to be confusing
and it does not excepts other tests. So i have modified it to make
more simpler by keep '(test_regex)' in that. ostestr will easy handle
it.
Change-Id: I0287ee7a19fca0298d9b934adde1a3112615cef3
This is the script that will send an email to a list of
user stored in the config file with information regarding the
results of a tempest run, together with the ansible task that
will execute it when tempest finishes run.
Change-Id: I57f5f0928e5efec106c793b71e335af1ac77fa91
Implements: blueprint send-mail-tool
This way to load the skip file will allow a more concise format
to be consumed not only by the skip file itself but for other
scripts
Change-Id: I09130b0e46711e1e01a6b5b5077ae9249f01c597
* From ocata release, In RDO, the tempest rpm has changed it's
source from redhat-openstack/tempest to upstream tempest as
well as config_tempest script is moved to
redhat-openstack/python-tempestconf.
* Remove tempest_source = rdo or upstream as it does not make
sense since we are using tempest_format = venv or packages,
As we select venv, for older release till newton upstream will
be redhat-openstack/tempest and for ocata will be upstream
tempest. If we select packages it will directly installs
from RDO packages.
* Use tempest_format to packages
* Use ostestr for running tempest tests as upstream tempest
run interface is currently unstable.
Change-Id: I177b32622500dab54896fda740080d07f5f65261
Don't exit with error automatically, write down tempest status in
file for later parsing and set variable tempest_status.
Change-Id: I986587ceadf58b13e9a2aeadb62f81c522e29b82