This change removes logic for releases older than train, because these
releases are already EOLed. This covers only tempest roles and
playbooks.
Change-Id: I0b50cad30107a83b562b8c22b2afd7af7636c615
Before switching the default machine type of instances to q35 in
tripleo-heat-templates via I9f60a73577ae7cd712e2a8285abc0c0788906112 we
first need to ensure that Cirros is updated to 0.5.2 that includes the
ahci module [1][2].
As set out in I12e0bdb3699e5343592ab834468ba6b2fcdcaaf4 for devstack and
Iad1adbc23b31dd54a96299e7a8a4b622c15eed8d for Nova this is required to
allow Cirros based instances to see the now SATA based config drive
cdroms attached to them during a Tempest run.
[1] https://github.com/cirros-dev/cirros/releases/tag/0.5.2
[2] https://github.com/cirros-dev/cirros/pull/65
Change-Id: Ia3f839a3c5e4e4b59898c11561fe7ef7126bba5f
We weren't testing containerized tempest in standalone job due the
missing logic on tempest-setup.sh script. This patch adds the logic for
when containerized tempest runs in standalone, copying the clouds.yaml
file instead of the stackrc/undercloudrc and also maintain the logic
when you execute tempest in the ovb, copying the rc file.
This also ensures that /var/lib/tempestdata and /var/log/tempest
directories exists in order to not fail the execution of containerized
tempest.
https://tree.taiga.io/project/tripleo-ci-board/issue/836
Depends-On: https://review.openstack.org/#/c/643212/
Change-Id: I90204150085a1c9b943f9e054c0420a8a5e66538
Related-Bug: 1819440
The current cirros image ssh setup fails due to lack of disk space
causing config-drive setup to fail forcing fallback to metadata
server. It is currently seen in OSA and devstack gates.
It might affect tripleo also.
Change-Id: I84a335921f3dd55c8e319c6bed260aa41201cdbb
Related-Bug: #1808010
In order to use it, in any FS, set tempest_format to venv
and then set:
tempest_plugins_venv:
- '<complete upstream openstack tempest plugin git url>'
Change-Id: I95a4449ca3976a914faf1ba5f3f4c0e756a6fefc
Allows us to slowly adopt yamllint by having to fix only the files
we would be touching. Once we lower the amount of differences we
can switch it to run on all files.
In addition to having a more consistent code-style this solves
*recurring* bugs like duplicate dictionary keys, like the one
fixes in this change. Only few weeks ago we had to fix a similar bug.
Change-Id: I2ef275700b420a0f700613568e43576079cd4d03
* https://review.openstack.org/#/c/600517 introduces undercloud_container_cli
to choose docker/podman container runtime. This changes introduces to use
both docker/podman for running tempest tests.
* Tempest user does not have permission to create tempest workspace let's
create the tempest workspace and own it with the tempest user.
* Introducing tempest_data directory to keep tempest related data
which can be used by tempestconf and tempest run to tests
* Cleaned up stuff which is not necessary for tempest container.
Depends-On: https://review.openstack.org/#/c/614664/
Change-Id: I1b19b2580d1d4c9844e9221f18b23655b36b5a23
This is a better way to install and run stackviz, also removing some
tasks that's no longer necessary:
* Remove stackviz static files from the repo, since it's already in the
tarball
* Remove stackviz cloning, and using tarball
* Install stackviz directly from pip module
* Removing tempest-status.yml since we don't use to anything
* Move tempest-results.yml to run before stackviz, so we can take
advantage of the testrepository.subunit file to generate stackviz data
* Use testrepository.subunit to generate subunit2html report instead
of stestr/testr stream
Closes-Bug: 1779562
Change-Id: I918e5401b4f46e947bbaca99a48c2b21bce9a4bd
Since the standalone deployment uses clouds.yaml, we need the validate
tempest role to allow for configuration using OS_CLOUD rather than the
traditional {stack,overcloud}rc files.
Change-Id: Ied55638618b718ee15a2ef5bc2f89cef232b5b7e
Related-Blueprint: all-in-one
* with release to tempestconf-2.0.0 introduces support for volume
microversion but in queens microversions were not configured and
related tests were skipped. But due to tempestconf-2.0.0 in queens
tests are failing. It removes the same from tempest.conf
Closes-Bug: #1791298
Change-Id: I42e436be9e7abbb47e9797735ebf5fabdeb6c2e5
* https://review.openstack.org/592276 marks the release of tempest-
19.0.0. Let's update the role to consume it
* Updated skiplist for rocky
Change-Id: I7a758d740d20ef8036d301aa612184186bac62f0
* On Containerized undercloud, tempest container is already pulled
and pushed to local docker registry through image prepare workflow
and it gets updated if there is an package update. So let's use that
once instead of pulling it again and test the stuff with that.
* new tempest_container_tag would be docker_image_tag + update_containers_append_tag
which comes from image prepare workflow.
* make container_build_id, update_containers_append_tag as cacheable as it is need in
different ansible run
Depends-On: https://review.openstack.org/#/c/589523/
Change-Id: Ibe5a285d6d26c9433d9ff6e7a85f266acc32815a
* In CI, when tempest container is used, It is good to dump the
logs in /var/log/tempest so that it would be easier to verify
tempest container is used.
* Introduced tempest_log_dir and tempest_log_file var to the
validate-tempest-playbook
Depends-On: https://review.openstack.org/#/c/566364/
Change-Id: Ifd72d902c623b3941a6c714e36b7cf104109a570
* undercloud_public_net_name is unused var in validate-tempest
role and is not used anywhere and there is no public network
on the undercloud installation. So it needs to be removed.
Change-Id: I18f63c79ddcf1cf3b1da02e5fef73c81876f2094
* https://review.openstack.org/551441 introduced test_black_regex
in order to skip test directly from featureset files. In order
to avoid confusion between test_regex and test_black_regex,
we are renaming test_regex to test_white_regex.
* Added deprecation message for the same.
Change-Id: I7e6240117c9fbab0e51d6248d74180c7a91dc9e2
* Introduced new workflow for validate-tempest role
- Remove previously created tempest artifacts
- Install tempest
- Create tempest workspace
- Create tempest resources
- Generate tempest configurations
- Run tempest
- Removed unused ironic undercloud fake services
- Removing public network from undercloud as there is no public network
- Added tempest_dir var to give path to tempest workspace
- use {{ working_dir }}/tempest_git for clonning stuff
- Added test_black_regex to skip a series of tests
- Fixed permission issue for tempest_dir to generate subunit files
- Install stestr subunit-filters and subunit for generating subunit files
- Added tempest_skip_master containing skip list for undercloud
- Introduced tempest_conf_version for installing particular version of
tempestconf from git
- Enable set -e to find the exit status of command running inside
container
Change-Id: I021432e222fae0e57a13ca4859ba77e990ac6c67
python-neutron-tests-tempest is only available starting in queens.
older releases use python-neutron-tests
Closes-Bug: #1737227
Change-Id: Iee9ba7ebfad3afbf886a5ac96ada623099d54736
keystone and neutron are the only two tempest plugins
that are required to be installed by default.
Change-Id: I58a955943191045324956bd61da0652076984e8c
Tempestmail is being used in other CI that don't work like the
openstack upstream ci, and logs are stored somewhere else.
This patch allows the possibility to pass --log-url option
to tempestmail script override the default log path.
Change-Id: Iab9b448551d1a2ed59b47f91d5a9bcc204e0a703
This patch gives the option to pass parameters to tempest to
remove certain keys while generating tempest.conf by passing
it as tempest_conf_removal.
The tempest_conf_removal is applied in the final stage and
as such overrides even if values were prevously set.
dvr is default value in tempest_conf_removal - neutron extension.
python-tempestconf gets all available neutron extensions, even if
they are not enabled in a deployment. dvr is not generally configured
so to avoid failed tests, it's removed from tempest.conf
Change-Id: I708866aa3218b2e3c25e9b47672237bc1838d760
Testing image must be configurable in tempest config scripts.
Set it default for temporary github mirror until
1713358 is resolved.
Related-Bug: #1713358
Change-Id: Idc979d60e7ed5d36646d2a9c3fd41e5f85ee15ca
* tempest_extra_config will contains a dict with section.option as
a key followed by values like that
tempest_extra_config:
identity.admin_v2_api: True
* The above defined values will be overriden in tempest.conf with
the help of discover-tempest-config tool
* With this flag we can have more control over tempest.conf while
using it in featureset files.
Change-Id: If787aaf2de0f5a243aceb510f91915db32dc48c1
Adding a .conf in the default name of white list file in order
to be collected by collect-logs role, since it collect all the
files with .conf extension.
Change-Id: I23799a8afe40d6a589d64e1796d7707396b37b2b
In order to run oooq jobs for pike release, we need to add
some variables in validate-tempest:
- add version of tempest to install when using venv (not default).
I'm setting last released version 16.1.0
- Create tests skip file for pike. I'm copying it from master under
the assumption that known issues for master apply also to pike.
Change-Id: If73d9fff826dc13c755041752d2ce1fc3e7d4a84
Since scenarios will start to run tempest, some of them requires
the installation of openstack services tempest plugin. Before, all
the plugins was being installed by default, however the tests being
executed had some conflicts and was decided to not install all at
once, but install only what is required. This patch enables this
behavior.
Change-Id: I9600024b1bd998a15d3aa2629212bdd6591c37d6
Due to scenario jobs starting to execute a set of tempest tests, it's
good to have an option to add these tests in a white list instead
of care about the complexity that a long regex line can bring.
Change-Id: Id1ba7af48173460ef41d9b4df3db723811dc67da
* --isolated, and --failing options does not work
with ostestr as these options are used with testr. Since ostestr
is used as a universal test runner for running tempest across the
board. If the above options got enabled by mistake it will break
the tempest run. So removing it.
* --load-list is replaced by whitelist-files and is taken care
by this review: https://review.openstack.org/#/c/485252/
Change-Id: I9745fdcf592c4dc5bf3cbaf11e9e6b11e209ea77
As Mitaka OpenStack release is now EOL, it is good time to remove
mitaka reference from validate-tempest.
Change-Id: I477b2b38bea79838093d2bd17adfea3f4e3aee11
This patch adds a script that will check all the bugs in the
yaml skip list file which has a bugzilla or launchpad attached
and verify if the bug still open or closed. If closed, it will
remove from the skip list.
This also can generate a report showing what is still open and
what was already closed, as well as generate the skiplist in
tempest/testr format.
Change-Id: Iadd508184c350d074600316ad15f5274df87f01b
This patch set run_tempest default to false. and it's also part of
scenario001 tempest enablement.
All the jobs running tempest explicty set this to true in extra-vars
so default to false won't make any difference.
Change-Id: Ifc26368f4a92c537343026cea9def627ca226cfd