Commit Graph

63 Commits

Author SHA1 Message Date
Jonathan Rosser 242203bafa Allow include/exclude lists to be defined in many variables
Previously the os_tempest role had a single variable for
defining the include and exclude lists in order to select the
tempest tests to run.

This works for simple scenarios, where a single service
is deployed and the tests for that service are enabled through
the necessary variables set in user_variables. This approach is
used in openstack-ansible CI / AIO.

More complicated scenarios such as magnum+barbican+octavia, would
create several user_variables files each with conflicting settings
for the test settings. It is possible in this scenario for there to
be no valid tempest tests to run and tempest to fail immediately.

This patch adds the possibility to have many variables defining
the include/exclude lists which have names using a common prefix.
Any variable names matching the prefix are gathered and combined
with the original role default to make extending the test lists
easy to do in an incremental/distibuted way in the ansible variables
instead of having to maintain a single point defining all necessary
tests.

Change-Id: Ie3a9a7be849171af042567ba8a152e5df5d2cb53
2023-08-17 11:20:14 +00:00
Jonathan Rosser 9ec945a0a3 Rename includelist/excludelist file path vars
These variable names collide with the best name prefix to use when
gathering many variable names into a combined include/exclude list.

Codesearch shows no use of these as overrides and the number of
cases when an override would be needed is small.

Change-Id: Ic64165e41d24ae8dc75061589de84fa57998f03d
2023-08-16 10:25:30 +01:00
Jonathan Rosser 47deba8c31 Remove deprecated variables
More suitable replacement variables have been available for
several releases now so we can remove the deprecated vars.

Tripleo still defines these deprecated variables in several
places but seems to have made no attempt to move to the new
vars, and tripleo is itself now deprecated so we should move
ahead with removal of these vars from os_tempest.

Change-Id: I5a4a90bc963acc8b44caf7eb060b763e0f90a50f
2023-08-16 09:03:23 +01:00
OpenStack Release Bot 62f7c7f89f Update master for stable/zed
Add file to the reno documentation build to show release notes for
stable/zed.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/zed.

Sem-Ver: feature
Change-Id: Ia808641ab5446707ec5296dab6198c260b1b4477
2022-12-13 13:21:37 +00:00
Damian Dabrowski e1ca4b83bf Allow to define extra tempest plugins
Allow to define extra tempest plugins without overriding the whole
`tempest_plugins` list.

Change-Id: Ie00ffb42ceb49331c997f1a700a9423c12a32684
2022-07-12 08:39:13 +00:00
Zuul 9e49925f3d Merge "Add support for both Credential Provider Mechanisms" 2022-04-27 14:47:34 +00:00
Zuul 601db553f8 Merge "Allow to create only specific tempest resources." 2022-04-27 14:36:42 +00:00
Damian Dabrowski 0566c20d99 Add support for both Credential Provider Mechanisms
Tempest currently has two different internal methods for providing
authentication to tests: dynamic credentials and pre-provisioned credentials.
Depending on which one is in use the configuration of Tempest is slightly
different.
We should provide a support for both of them.

https: //docs.openstack.org/tempest/latest/configuration.html#credential-provider-mechanisms
Change-Id: I26d69caa3f96a530bc0a4a21365404b1a84e489a
2022-04-26 16:45:14 +00:00
Damian Dabrowski 1baf863bf6 Allow to create only specific tempest resources.
In most cases, many of tempest resources are not needed.
From the other hand, some of our contributors still need them so we
can't just drop this functionality from this role.
The best solution I see is to add more flexibility and let users
choose which resources should be created by tempest role.

- public network:
  new variable: bool tempest_public_net_create (default: true)
  if false: require tempest_neutron_public_network_id to be set
  if true: require all other tempest_public_net_* variables to be set
  explanation: it'd be useful to choose if user wants to use already
    existing public net or create a new one/
- private network:
  new variable: bool tempest_private_net_create (default: false)
  if true: require all tempest_private_* variables to be set
  explanation: by default tempest has use_dynamic_credentials &
    create_isolated_networks enabled, so it spawns an usable network,
    subnet, and router when needed for each project it creates, so in
    most cases it doesn't make sense to create special 'private' net.
- router:
  new variable: bool tempest_router_create (default: false)
  if true: both tempest_public_net_create and
    tempest_private_net_create should be enabled
  explanation: same case as for private network
- image:
  new variable: bool tempest_images_create (default: true)
  if false: require tempest_glance_image_id_1,
    tempest_glance_image_id_2 to be set
  explanation: tempest needs public images, so it'd be useful to just
    use already existing ones
- flavor:
  new variable: bool tempest_flavors_create (default: true)
  if false: require tempest_nova_flavor_id_1, tempest_nova_flavor_id_2
    to be set
  explanation: tempest needs public flavors, so it'd be useful to just
    use already existing ones
- projects:
  new variable: bool tempest_projects_create (default:
    "{{ tempest_public_net_create or tempest_private_net_create or
      tempest_public_router_create }}")
  explanation: by default tempest has use_dynamic_credentials &
    create_isolated_networks enabled, so we don't need to create any
    'static' projects, but network resources spawned by tempest
    playbook belong to this project, so it should exist if any of
    these resources are going to be created

I don't see any reason why users and tempest role should be created
so i dropped this functionality.

Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/825166
Change-Id: Icb46f5cb9e2dda6511cc49e30587f541aa7e399f
2022-04-26 16:44:02 +00:00
Zuul 4ae36d5828 Merge "do not include [*-feature-enabled] sections in tempest.conf" 2022-04-26 15:42:26 +00:00
Damian Dabrowski f82ea26fcb do not include [*-feature-enabled] sections in tempest.conf
We should not include [*-feature-enabled] sections in tempest.conf and use
``tempest_tempest_conf_overrides`` for these customizations instead.

There is no way we can keep feature list updated all the time and cover all
possible features in defaults/main.yml.
That's why I think we shouldn't cover them here at all.

If we need these customizations for testing purposed(tested it and I think we
don't), we should define them using ``tempest_tempest_conf_overrides`` only for
these tests.

Change-Id: If7d7ca5ae76cb3cb24f95fb9608ce3e1182d5e5a
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/825166
2022-04-14 22:52:04 +02:00
Zuul 169e8c1452 Merge "Implement variable: tempest_endpoint_type" 2022-04-14 02:43:36 +00:00
Zuul 2402146bf7 Merge "Deprecate unnecessary variables" 2022-03-21 18:39:17 +00:00
Dmitriy Rabotyagov 1900f0ff6b Deprecate congress and lxd tempest plugins
Congress is no longer maintained and no reason to carry on it's
tempest testing.

LXD driver is also not supported by nova and project has been deprecated
back in Train.

Tempest tests for neutron vpnaas and bgpvpn were moved to neutron tempest
plugin with [1] and [2]

[1] 4077093c10
[2] 44866dadba

Change-Id: I161167b995a4cd3ee532719fa5fcf1f1a6cacb73
2022-01-24 16:08:46 +00:00
Damian Dabrowski 23387a5fbd Implement variable: tempest_endpoint_type
``tempest_endpoint_type`` variable has been implemented to avoid having endpoint type hardcoded in tempest.conf

Change-Id: Id218dcdfa788c184074d15519ac170d938290bd4
2022-01-19 17:32:06 +00:00
Damian Dabrowski 18b2cba8cc Deprecate unnecessary variables
It is really unlikely to change these variables:
``tempest_network_tenant_network_cidr``, ``tempest_network_tenant_network_mask_bits``
and ``tempest_fatal_deprecations``

To keep tempest role as simple as it is possible, we should deprecate them.
They still can be changed using ``tempest_tempest_conf_overrides``.

Change-Id: Ia95f10c3548a5773c286e7d7075b71861fcef28b
2022-01-18 23:54:43 +01:00
Jonathan Herlin 0e71336651 Rename whitelist and blacklist in role
'--whitelist-file' option is deprecated, use '--include-list' instead which is functionally equivalent.
Right now Tempest still supports this option for backward compatibility, however, it will be removed soon.

This change addresses the occurences of whitelist and blacklist in the os_tempest OSA role.

Change-Id: Id021d084867ef2891c381c5863b82421e90ee2ca
2021-05-25 08:15:40 +00:00
Martin Kopec f82d248332 Remove deprecated scenario image option
Removing deprecated CONF.scenario.img_dir option as it's beeing removed
from Tempest after ~4 year depreation period [1].

Currently tempest is throwing a warning if img_dir is still used.
Both behaviors (img_dir and a full pathin img_file) are supported until
tempest 25.0.0, starting tempest 26.0.0 a full path in
CONF.scenario.img_file config option will have to be specified.

[1] https://review.opendev.org/#/c/710996/

Related-Bug: #1393881
Change-Id: I949cbc6347030461a2daf0658d1ffef1d7aa0e87
2020-07-13 09:17:25 +00:00
Zuul f88a90e7ac Merge "Use newer openstackdocstheme and reno versions" 2020-06-05 11:45:08 +00:00
Zuul afb8aa66a6 Merge "Update docstheme for style" 2020-06-03 18:20:45 +00:00
Andreas Jaeger a0c9e56003 Use newer openstackdocstheme and reno versions
The sync from https://review.opendev.org/733244 updated to
openstackdocstheme 2.2.1 and reno 3.1.0 versions.

Set openstackdocs_pdf_link to link to PDF file. Note that
the link to the published document only works on docs.openstack.org
where the PDF file is placed in the top-level html directory. The
site-preview places the PDF in a pdf directory.

openstackdocstheme renames some variables, so follow the renames
before the next release removes them. A couple of variables are also
not needed anymore, remove them.

See also
http://lists.openstack.org/pipermail/openstack-discuss/2020-May/014971.html

Change-Id: I83e11880da07e44af82df61e3bc542058561fe4c
2020-06-03 20:06:11 +02:00
OpenStack Release Bot 9866c53c26 Update master for stable/ussuri
Add file to the reno documentation build to show release notes for
stable/ussuri.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/ussuri.

Change-Id: Ic24456451d2e7724f2a48e9b102d97482829a2c0
Sem-Ver: feature
2020-06-01 13:19:06 +00:00
Andreas Jaeger bb0e2244cc Update docstheme for style
New version of openstackdocstheme (Victoria+) respects pygments_style.
Since this repo is using now Victoria (master) requirements but has
not branched for Ussuri yet, it uses the new version.

Change pygments_style to 'native' since old theme version always used
'native' and the theme now respects the setting and using 'sphinx' can
lead to some strange rendering.

Change-Id: Ia33e56d772826744a0ae4962fb4541a54a567b4e
2020-05-20 18:52:11 +02:00
Dmitriy Rabotyagov 1003438006 Added tempest_tempestconf_profile_ specific vars
Added tempest_tempestconf_profile_extras and
tempest_tempestconf_profile_overrides variables

tempest_tempestconf_profile_extras is designed to extend basic
tempest_tempestconf_profile dictionary when required. New var has
presedence in case of having same keys defined. It can be used
to define image or other extra params without need of overriding
defult things

tempest_tempestconf_profile_overrides var is a replacement for
tempest_tempest_conf_overrides which was used in inapropriate way
as tempestconf profile overrides have pretty different format from what
is used by config_template.

[1] https://docs.openstack.org/python-tempestconf/latest/user/profile.html

Related-Bug: #1868938

Change-Id: Ib1e78a6ce2085df82d2a22db6817e800432f2f05
Signed-off-by: Chandan Kumar (raukadah) <chkumar@redhat.com>
Co-authored-by: Dmitriy Rabotyagov (noonedeadpunk) <noonedeadpunk@ya.ru>
2020-03-29 12:55:29 +05:30
OpenStack Release Bot 44d5b2a240 Update master for stable/train
Add file to the reno documentation build to show release notes for
stable/train.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/train.

Change-Id: I892eb9c004666603873052e33e51610fca2082c0
Sem-Ver: feature
2019-10-22 19:02:37 +00:00
Chandan Kumar ecdc0eedaf Update openstack.org -> opendev.org
It updates the url from git.openstack.org to opendev.org and
also remove cgit from opendev.org.

Keeping vagrantfile, run_tests.sh and bindep.txt as itis
as these are updated automatically by openstack-ansible-tests.

Change-Id: Ia449079776e0339515ae1a2bca7553b03d9cde1c
Signed-off-by: Chandan Kumar <chkumar@redhat.com>
2019-05-30 15:29:22 +05:30
Zuul 77a6739bec Merge "Update master for stable/stein" 2019-04-08 21:48:17 +00:00
Chandan Kumar 8e5086e090 tempest_services var for setting tempest_service_available_{}
Currently os_tempest is used by multiple deployement tools
and each of them have different methods of configuring
services and currently tempest_service_available_{service_name}
are setted based on group vars and group vars is not used at
all places which leads enable user to specifically define a
list of tempest_service_available_{service_name} which is too
hectic.

By introducing tempest_services:
                 - aodh
                 - neutron
will set those vars automatically and makes it easier for
handling tempest_service_availabe vars.

Depends-On: https://review.openstack.org/#/c/649653/

Change-Id: I8734f38206b73e6547a6ec545f07f625e844ed66
Signed-off-by: Chandan Kumar <chkumar@redhat.com>
2019-04-08 16:51:52 +05:30
OpenStack Release Bot 00893906a5 Update master for stable/stein
Add file to the reno documentation build to show release notes for
stable/stein.

Use pbr instruction to increment the minor version number
automatically so that master versions are higher than the versions on
stable/stein.

Change-Id: I71cfa11cc27aa4f97301f550efa1724ff80a2063
Sem-Ver: feature
2019-04-08 09:21:50 +00:00
Chandan Kumar 2dd874a7f3 Rename tempest_public_net_physical_{type to name}
tempest_public_net_physical_type var stores the physical network
name but due to the name of the var it looks confusing. This
patch renames the var and providing that user can still use
tempest_public_net_physical_type var.

Change-Id: I45f63426a105ada0715c665a1289ed6d09911202
2019-01-16 16:58:13 +05:30
Zuul 813266595e Merge "Remove tempest_image_dir_owner var" 2019-01-11 15:15:26 +00:00
Chandan Kumar 2b9a63e1f6 Remove tempest_image_dir_owner var
tempest_image_dir_owner is used to create tempest_image_dir.
and sets proper directory permission, user and group.
Let's use default ansible user to create the directory instead
of restricting on tempest_image_dir_owner user.

Change-Id: I4ccad998f2b9ef84dbe7f645f31960b38c6fd27a
2019-01-11 16:53:34 +05:30
Arx Cruz 4c3e0ba4e4 Better tempest blacklist management
This patch set the information regarding how the tempest_test_blacklist
variable should be configured. From now, each test in the blacklist
should have a launchpad and/or bugzilla tighted on it in order to
trackdown the progress of the test, as well as use other tools to
automate the verification of the status of those bugs.

Change-Id: I36c41d97ca778d892c1655731cca9ac0112851c1
2019-01-08 16:47:32 +01:00
Arx Cruz 55a8b2b34d Enable python-tempestconf support
This is an initial support for tempestconf tool.
Initially, tempestconf tool is being executed to generate the
tempestconf as it is, furthermore will be added the option to override
values, generate resources, hardcoded options, etc.
This is will be done in an incremental way in order to not have big
patches full of functionalities that might not be able to be fully
tested in one single patch.

Change-Id: I08cfa29361804de3bdb9077a7d8d2b8b417e50d8
2018-11-07 15:42:34 +01:00
Chandan Kumar 7e6e614ff0 Added support for installing tempest from distro
In order to consume os_tempest role across different deployment
tools, following changes are introduced:
- tempest_install_method:distro|source through which user can install
  tempest from source as well as distro
- tempest_workspace directory is added as when tempest is installed
  from distro, we need a place where tempest can be executed. It
  generalised the use of tempest_workspace at each place.
- The default location of the default tempest configuration is now
 /etc/tempest/tempest.conf rather than the previous default of
  $HOME/.tempest/etc.

Change-Id: Ie280cfa6b93fa65c56ed98066deb4f9c82a1a8db
2018-10-26 08:19:51 +00:00
Zuul 360620691f Merge "Update reno for stable/rocky" 2018-08-10 18:32:23 +00:00
OpenStack Release Bot 8be55786e3 Update reno for stable/rocky
Change-Id: I33f3b983b8f05f6b7ae192c5bf99d81fdcb916e9
2018-08-10 16:48:37 +00:00
Jesse Pretorius 81f6bda4c0 Execute service setup against a delegated host
In order to reduce the packages required to pip install
on to the hosts, we allow the service setup to be delegated
to a specific host, defaulting to the deploy host.

1. We no longer need tempest_requires_pip_packages,
   so it is removed.
2. The openstack_openrc role is executed in the playbook,
   so we do not need it as a meta-dep any more.
3. All uses of our custom keystone module have been
   replaced with standard Ansible modules.
4. Using the service delegation model, it makes no sense
   to run the resource creation on all hosts, so we
   use run_once instead.
5. Given that we're using developer mode for master
   integrated builds, and pip.conf is present, we need
   to use the CLI to create the virtualenv so that we
   can use the '--never-download' option.

Change-Id: I572db645b64a4a0ac6255bc36eaac1be13336989
2018-08-10 11:57:13 +00:00
Jesse Pretorius 2cda0e178e Add tempest test blacklist capability
While the whitelist may contain regex, sometimes using regex
is difficult and it is simpler just to blacklist the specific
test you're looking to block.

This patch adds the capability to specify a list of tests to
blacklist.

Change-Id: I16bd158ff8cd2e9b1ff42876dcf7796676f54b68
2018-08-02 20:24:48 +01:00
OpenStack Release Bot c7fdd124ac Update reno for stable/queens
Change-Id: Id9959e58d8cbf11bb8ed1c8384d45b4b9d7eb663
2018-02-14 15:50:57 +00:00
Jimmy McCrory 960422c5b9 Add task to allow creation of provided roles
Some tempest tests require that a specific role is available for use.
Add a new variable, `tempest_roles`, to allow users to provide a list of
roles to create.

The existing tasks that create the 'reseller_admin' and 'remote_image'
roles have been removed. The correctly named 'ResellerAdmin' role is
already created by the os_swift role and the 'remote_image' is no longer
required since the glance tests make use of an admin user.

Depends-On: I66b7c639494d98fdb3baa52fe56c670e32dea3ce
Change-Id: I7987c5a4dbf50e97322a5b348c3cbd817b2672d6
2017-11-21 23:49:03 +00:00
Andreas Jaeger 1b4770087b Remove setting of version/release from releasenotes
Release notes are version independent, so remove version/release
values. We've found that projects now require the service package
to be installed in order to build release notes, and this is entirely
due to the current convention of pulling in the version information.

Release notes should not need installation in order to build, so this
unnecessary version setting needs to be removed.

This is needed for new release notes publishing, see
I56909152975f731a9d2c21b2825b972195e48ee8 and the discussion starting
at
http://lists.openstack.org/pipermail/openstack-dev/2017-November/124480.html
.

Change-Id: I4c63f5907d2b698b7025266829278bb434b921bd
2017-11-17 08:06:19 +01:00
Jesse Pretorius 5c307f9b20 Use the deployment host to download images for tempest
Currently there are two sets of image downloads happening,
which is a common cause of failure in jobs due to the fact
that they're downloaded from a URL outside of OpenStack-CI.

This patch consolidates them into a single single download,
but also uses the deploy host to do the download. The download
gets done to the same location as where the image is cached
in OpenStack-CI, so as long as the SHA256SUM matches, it
will skip the download - making our tests faster and more
reliable.

As part of this, some legacy and unnecessary variables have
been removed. Also, we were uploading the same image twice
into Glance which is unnecessary. Instead we re-use the same
image ID twice in the tempest configuration.

Change-Id: I8d0afd08f6c80594d69b8d711261ae6365fad917
2017-10-23 10:39:16 +01:00
Jesse Pretorius 37439954e4 Initial OSA zuul v3 role jobs
This patch implements an initial set of jobs intended to match
the current job execution method. It does not intend to improve
how the jobs are executed - only to replicate what is currently
in openstack-infra/openstack-zuul-jobs and provide the platform
to iterate on.

Depends-On: I0391ec310c4eede436011a48490e3c524c8ddf4d
Change-Id: I695084aa70d6bd48cd8d498556918a1fbbc3c2a2
2017-10-18 17:32:14 +00:00
OpenStack Release Bot 2c065ca5a6 Update reno for stable/pike
Change-Id: I4e5ef394f564126422b9b8bd05964676a4651c74
2017-08-25 10:46:43 +00:00
ZhongShengping 62ed92e7b1 Fix openstackdocstheme settings
To use openstackdocstheme 1.11.0 properly, this patch fixes
some settings according to follow[0].

[0]https://docs.openstack.org/openstackdocstheme/latest/

Change-Id: I7ba88c5498478bef3c601620a20be8425c79c611
2017-07-03 16:22:41 +08:00
ZhongShengping 6cd1f12c58 Switch from oslosphinx to openstackdocstheme
As part of the docs migration work[0] for Pike we need to switch to use the
openstackdocstheme.

[0]https://review.openstack.org/#/c/472275/

Change-Id: Id5969a88aa5a3230433dd8f6f2a6ff5dd141f278
2017-06-26 11:39:49 +08:00
Jesse Pretorius e33c72c36e Force the use of the new tempest testing method
The old tempest execution method was deprecated in
Newton. We need to shift over to using the new method
which is more reliable and easier to configure.

Change-Id: Idb750cd75f82b4af7cc7c66ce3dca21d2a7066c8
2017-03-06 15:13:10 +00:00
Matt Thompson 28d2535227 Refresh multi-backend variables
This commit removes some deprecated tempest multi-backend variables
(tempest_volume_backend1_name, tempest_volume_backend2_name)
and adds a new variable tempest_volume_backend_names.

Change-Id: Ie62466940f81a96a9e42e237fc2609a98596f156
2017-02-15 17:04:34 -05:00
Cao Xuan Hoang 34823718a7 Use https instead of http for git.openstack.org
Trivialfix

Change-Id: Id966d1935b94b7f19bca4987093314e978e9daca
2017-02-06 17:14:03 +07:00