Commit Graph

334 Commits

Author SHA1 Message Date
Ghanshyam Mann cfdaf9a4b9 Add releasenote to tag the Tempest for 2024.1 release
Updating the doc for the same.

Change-Id: Id1e7ac3e21aa713aa1a21740d59b547fd566f23b
2024-03-21 23:20:29 -07:00
Ghanshyam Mann a7b530ce7a End support for stable/yoga
This marks the end of support for stable/yoga
which is moving to 'unmaintained' stage[1].
This remove the stable/yoga jobs and also adds
relesenote to mark the end of support of Yoga.

[1] https://review.opendev.org/q/topic:%22yoga-unmaintained%22

Change-Id: I364bcd69f55753e4c99f66240f743e3bc7205832
2024-02-07 12:34:55 -08:00
Martin Kopec d76178e682 General doc updates
Added missing doc, for example for serial tests, fixed 404 links,
spellings, formatting and etc

Change-Id: I07a8cf9508d3cba33bd04afdc301bb7145153d59
2024-01-31 09:00:16 +01:00
Ghanshyam Mann 2d5987fbe6 Add releasenote to tag the Tempest for 2023.2 release
Updating the doc for the same.

Change-Id: Idfdaabd01cc4e872efca3939e0494b059c14f431
2023-09-22 03:06:56 +00:00
Ghanshyam Mann 3bf5e5b243 Update Tempest supported releases and python versions
Tempest support from stable/yoga to stable/2023.1 and
till python 3.11 but it seems like doc is not updated
for the same.

Change-Id: I7bb5c7b0a39bb833219ed7b0947b88980b4e9f79
2023-09-20 14:29:33 -07:00
Ghanshyam Mann 35fc95dbd0 Fix creation of requested creds within the same project
We have a bug in dynamic creds creation where project creds
with different roles are created under a new projects. Creds
of different role of projects must be created within the same
project.

Fixing the creation of 'project_admin', 'project_member',
'project_reader', 'primary' creds in same projects. All the alt
creds will be created under same projects. but main and alt creds
will use different project, for example, 'project_alt_member'
and 'project_member' creds will be created in different project.

'admin' creds will continue be in new project as many test
use it as legacy admin.

Closes-Bug: #1964509
Change-Id: I9af005e2900195c42ecbbf7434facae2d3952f30
2023-02-22 15:46:42 +00:00
Balazs Gibizer dfb304355b Introduce @serial test execution decorator
Tempest provides a LockFixture to avoid two potentially interfering
tests to run in parallel. However, this solution does not scale when
we want to separate a set of tests from many other test cases. For
example, host aggregate and availability zone testing needs compute
hosts without any nova servers to be able to test moving computes
between aggregates but a lot of other tests are creating nova
servers. To fully separate these aggregate tests from the rest of
the tempest test cases, this patch proposes a @serial class decorator
to mark a test class to be run totally independently of any other test
classes.

Under the hood, the @serial decorator is implemented with a tempest-wide
interprocess read-write lock. The serial test classes always take the
write lock, while the non-serial classes take the read lock. The lock
allows in many readers OR a single writer. So the serial tests are run
independently from the rest.

To minimize the time a serial test blocks other tempest tests run in
parallel, this patch also introduced a serial_tests test directory to
store the serial tests. The current test ordering in a fresh env
uses alphabetical order so the serial tests will run at the end of
the execution not randomly in the middle. The gate uses fresh VMs
for every run so we can rely on this optimization there. In local
envs where tests are re-run, the subsequent runs will be ordered at
runtime by stestr. Therfore, a longer runtime might be observed due to
locking, but the correctness of the test execution is still kept.

Related-Bug: #821732
Change-Id: I0181517edab75f586464a38c4811417f888783b1
2023-01-18 02:45:43 +00:00
Zuul 5f7b9d40fc Merge "Remove references to 'all-plugin' tox environment" 2023-01-05 10:33:34 +00:00
Masayuki Igawa 5fbca522de
[Doc] Fix python version in unit tests document
This commit fixes the python version which is supported officially. We
dropped Python 3.6 in the past commit[1] already and Python 2.7 is not
no longer supported. So we should be able to drop and replace the tox
job in the document.

[1] Ie844887a798a006794729e733748ceaa5e06d2db

Change-Id: If2998dd984aeb7d93bd49659d08a7212e16069c0
2022-10-12 13:39:12 +09:00
Ghanshyam Mann 1d94c7b708 Updated document for the correct stable branches support
With latest Tempest 32.0.0, Tempest master started supporting
Zed and vistoria is not supported as that is in EM state.

Change-Id: I70303bcc8322da1ed9dfcfc1ea0865a70d287237
2022-10-10 13:16:13 -05:00
Lukáš Piwowarski be19f21813 Remove references to 'all-plugin' tox environment
The 'all-plugin' tox environment was deprecated by this patch [1].
Instead of the 'all-plugin' it is recommended to use the 'all' tox
environment.

This patch removes any reference to 'all-plugin' tox environment and
updates the documentation so that the installation steps work with
the 'all' tox venv.

[1] https://review.opendev.org/c/openstack/tempest/+/543974

Related-Bug: 1862240
Change-Id: I0a86d9cf33bff6576a6337967d6b663ab7d9f4a6
2022-10-10 14:01:49 +02:00
Martin Kopec 991866fd75 Fix supported releases and py versions in the doc
Our documentation showed that Victoria, Ussuri and Train are the
supported releases by tempest's master, which wasn't the case.
It also showed that tempest is compatible with python 3.6, 3.7
and 3.8. We dropped py 3.6 and 3.7 support by [1].

[1] https://review.opendev.org/c/openstack/tempest/+/843182

Closes-Bug: #1976282
Change-Id: I0ccfa824f08cbdf58f26a142dac3a0881605ef08
2022-05-31 16:37:29 +02:00
Ghanshyam Mann 38fcb5f026 Remove reference of openstack-health
openstack-health is retiring[1] so removing the
reference of it from doc.

[1] https://review.opendev.org/c/openstack/governance/+/836706/2

Change-Id: I5825fed546283ea23348cfb7371b10c39ad2f88a
2022-04-14 14:33:48 -05:00
Zuul fd708dc272 Merge "Add compute response schema for microversion 2.75" 2022-03-09 13:44:19 +00:00
Zuul 5a7cc775a3 Merge "Add schema for compute 2.45 microversion" 2022-03-09 06:23:32 +00:00
Ghanshyam Mann 1a65c4f0b1 Add compute response schema for microversion 2.75
Compute microversion 2.75 made the rebuild and update server
response same as get servers
- https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id68

Partially Implements: blueprint fix-microversion-gap

Change-Id: I1728e5de0e815def0fe31c242203bd9f36dc41f4
2022-03-05 10:28:21 -06:00
Zuul 5549fd2a04 Merge "Fix compute quota classes schema for v2.50 and v2.57" 2022-03-04 09:17:34 +00:00
Ghanshyam Mann 3ee9f2167d Add schema for compute 2.45 microversion
Compute microversion 2.45 changed the response
of create image and create backup APIs.

create image schema already covered
- fec2c93cdc/tempest/lib/api_schema/response/compute/v2_45/images.py

This patch adds the schema for create backup API.

Partially Implements: blueprint fix-microversion-gap

Change-Id: Icfeec8957c58472e7f176680d48745b5c5350b55
2022-03-03 21:53:05 -06:00
Zuul b87a29b689 Merge "Fix server group schema for compute microversion 2.64" 2022-03-03 22:59:29 +00:00
Zuul 268606a720 Merge "Add fields in hypervisor schema for 2.33 and 2.53" 2022-01-17 02:53:57 +00:00
Ghanshyam Mann dc517e0a1c Fix compute quota classes schema for v2.50 and v2.57
For Compute microversion 2.50:
1. fixed_ips, floating_ips, security_group_rules and security_groups
    are removed from:
      * GET /os-quota-class-sets/{id}
      * PUT /os-quota-class-sets/{id}
2. server_groups and server_group_members are added to:
      * GET /os-quota-class-sets/{id}
      * PUT /os-quota-class-sets/{id}

For Compute microversion 2.57:
1. injected_file_content_bytes, injected_file_path_bytes, injected_files
    are removed from:
      * GET /os-quota-class-sets/{id}
      * PUT /os-quota-class-sets/{id}

Partially Implements: blueprint fix-microversion-gap

Co-authored-by: Ghanshyam Mann <gmann@ghanshyammann.com>

Change-Id: I2a9dcb20251d9682fbcafd706ac60325a60e35ef
2021-12-17 00:45:43 +00:00
zhufl ed6d10235c Fix server group schema for compute microversion 2.64
For Compute microversion 2.64:
1. change policies to policy in:
   * GET /os-server-groups
   * POST /os-server-groups
   * GET /os-server-groups/{server_group_id}
2. add rules in:
   * GET /os-server-groups
   * POST /os-server-groups
   * GET /os-server-groups/{server_group_id}
3. remove metadata from:
   * GET /os-server-groups
   * POST /os-server-groups
   * GET /os-server-groups/{server_group_id}

The testcases of create_server_group should also be modified.

Co-authored-by: Ghanshyam Mann <gmann@ghanshyammann.com>

Change-Id: I8fc37415c968eb64380b8bd66b713aa6b733578f
2021-12-17 00:45:11 +00:00
zhufl 6bc98ccfaa Add fields in hypervisor schema for 2.33 and 2.53
1. For microversion 2.33
   hypervisor_links is added to the response body for the following APIs:
     - GET /os-hypervisors
     - GET /os-hypervisors/detail

2. For microversion 2.53
   servers is added to the response body for the following APIs:
     - GET /os-hypervisor
     - GET /os-hypervisors
     - GET /os-hypervisors/detail

This is to fix the hypervisor schema for microversion 2.33 and 2.53.

Partially Implements: blueprint fix-microversion-gap

Co-authored-by: Ghanshyam Mann <gmann@ghanshyammann.com>

Change-Id: Ifeb26a50b604d257e9ab00cceed7cd2f34621419
2021-12-17 00:45:04 +00:00
James Parker ef0da21010 Add flavor extra spec validation tests
Tests cover the addition of flavor extra spec validation [1,2]. Test
validates the ability to create a flavor with extra specs using a custom
namespace. This change was introduced in 2.86 [3].

[1] https://blueprints.launchpad.net/nova/+spec/flavor-extra-spec-image-property-validation-extended
[2] https://review.opendev.org/c/openstack/nova-specs/+/682655/
[3] https://docs.openstack.org/nova/latest/reference/api-microversion-history.html#id79

Change-Id: I20267054af04da3753795f7008abcf1118a31bdd
2021-12-16 11:22:24 -05:00
Martin Kopec 425b486d9b Update Interop doc
The commit replaces DefCore committee (a former name) by
Interop Working Group (the current name) and updates a few
more old interop references.

Change-Id: I8bd05246e42c0e98a9825a4621ac9c3ed714d1dc
2021-11-08 10:41:24 +00:00
Zuul 165586d190 Merge "Add tests for "new" Transfers API (mv 3.55)" 2021-06-02 18:11:25 +00:00
Ghanshyam Mann 3e12764065 Update IRC network to OFTC
Change-Id: I615c6a8c78cb293903f9a778ddc2230c2d5edfd2
2021-05-31 15:04:48 -05:00
Brian Rosmaita cafd2d1656 Add tests for "new" Transfers API (mv 3.55)
Add a client and test class for Block Storage API v3.55

Change-Id: I4179799bc40a50bdbeecaee461d32e827269cc1f
2021-05-20 22:28:26 -04:00
songwenping c7845314f5 Remove unicode from code
In python 3, all strings are considered as unicode string.

This patch drops the explicit unicode literal (u'...')
or (u"..") appearances from the unicode strings.

Change-Id: I64af1c7c7415cd4ab99b6a913fb84fecbc0a3bf1
2021-04-02 08:49:22 +00:00
Zuul d4f7129d57 Merge "Add tempest.scenario.manager as stable interface for plugins" 2021-03-26 12:52:08 +00:00
Ghanshyam Mann c2ca52dced Add tempest.scenario.manager as stable interface for plugins
Implements: blueprint tempest-scenario-manager-stable
Change-Id: I5da35e4aafeb55b6754fd9cb86eeea04f869686e
2021-03-25 21:29:14 -05:00
zhufl e9ed6092ec Add delete_on_termination for compute API v2.79 - os-volume_attachments
This adds delete_on_termination field for the compute API response schema
for the 2.79 microversion in the following APIs:

  - GET /servers/{server_id}/os-volume_attachments (list)
  - GET /servers/{server_id}/os-volume_attachments/{volume_id} (show)
  - POST /servers/{server_id}/os-volume_attachments (attach)

Change-Id: I7a9b22477b800c6d60acb67c27aff5dcb61d7875
2021-03-25 13:28:24 -05:00
Ghanshyam Mann 672eee1d31 Add documentation on the usage of keystone's scope & roles
As Tempest support the keystone's scope and new default roles
like reader, this commit document that how those can be
requested and used in Tempest or its plugins tests.

Change-Id: Iebacbeda231f82d6d16dbdcde635f19ae862181f
2021-03-12 09:54:39 -06:00
Zuul 4de12b1113 Merge "Update microversion doc for compute microversion 2.3" 2021-02-10 16:36:48 +00:00
Zuul eae384ac1d Merge "Inclusive jargon" 2021-02-09 22:33:47 +00:00
Ghanshyam Mann bfce1f219b Change Review merge policy from two +2 to single +2 requirement
Now a days, Tempest does not have much active core reviewers due
to that merging the incoming code change is taking too much time.

As you can see we have lot of backlogs in open review ~400

If situation improve in future and we have more Core review
then we can change the policy back to two +2 requirement.

Discussion in QA office hour:
-  http://eavesdrop.openstack.org/meetings/qa/2021/qa.2021-01-26-14.00.log.html#l-147

Change-Id: Ic005de3e56cb4fde9cfce183ee598c7e9f77f464
2021-02-03 16:50:34 -06:00
Martin Kopec dc84423b72 Inclusive jargon
Following stestr's example where arguments such as --blacklist-file,
--black-regex and --whitelist-file are deprecated since its
3.1.0 release, let's do the change here as well in order to
get tempest consumers some time for the transition.

This change deprecates the following arguments and replaces them
by new ones which are functionally equivavelnt:
* --black-regex is replaced by --exclude-regex
* --blacklist-file is replaced by --exclude-list
* --whitelist-file is replaced by --include-list

For now, Tempest will accept both (new and old) arguments to make
the transition smoother for all consumers.

The patch also bumps min version of tox to 3.18.0 in order to
replace tox's whitelist_externals by allowlist_externals option:
https://github.com/tox-dev/tox/blob/master/docs/changelog.rst#v3180-2020-07-23

Change-Id: I3e09b31f63d2cd7ea41c48e62432bd3bc54fcf44
2021-01-20 08:40:57 +00:00
Ghanshyam Mann 983e1ceb44 Update microversion doc for compute microversion 2.3
compute microversion 2.3 schema is already covered in
I2f36a51fa47b28df1afada5396dec3ce7dc43d78

This commit update the doc for the same.

Change-Id: I58160cc2bd42cabce1abc482dd769e2323c90026
2020-12-29 14:36:10 -06:00
Ghanshyam Mann f8625f0de5 Remove stable/stein testing jobs
stable/stein is not suported in Tempest now, so
let's remove their jobs from master gate.

Also, update doc to reflect it.

Depends-On: https://review.opendev.org/c/openstack/stackviz/+/768212

Change-Id: I0cbc2883e1cfdb3b1d6b10a7ecea83a4bae32afb
2020-12-22 14:46:23 +00:00
Roman Dobosz edda62ab35 Fix indentation for the example code in documentation.
Change-Id: I454c22aaf1ffa56c37c0105ab76aa70d28983f43
2020-12-07 10:18:42 +01:00
zhufl fb1e4db2bf Fix incorrect compute api microversion history links
Some compute api microversion history links are incorrect, this
is to fix them.

Change-Id: Ied16a88afa2799d6b5b5cb858851ead44ce2e081
2020-10-29 14:34:13 +08:00
Lukas Piwowarski a9055010fc Create documentation for tests
Generate documentation for each test from its docstring.

The sphinx-apidoc tool is used to generate the .rst
files necessary for the generation of the documentation.

When the documentation is being build only functions that
start with "test" are passed to the resulting documentation.

Change-Id: I1878f67e10df9bb1f8289beda8e59a56b5057fe4
2020-09-02 09:46:00 +02:00
Zuul b652e78103 Merge "Enable xindy for PDF build" 2020-08-04 10:05:45 +00:00
Zuul 50ec7d74c3 Merge "Document the process to pin the constraint in tox.ini" 2020-07-22 03:23:41 +00:00
Ghanshyam Mann 0aa0636a09 Fix tempest plugin cookicutter repo path
cookicutter expect ether git+<repo-url> or it has the
'git' in url, otherwise it give error
- http://paste.openstack.org/show/795952/

plugin cookicutter README is fine but tempest doc does
not mention about it and end up having error
- https://opendev.org/openstack/tempest-plugin-cookiecutter

Change-Id: Ifb12011ac901be57d49a6ae623a24a916837c07d
2020-07-15 13:42:07 -05:00
Zuul ea55a17ee2 Merge "Fix pygments style" 2020-07-14 16:16:59 +00:00
Ghanshyam Mann d89122be41 Document the process to pin the constraint in tox.ini
As discussed in Victoria PTG[1], we need to start pining
the constraint in tox.ini on Tempest new release so that our
tox.ini can match with compatible constraints with Tempest version.

[1] https://etherpad.opendev.org/p/qa-victoria-ptg

Change-Id: Ib705f59ef3aea93b2778cc87fc44d390b5a62c94
2020-07-07 13:11:42 +00:00
Zuul 3800158de0 Merge "Remove deprecated scenario image options" 2020-07-06 13:04:36 +00:00
Martin Kopec 02af6a44cc Remove deprecated scenario image options
Let's remove deprecated scenario image options which have been
deprecated for more than 4 years.

From now on CONF.scenario.img_file will need a full path for the image.
CONF.scenario.img_dir was deprecated and will be removed in the next
release. Till Tempest 25.0.0, old behavior is maintained and keep working
but starting Tempest 26.0.0, you need to specify the full path
in CONF.scenario.img_file config option.

Change-Id: I96d26ad3889f55d867fdea016953cce522a6fbdf
Related-Bug: #1393881
2020-07-05 10:16:18 +00:00
Ghanshyam Mann aaaafbb5b8 Fix pdf-doc build by skiping the sample file
Tempest pdf doc build includes the sample config
file which is large in size and end up with
error 'Dimension too large'

- https://zuul.opendev.org/t/openstack/build/84d73ec987d04c05acc54b231d33067e/log/sphinx-build-pdf.log#1874

Its happening while adding new config option in
https://review.opendev.org/#/c/686073/ which means we
are at the edge of pdf build limit.

We should not include the sample config file as part
of pdf build.

Change-Id: I59383505989705d35d86b19b5c1aaafebd2bd307
Closes-Bug: #1886212
2020-07-03 11:24:43 -05:00