As per 2023.1 cycle testing runtime and community goal[1] we need to migrate
upstream CI/CD to Ubuntu Jammy(22.04). Most of the Tempest jobs will be migrate
automatically once devstack base job start running on Jammy(Depends-On).
As we need to run at least one job on Ubuntu Focal(20.04), I am adding
a new job tempest-full-ubuntu-focal.
Few job like tempest-full-multinode-py3 and plugin sanity job needs to override
the nodeset. multinode job needs to run on Focal till stable/zed and on Jammy
from master(2023.1) onwards. Stable branch jobs testing stable branch needs to
keep running on their supported distro version which is Focal.
Depends-On: https://review.opendev.org/c/openstack/devstack/+/860795
Depends-On: https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/861116
[1] https://governance.openstack.org/tc/goals/selected/migrate-ci-jobs-to-ubuntu-jammy.html
Change-Id: I2e37603b098880b778100613800496b01167b22f
As Tempest is branchless and needs to support older
supported stable branches, let's test it with all the
python version supported in those supported stable branches.
For that, we need to explicitly dd the py jobs instead of using
the template. This way, we will explicitly know that we are testing
all supported python versions when the new release template bumps the
min python version in the template.
In Yoga testing, we are making py3.9 job to run as voting, so
adding py3.9 job as voting.
Change-Id: I422da8da06509e529f4091e21cdc3d392d3cf910
Setuptools v54.1.0 introduces a warning that the use of dash-separated
options in 'setup.cfg' will not be supported in a future version [1].
Get ahead of the issue by replacing the dashes with underscores. Without
this, we see 'UserWarning' messages like the following on new enough
versions of setuptools:
UserWarning: Usage of dash-separated 'description-file' will not be
supported in future versions. Please use the underscore name
'description_file' instead
[1] https://github.com/pypa/setuptools/commit/a2e9ae4cb
Change-Id: I532be460dc673548be9ed3b540e9feb8ca832fbe
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Tempest CLI tempest-account-generator was deprecated
4 years ago.
devstack still uses the deprecated CLI for which
patch has been proposed.
As deprecation is done long time back,
'tempest-account-generator' should be now used as
'tempest account-generator'
The following commit gives the idea of deprecation
message:-
68b8b9d26f
Depends-on: I624e1dc57a3d3533322fb298c01f70241d0400ed
Signed-off by: Soniya Vyas<svyas@redhat.com>
Change-Id: I7c18998dd877d7fd065e4d1d0950b6f6413587c3
Tempest, its plugins and their dependencies in ussuri
cycle are python-requires>=py3.6 which makes distro not
having python 3.6 to do hack to install py3.6 etc.
We cannot do >py3.6 in basepython so one way is to drop the
py3.5 support and basepython to py3 so that it can choose
the available py version in env.
It time to drop the py3.5 from Tempest. Last supported version of
Tempest for py3.5 is 23.0.0.
oslo drop py2 so we need to remove the py2 unit test job which
is failing and blocking gate.
There is seoarate work to drop the py2 things from Tempest in
compplete set.
Change-Id: I872dbb57df46d1193c0d409bb00d56f372d295fe
Related-Bug: 1861308
Tempest CLI verify-tempest-config was deprecated
4 years ago.
With assumption of all the users of this CLI have
moved to new CLI 'tempest verify_config', it is
time to remove the old deprecated CLI now.
'verify_tempest_config' should be now used as
'tempest verify_config'
The following commit gives the idea of deprecation
message:-
e45aa842bc
Signed-off by: Soniya Vyas<svyas@redhat.com>
Change-Id: Ic3dbd6f5e854e017a5bec4b1a625730ab68e767d
We are adding a subcommand to tempest that is a parser for subunit
streams to determine what REST API calls are made inside of a test
and in what order they are called. Right now the implementation of
utility is as a console script and which is in argparse.
We are porting it to cliff to give a similar experience across
the codebase.
Change-Id: If95042898d8b2a148246c991d73d1a577fbcf7a0
Signed-off-by: Soniya Vyas <svyas@redhat.com>
This is a mechanically generated patch to ensure unit testing is in place
for all of the Tested Runtimes for Train.
See the Train python3-updates goal document for details:
https://governance.openstack.org/tc/goals/train/python3-updates.html
Note: This diverges from the Train goal as it does not drop the
openstack-python35-jobs. As tempest is branchless it needs to continue
supporting Python 3.5.
Change-Id: Ie702ca637afb8229b6e605abcec25550bc9a0673
Story: #2005924
Task: #34240
If GENERATE_TEMPEST_PLUGIN_LIST=False, the plugin-registry page is not
generated, and then sphinx fails with:
doc/source/index.rst:69:toctree contains reference to nonexisting
document u'plugin-registry'
Setting the variable to False is a legit use case, used in isolated
build environments like CBS used for RDO packages and should be
supported.
Alternatively, we could make inclusion of the page conditional to the
environment variable value, for example using 'only' directive for
sphinx and passing an appropriate 'tag' as a flag indicating the page is
generated; setup.py interface doesn't seem to provide an explicit way to
pass tags but SPHINXOPTS variable would be handy. Sadly, table of
contents can't be controlled with the directive:
https://github.com/sphinx-doc/sphinx/issues/1420
Another alternative would be to generate a stub page with no useful
info just to upease sphinx. I figured it's easier to tweak sphinx
configuration though.
The fix still leaves warnings-as-errors enabled for tox environments
with assumption that if tox is used, the build environment is not as
constrained.
Change-Id: Id174d1679c7b141b0327d8c23d6b14f7db83fae8
Due to the change from Ubuntu Trusty to Xenial, only python3.5 is
available(Python3.4 is restricted to <= Mitaka). So, there is no
need to continue to keep these settings.
Change-Id: I07bf6c3299a2e509a2d30342922459dc79c8f281
This commit makes tempest workspace command use cliff.lister to show
various formats. And this commit splits the original class -
TempestWorkspace into the subcommand classes to use the cliff.lister
class. With this splitting, we can get the subcommand helps as a side
effect.
Change-Id: I07c69c467743a2f132c99d992773a53bda3bec7b
With pbr 2.0 and Sphinx 1.5, the setting for treat sphinx warnings as
errors is setting warning-is-error in build_sphinx section. Migrate
the setting from the old warnerrors one.
There are a couple of small items to be cleaned up.
Change-Id: I01b1ac49c69d001ea25a475d903311d0b592656e
It's not used anymore. There was general consensus in Feb 2016
to deprecate it (see [1]) and remove it in Newton.
[1] [qa] deprecating Tempest stress framework
Change-Id: Ib229985ea2a1fee495c9492c9ce1781e6bac1dc6
The CLI entry point removed here points to the main() class which was removed
so that tempest-cleanup is not a standalone command anymore.
See I2119acd402b0985d4a81c85501ce7265f782d37b
Change-Id: I3421667916bf3652ab0e777fa78c382a3f6e849d
Javelin CLI utilities has been deprecated in Jan 2016
- Ie4ab5258946271a16e96a070abeedfafa5423714
which is over 6 months of Tempest deprecation period.
This patch removes the javelin CLI and corresponding doc/sample
files etc.
Change-Id: Ia59d3e81f245f29a7bf0f56b690586dfc55f16a0
Now that there is a passing gate job, we can claim support for
Python 3.5 in the classifier. This patch also adds the convenience
py35 venv.
Change-Id: Iaec0c9e88cb38ab876b5bc957f53989615cab246
Adds new command subunit-describe-calls, documentation,
and unittests.
subunit-describe-calls is a parser for subunit v1 & v2 streams to
determine what REST API calls are made inside of a test and in what
order they are called.
Later commits will add additional functionality relating to request
& response headers & body data along with a stdout output option and
stdin input.
Change-Id: I468d0d3e3b6098da95a81cc86d9bdd1b47ee1f03
This commit adds a new run command to the unified cli endpoint. The
intent here is for tempest to control it's own run story. This
implements the basic runner and selection functionality to use the
command, however it's not necessarily the end state of the command.
The functionality in this patch is just a starting point to add the
command and the basic functionality needed. It is starting with a
limited feature set with the intent to add additional, more complex
functionality in self contained patches after the command exists.
Co-Authored-by: David Paterson <davpat2112@yahoo.com>
Co-Authored-by: Stephen Lowrie <stephen.lowrie@rackspace.com>
Partially-Implements bp tempest-run-cmd
Depends-On: I09299043e536521d48dbe10632621138e3a366e0
Change-Id: I24588b5c00d005320e8719cf82b5dd95662572cf
Create a consistent means for creation and management of Tempest
workspaces.
Creates a file located at ~/.tempest/workspaces.yaml which stores existing
workspaces.
Available subcommands: list, register, rename, move, remove
bp centralized-workspaces
Change-Id: I9595e3ba809e457951a0ffdf4b15f641f2fec4f4
Now that tempest_lib code is copied back into tempest, stop using
tempest_lib in tempest, and start using the copied code.
Remove the dependency to tempest_lib from requirements, and drop
the script to use tempest_lib in tempest. Add os-testr to the
test-requirements.
Partially implements bp tempest-lib-reintegration
Change-Id: I21ab5fe6349f72c98ac9f960a29bf62e813f8b1b
Leave existing endpoint for now as to not disrupt existing
workflows and deprecate next cycle.
Add deprication warning for old usage.
Change-Id: I9db27a853184126a7f751800bc8cacd7dd4e40fd
Implements: blueprint tempest-cli-improvements
Leave existing endpoint for now as to not disrupt existing
workflows and deprecate next cycle.
Added deprecation message for old entry point.
Change-Id: If8afab441f51e5cf0f3bf1e2b95d76b2c1d0c474
Implements: blueprint tempest-cli-improvements
Also disable ability to run script in stand-alone mode. So users need to
use 'tempest run-stress' instead.
Change-Id: I3effd1b71b2375f75a11f5924205741be0903361
Implements: blueprint tempest-cli-improvements
Also disable ability to run script in stand-alone mode so
users will use 'tempest cleanup' instead.
Change-Id: I2119acd402b0985d4a81c85501ce7265f782d37b
Implements: blueprint tempest-cli-improvements
This commit fixes a few things in the tempest docs. First it fixes all
of the sphinx warnings and enables fail on warn to ensure we're using
valid sphinx everywhere. It also adds a link from the configuration
guide to the sample config file.
Change-Id: I3ad645a6bbfa46b4498e1732410743e46d6eb0cb
The absolute path causes failures when installing in a virtual
environment, not having root privileges.
This change solves the installation in a virtual environment,
but it does not fix yet the tempest init workflow when installing
into a virtual environment.
Partially-fixes bug #1480389
Change-Id: Iddefb8db77ffe35a84fb35b6f9e7e823de4ab45f
This commit adds a new 'tempest init' command to the unified cli.
This command tempest init will be the first step of the new workflow
for running tempest. It is used to initilize a local working directory
to run tempest from. The idea being you can setup different local
working dirs for different tempest envs to test against several
different clouds.
The basic idea for this new workflow after installing tempest is
something like:
1. mkdir tempest-newcloud
2. cd tempest-newcloud
3. tempest init
4. edit local config file
5. tempest run
The next step after this is to start working on the 'tempest run'
command. (which will include the hooks for the tempest plugin
interface) But, until that is added a test runner can be called
directly inside a local tempest working directory.
Partially Implements bp tempest-cli-improvements
Change-Id: I572267e61ba8ca86f92299e174864a7a89597207
This commit adds the necessary bits to tempest's setup.cfg file to
instruct pbr on how to make sure that the sample config files and
a /etc/tempest dir get setup when installing tempest. In the new
cli workflow it is expected that by default /etc/tempest will exist
as a source for local working dir configs.
Partially implements: bp tempest-cli-improvements
Change-Id: I2bf93f6789601caaebdfb85ac2b9f34b82f87980
This commit adds the basic framework to tempest for a unified modular
CLI endpoint using cliff. As of right now this entry-point doesn't
really do anything but exist. It will be expanded in the future first
all the existing commands will be refactored to be used through this.
Secondly a new interface and workflow for running tempest will be
created on top of this.
Co-Authored-By: Matthew Treinish <mtreinish@kortar.org>
Change-Id: I1ff8d22c120dbc81d812f1f107db7c2d9b15a505
Implements: blueprint tempest-cli-improvements
Implemented:
https://etherpad.openstack.org/p/accounts-yaml-generation
This tool can be used for creating user accounts for
tempest runs in concurrency.
Basic usage:
./account_generator.py
--tag foo
-c tempest.conf
--os-username <admin-name>
--os-password <admin-password>
--os-tenant-name <admin-tenant>
-r <concurrency count>
--with-admin (optional)
accounts.yaml
Change-Id: I3c313787b0f6f69edcc5ad16c6c186226f619692
This commit adds references explaining the current state of python 3.4
and tempest to the readme, pkg metadata, and adds an etry to the tox
envlist for running the unit tests on python 3.4.
Change-Id: I9c1e3cdf53dbae723a9fee67ad96932709d2305f
Ghanshyam Mann