This change mainly fixes incorrect use of backticks
but also adress some other minor issues like unbalanced
backticks, incorrect spacing or missing _ in links.
This change add a tox target to run sphinx-lint
as well as adding it to the relevent tox envs to enforce
it in ci. pre-commit is leveraged to install and execute
sphinx-lint but it does not reqiure you to install the
hooks locally into your working dir.
Change-Id: Ib97b35c9014bc31876003cef4362c47a8a3a4e0e
Nova Hyper-V driver is not tested in OpenStack upstream and no maintianers.
This driver has been marked as deprecated in Antelope release. It has dependency
on the OpenStack Winstacker project which has been retired[1].
As discussed in vPTG[2], removing the HyperV driver, tests, and its config.
[1] https://review.opendev.org/c/openstack/governance/+/886880
[2] https://etherpad.opendev.org/p/nova-caracal-ptg#L301
Change-Id: I568c79bae9b9736a20c367096d748c730ed59f0e
Add mypy so we can run these checks as part of our linting step. Note
that mypy is the last item in the list of checks since flake8 is going
to catch e.g. syntax issues which should be addressed first.
This also allows us to drop use of mypywrap: rather than wrapping mypy,
we can make use of mypy's configuration to specify the files we wish to
run checks on.
Change-Id: Ie51401ead4e31ccd70ab833c59cb397f802ff188
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
pre-commit is an effective linter runner. No need to duplicate things.
We are able to remove our homegrown "is this valid JSON" check by using
the one built into pre-commit. Beautiful!
Change-Id: I89398d4a1c5d4e7205f5f9c499cba0b4ff90fa46
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
This chnage adds the pre-commit config and
tox targets to run codespell both indepenetly
and via the pep8 target.
This change correct all the final typos in the
codebase as detected by codespell.
Change-Id: Ic4fb5b3a5559bc3c43aca0a39edc0885da58eaa2
We leak due to running background operations like server create, rebuild
and in one case a sleeping claim. So this patch removes the leaks by
making sure that the background operations stop before the test
finishes.
As there is no more leak in functional test too this patch makes the
leak an error there too.
Change-Id: I6905999050e8d09b772837034a212c534e9c3226
This change add a global greenpool which is used to manage
the greenthreads created via nova.utils.spawn(_n).
A test fixture is also added to use an isolated greenpool
which will raise an exception if a greenthread is leaked.
the fixture will optionally raise if greenlets are leaked.
This is enabled by unit test by default and is configurable
for functional tests.
This change removes all greenthread leaks from the unit
and functional tests that were detected. 7 functional
tests still leak greenlets but they have no obvious
cause. as such greenlet leaks are not treated as errors
for funtional tests by default. Greenthread leaks
are always treated as errors.
Set NOVA_RAISE_ON_GREENLET_LEAK=1|true|yes when invoking
tox to make greenlet leaks an error for functional tests.
Change-Id: I73b4684744b340bfb80da08537a745167ddea106
When the upper constraint usage moved to install_command in tox.ini, it
was not removed from docs target's deps list, where it's now not
necessary, and even could cause issues when overriding
TOX_CONSTRAINTS_FILE.
This patch replaces the old UPPER_CONSTRAINTS_FILE to the "new"
TOX_CONSTRAINTS_FILE in the install_command, too.
Change-Id: I9c27caa869d72e06e84bcf84ceedb586ed3bc8e9
This makes us able to run functional on python 3.11. Without this,
tox will happily (and silently) run the default venv, which is unit
tests.
Change-Id: I544a29ae78814f9a454daba8c1978f7ab2c2505c
It seems that with tox 4.2.6 the missing interpreter error was fixed but
the generative testenv feature is broken and the
[testenv:functional{,-py38,-py39,-py310}] format is leads to missing
interpreter error. It is due to a conflict between basepython = python3
and the version fragment in the generative target suppressed by
ignore_basepython_conflict = true.
This patch removes basepython = python3 assuming that developers already
switched for python3 in their environment as python2.7 is EOL.
Also we took the opportunity to add the global constraints via the
install_command instead of deps as deps is not used during the
installation of the editable package.
Change-Id: I258a7c13434b29402804181dea275b42d5539df0
As per 2023.1 testing runtime[1], we need to test on Ubuntu
Jammy (which will be taken care by tempest and devstack patches
to move base jobs to Jammy) and at least single job to run on
Ubutnu Focal (for smooth upgrade). Also, python 3.10 testing is
voting now.
This commit adds a new job to run on focal which can be removed
in future cycle when testing runtime drop the requirement of Focal
testing. Also, make python 3.10 functional and unit test job as voting
(openstack-tox-py310 is running as part of generic template so we do
not need to explicitly add that)
[1] https://governance.openstack.org/tc/reference/runtimes/2023.1.html
Change-Id: Ia43f73dba00b0b5932939bcc7d11b97a83072ee3
Add the following hacking rule.
* N372: Don't use the setDaemon method.
Use the daemon attribute instead.
Change-Id: Idb45421205f76d2d3b0576bd0504d261ed249edd
Related-Bug: 1987191
Signed-off-by: Takashi Natsume <takanattie@gmail.com>
This change extends the guest xml parsing such that
the source device path can be extreacted from interface
elements of type vdpa.
This is required to identify the interface to remove when
detaching a vdpa port from a domain.
This change fixes a latent bug in the libvirt fixutre
related to the domain xml generation for vdpa interfaces.
Change-Id: I5f41170e7038f4b872066de4b1ad509113034960
The PowerVM driver was deprecated in November 2021 as part of change
Icdef0a03c3c6f56b08ec9685c6958d6917bc88cb. As noted there, all
indications suggest that this driver is no longer maintained and may be
abandonware. It's been some time and there's still no activity here so
it's time to abandon this for real.
This isn't as tied into the codebase as the old XenAPI driver was, so
removal is mostly a case of deleting large swathes of code. Lovely.
Change-Id: Ibf4f36136f2c65adad64f75d665c00cf2de4b400
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Since we dropped support for python 2 [1], we no longer need to use the
mock library, which existed to backport py3 functionality into py2.
Change Ib44b5bff657c8e76c4f701e14d51a4efda3f6d32 cut over to importing
the stock mock, which must be done by saying::
from unittest import mock
...because if you say::
import mock
...you will be using the third party mock library instead, which may or
may not be installed.
This commit adds hacking check N371 to enforce the former.
[1] https://review.opendev.org/#/c/687954/
Change-Id: I71439580e80d33cff62aba807df2b35164a47cbe
These are currently non-voting since we don't care about this stuff for
Zed. It does get us ready for a 3.10-having future, however.
Change-Id: I7740dafd6523eca27fa4e725d7eaf8558e434779
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
When tox 'docs' target is called, first it installs the dependencies
(listed in 'deps') in 'installdeps' phase, then it installs nova (with
its requirements) in 'develop-inst' phase. In the latter case 'deps' is
not used so that the constraints defined in 'deps' are not used.
This could lead to failures on stable branches when new packages are
released that break the build. To avoid this, the simplest solution is
to pre-install requirements, i.e. add requirements.txt to 'docs' tox
target.
Change-Id: I4471d4488d336d5af0c23028724c4ce79d6a2031
Spotted this in a review recently. We don't want people using six
anymore.
Change-Id: Ie107a95bc06390ab519d3b3af9b07103a9a14316
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Install these via tox.ini when required. Note that we don't add them to
'test-requirements.txt' since packagers consume those and would be
forced to package these dependencies or modify this file.
A small bug in the tox file is corrected: you can't share an environment
directory if the dependencies in that directory are different. As such,
pep8 must go in its own directory again, not the 'shared' directory.
Change-Id: Iffb9ff2e300213ec01cccaf3b967e73331c9c9ff
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
This is a follow up change to I168fffac8002f274a905cfd53ac4f6c9abe18803
which added a hackaround to enable our tests to pass with
fasteners>=0.15 which was upgraded recently as part of a
openstack/requirements update.
The ReaderWriterLock from fasteners (and thus lockutils) cannot work
correctly with eventlet patched code, so this adds a wrapper containing
the aforementioned hackaround along with a hacking check to do our best
to ensure that future use of ReaderWriterLock will be through the
wrapper.
Change-Id: Ia7bcb40a21a804c7bc6b74f501d95ce2a88b09b5
Yoga testing runtime is updated now
- https://governance.openstack.org/tc/reference/runtimes/yoga.html
which needs to test py38 and py39. Unit tests update are
handled by the job template change in openstack-zuul-job and
this commit makes changes to fucntional job to run py39 as voting
and updating the metdata in setup file.
Change-Id: I314fd61f20f2c3551f6231d191b7dcaaefabd2b5
Well, sort of. We enable them but immediately filter out the ones we're
actually seeing, the rationale being that we can address these in a
piecemeal fashion without the risk of introducing new issues.
There's a lot more to be done here. However, the work done in oslo.db
[1] should provide a guide for how to resolve the outstanding issues.
[1] https://review.opendev.org/q/topic:%2522sqlalchemy-20%2522+project:openstack/oslo.db
Change-Id: Iafe726d3819031c357460fd131bb2bb58babb4e2
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
autopep8 is a code formating tool that makes python code pep8
compliant without changing everything. Unlike black it will
not radically change all code and the primary change to the
existing codebase is adding a new line after class level doc strings.
This change adds a new tox autopep8 env to manually run it on your
code before you submit a patch, it also adds autopep8 to pre-commit
so if you use pre-commit it will do it for you automatically.
This change runs autopep8 in diff mode with --exit-code in the pep8
tox env so it will fail if autopep8 would modify your code if run
in in-place mode. This allows use to gate on autopep8 not modifying
patches that are submited. This will ensure authorship of patches is
maintianed.
The intent of this change is to save the large amount of time we spend
on ensuring style guidlines are followed automatically to make it
simpler for both new and old contibutors to work on nova and save
time and effort for all involved.
Change-Id: Idd618d634cc70ae8d58fab32f322e75bfabefb9d
We have placement-nova-tox-functional-py38 job defined and run
on placement gate[1] to run the nova functional test excluding
api and notification _sample_tests, and db-related tests but that
job skip those tests via tox_extra_args which is not right way
to do as we currently facing error when tox_extra_args is included
in tox siblings task
- c02c28a982
- https://zuul.openstack.org/build/a8c186b2c7124856ae32477f10e2b9a4
Let's define a new tox env which can exclude the required test
in stestr command itself.
[1] bd5b19c00e/.zuul.yaml (L83)
Change-Id: I20d6339a5203aed058f432f68e2ec1af57030401
As the bug and fix If71620e808744736cb4fe3abda76d81a6335311b showed
it is dangerous to forget instantiating the Mock class before it is
used in the test as changes on the class directly leaks out from the
test. In almost all the cases using Mock class directly is a bug and the
author original intention is to use an instance instead, just forgot
about the parents. So this patch adds two new hacking rules:
N367: catches the case when Mock class is aliased in the test:
self.mock_mystuff = mock.Mock
N368: catches when mock.patch instructed to use the Mock class as
replacement value during patching:
mock.patch('Bar.foo', new=mock.Mock)
For N367 the previous patch removed the last hit. For N368 this patch
removes the two hits exists.
Change-Id: Id42ca571b1569886ef47aa350369e9d2068e77bc
Related-Bug: #1936849
Some things that were missed in previous patches and are thrown together
here:
- Add alembic as an explicit dependency (we were getting it transitively
from oslo.db). We also bump the sqlalchemy dependency to a 1.4.x
release, which is the minimum supported by our chosen version of
alembic (more on this below)
- Remove tooling related to the old migrations
- Fix the tox whitelisting of the flaky MySQL tests
On the SQLAlchemy front, we opt for 1.4.13. Technically alembic should
support anything from 1.4.0, however, with SQLAlchemy >= 1.4.0, < 1.4.13
we see errors like the following in some tests:
sqlalchemy.exc.InvalidRequestError: Entity namespace for
"count(instance_mappings.id)" has no property "queued_for_delete"
There's nothing specific about this in the release notes for 1.4.13 [1]
but it definitely fixes things.
[1] https://docs.sqlalchemy.org/en/14/changelog/changelog_14.html#change-1.4.13
Change-Id: I4c8eb13f11aa7471c26a5ba326319aef245c9836
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
This reverts commit 89e321b7b5.
Reason for revert: The previous change require develper to install multiple spcific version python
locally to run the test envs and it arbitrally pins lower constratins to the
oldest python we support but lower contratis shoudl work with the newwst python we support too.
on fedora or other operating systems that ship with default python version that exceed our supported
versions test can be run with tox -e py38 if required.
similarly our updated constratins shoudl work with the older python.
Change-Id: I358ff8128746c9d0c584a03381e55817c4aace3f
Declare specific versions of Python for environments that don't
specifically encode a version. We use the highest version officially
supported, Python 3.8, for most environments except lower-constraints,
which uses the lowest version, Python 3.6. This is necessary for users
on Fedora, where the default Python version is Python 3.9.
This will unfortunately introduce a small amount of busy work whenever
we change supported Python versions, as tox will need to be updated, but
that's a small price to pay for the usability wins that Fedora users
will see.
Change-Id: I240d5b8aa3eb1925b97af57e2644be61bfc106c2
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Depends-On: https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/801773
This currently runs in the 'check' pipeline, as part of the pep8 job,
which causes otherwise perfectly valid backports to report as failing
CI. There's no reason a stable core shouldn't be encouraged to review
these patches: we simply want to prevent them *merging* before their
parent(s). Resolve this conflict by moving the check to separate voting
job in the 'gate' pipeline as well as a non-voting job in the 'check'
pipeline to catch more obvious issues.
Change-Id: Id3e4452883f6a3cf44ff58b39ded82e882e28c23
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Just removing mypy and then we will revert this patch once
I035d6dc752eaa83105cc12797765b304d843e1f7 is merged.
Change-Id: Iaa02e9d35419bf7100cef1014217210c1d5b6855
I4f551dc4b57905cab8aa005c5680223ad1b57639 introduced the environment
variable to disable the check-cherry-pick.sh script but forgot to allow
it to be passed into the pep8 tox env.
Change-Id: Ie8a672fd21184c810bfe9c0e3a49582189bf2111
This significantly speeds up our doc build process. This requires a
newer version of 'sphinx-feature-classification' and some tweaks to our
own in-tree extensions. While we're here, we drop the '-d DOCTREE_DIR'
parameter since it's of no use when we blast away our previously built
docs each time we build.
Change-Id: I679da65d44c40880f720df8a2f06286a19eb8d22
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Introduce API microversion 2.88, which makes the following changes to
a number of 'os-hypervisors'. Specifically, the following fields are
dropped from both the '/os-hypervisors/detail' (detailed list) and
'/os-hypervisors/{hypervisor_id}' (show) APIs:
- current_workload
- cpu_info
- vcpus
- vcpus_used
- free_disk_gb
- local_gb
- local_gb_used
- disk_available_least
- free_ram_mb
- memory_mb
- memory_mb_used
- running_vms
In addition, the '/os-hypervisors/statistics' API, which provided a
summary of the above stats but for all hypervisors in the deployment, is
dropped entirely.
Finally, the '/os-hypervisors/{hypervisor}/uptime' API, which provided a
similar response to the '/os-hypervisors/{hypervisor}' API but with an
additional 'uptime' field, has been removed in favour of including this
field in the primary '/os-hypervisors/{hypervisor}' API.
A small tweak to 'tox.ini' that allows us to share some venvs is
included.
Part of blueprint modernize-os-hypervisors-api
Change-Id: I515e484ade6c6455f82a3067940a418a0d7d965a
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
In a heavily IO deprived CI VM the db migration tests could take a
significant amount of time and eventually time out. This patch moves the
tests into the same test executor worker process to spread the load
generated by these test in time until a final solution is found. For
example we hope that [1] will help eventually to decrease the load.
[1] https://review.opendev.org/q/topic:bp/compact-db-migrations-wallaby
Change-Id: I6ce930fa86c82da1008089791942b1fff7d04c18
Related-Bug: #1823251
Nova does not run gabbi test since placement is moved to a separate
git repository. So the gabbi related tox.ini comment is removed.
Change-Id: Ic324e3e32fa03478895b32fa583e805ee6c721e2
This patch adds both a tox target and a zuul job to run functional tests
with python3.9
Depends-On: https://review.opendev.org/760932
Change-Id: I672904e9bfb45a66a82331063c7d49c4bc0439df
This linting makes building docs an even more painful process than it
would otherwise be. We already do this as part of the pep8 target,
which the gate runs, so there's no need to do this here.
Change-Id: I36ae872dd21299ad5d165422cb83564eafd89bea
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Add the following hacking rule.
* N366: The assert_has_calls is a method rather than a variable.
Not correct: mock_method.assert_has_calls = [mock.call(0)]
Correct: mock_method.assert_has_calls([mock.call(0)])
This patch is a follow-up patch for
Id094dd90efde09b9a835d4492f4a92b8f8ad296e.
Change-Id: I892f8c23ee44f2b3518776a9705e3543f3115cae
Signed-off-by: Takashi Natsume <takanattie@gmail.com>
Sean Mooney