Add mypy so we can run these checks as part of our linting step. Note
that mypy is the last item in the list of checks since flake8 is going
to catch e.g. syntax issues which should be addressed first.
This also allows us to drop use of mypywrap: rather than wrapping mypy,
we can make use of mypy's configuration to specify the files we wish to
run checks on.
Change-Id: Ie51401ead4e31ccd70ab833c59cb397f802ff188
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
This bumps the version of flake8 and resolves some erroneous failures in
f-strings. A number of new E721 (do not compare types) class errors are
picked up, which are all addressed.
Change-Id: I7a1937b107ff3af8d1e5fe23fc32b120ef4697f7
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
We no longer need this since everything is using SDK. The removal of
this highlights a couple of areas where there were outstanding unused
mocks. These are cleaned up in the process.
Change-Id: I6247bfbb157b37eff66ac3974bf91780a3d01c1d
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Hacking has bumped the version of flake8 that it's using to 5.0 in its
6.0.1 release. This turns up quite a few pep8 errors lurking in our
code. Fix them.
Needed-by: https://review.opendev.org/c/openstack/hacking/+/874516
Change-Id: I3b9c7f9f5de757f818ec358c992ffb0e5f3e310f
The PowerVM driver was deprecated in November 2021 as part of change
Icdef0a03c3c6f56b08ec9685c6958d6917bc88cb. As noted there, all
indications suggest that this driver is no longer maintained and may be
abandonware. It's been some time and there's still no activity here so
it's time to abandon this for real.
This isn't as tied into the codebase as the old XenAPI driver was, so
removal is mostly a case of deleting large swathes of code. Lovely.
Change-Id: Ibf4f36136f2c65adad64f75d665c00cf2de4b400
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
...and start skipping the relevant tests in 'nova.tests.unit.virt' if
these dependencies are not present. Thanks to hacking rules N311 and
N312, which prevent virt drivers using code from other virt drivers,
simply skipping the unit tests for a virt driver module is enough to
ensure we never load those modules.
This means users that want to use the powervm driver can install the
required dependencies using e.g. 'pip install .[powervm]', and packagers
can choose to skip packaging a dependency safe in the knowledge that the
relevant tests will be skipped.
Change-Id: I3787f9afd78cd0a7c7feb4dfe1bcb21437b5a128
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
This release removed the use of unittest2 in favour of the stdlib
unittest module. unittest2 hasn't been maintained in years and has some
quirky behavior differences to stdlib unittest. By bumping to 2.5.0, we
avoid anyone having to use this.
Change-Id: Ic3b257c799fe1669933c13f9571208740cd1cd1a
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
The commit I168fffac8002f274a905cfd53ac4f6c9abe18803 added a wrapper
around fasteners.ReaderWriterLock to fix up an issue with eventlet. But
the wrapper was added to nova.utils module that is use not only by the
nova tests but also the nova production code. This made the fixture
library a dependency of the nova production code. While the current
ReaderWriterLock usage only limited to the nova test sub tree. The
I712f88fc1b6053fe6d1f13e708f3bd8874452a8f commit fix the issue of not
having fixtures in the nova requirements.txt. However I think a better
fix is to move the wrapper to the test subtree instead. This patch does
that and restores the state of the requirements.txt
Change-Id: I6903ce53b9b91325f7268cf2ebd02e4488579560
Related-Bug: #1958075
The commit 887c445a7a made the nova.utils
module dependent on the fixtures library but the change missed updating
requirements and the fixtures library is not installed automatically.
This change migrates the fixtures library from test-requirements.txt to
requirements.txt so that the library is installed without test codes.
Closes-Bug: #1958075
Change-Id: I712f88fc1b6053fe6d1f13e708f3bd8874452a8f
With switching the SOAP library backing oslo.vmware [1], the internal
representation of ManagedObjectReference's attributes changes. To be able
to make the switch without interruption, we introduced helper functions
in oslo.vmware. This commit uses one of those - get_moref_value()
- to make the access to the "value" attribute compatible with both
backing libraries.
Bump oslo.vmware lower-constraint to a version containing the helper
function. We bump one version higher, because we already merged a patch
(I3c769d3499f906b33725171a57313c8ae35b6a1d) that uses the cookiejar
access contained in that version.
[1] https://specs.openstack.org/openstack/oslo-specs/specs/victoria/oslo-vmware-soap-library-switch.html
Change-Id: I48841a777aaa2e515c141857099c16483ab619e0
this change replaces psycopg2 with psycopg2-binary
to avoid requiring the build depencyies for the postgres client.
instead the tests now use the prebuilt binary wheel.
Change-Id: Id6d920508b9ee9597a51616e66c47d4cb7f48ca0
Several version specified in our lower-constraints files had conflicting
dependencies. This updates a few packages to avoid those conflicts. It
also removes the linters that are tracked in the global requirements
blocklist since we do not need to enforce lower-constraints for linters.
Change-Id: Iae93f0631d77794f093fd330e849d2fd805b238f
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
This series implements the referenced blueprint to allow for specifying
custom resource provider traits and inventories via yaml config files.
This first commit includes schema version 1.0 and functions to load and
validate against that schema.
Since this patch, package ddt>=1.2.1 is required to run tests.
Co-Author: Tony Su <tao.su@intel.com>
Author: Dustin Cowles <dustin.cowles@intel.com>
Blueprint: provider-config-file
Change-Id: I58099726f799c427e9174a0dcce9889344d51e7c
This has introduced a new version of pycodestyle, which has improved its
parser to catch new corner cases and introduced some new tests.
Closes-Bug: #1878317
Change-Id: I853cf4dbd7ad6b7903a7f444f5c1da3f0fb21f77
mypy is an experimental optional static type checker for Python that
aims to combine the benefits of dynamic (or "duck") typing and static
typing. While still in development, most features are supported and it's
already being used by real world projects like Sphinx. Let's start small
by integrating it into some of the interfaces that nova exposes. We can
further build upon this if it works out.
This change sets up the boilerplate necessary to use mypy in nova. Type
annotations are not included for any module - these will be added
separately. We're calling mypy by way of a script, as this allows us to
store a list of files that we have converted while we're in the process
of adding type annotations where necessary.
Change-Id: I75ab46a6768c4ca2050fdde2b7f8eeb90724c8c6
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
hacking 3.0.1 fix the pinning of flake8 to avoid bringing in a new
version with new checks.
bumping the min version for hacking so that any older hacking versions
which auto adopt the new checks are not used.
Depends-On: https://review.opendev.org/#/c/728335/
Change-Id: Ie00c10332bd7110169dbb150d601c157b6694d05
It is possible, though unlikely, that hacking might introduce new rules
in a minor version that would break the gate for nova. See this off
before it can ever be an issue by doing $subject.
Change-Id: I063488a2ad4924ffe2a48cbfc0c3c750221d37b7
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
This bumps the version of flake8 and pycodestyle to something much
newer, which resolves a long-standing warning about nested sets and
allows us to use new fangled features like f-strings if we so choose.
Change-Id: I0bb9077f1cea2243b7945e87cfa140f9cf89d558
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Finish the job by removing all the now-unused modules. This also allows
us to - wait for it - kill mox at long last. It's a great day in the
parish.
Partial-Implements: blueprint remove-nova-network-ussuri
Partial-Implements: blueprint mox-removal-ussuri
Change-Id: Ia33ec2604b2fc2d3b6830b596cac669cc3ad6c96
This reverts commit 33a2a5f2e4. We're now
at M1 and the DevStack default has been changed. We no longer need to
pretend to support Python 2.7. This is slightly modified to drop an
additional Python 2.7-only dependency from lower-constraints.txt.
[1] I52b03caee0ba700da3a15035201ea6cd91baa06b
Change-Id: I1068850072e639ee268d68ca82249f569a40310e
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Revert part of the changes from
Ie1a0cbd82a617dbcc15729647218ac3e9cd0e5a9
This way it is possible again for nova to be installed
using python 2, thus allowing other devstack-based
jobs to work until everything switches to python 3
according the plans.
While py2 support may break anytime, its days are limited anyway.
Change-Id: I99ff84e6c1c033c28d499b0b8ac17ac5b0565f61
It's Ussuri. We can *finally* stop testing Python 2 [1]. Time to party.
We don't attempt any cleanup but simply stop testing with Python 2,
indicate that we only support Python 3 via 'setup.cfg' and remove any
Python 2 only dependencies. Our 'tox.ini' is modified such that
'functional' now runs with 'python3', whatever that may point to, though
the gate will only use a versioned variant (currently
'functional-py36').
This should free up a significant amount of resources from the gate and
let us start using Python 3 idioms in our code. Win-win.
[1] https://governance.openstack.org/tc/resolutions/20180529-python2-deprecation-timeline.html#python2-deprecation-timeline
Change-Id: Ie1a0cbd82a617dbcc15729647218ac3e9cd0e5a9
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
The following file should have been removed
in I4181f39dea7eb10b84e6f5057938767b3e422aff.
* etc/nova/placement-policy-generator.conf
A description of generating the placement policy.yaml file
should also have been removed in the commit.
Therefore remove them.
And the 'wsgi-intercept' package is used for OSAPIFixture and
not for placement functional tests currently.
So a comment is removed in test-requirements.txt.
Change-Id: Ie8d0d7085ccd82a47ea27b98be75c1bc9ba6195b
By default, mock.patch's autospec argument is None, meaning that
there's no signature checking for the patched methods and functions.
oslotest.mock_fixture.patch_mock_module fixes a few issues within
mock.patch functions, as well as setting autospec=True by default,
unless otherwise specified or new_callable, create, spec arguments
are passed in.
Co-Authored-By: Claudiu Belu <cbelu@cloudbasesolutions.com>
Change-Id: I4484e63c97bd1cdde3d88855eabe7545784f365e
Closes-Bug: #1735588
Several existing tests patch open() to fake the contents of a file
outside the test virtualenv, whilst avoiding interfering with reading
and writing of other files inside the test virtualenv. Currently they
do it by duplicating logic.
Furthermore, in the near future, more tests (specifically, some SEV
tests) will want to do the same selective patching, and similarly will
need to avoid impacting reads of other files within the test
virtualenv, e.g. placement-policy.yaml.
So create new patch_open() context manager / decorator in nova.test
for selectively patching open based on the path parameter, and reuse
this for existing tests.
Also add unit tests for all these cases.
mock >= 3.0.0 is required because configparser.RawConfigParser._read()
uses enumerate() to iterate over the lines of the (mocked)
/etc/nova/release config file, and this uses __iter__() under the hood
which was not supported via mock_open until a bug was fixed and
backported to the external mock library for 3.0.0:
https://bugs.python.org/issue21258https://bugs.python.org/issue3293373f6eed0d6
Change-Id: I19f49c923d2c41eab0c7b4cab28c50498dc07046
Several existing tests patch os.path.exists() to return a given value,
whilst avoiding interfering with other checks for path existence
inside the test virtualenv. Currently they do it by duplicating
logic.
Furthermore, in the near future, more tests (specifically, some SEV
functional tests) will want to do the same selective patching, and
similarly will need to avoid impacting existence checks on other files
within the test virtualenv, e.g. placement-policy.yaml.
TestGuestConfigSysinfoSerialOS already had a nice patch_exists()
context manager for selectively patching os.path.exists based on the
path parameter, so move this to nova.test.TestCase so that it can be
reused to reduce duplication, and add appropriate unit test cases.
In order to make patch_exists() generic, the path has to be introduced
as an extra parameter alongside the desired return value.
Additionally make it usable as a decorator, not just a context
manager. This is achievable natively in Python 3 with contextlib, and
in Python 2 via contextlib2.
Change-Id: Ibe7cb29620f06d31059f2a5f94ca180b8671046e
This brings in a couple of new checks which must be addressed, many of
which involve a rather large amount of changes, so these are ignored for
now. A series of follow-up changes will resolved these.
'pycodestyle' is added as a dependency rather than it being pulled in
transitively. This is necessary since we're using it in tests.
Change-Id: I35c654bd39f343417e0a1124263ff31dcd0b05c9
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
In the review of a similar change in placement [1], it was realized that
the nova lower-constraints tox job probably had the same problems.
Testing revealed this to be the case. This change fixes the job and
updates the related requirements problems accordingly.
The are two main factors at play here:
* The default install_command in tox.ini uses the upper_contraints.txt
file. When there is more than one constraints.txt they are merged and
the higher constraints win. Using upper and lower at the same time
violates the point of lower (which is to indicate the bare minimum
we are capable of using).
* When usedevelop is true in tox, the command that is run to install the
current projects code is something like 'python setup.py develop',
which installs a project's requirements _after_ the install_command has
run, clobbering the constrained installs. When using pbr,
'python setup.py install' (used when usedevelop is False) does not do
this.
Fixing those then makes it possible to use the test to fix the
lower-constraints.txt and *requirements.txt files, changes include:
* Defining 'usedevelop = False' in the 'lower-constraints' target and
removing the otherwise superfluous 'skipsdist' global setting to
ensure requirements aren't clobbered.
* Removing packages which show up in lower-constraints.txt but not in
the created virtualenv. Note that the job only runs unit tests, so
this may be incomplete. In the placement version of this both unit and
functional are run. We may want to consider that here.
* Updating cryptography. This version is needed with more recent
pyopenssl.
* Updated keystonemiddleware. This is needed for some tests which
confirm passing configuration to the middleware.
* Update psycopg2 to a version that can talk with postgresql 10.
* Add PyJWT, used by zVMCloudConnector
* Update zVMCloudConnector to a version that works with Python 3.5 and
beyond.
* Update olso.messaging to versions that work with the tests, under
Python 3.
* Adding missing transitive packages.
* Adjusting alpha-ordering to map to how pip freeze does it.
* setuptools is removed from requirements.txt because the created
virtualenv doesn't contain it
NOTE: The lower-constraints.txt file makes no commitment to expressing
minimum requirements for anything other than the current basepython.
So the fact that a different set of lower-constraints would be present
if we were using python2 is not relevant. See discussion at [1].
However, since requirements.txt _is_ used for python2, the
requirements-check gate job requires that enum34 be present in
lower-constraints.txt because it is in requirements.txt.
NOTE: A test is removed because it cannot work in the
lower-constraints context: 'test_policy_generator_from_command_line'
forks a call to 'oslopolicy-policy-generator --namespace nova' which
fails because stevedore fails to pick up nova-based entry points when
in a different process. This is because of the change to usedevelop.
After discussion with the original author of the test removal was
considered an acceptable choice.
[1] http://eavesdrop.openstack.org/irclogs/%23openstack-dev/%23openstack-dev.2019-03-05.log.html#t2019-03-05T13:28:23
Closes-Bug: #1822575
Change-Id: Ic6466b0440a4fe012731a63715cf5d793b6ae4dd
This is related to change I5f78d21c39ed2fd58d2a0f3649116e39883d5a2c
which depends on python-ironicclient change
I610836e5038774621690aca88b2aee25670f0262 which was released in
2.7.0. The former did not have a direct dependency or version
bump since it is being backported to stable branches where we
don't bump minimum required versions of dependencies.
Change-Id: I5edf350b29ef3bd1f103cdd4b0b997cb5ca32117
Related-Bug: #1818295
In preparation for stein, use 'endpoint' instead of 'ironic_url'
when calling get_client in order to remove the following warning:
WARNING ironicclient.client The argument "ironic_url" passed to
get_client is deprecated and will be removed in Stein release,
please use "endpoint" instead.
For reference:
In the python-ironicclient code, in the ironicclient/client.py#L24
TODO(vdrok): remove in Stein
[...]
('ironic_url',): 'endpoint',
Introduced in commit:
58c39b7a80
Change-Id: I1b3ce1955622c40b780c0b15ec7e09be3e8ace72
With a fix in wsgi-intercept 1.7.0 we can properly
use the PlacementFixture as a context manager to
simulate when placement is configured for a given
operation. This allows us to remove the ugly stub
that one of the tests in here had to rely on.
While in here, the CastAsCall fixture is removed
since we shouldn't rely on that in these tests
where we're trying to simulate the user experience.
Change-Id: I2074b45126b839ea6307a8740364393e9dddd50b
The Project Testing Interface [1] asks that we list requirements in
'doc/requirements.txt' and build docs by calling 'sphinx-build' directly
instead of via the 'build_sphinx' setuptool/distutils wrapper. Start
doing this.
[1] https://governance.openstack.org/tc/reference/project-testing-interface.html
Change-Id: If9342c8ea757b1735f2488db751008984fb33baf
Nova switched to using stestr for all it's in tree testing except for
the coverage job. This commit updates that job so it also uses stestr
and cleans up the last bits of testr setup left in tree. As part of
this change this moves all the one off per job configuration into the
tox job definitions instead of hiding it in either the .stestr.conf or
in a wrapper script.
Change-Id: I36e0e791d485b44641a1ed8770eca8e95aca19be
Use whereto to test that the redirect rules do what we expect. The
test coverage is a bit minimal for now, but should be useful as an
example of how it can be extended. The test files in the
openstack-manuals repo provide some other examples.
Depends-On: I36003d3525dcc9fef8e5f008ecabb2b2ac78088e
Change-Id: Ia14bfa6c89fd04844842d727d11ff8874ec7fc2a
Signed-off-by: Doug Hellmann <doug@doughellmann.com>
Stephen Finucane