Now that we no longer support py27, we can use the standard library
unittest.mock module instead of the third party mock lib. Most of this
is autogenerated, as described below, but there is one manual change
necessary:
nova/tests/functional/regressions/test_bug_1781286.py
We need to avoid using 'fixtures.MockPatch' since fixtures is using
'mock' (the library) under the hood and a call to 'mock.patch.stop'
found in that test will now "stop" mocks from the wrong library. We
have discussed making this configurable but the option proposed isn't
that pretty [1] so this is better.
The remainder was auto-generated with the following (hacky) script, with
one or two manual tweaks after the fact:
import glob
for path in glob.glob('nova/tests/**/*.py', recursive=True):
with open(path) as fh:
lines = fh.readlines()
if 'import mock\n' not in lines:
continue
import_group_found = False
create_first_party_group = False
for num, line in enumerate(lines):
line = line.strip()
if line.startswith('import ') or line.startswith('from '):
tokens = line.split()
for lib in (
'ddt', 'six', 'webob', 'fixtures', 'testtools'
'neutron', 'cinder', 'ironic', 'keystone', 'oslo',
):
if lib in tokens[1]:
create_first_party_group = True
break
if create_first_party_group:
break
import_group_found = True
if not import_group_found:
continue
if line.startswith('import ') or line.startswith('from '):
tokens = line.split()
if tokens[1] > 'unittest':
break
elif tokens[1] == 'unittest' and (
len(tokens) == 2 or tokens[4] > 'mock'
):
break
elif not line:
break
if create_first_party_group:
lines.insert(num, 'from unittest import mock\n\n')
else:
lines.insert(num, 'from unittest import mock\n')
del lines[lines.index('import mock\n')]
with open(path, 'w+') as fh:
fh.writelines(lines)
Note that we cannot remove mock from our requirements files yet due to
importing pypowervm unit test code in nova unit tests. This library
still uses the mock lib, and since we are importing test code and that
lib (correctly) only declares mock in its test-requirements.txt, mock
would not otherwise be installed and would cause errors while loading
nova unit test code.
[1] https://github.com/testing-cabal/fixtures/pull/49
Change-Id: Id5b04cf2f6ca24af8e366d23f15cf0e5cac8e1cc
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Current tests do not have good test coverage of existing policies.
Either tests for policies do not exist or if they exist then they
do not cover the actual negative and positive testing.
To adopt the keystone's scope_type and new defaults in deprecated
API policies, we need to first write test coverage for the same to
know the complete effect of policies changes.
Partial implement blueprint policy-defaults-refresh-deprecated-apis
Change-Id: Ib89d65f93363f36de15a5a722fe8991c16ad6235
FIP API policy is default to admin_or_owner[1] but API
is allowed for everyone.
We can see the test trying with other project context can access the API
- https://review.opendev.org/#/c/741799
This is because API does not pass the server project_id in policy target[2]
and if no target is passed then, policy.py add the default targets which is
nothing but context.project_id (allow for everyone who try to access)[3]
This commit fix this policy by passing the server's project_id in policy
target.
Partial implement blueprint policy-defaults-refresh-deprecated-apis
[1] cd16ae25c8/nova/policies/floating_ips.py (L27)
[2] cd16ae25c8/nova/api/openstack/compute/floating_ips.py (L189)
[3] c16315165c/nova/policy.py (L191)
Change-Id: Ib4e81ffec9c7ace0c18062ebcd6bfb7e4b5c1880
With the removal of nova-network, a whole swathe of exceptions are no
longer raised on certain code paths. Start cleaning things up by
removing these. Some of these features can be removed now while the
remainder will be removed once we have removed the (now unused) code
that calls them.
Change-Id: I131062ded9ddedc31cf3b448b2c38306b55e874b
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
We want to get rid of the 'FloatingIP' object. Alas, that's easier said
than done because there are still a few users of this. The
'get_floating_ip', 'get_floating_ip_by_address', and
'get_floating_ips_by_project' methods are examples. These are only
called by the legacy 'os-floating-ips' API and the 'FloatingIP' object
is simply used as an unnecessary container between the two. Convert
things so the aforementioned API can handle mostly intact responses from
neutron instead, eliminating this user of the 'FloatingIP' object.
Change-Id: I77b1cfeab3c00c6c3d7743ba51e12414806b71d2
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Finish the job by removing all the now-unused modules. This also allows
us to - wait for it - kill mox at long last. It's a great day in the
parish.
Partial-Implements: blueprint remove-nova-network-ussuri
Partial-Implements: blueprint mox-removal-ussuri
Change-Id: Ia33ec2604b2fc2d3b6830b596cac669cc3ad6c96
There's more to be done here around making the cached objects look like
something that would be generated by neutron, but this is good enough
for now.
Change-Id: I08aa4d581720b0f6cd1dabccc98dac210d1a1663
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Pretty self-explanatory. This is blocking us removing a lot of other
things like the nova-network security group driver, so switch things
over now. The main change is in 'test_serversV21', where we need to
stub the neutron-based security group driver.
Change-Id: I03fe7d2c7ac2c58fc836e2835a3b85d17f15aa5e
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
This patch is a follow-up for I3eac9ac0a2a698a3c72b4e646cdcaf293ec4b057.
Replace 'fake' with a real project ID in URL paths and related places
in unit tests.
Change-Id: I603212f2d3cbbd482a2a68cd835af0261a85c392
This brings in a couple of new checks which must be addressed, many of
which involve a rather large amount of changes, so these are ignored for
now. A series of follow-up changes will resolved these.
'pycodestyle' is added as a dependency rather than it being pulled in
transitively. This is necessary since we're using it in tests.
Change-Id: I35c654bd39f343417e0a1124263ff31dcd0b05c9
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
This patch sets the stage for modifying the behavior of nova show
which currently gives a 500 when the cell in which the instance
lives is down. The new behavior will return a partial construct
consisting of uuid, project_id, created_at from instance_mappings
table and user_id, flavor, image_ref and availability_zone info
from request_specs table. Note that the rest of the keys will be
missing. This behavior will be enabled by passing a new enough
microversion, handling for which is introduced later in this series.
Related to blueprint handling-down-cell
Change-Id: Iaea1cb4ed93bb98f451de4f993106d7891ca3682
nova/db/__init__.py was importing * from nova.db.api. This meant that
any time any code anywhere within the nova.db package was imported
then nova.db.api was too, leading to a cascade of imports that may
not have been desired. Also, in general, code in __init__.py is a pain.
Therefore, this change adjusts code that so that either:
* nova.db.api is used directly
* nova.db.api is imported as 'db'
In either case, the functionality remains the same.
The primary goal of this change was to make it possible to import the
model files without having to import the db api. Moving the model files
to a different place in the directory hierarchy was considered, but
given that "code in __init__.py is a pain" this mode was chosen.
This looks like a very large change, but it is essentially adjusting
package names, many in mocks.
Change-Id: Ic1fd7c87ceda05eeb96735da2a415ef37060bb1a
The old legacy v2 extension mechanism isn't used by v2.1 API
anymore. The fake ExtensionManager is used to test that, so
now we can remove it.
Partial-implement-blueprint api-no-more-extensions-pike
Change-Id: I5b1ff4439a4441a90f1c77804b5521f7c3c4830d
This changes floating ips from a ReservableResource to a
CountableResource and replaces quota reserve/commit/rollback with
check_deltas accordingly. Note that floating ip quota is only
relevant to nova-network and will be obsolete when nova-network is
removed.
Part of blueprint cells-count-resources-to-check-quota-in-api
Change-Id: I9e6c16ebe73f2af11bcc47899f25289f08c1204a
This patch removes get_nw_info_for_instance(instance) from nova.compute.utils
as it was just a proxy for instance.get_network_info() call.
Change-Id: Iddae8074554995df22b656bb2e9bddaec6d775cc
In some cases, trying to delete a floating IP multiple times in a short
delay can trigger an exception beacause the floating ip deletion
operation is not atomic. If neutronclient's call to delete fails with a
NotFound error, we raise a 404 error to nova's client instead of a 500.
Change-Id: I49ea7e52073148457e794d641ed17d4ef58616f8
Co-Authored-By: Stephen Finucane <sfinucan@redhat.com>
Closes-Bug: #1649852
Neutron can raise a Conflict exception when attempting
to associate a floating IP to a server when the fixed
address is already associated to another floating IP.
This has always resulted in a 400 response, however, it
would also trace an ERROR in the nova-api logs, which is
something we shouldn't be doing for an expected type of
failure.
This handles the Conflict in the neutronv2 API client code
and re-raises an exception that the REST API controller code
can handle and return as a 400 without the stacktrace in the
logs.
Change-Id: I27d3241300f75e2aa79a32348a3843e09123cb10
Closes-Bug: #1693576
The following APIs which are considered as proxies of Neutron networking
API, are deprecated and will result in a 404 error response in new
Microversion:
POST /servers/{server_uuid}/action
{
"addFixedIp": {...}
}
POST /servers/{server_uuid}/action
{
"removeFixedIp": {...}
}
POST /servers/{server_uuid}/action
{
"addFloatingIp": {...}
}
POST /servers/{server_uuid}/action
{
"removeFloatingIp": {...}
And the nova-network specific API for query server's interfaces is
deprecated also:
GET /servers/{server_uuid}/os-virtual-interfaces
Partial implement blueprint deprecate-multinic-proxy-api
Change-Id: I1848db384a825d3b166f113b30ebad92113af8e5
Some pretty simple changes:
- test_floating_ip_dns can only be tested with nova-network as the
API isn't implemented for Neutron.
- test_floating_ip_pools just needed some more generic mocking.
- test_floating_ips just needed more generic mocking and to make
the non-Neutron-specific test force use_neutron=False.
Part of blueprint use-neutron-by-default
Change-Id: If8b8a6662ae01f23bde09fd5804fe635b6340eb4
As of now, in most of the test cases, uuidsentinel is used for
generating a UUID except at some places where uuid4() is used.
In order to maintain consistency, we propose to use uuidsentinel
module for generating UUIDs throughout the test cases.
There are some cases where unique UUIDs are required. For such
cases, generate_uuid() from oslo_utils.uuidutils is used.
Change-Id: I61914796aa94b453669add2e71b3d5d704633176
This patch deprecates all the proxy APIs which related floating ip. All those
APIs will return 404 in new Microversion.
The deprecated API endpoints are
'/os-floating-ip-dns'
'/os-floating-ip-dns/{domain}/entries'
'/os-floating-ip-pools'
'/os-floating-ips'
'/os-floating-ips-bulk'
This patch doesn't bump the max api version, due to the patch separation.
The max api version will bump in the last patch.
Partially implements blueprint deprecate-api-proxies
Change-Id: Idca478c566f9a7b5b30a3172453ce7c66d9fd8f0
In module netaddr valid_ipv4 raise exception AddrFormatError
if address is specified as empty string. Method is_valid_ipv4
catches the exception and returns boolean value. So we'd better
use it from oslo.utils.
Change-Id: I452d437692e44f4e20ff1b064032d247c0ab7530
There are 2 try/except blocks within _translate_floating_ip_view() that
cause in different resulting dictionaries. These situations are
untested, so now we can test for different scenarios of missing
attributes on the floating IP. An if check was also added to the
function because the types passed in are objects, so if the attribute is
not set on the object, it raises a NotImplementedError. So instead of
catching that too, we check to see if the key is in the object (which
checks if it is set).
Change-Id: Ia5739f214f57525b3083ef3f201f6f08198703e5
This makes the neutronapi module return NeutronFloatingIP objects from
the relevant methods, instead of bare dicts.
Note that the API is returning string floating ip identifiers when neutron
is in use, as opposed to the integer ones returned when nova-network is in
use. This patch does not address that problem, but uses the new
NeutronFloatingIP.
Needed for blueprint rm-object-dict-compat-newton
Co-Authored-By: Ryan Rossiter <rlrossit@us.ibm.com>
Change-Id: Ie60fb8661195eec9c01a57129efe8145504789e1
There are two implementation code for similar API in Nova repository.
One is newer: v2.1 API, another is legacy: v2 API. v2.1 API has been
used as the default API since Liberty and legacy v2 API has been marked
as deprecated. We have used and tested v2.1 API so well and now is nice
time to remove legacy API code based on the consensus of the design
summit of Austin. This patch removes unit tests of legacy v2 API[f-n].
Partially implements blueprint remove-legacy-v2-api-code
Change-Id: I543bc2a9c068aae2c755f8159c7d2a9fff2c67ee
Update all the tests to use_neutron=True, in doing so this exposed an
issue with the existing nova.utils.is_neutron function, as it only
worked off of the network_api_class option.
This was adjusted to provide a new nova.network.is_neutron function
which returns True / False / None (None when we have no idea because
it's not a class we know). nova.utils.is_neutron now uses this.
Remove tests that specifically were using Neutron API subclasses. In
digging through git history this comes from
I6b7b7389e127ff8b084ac379cb9aebccc102620c which was the transition
patches to get us from Quantum -> Neutron without breaking the
world. They need not be supported anymore.
Return the support for network_api_class actually working, as the
previous patch did a little more than deprecate it, as the option
provided was never respected.
Change-Id: I5ddf56bac99b16a8c6752303a532f132c3e5f14d
In order to add a hacking check which ensures that closures defined in
tests are actually used a few instances where they are not must be
cleaned up. This removes a few that were not used and uses a few that
should be used.
Change-Id: Ifb76525b96de49eccda5311c5d96c7644d99336b
As part of the ongoing effort to stop using mox, start replacing
the stubs.Set calls with stub_out.
Limit the scope of this patch to the ~400 db stubs.
Part of bp:remove-mox
Change-Id: I449726ede61a18d2c504cf370bebc2b3291fcb04
As part of the ongoing effort to stop using mox, start replacing
stubs.Set calls with stub_out.
Limit the scope of this patch to the following fake functions:
- fake_get_instance_nw_info
- stub_nw_info
- stub_out_instance_quota
- stub_out_networking
- stub_out_nw_api
- stub_out_nw_api_get_instance_nw_info
- set_stub_network_methods
- stub_out_network_cleanup
- unset_stub_network_methods
Part of bp:remove-mox
Change-Id: I70215fb25ef25422786b96d33c91d8f1d4760a23
It should be "IP" not "ip". Change strings
that are user visible or documentation strings.
Include some small updates in updated strings.
This does not change any comments.
Closes-Bug: #1524276
Change-Id: I16147e60b60be0f51c2aeb404ea9ed9fd3066795
contextlib.nested is deprecated in Python 2.7 and removed from
Python 3. This change will facilitate enabling Python 3 unit tests.
Enables some Python 3 unit tests in gate-nova-python34.
Partially implements blueprint: nova-python3-mitaka
Change-Id: I9b7d27be9404d5a87e4816e77649b87e09607f1b
This commit adds a new nova exception to handle a floating ip create
failure with a BadRequest exception from neutronclient. This is then
used in all 3 nova api implementations to ensure a 400 response is
returned just as neutron is returning to nova.
Change-Id: I36e8ef0113ae91e3b15c846c6d10e0b766cf2a37
Closes-Bug: #1482816
This patch moves the tests in contrib/ and plugins/v3/ into the
base directory.
Note that some of the tests have both v2 and v21 tests, The v2
tests will be deleted when V2 API removed.
Co-Authored-By: Ed Leafe <ed@leafe.com>
Change-Id: I6ff1d6594e7a44f2bcb6bbb04a4277b98d1cac74
Partial-Bug: #1462901
Stephen Finucane