Now that we no longer support py27, we can use the standard library
unittest.mock module instead of the third party mock lib. Most of this
is autogenerated, as described below, but there is one manual change
necessary:
nova/tests/functional/regressions/test_bug_1781286.py
We need to avoid using 'fixtures.MockPatch' since fixtures is using
'mock' (the library) under the hood and a call to 'mock.patch.stop'
found in that test will now "stop" mocks from the wrong library. We
have discussed making this configurable but the option proposed isn't
that pretty [1] so this is better.
The remainder was auto-generated with the following (hacky) script, with
one or two manual tweaks after the fact:
import glob
for path in glob.glob('nova/tests/**/*.py', recursive=True):
with open(path) as fh:
lines = fh.readlines()
if 'import mock\n' not in lines:
continue
import_group_found = False
create_first_party_group = False
for num, line in enumerate(lines):
line = line.strip()
if line.startswith('import ') or line.startswith('from '):
tokens = line.split()
for lib in (
'ddt', 'six', 'webob', 'fixtures', 'testtools'
'neutron', 'cinder', 'ironic', 'keystone', 'oslo',
):
if lib in tokens[1]:
create_first_party_group = True
break
if create_first_party_group:
break
import_group_found = True
if not import_group_found:
continue
if line.startswith('import ') or line.startswith('from '):
tokens = line.split()
if tokens[1] > 'unittest':
break
elif tokens[1] == 'unittest' and (
len(tokens) == 2 or tokens[4] > 'mock'
):
break
elif not line:
break
if create_first_party_group:
lines.insert(num, 'from unittest import mock\n\n')
else:
lines.insert(num, 'from unittest import mock\n')
del lines[lines.index('import mock\n')]
with open(path, 'w+') as fh:
fh.writelines(lines)
Note that we cannot remove mock from our requirements files yet due to
importing pypowervm unit test code in nova unit tests. This library
still uses the mock lib, and since we are importing test code and that
lib (correctly) only declares mock in its test-requirements.txt, mock
would not otherwise be installed and would cause errors while loading
nova unit test code.
[1] https://github.com/testing-cabal/fixtures/pull/49
Change-Id: Id5b04cf2f6ca24af8e366d23f15cf0e5cac8e1cc
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
As agreed in the spec, we will both drop the generation support for a keypair
but we'll also accept @ (at) and . (dot) chars in the keyname, all of them in
the same API microversion.
Rebased the work from I5de15935e83823afa545a250cf84f6a7a37036b4
APIImpact
Implements: blueprint keypair-generation-removal
Co-Authored-By: Nicolas Parquet <nicolas.parquet@gandi.net>
Change-Id: I6a7c71fb4385348c87067543d0454f302907395e
When using unified limits, we add enforcement of those limits on all
related API calls. Note: we do not yet correctly report the configured
limits to users via the quota APIs, that is in a future patch.
Note the unified limits calls are made alongside the existing legacy
quota calls. The old quota calls will be handed by the quota engine
driver, that is basically a no-op. This is to make it easier to remove
the legacy code paths in the future.
Note, over quota exceptions raised with unified limits use the standard
(improved) exception message as those raised by oslo.limit. They
however do use the existing exception code to ease integration. The
user of the API will see the same return codes, no matter which code is
enabled to enforce the limits.
Finally, this also adds test coverage where it was missing. Coverage
for "quota recheck" behavior in KeypairAPI is added where all other
KeypairAPI testing is located. Duplicate coverage is removed from
nova/api/openstack/compute/test_keypairs.py at the same time.
blueprint unified-limits-nova
Change-Id: I36e82a17579158063396d7e55b495ccff4959ceb
The KeypairLimitExceeded exception has a message string which is never
used. We raise this exception and then return a different message to
the API user. For the unified limit work, we want to move to using
oslo.limit's better error messages when available, which means we
need to honor the message in the exception. This just moves the
legacy string into the exception and makes the API use that instead
of overriding it.
Related to bp/unified-limits-nova
Change-Id: I217b3d0551291498191b556f62d78abf159778c2
This was migrated to the API DB during the 14.0.0 (Newton) release [1]
and the 345 migration introduced during the 15.0.0 (Ocata) release [2]
ensures we should no longer have any entries left in the main database.
The actual model isn't removed yet. That will be done separately.
[1] I5f6d88fee47dd87de2867d3947d65b04f0b21e8f
[2] Iab714d9e752c334cc1cc14a0d524cc9cf5d115dc
Change-Id: I15efc38258685375284d8a97004777385023c6e8
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Merge these, removing an unnecessary layer of abstraction, and place
them in the new 'nova.db.main' directory. The resulting change is huge,
but it's mainly the result of 's/sqlalchemy import api/main import api/'
and 's/nova.db.api/nova.db.main.api/' with some necessary cleanup. We
also need to rework how we do the blocking of API calls since we no
longer have a 'DBAPI' object that we can monkey patch as we were doing
before. This is now done via a global variable that is set by the 'main'
function of 'nova.cmd.compute'.
The main impact of this change is that it's no longer possible to set
'[database] use_db_reconnect' and have all APIs automatically wrapped in
a DB retry. Seeing as this behavior is experimental, isn't applied to
any of the API DB methods (which don't use oslo.db's 'DBAPI' helper),
and is used explicitly in what would appear to be the critical cases
(via the explicit 'oslo_db.api.wrap_db_retry' decorator), this doesn't
seem like a huge loss.
Change-Id: Iad2e4da4546b80a016e477577d23accb2606a6e4
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Back in change Id4a4e1ae4c0acd40c1fc32c3b82a8d8a62d4624d we bumped our
required cryptography package version to one that has support for
ed25519 ssh keys.
This adds a unit test to assert that ed25519 ssh key import works.
Related-Bug: #1555521
Change-Id: I5b37caa6d3b1b209722e9d74b127ea2f29b37563
Replace six.text_type with str.
This patch completes six removal.
Change-Id: I779bd1446dc1f070fa5100ccccda7881fa508d79
Implements: blueprint six-removal
Signed-off-by: Takashi Natsume <takanattie@gmail.com>
nova/db/__init__.py was importing * from nova.db.api. This meant that
any time any code anywhere within the nova.db package was imported
then nova.db.api was too, leading to a cascade of imports that may
not have been desired. Also, in general, code in __init__.py is a pain.
Therefore, this change adjusts code that so that either:
* nova.db.api is used directly
* nova.db.api is imported as 'db'
In either case, the functionality remains the same.
The primary goal of this change was to make it possible to import the
model files without having to import the db api. Moving the model files
to a different place in the directory hierarchy was considered, but
given that "code in __init__.py is a pain" this mode was chosen.
This looks like a very large change, but it is essentially adjusting
package names, many in mocks.
Change-Id: Ic1fd7c87ceda05eeb96735da2a415ef37060bb1a
The keypair.import.start and keypair.import.end notifications
has been transformed to the versioned notification framework.
Change-Id: Icadc53d84fa021aae0f9ec2e18e9125de98e00f6
Implements: bp versioned-notification-transformation-queens
The keypair.delete.start and keypair.delete.end notifications
has been transformed to the versioned notification framework.
Change-Id: I48f28a56f9232faf52f458588500a9d77eb6b679
Implements: bp versioned-notification-transformation-queens
The keypair.create.start and keypair.create.end notifications
has been transformed to the versioned notification framework.
Change-Id: I71e9d8dae55653ad3ee70f708a6d92c98ed20c1c
Implements: bp versioned-notification-transformation-pike
This adds two new methods to the Quotas object in preparation for the
switch to counting resources for checking quota.
The existing limit_check() method only supports checking quota against
per-user limits because currently, the only CountableResources that
exist are not ever counted across a project.
To implement resource counting for all resources, we will change each
resource to a countable resource and some of the resources will be
counted across a project and across a user and checked against
per-project and per-user limits, respectively. To do this, we add a new
method: count_as_dict that returns a count as a dictionary to support
multiple counts (over a project and over a user) and a new method:
limit_check_project_and_user() that supports checking quota against
both per-project and per-user limits.
The check_deltas() method is a wrapper around the count_as_dict() and
limit_check_project_and_user() calls where only the deltas are passed
in and the counting and limit check are done inside the wrapper. This
cleans up the call sites and enables us to potentially make the
count + check atomic in the future behind the check_deltas() interface.
A change has also been made to the _validate_method_call_check_resource
methods to let them take the Resources dict as an argument instead of
assuming and hard-coding the global QuotaEngine's resources. This was
mainly needed for unit testing because in order to cover both the
per-project and per-user limit checking, a CountableResource counted
across both a project and a user is needed, and currently none exist
in the global QuotaEngine. Resources will be changed over in
subsequent patches.
Co-Authored-By: melanie witt <melwittt@gmail.com>
Part of blueprint cells-count-resources-to-check-quota-in-api
Change-Id: Ib52a21836c4638cb9f7a5a55a5befbf0963fe6bf
As suggested in John Garbutt in I7e1986c5f11356060cc9db12605b1322c39e79c0,
move the quota config options into a config group of their own.
Change-Id: Ie06a370868f01fb9a1cc246a77d7823fac83e70e
keypair object and db_api supports 'limit' and 'marker'
parameters, these changes are required for keypairs pagination
for Nova API.
Part of blueprint keypairs-pagination
Change-Id: I9776efc609f01d053824b31f64126cfcd6dadc18
As part of the ongoing effort to stop using mox, start replacing
the stubs.Set calls with stub_out.
Limit the scope of this patch to the ~400 db stubs.
Part of bp:remove-mox
Change-Id: I449726ede61a18d2c504cf370bebc2b3291fcb04
Currently, the Subject created for the X509 certificate
is too long, resulting in exceptions and failing to
create the keypair.
This change shortens the Subject.
Unit test added to prove the issue's existence.
Issue found by: Andrey Kurilin
Co-authored-by: Andrey Kurilin <akurilin@mirantis.com>
Change-Id: I7885c120ce81a22d416f806779bf9a12092d5040
Closes-Bug: #1447653
Removes unused arguments in test_keypairs.
Sets the keypair 'type' field to enum in the v2.2 validation
schema.
Removes check if the keypair 'type' is valid, as it will be done
by the validation schema.
Adds functional tests for ssh and x509 keypairs.
Closes-Bug: 1434033
Change-Id: I39f0a363458a17a4011a0661a45dc449f122ba1a
X509 certificates are used by Windows for passwordless
authentication (WinRM) in a way which can be considered
consistent with the usage of SSH keys on Linux, as both
are based on public / private keypairs.
Adds x509 certificate creation implementation.
Adds 'keys' in InstanceMetadata.
Implements: blueprint keypair-x509-certificates
Depends-On: Id5b210d7afe5c0a590abcbd42b9ff85b071a5c55
Change-Id: I1ab518553cf757ae35548fa89b6c8fee7ec32b8d
X509 certificates are used by Windows for passwordless
authentication (WinRM) in a way which can be considered
consistent with the usage of SSH keys on Linux, as both
are based on public / private keypairs.
Enables nova-api to return the keypair type, updates
nova-api version to reflect the changes and updates the
unit and functional tests to validate the API changes.
Unit tests have been updated to ensure that the keypair
type is not being returned on previous API versions.
Note: x509 keypair implementation is added in the next
commit.
DocImpact - See nova/api/openstack/rest_api_version_history.rst
for details
APIImpact
Depends-On: Id5b210d7afe5c0a590abcbd42b9ff85b071a5c55
Co-Authored-By: Chris Yeoh <cyeoh@au1.ibm.com>
Partially implements: blueprint keypair-x509-certificates
Change-Id: I215662f2f92a01921a866c3218031787a9eaf915
The oslo team is recommending everyone to switch to the
non-namespaced versions of libraries. Updating the hacking
rule to include a check to prevent oslo.* import from
creeping back in.
This commit includes:
- using oslo_utils instead of oslo.utils
- using oslo_serialization instead of oslo.serialization
- using oslo_db instead of oslo.db
- using oslo_i18n instead of oslo.i18n
- using oslo_middleware instead of oslo.middleware
- using oslo_config instead of oslo.config
- using oslo_messaging instead of "from oslo import messaging"
- using oslo_vmware instead of oslo.vmware
Change-Id: I3e2eb147b321ce3e928817b62abcb7d023c5f13f
Exception lines in unit tests won't ever be run in production, no reason to
translate them.
Added hacking rule for not importing i18n translation in tests.
Change-Id: I92f546166d4e0b2fa8dc2018c6d3e268b8ec4c0b
As part of the split of functional and unit tests we need to isolate
the unit tests into a separate directory for having multiple test
targets in a sane way.
Part of bp:functional-tests-for-nova
Change-Id: Id42ba373c1bda6a312b673ab2b489ca56da8c628
Stephen Finucane