This makes our API fixture pass roles in line with the user that
is being used. For admin_api, the admin role is included, and two
other clients are added for "reader" and "other".
Change-Id: I4aa985072103aeab50a1a3db4784081a492dcb7b
This adds a regression test for a bug where quota limit checking during
server creates is not properly scoped per-user when per-user quota has
been defined.
As a result, users who should be able to create a server are rejected
with a 403 "quota exceeded" error when they should be allowed to create
a server because servers owned by other users in the project are
incorrectly being counted for the current user.
Related-Bug: #1893284
Change-Id: I615ada45ffcbac081474c0a0cf005afdb8eec953
Enable support for API-based extra spec validation. Since most of the
hard work has been done in previous patches, all that's necessary here
is to wire up the microversion handling and turn things on.
Part of blueprint flavor-extra-spec-validators
Change-Id: If67f0d924ea372746a6dc440ea7bdc655e4f0bea
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
In microversion 2.80, the ``GET /os-migrations`` API will have
optional ``user_id`` and ``project_id`` query parameters for
filtering migrations by user and/or project:
* GET /os-migrations?user_id=ef9d34b4-45d0-4530-871b-3fb535988394
* GET /os-migrations?project_id=011ee9f4-8f16-4c38-8633-a254d420fd54
* GET /os-migrations?user_id=ef9d34b4-45d0-4530-871b-3fb535988394&project_id=011ee9f4-8f16-4c38-8633-a254d420fd54
And expose the ``user_id`` and ``project_id`` fields in the following APIs:
* GET /os-migrations
* GET /servers/{server_id}/migrations
* GET /servers/{server_id}/migrations/{migration_id}
Co-Authored-By: Qiu Fossen <qiujunting>
Part of blueprint add-user-id-field-to-the-migrations-table
Change-Id: I7313d6cde1a5e1dc7dd6f3c0dff9f30bbf4bee2c
This rips NoAuthMiddleware completely out of functional tests by:
- Removing our override of [api]auth_strategy in ConfFixture, allowing
it to default to ``keystone``. This causes the fake wsgi setup to go
through the same pipeline as real API requests; so making that work
entails...
- Mocking out the keystonecontext piece of the paste pipeline to create
the context previously mashed (somewhat inappropriately) into
NoAuthMiddleware. In the future we may want to mock this more
shallowly or find a way to make the req more realistic so it needn't
be mocked at all, but for now this is close to what the noauth2
pipeline used to do.
- Stubbing out the keystonemiddleware piece of the paste pipeline. In
the future we should try to use keystonemiddleware's AuthTokenFixture
so our tests can occur in a more realistic environment, but for now
this is just mimicking what the noauth2 pipeline used to do; except
for...
- Removing the authentication portion of the TestOpenStackClient. This
used to make an actual request(), which landed in NoAuthMiddleware,
which was hacking together some headers (based, it appears, on a
protocol which is many years out of date and no longer approximates
what keystone does, which should be the point if it's going to exist
at all). So now we just hack up the necessary headers inline.
- Doing the addition of project_id in request URIs in OSAPIFixture.
This is another thing that NoAuthMiddleware was doing inappropriately
(IRL the project_id will either be part of the request from the start,
or it won't). It was also only doing it part of the time; as a result,
a couple of tests requesting version documents, which were previously
not expecting the project ID to be present, needed to be modified to
expect it. This better reflects reality.
Change-Id: I459a605b4a9390f0e36356ca1fe432948159acd4
This patch enables counting both VCPU and PCPU when
`CONF.quota.count_usage_from_placement` enabled. Also add functional
test ensure the quota usage as expected.
Change-Id: I8377cbd1fb601d91deb7bb667a1ad5271c667399
Part of blueprint cpu-resources
This patch adds a new external event called "power-update"
through which ironic will convey all (power_off and power_on)
power state changes (running -> shutdown or shutdown -> running
will be the only ones handled by nova and the rest will be ignored)
on a physical instance to nova. The database will be updated
accordingly to reflect the real vm_state and power_state of the
instance. This way nova will not be able to enforce
an incorrect power state on the physical instance during
the periodic "sync_power_states" task.
Implements blueprint nova-support-instance-power-update
Story: 2004969
Task: 29423
Change-Id: I2b292050cc3ce5ef625659f5a1fe56bb76072496
get_instance_diagnostics expected all interfaces
to have a <target> element with a "dev" attribute in
the instance XML. This is not the case for VFIO
interfaces (<interface type="hostdev">).
This caused an IndexError when looping over
the interfaces.
This patch fixes this issue by retrieving interfaces
data directly from the guest XML and adding nics
appropriately to the diagnostics object.
Change-Id: I8ef852d449e9e637d45e4ac92ffc5d1abd8d31c5
Closes-Bug: #1821798
Adjust the fixtures used by the functional tests so they
use placement database and web fixtures defined by placement
code. To avoid making redundant changes, the solely placement-
related unit and functional tests are removed, but the placement
code itself is not (yet).
openstack-placement is required by the functional tests. It is not
added to test-requirements as we do not want unit tests to depend
on placement in any way, and we enforce this by not having placement
in the test env.
The concept of tox-siblings is used to ensure that the
placement requirement will be satisfied correctly if there is a
depends-on. To make this happen, the functional jobs defined in
.zuul.yaml are updated to require openstack/placement.
tox.ini has to be updated to use a envdir that is the same
name as job. Otherwise the tox siblings role in ansible cannot work.
The handling of the placement fixtures is moved out of nova/test.py
into the functional tests that actually use it because we do not
want unit tests (which get the base test class out of test.py) to
have anything to do with placement. This requires adjusting some
test files to use absolute import.
Similarly, a test of the comparison function for the api samples tests
is moved into functional, because it depends on placement functionality,
TestUpgradeCheckResourceProviders in unit.cmd.test_status is moved into
a new test file: nova/tests/functional/test_nova_status.py. This is done
because it requires the PlacementFixture, which is only available to
functional tests. A MonkeyPatch is required in the test to make sure that
the right context managers are used at the right time in the command
itself (otherwise some tables do no exist). In the test itself, to avoid
speaking directly to the placement database, which would require
manipulating the RequestContext objects, resource providers are now
created over the API.
Co-Authored-By: Balazs Gibizer <balazs.gibizer@ericsson.com>
Change-Id: Idaed39629095f86d24a54334c699a26c218c6593
We allowed add instance tags when booting in
Iac54b9627cb4398e45f1f15a2f4e7d6f86242093, conductor
will create tags for us in the selected cell, but
we forgot to consider the case that no valid host
was found.
So we will be unable to filter those instances in
cell0 with tags.
Change-Id: Idfe37d356b349024dfe963c934728111e1a1d314
closes-bug: #1806515
There are cases where ``root_provider_id`` of a resource provider is
set to NULL just after it is upgraded to the Rocky release. In such
cases getting allocation candidates raises a Keyerror.
This patch fixes that bug for cases there is no sharing or nested
providers in play.
Change-Id: I9639d852078c95de506110f24d3f35e7cf5e361e
Closes-Bug:#1799892
Later patches will introduce a field in RequestSpec using this type as
the field type to store the resource requests coming from outside of
Nova like the bandwidth request coming from the Neutron ports.
This patch refactors the usage of placement.lib.RequestGroup. Until now
this class was used both by placement and nova services and they used
it only as a util class. However after this series the nova services
would like to use such a class via RPC which requires an OVO. This
patch makes sure that the new OVO is used by nova and the old plain
object is used by placement. This way placement is not forced to use
an OVO where no OVO functionality is required.
The minimum required version of oslo.versionedobjects is updated to
1.33.3 to include the fix for bug 1771804.
Change-Id: I46c97d2641d9685ef59771314665a17a5236097d
blueprint: bandwidth-resource-provider
There are cases where ``root_provider_id`` of a resource provider is
set to NULL just after it is upgraded to the Rocky release. In such
cases getting allocation candidates raises a Keyerror.
This patch recreate that bug by simulating the situation by
inserting the records to the database directly.
Change-Id: Iaed912314f3e8fef2f46453a6bf12011702ae1dd
Related-Bug:#1799892
If there are multiple consumers having allocations to the same
resource provider, with different classes, it will attempt
multiple INSERTs with the same consumer_id which is not allowed
because of the database constraints.
This patch adds a simple GROUP BY in order to ensure that the
database server only provides us with unique values to avoid
trying to INSERT duplicate values.
Change-Id: I1acba5e65cd562472f29e354c6077f82844fa87d
Closes-Bug: #1798163
Change Icae5038190ab8c7bbdb38d54ae909fcbf9048912 in Rocky
attempts to online migrate missing consumers table records
when listing allocations for a given resource provider. The
problem is when it's doing an insert-from-select, it's not
handling multiple allocations on the same provider for the
same consumer, like you'd have with a compute instance that
has VCPU, MEMORY_MB and DISK_GB allocations against a single
compute node resource provider. As a result, the insert
statement has duplicate consumer IDs in it which results in
a unique constraint violation.
The existing tests never caught this because they tested with
3 unique consumers with a single allocation each.
The functional test added here hits both online data migration
routines: via the API when listing allocations for a resource
provider and the direct online data migration CLI.
Change-Id: Iba56aa6b227b6455d2437e4fabcd296b1b0f06ee
Related-Bug: #1798163
The Iddbe50ce0ad3c14562df800bbc09ec5a7e840485 patch only considered
instance delete in the happy case when the instance is scheduled to a
compute successfully and the compute is available when the delete
action is executed. If the instance is never scheduled to a compute or
the compute is not available when the instance is deleted legacy delete
notifications are emitted from different places, compute.api instead of
compute.manager. The original patch missed these places.
There will be subsequent patch(es) handling the same edge cases
for soft_delete and force_delete.
Change-Id: If0693eab2ed31b5fbfe6cbafa5d67b69c2ed8442
Implements: bp versioned-notification-transformation-stein
Per the referenced bug, we weren't accounting for the scenario where a
reshape operation was removing *all* inventories for a provider (which
could be fairly common). With this fix, we do a three-stage lookup of
the provider object: If it's not in the inventories, we look in the
allocations; if it's not in the allocations, we look it up in the
database.
Change-Id: I594bb64f87c61b7ffd39c19e0fd42c4c087a3a11
Closes-Bug: #1783130
Add a test that traverses all available placement URLs at the latest
microversion and tries to access them as non-admin. If something other
than a 403 response is given a failed test with a message like
method POST on route /resource_providers/{uuid}/inventories
is open for user, status: 404
is produced.
This works because we do authZ handling early in each handler, before
data processing and path parameter handling.
The method is pretty straightforward: traverse ROUTE_DECLARATIONS, visit
every url with each the declared methods, except the root version document,
and confirm a 403 response when the provided auth token is non-admin.
This has been created to avoid situations where a route is added without
policy like happened on https://review.openstack.org/#/c/576927/ . Until
recently we had a failover where any route not defined to have policy
would default to admin. That went away so now we need some test
automation to catch our forgetful humanness.
Change-Id: Id582886ec4b621b97d7cc7237b4670ad7bb12295
When creating resource provider with '--uuid' argument, nova
accept uuid without dash('-') too, which some time results in,
resource provider with same uuid i.e one with dash and one without.
This patch attempts to fix it by transforming dashless UUID into
dashed one before inserting it into the database.
Co-Authored-By: Chen <dstbtgagt@foxmail.com>
Change-Id: I2685eb65907adbd22b2d09264b110692e100eaf9
Closes-Bug: #1758057
/reshaper provides a way to atomically modify some allocations and
inventory in a single transaction, allowing operations like migrating
some inventory from a parent provider to a new child.
A fair amount of code is reused from handler/inventory.py, some
refactoring is in order before things get too far with that.
In handler/allocation.py some code is extracted to its own methods
so it can be reused from reshaper.py.
This is done as microversion 1.30.
A suite of gabbi tests is provided which attempt to cover various
failures including schema violations, generation conflicts, and
data conflicts.
api-ref, release notes and rest history are updated
Change-Id: I5b33ac3572bc3789878174ffc86ca42ae8035cfa
Partially-Implements: blueprint reshape-provider-tree
Without this change, tests can intermittently fail with NoSuchOptError
when a single process does not have other tests running prior to
gabbi tests. This change ensure the opts are registered and defaulted.
Change-Id: I1c7e347b6e788928bef96e32c3365d0fdc5ba00f
Related-Bug: #1786498
Closes-Bug: #1788176
This patch adds a test for _get_provider_ids_matching()
to verify it works correctly with required traits.
Related-Bug: #1786519
Change-Id: I2512e361f5eaa4e60701be7c8bf57b2e0a02a146
For code refactoring purpose in the placement granular fixture
setup, this patch substitutes common functions in test_base.py
for existing local functions of _create_providers(),
_add_inventory(), and _set_traits().
Change-Id: I76a3f7d7e446e0f3af379f83c9d8333279884c73
`GET /resource_provider/{uuid}/allocations` API didn't
return all the allocations made by multiple users.
This was because the placement wrongly used project table
for user table. This patch fixes it with the test case.
Change-Id: I7c808dec5de1204ced0d1f1b31d8398de8c51679
Closes-Bug: #1785382
`GET /resource_provider/{uuid}/allocations` API currently doesn't
return all the allocations made by multiple users.
This patch adds a test to describe this bug. The fix for this
is coming in a follow up.
Change-Id: I2b01e27922f11bef2defcb01fe415692de1578ea
Partial-Bug: #1785382
This patch refactors the allocation fixture used in some placement
api tests by substituting common functions in test_base.py for the
existing object management functions.
Change-Id: Ide6544d1cf9e1ed154b42075acdd7af986c3afe8
Dan Smith