This makes our API fixture pass roles in line with the user that
is being used. For admin_api, the admin role is included, and two
other clients are added for "reader" and "other".
Change-Id: I4aa985072103aeab50a1a3db4784081a492dcb7b
This adds a regression test for a bug where quota limit checking during
server creates is not properly scoped per-user when per-user quota has
been defined.
As a result, users who should be able to create a server are rejected
with a 403 "quota exceeded" error when they should be allowed to create
a server because servers owned by other users in the project are
incorrectly being counted for the current user.
Related-Bug: #1893284
Change-Id: I615ada45ffcbac081474c0a0cf005afdb8eec953
Enable support for API-based extra spec validation. Since most of the
hard work has been done in previous patches, all that's necessary here
is to wire up the microversion handling and turn things on.
Part of blueprint flavor-extra-spec-validators
Change-Id: If67f0d924ea372746a6dc440ea7bdc655e4f0bea
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
In microversion 2.80, the ``GET /os-migrations`` API will have
optional ``user_id`` and ``project_id`` query parameters for
filtering migrations by user and/or project:
* GET /os-migrations?user_id=ef9d34b4-45d0-4530-871b-3fb535988394
* GET /os-migrations?project_id=011ee9f4-8f16-4c38-8633-a254d420fd54
* GET /os-migrations?user_id=ef9d34b4-45d0-4530-871b-3fb535988394&project_id=011ee9f4-8f16-4c38-8633-a254d420fd54
And expose the ``user_id`` and ``project_id`` fields in the following APIs:
* GET /os-migrations
* GET /servers/{server_id}/migrations
* GET /servers/{server_id}/migrations/{migration_id}
Co-Authored-By: Qiu Fossen <qiujunting>
Part of blueprint add-user-id-field-to-the-migrations-table
Change-Id: I7313d6cde1a5e1dc7dd6f3c0dff9f30bbf4bee2c
This rips NoAuthMiddleware completely out of functional tests by:
- Removing our override of [api]auth_strategy in ConfFixture, allowing
it to default to ``keystone``. This causes the fake wsgi setup to go
through the same pipeline as real API requests; so making that work
entails...
- Mocking out the keystonecontext piece of the paste pipeline to create
the context previously mashed (somewhat inappropriately) into
NoAuthMiddleware. In the future we may want to mock this more
shallowly or find a way to make the req more realistic so it needn't
be mocked at all, but for now this is close to what the noauth2
pipeline used to do.
- Stubbing out the keystonemiddleware piece of the paste pipeline. In
the future we should try to use keystonemiddleware's AuthTokenFixture
so our tests can occur in a more realistic environment, but for now
this is just mimicking what the noauth2 pipeline used to do; except
for...
- Removing the authentication portion of the TestOpenStackClient. This
used to make an actual request(), which landed in NoAuthMiddleware,
which was hacking together some headers (based, it appears, on a
protocol which is many years out of date and no longer approximates
what keystone does, which should be the point if it's going to exist
at all). So now we just hack up the necessary headers inline.
- Doing the addition of project_id in request URIs in OSAPIFixture.
This is another thing that NoAuthMiddleware was doing inappropriately
(IRL the project_id will either be part of the request from the start,
or it won't). It was also only doing it part of the time; as a result,
a couple of tests requesting version documents, which were previously
not expecting the project ID to be present, needed to be modified to
expect it. This better reflects reality.
Change-Id: I459a605b4a9390f0e36356ca1fe432948159acd4
This patch enables counting both VCPU and PCPU when
`CONF.quota.count_usage_from_placement` enabled. Also add functional
test ensure the quota usage as expected.
Change-Id: I8377cbd1fb601d91deb7bb667a1ad5271c667399
Part of blueprint cpu-resources
This patch adds a new external event called "power-update"
through which ironic will convey all (power_off and power_on)
power state changes (running -> shutdown or shutdown -> running
will be the only ones handled by nova and the rest will be ignored)
on a physical instance to nova. The database will be updated
accordingly to reflect the real vm_state and power_state of the
instance. This way nova will not be able to enforce
an incorrect power state on the physical instance during
the periodic "sync_power_states" task.
Implements blueprint nova-support-instance-power-update
Story: 2004969
Task: 29423
Change-Id: I2b292050cc3ce5ef625659f5a1fe56bb76072496
get_instance_diagnostics expected all interfaces
to have a <target> element with a "dev" attribute in
the instance XML. This is not the case for VFIO
interfaces (<interface type="hostdev">).
This caused an IndexError when looping over
the interfaces.
This patch fixes this issue by retrieving interfaces
data directly from the guest XML and adding nics
appropriately to the diagnostics object.
Change-Id: I8ef852d449e9e637d45e4ac92ffc5d1abd8d31c5
Closes-Bug: #1821798
We allowed add instance tags when booting in
Iac54b9627cb4398e45f1f15a2f4e7d6f86242093, conductor
will create tags for us in the selected cell, but
we forgot to consider the case that no valid host
was found.
So we will be unable to filter those instances in
cell0 with tags.
Change-Id: Idfe37d356b349024dfe963c934728111e1a1d314
closes-bug: #1806515
The Iddbe50ce0ad3c14562df800bbc09ec5a7e840485 patch only considered
instance delete in the happy case when the instance is scheduled to a
compute successfully and the compute is available when the delete
action is executed. If the instance is never scheduled to a compute or
the compute is not available when the instance is deleted legacy delete
notifications are emitted from different places, compute.api instead of
compute.manager. The original patch missed these places.
There will be subsequent patch(es) handling the same edge cases
for soft_delete and force_delete.
Change-Id: If0693eab2ed31b5fbfe6cbafa5d67b69c2ed8442
Implements: bp versioned-notification-transformation-stein
This adds the "nova-manage placement sync_aggregates"
command which will compare nova host aggregates to
placement resource provider aggregates and add any
missing resource provider aggregates based on the nova
host aggregates.
At this time, it's only additive in that the command
does not remove resource provider aggregates if those
matching nodes are not found in nova host aggregates.
That likely needs to happen in a change that provides
an opt-in option for that behavior since it could be
destructive for externally-managed provider aggregates
for things like ironic nodes or shared storage pools.
Part of blueprint placement-mirror-host-aggregates
Change-Id: Iac67b6bf7e46fbac02b9d3cb59efc3c59b9e56c8
Keystone deprecated 'auth_uri' to bring 'www_authenticate_uri' in
https://review.openstack.org/#/c/508522/, so 'auth_uri'
should be removed from all openstack projects.
The auth_url is the parameter passed to the keystoneauth plugin
and the auth_uri is the parameter specific to keystonemiddleware,
so the auth_url would be appropriate in api/client since that's
going to be passed straight to keystoneauth, and it should be
the same in api/client and in the fixture. So we change the
parameter to auth_url in api/client instead of www_authenticate_uri.
Change-Id: I7f878974fe8bebb8fe5803ca4e2f8839cdc971d0
Related-Bug: 1755728
Change I9c2111f7377df65c1fc3c72323f85483b3295989 fixed it so we stop
allocating the flavor's root_gb if we're booting from volume. However,
we're still reporting it against the compute node's capacity.
Furthermore, we're not reporting swap space at all.
This adds functional test coverage demonstrating these bugs.
Change-Id: Ia46754d8ff65c501545853e6c2909a5a900d45c0
Related-Bug: #1782393
Related-Bug: #1782386
The aggregate.update_prop.start and aggregate.update_prop.end
notifications has been transformed to the versioned notification
framework.
Co-Authored-By: Takashi Natsume <natsume.takashi@lab.ntt.co.jp>
Change-Id: I37b19573b6d0e1131c446fcec361f01fa2560f82
Implements: bp versioned-notification-transformation-rocky
Currently when deleting a nova-compute service via the API, we will
(soft) delete the service and compute_node records in the DB, but the
placement resource provider and host mapping records will be orphaned.
This patch deletes the resource provider and host_mapping records
before deleting the service/compute node.
Change-Id: I7b8622b178d5043ed1556d7bdceaf60f47e5ac80
Closes-Bug: #1756179
The instance.resize_confirm.start and instance.resize_confirm.end
notifications are transformed to the versioned framework.
Change-Id: I6a51542216340299d250576714e303f74e0ceb0f
Co-Authored-By: Takashi Natsume <natsume.takashi@lab.ntt.co.jp>
Co-Authored-By: stewie925 <st3wty@att.com>
Co-Authored-By: MikeG451 <michael.glaser@att.com>
Implements: bp versioned-notification-transformation-queens
During a rebuild we bypass the scheduler because we are
always rebuilding the instance on the same host it's already
on. However, we allow passing a new image during rebuild
and that new image needs to be validated to work with the
instance host by running it through the scheduler filters,
like the ImagePropertiesFilter. Otherwise the new image
could violate constraints placed on the host by the admin.
This change checks to see if there is a new image provided
and if so, modifies the request spec passed to the scheduler
so that the new image is validated all while restricting
the scheduler to still pick the same host that the instance
is running on. If the image is not valid for the host, the
scheduler will raise NoValidHost and the rebuild stops.
A functional test is added to show the recreate of the bug
and that we probably stop the rebuild now in conductor by
calling the scheduler to validate the image.
Co-Authored-By: Sylvain Bauza <sbauza@redhat.com>
Closes-Bug: #1664931
Change-Id: I11746d1ea996a0f18b7c54b4c9c21df58cc4714b
In I6ddcaaca37fc5387c2d2e9f51c67ea9e85acb5c5 we forgot to update the
legacy filter properties dictionary so the requested target wasn't
passed to the scheduler when evacuating.
Adding a functional test for verifying the behaviour.
NOTE(sbauza): The issue has been incendentally fixed in Pike by
I434af8e4ad991ac114dd67d66797a562d16bafe2 so the regression test just
verifies that the expected behaviour works.
The Newton and Ocata backports will be slightly different from that one as we
need to verify that host3 will be preferred eventually over host2.
Related-Bug: #1702454
Change-Id: Id9adb10d2ef821c8b61d8f1d5dc9dd66ec7aaac8
This patch adds test that checkes that the resource allocation is properly
handled during live migration force complete.
Co-Authored-By: Dan Smith <dms@danplanet.com>
Related-Bug: #1712045
Change-Id: If958e64eda1260a8dd08078de2dae167c6a147bc
The aggregate.add_host.start and aggregate.add_host.end notifications
has been transformed to the versioned notification framework.
Note that the legacy event_type was aggregate.addhost but the versioned
notification is named to aggregate.add_host to keep the naming consistent
with the other event_types.
Implements: bp versioned-notification-transformation-queens
Change-Id: I980d89f47b944116e5fc9cedd74c43bbd88fbeae
The keypair.delete.start and keypair.delete.end notifications
has been transformed to the versioned notification framework.
Change-Id: I48f28a56f9232faf52f458588500a9d77eb6b679
Implements: bp versioned-notification-transformation-queens
Add functional test for evacuation, when no valid host available.
Migration should end up in 'error' state.
Change-Id: I1adc20f2a5261e6906a18b9aee5cd2c8ecf0cf4d
Related-bug: #1713783
Test cases for resize to same host with confirm and revert scenarios
have been added.
The accounting of the current code is very incorrect at the moment, so this
asserts the existing (wrong) behavior, but marks the actual expected
values everywhere so that we can follow up with fixes to remove all of the
annotations.
Related-Bug: #1707252
Change-Id: Ica83ce5f43c0e7b84fd4117852e81e4d2979dea3
Test cases for resize with confirm and revert scenarios have been
added.
This test depends on the periodic update_available_resources call to heal
the resource allocations. The fake virt driver needed to be tweaked to
work when more than one compute service is created and periodic tasks
are enabled.
The accounting of the current code is very incorrect at the moment, so this
asserts the existing (wrong) behavior, but marks the actual expected
values everywhere so that we can follow up with fixes to remove all of the
annotations.
Change-Id: If7b02bcd8d77e94c7fb42b721792c1391bc0e3b7
blueprint: placement-claims
In functional tests, this method will be used in some places. This
method waits for the migrations and if there is at least one active
migration in progress, the method return with the migrations.
Change-Id: Ia068de2dac34f2399025fb28fec9af75c1442d63
Co-Authored-By: Alex Szarka <szarka@inf.u-szeged.hu>
Implements: bp versioned-notification-transformation-pike
Due to eventlet threading weirdness, it appears that sometimes a request
that is destined for the compute api fixture ends up on the placement
api fixture. The latter uses a no auth middleware that 401s when it does
not get a token.
The change retries the auth request a few times before failing.
If this has no effect, doing an eventlet.sleep() before the retry might
be something worth trying.
Change-Id: I9ca2022ca956be5ea67f4c40bed80115f2fae29d
Related-Bug: #1705753
When running py27 tests, you will see a bunch of the following
warnings:
BaseException.message has been deprecated as of Python 2.6
Because of some old code in the OpenStackApiException base class that
was setting the message attribute of the base class. This code isn't
necessary.
Change-Id: I24357d3069f653a707f15abb34d9d2546c3ee044
The keypair.create.start and keypair.create.end notifications
has been transformed to the versioned notification framework.
Change-Id: I71e9d8dae55653ad3ee70f708a6d92c98ed20c1c
Implements: bp versioned-notification-transformation-pike
After the cellification of the Aggregates API, we introduced a requirement
that the service must have a HostMapping record, so that we know which
cell the service is in. This is normally fine, but for weird drivers such
as ironic, there may be valid cases (i.e. during setup) where no ironic
nodes are present, thus one or more services may not have any compute node
records, and thus cannot be added to aggregates.
This adds a cell scan, only if necessary, to find the desired service so
that the operation may proceed as it did before. To do this, we refactor
the _find_service() helper to a more generic utility and use that if we
don't find a HostMapping during the add operation.
Change-Id: Idc97126d63684e7d638b974d7226ff210c744404
Closes-Bug: #1686744
The negative tests of interfaces API are not covered enough
in functional tests. We want to add the conflict tests of when
this API runs in the unexpected state (e.g. not ACTIVE) of server.
Change-Id: I4bea067a729db735524689a5cbe49cceab23b51c
There was a latent bug in the DB API code such that when we
process filters when listing instances, the various tags
filters have their values popped out of the filters dict and
the values (which are lists) for the filter also have the
first item in the list popped out to build the query.
This latent bug was exposed in Newton when we started listing
instances in the API from both cell0 and the main cell database,
because the query to cell0 would pop an item and then it
would not be in the 2nd query to the main cell database. If we
only had one tag in the filter list, we get an IndexError on
the 2nd pop() call.
Note that we also use the build_requests table in the API to
list instances, but there would not be any tagged servers in
that table since a server has to be ACTIVE before you can tag it,
and build_requests only exist until the instance is put into a
cell and starts building there (so it won't be ACTIVE yet).
Change-Id: I65bdfadd406368775c794ee4d8a16302a8a134bc
Related-Bug: #1682693
Once an instance is in a cell and we do a local delete from
the API, we aren't actually detaching volumes and destroying
BDMs.
Related-Bug: #1675570
Change-Id: Ie3e2dfd4b0f1bb3dff4080f460bf8bb40d69f4f4
This adds a functional regression test for bug 1670627.
This is the recreate scenario. Patches that are proposed to
fix the bug will build on top of this and change it's assertions
to know when it's properly fixed.
Change-Id: I872a3fd5cfd3dd869f74cd3fd0aa5da411b1fec3
Related-Bug: #1670627
1.As mentioned in [1], we should avoid using
six.iteritems to achieve iterators. We can
use dict.items instead, as it will return
iterators in PY3 as well. And dict.items/keys
will more readable. 2.In py2, the performance
about list should be negligible, see the link [2].
[1] https://wiki.openstack.org/wiki/Python3
[2] http://lists.openstack.org/pipermail/openstack-dev/2015-June/066391.html
The patch list:
tests.
Partial-Implements: blueprint replace-iteritems-with-items
Change-Id: I2d09c1d06370fed9c7c116a9eedea929fa5b5dc2
The aggregate.delete.start and aggregate.delete.end notifications
has been transformed to the versioned notification framework.
Implements: bp versioned-notification-transformation-ocata
Change-Id: I911e7aef813380291e6cf166eac91b53215a3726
The aggregate.create.start and aggregate.create.end notifications
has been transformed to the versioned notification framework.
The notification payload contains the 'id' of the aggregate which
is the db primary key. It is added because the REST API also
uses the db id instead of the uuid in the os-aggregates' requests
and responses.
Implements: bp versioned-notification-transformation-ocata
Change-Id: I92fe504a8f7dc19b0e1df5884045d4bc0d9e4f98
Dan Smith