Finish up removing these entries from the versioned instance
notifications. They're useless since we dropped support for the XenAPI
virt driver. The underlying model is retained for now: that will be
handled separately.
Change-Id: I774c50fca99bc655ca5010e3b9d8247b739293b3
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
delete arqs:
- delete arq while port unbind
- create ops failed and arqs did not bind to instance
- arq bind to instance but not bind to port
Implements: blueprint sriov-smartnic-support
Change-Id: Idab0ee38750d018de409699a0dbdff106d9e11fb
The fake_notifier uses module globals and also needs careful stub and
reset calls to work properly. This patch wraps the fake_notifier into a
proper Fixture that automates the complexity.
This is fairly rage patch but it does not change any logic just redirect
calls from the fake_notifier to the new NotificationFixture
Change-Id: I456f685f480b8de71014cf232a8f08c731605ad8
Glance uses properties like os_glance_importing_to_stores for internal
purposes. These should never be copied to a snapshot by nova. Further,
Glance would like to disallow setting these properties by namespace
entirely in the future.
This patch adds namespace support to our existing list of static
properties which are non-inheritable.
Change-Id: Ib5d9da46b1a0e467b968e3ec3e61125e06f99bfc
Make update_pci_request_spec_with_allocated_interface_name only depend
on a list of IntancePCIRequest o.vos instead of a whole Instance object.
This will come in handy for the qos interface attach case where we only
need to make the changes on the Instance o.vo after we are sure that
the both the resource allocation and the pci claim is succeeded for the
request.
Change-Id: I5a6c6d3eed61895b00f9e9c3fb3b5d09d6786e9c
blueprint: support-interface-attach-with-qos-ports
Replace six.text_type with str.
This patch completes six removal.
Change-Id: I779bd1446dc1f070fa5100ccccda7881fa508d79
Implements: blueprint six-removal
Signed-off-by: Takashi Natsume <takanattie@gmail.com>
Migrate unit tests over to the new GlanceFixture, allowing us to fully
delete '_FakeImageService' and various helpers.
Change-Id: I3e3d9bbd32aa2fe0777681f21ba827610d5b3040
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
The Quotas.check_deltas method needs a user_id keyword arg in order to
scope a quota check to a particular user. However, when we call
check_num_instances_quota we don't pass a project_id or user_id because
at the time of the quota check, we have not yet created an instance
record and thus will not use that to determine the appropriate project
and user.
Instead, we should rely on the RequestContext.project_id and
RequestContext.user_id as defaults in this case, but
check_num_instances_quota only defaults project_id and not user_id.
check_num_instances_quota should also default user_id to the
RequestContext.user_id when user_id is not explicitly passed.
check_num_instances_quota should also check whether any per-user quota
limits are defined for instance-related resources before passing along
the user_id to scope resource counting and limit checking. Counting
resources across a user is costly, so we should avoid it if it's not
needed.
Closes-Bug: #1893284
Change-Id: I3cfb1edc30b0bda4671e0d2cc2a8993055dcc9ff
The 'inspect.trace()' function is expected to be called within the
context of an exception handler. The 'from_exc_and_traceback' class
method of the 'nova.notification.objects.exception.ExceptionPayload'
class uses this to get information about a provided exception, however,
there are cases where this is called from outside of an exception
handler. In these cases, we see an 'IndexError' since we can't get the
last frame of a non-existent stacktrace. The solution to this is to
fallback to using the traceback embedded in the exception. This is a bit
lossy when decorators are involved but for all other cases this will
give us the same information. This also allows us to avoid passing a
traceback argument to the function since we have it to hand already.
Change-Id: I404ca316b1bf2a963106cd34e927934befbd9b12
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Closes-Bug: #1881455
This patch series now works for many VM operations with libvirt:
* Creation, deletion of VM instances.
* Pause/unpause
The following works but is a no-op:
* Lock/unlock
Hard reboots are taken up in a later patch in this series.
Soft reboots work for accelerators unless some unrelated failure
forces a hard reboot in the libvirt driver.
Suspend is not supported yet. It would fail with this error:
libvirtError: Requested operation is not valid:
domain has assigned non-USB host devices
Shelve is not supported yet.
Live migration is not intended to be supported with accelerators now.
Change-Id: Icb95890d8f16cad1f7dc18487a48def2f7c9aec2
Blueprint: nova-cyborg-interaction
This patch adds support for unshelving an offloaded server with qos ports.
To do that this patch:
* collects the port resource requests from neutron before the scheduler
is called to select the target of the unshelve.
* calculate the request group - provider mapping after the scheduler
selected the target host
* update the InstancePCIRequest to drive the pci_claim to allocate VFs
from the same PF as the bandwidth is allocated from by the scheduler
* update the binding profile of the qos ports to so that the allocation
key of the binding profile points to the RPs the port is allocated
from.
As this was the last move operation to be supported the compute service
version is bumped to indicate such support. This will be used in a later
patches to implement a global service level check in the API.
Note that unshelve does not have a re-schedule loop and all the RPC
changes was committed in Queens.
Two error cases needs special care by rolling back allocations before
putting the instance back to SHELVED_OFFLOADED state:
* if the IntancePCIRequest cannot be updated according to the new target
host of unshelve
* if updating port binding fails in neutron during unshelve
Change-Id: I678722b3cf295c89110967d5ad8c0c964df4cb42
blueprint: support-move-ops-with-qos-ports-ussuri
Inheritance of image properties from the image an instance was booted
from to an image created from that instance is governed by the
non_inheritable_image_properties configuration option. However, there
are some image properties (for example, those used for image signature
validation or to reference a cinder encryption key id) which it makes
no sense to inherit under any circumstances. Additionally,
misconfiguration of the non-inheritable properties can lead to data
loss under the circumstances described in Bug #1852106. So it would
be better if these properties were not subject to configuration.
The initial set of absolutely non-inheritable image properties
consists of those associated with cinder encryption keys and image
signature validation.
Change-Id: I4332b9c343b6c2b50226baa8f78396c2012dabd1
Closes-bug: #1852106
Strip out everything matching '(is|use)_neutron', except the tests for
nova-network code and two other places that these tests rely on. Along
the way, remove a whole load of apparently unnecessary mocking that
clearly wasn't caught when we switched over the bulk of testing to use
the neutron network driver.
Change-Id: Ifa9c5c468400261a5e1f66b72c575845173a4f8f
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
This adds a kwarg to wrap_instance_event to be used in the
EventReporter to allow the caller to tell EventReporter to
gracefully handle InstanceActionNotFound on __exit__.
This will be used by ComputeTaskManager.revert_snapshot_based_resize
which starts an action in the target cell DB but upon successful
exit of the RevertResizeTask the instance in the target cell DB
will be hard destroyed resulting in an InstanceActionNotFound
traceback which should be avoided.
Part of blueprint cross-cell-resize
Change-Id: Ie48a9c0a285f77e260f675fbe9282df9f02282b1
Since [1] the only thing still using this utility method
is some functional report client test code so this change
moves it to the test class that needs it.
[1] Ib62ac0b692eb92a2ed364ec9f486ded05def39ad
Change-Id: I016765112b4d7a811a855da5e503a8cb870afbbe
This refactors the meat from delete_image_on_error()
to nova.compute.utils so it can be re-used in other
places.
Change-Id: Idee91453289dd512eddfc4903ce00e4e20b76df8
When deleting an instance, if the network info is empty, we should
refresh the info because we can't be sure the copy of the cache we
have when we fetched the instance to delete is up-to-date, i.e. if
we're racing to delete the server while it's building and the
network info cache was updated in the database after we started the
delete operation and got the instance from the DB, then we could
fail to unplug VIFs.
Closes-Bug: #1830081
Change-Id: I99601773406c61f93002e2f7cbb248cf73cef0ab
Signed-off-by: Arnaud Morin <arnaud.morin@corp.ovh.com>
The deprecated 'default_flavor' option has been removed.
The following methods in nova/compute/flavors.py
have been removed because they are only used in unit tests.
* get_default_flavor
* get_flavor_by_name
Change-Id: If1e461da382f707be2b5ba89f74f77269f0909dd
There were a bunch of report client methods around updating inventory to
placement which were only being used in the non-update_provider_tree
code paths of the resource tracker's update routine. Those code paths
had already been retrofitted to produce a placement-shaped inventory
object.
update_from_provider_tree gives us another way to flush these inventory
changes.
This patch simply takes the inventory object produced by the
get_inventory() and update_compute_node() code paths and updates the
provider tree object in the same fashion as update_provider_tree does.
So now all three code paths can commonly invoke
update_from_provider_tree.
And we can get rid of a ton of redundant code in the report client.
This includes the former incarnation of set_inventory_for_provider; so
we rename the artist formerly known as _set_inventory_for_provider to
match its brethren, set_traits_for_provider and
set_aggregates_for_provider.
Change-Id: I1a305847f0310c8d4babd5a625e4cc7bffe5b086
The volume.usage notification has been transformed to
the versioned notification framework.
Change-Id: Ica45a95d26b602f9a149d42516baf4b84fc01cec
Implements: bp versioned-notification-transformation-stein
This patch removes the host parameter of the
notify_about_instance_delete() call as it is always filled with
CONF.host value.
Change-Id: Iff3b605b9410d5a097b53f532870df65780bc1e4
Implements: bp versioned-notification-transformation-stein
This patch moves soft_delete notification sending to the
notify_about_instance_delete context manager to unify the
delete notification sending code practice.
Implements: bp versioned-notification-transformation-stein
Change-Id: Ib6f95c22ffd3ea235b60db4da32094d49c2efa2a
The Iddbe50ce0ad3c14562df800bbc09ec5a7e840485 patch only considered
instance delete in the happy case when the instance is scheduled to a
compute successfully and the compute is available when the delete
action is executed. If the instance is never scheduled to a compute or
the compute is not available when the instance is deleted legacy delete
notifications are emitted from different places, compute.api instead of
compute.manager. The original patch missed these places.
There will be subsequent patch(es) handling the same edge cases
for soft_delete and force_delete.
Change-Id: If0693eab2ed31b5fbfe6cbafa5d67b69c2ed8442
Implements: bp versioned-notification-transformation-stein
Change I9c2111f7377df65c1fc3c72323f85483b3295989 sets the
RequestSpec.is_bfv flag for newly created instances so
that scheduling (using placement) does not allocate DISK_GB
resources for the Flavor's root_gb when booting from volume.
RequestSpecs for old instances created before that change will
not have the is_bfv field set, so this change adds a check for
that in the various move operations (evacuate, unshelve, cold
migrate and live migrate) and sets the RequestSpec.is_bfv flag
accordingly.
The related functional test is updated for the legacy cold
migrate and heal scenario.
Change-Id: I8e529ad4d707b2ad012328993892db83ce464c4b
Closes-Bug: #1469179
In this patch, the ServerGroupPayload is updated to include
the new ``policy`` field; the ``policies`` field is deprecated
for removal but still put into the notification payload for
backward compatibility.
Related to blueprint complex-anti-affinity-policies
Change-Id: Ie739ee8dec4685cd70e735ff83f7f30bc7e95a57
This patch adds full traceback to ExceptionPayload in versioned
notifications.
The instance fault field and instance-action REST API has already
provide the traceback to the admin users (controlable through policy)
and the notifications are also admin only things as they are emitted
to the message bus by default. So it is assumed that security is not
a bigger concern for the notification than for the REST API.
On the ML [1] post there was no objection to add new string field to the
ExceptionPayload that will hold the serialized traceback object.
[1] http://lists.openstack.org/pipermail/openstack-dev/2018-March/128105.html
Implements: blueprint add-full-traceback-to-error-notifications
Change-Id: Id587967ea4f9980c292492e2f659bf55fb037b28
This patch contains the instance.exists related notification
transformation. The functional test for instance_usage_audit
periodic task case is added in a subsequent patch.
Change-Id: I1a0afa0e8740c229db77c18b932e316196880de5
Implements: bp versioned-notification-transformation-rocky
The following methods in nova/compute/utils.py
have not been used since
I9269ffa2b80e48db96c622d0dc0817738854f602.
* reverse_upsize_quota_delta
* downsize_quota_delta
In addition, remove an unused argument of
the 'upsize_quota_delta' method in the file.
Change-Id: Ia8eed927e3946ee8d7c90f6aead2883728772aca
The system_metadata argument to info_from_instance() was not used
so it's removed in this change, along with all callers of that
method, which goes quite a ways.
Also, the docstring for the system_metadata argument to
notify_usage_exists() is updated since it is passed in one
specific place: rebuild with a new image. In that case, nova-api
saves off the original instance system_metadata before resetting
the system_metadata based on the new image to rebuild, which is
then passed down through nova-conductor and nova-compute where
it eventually gets used to override "image_meta" in the payload
created in info_from_instance(). It's weird, yes, and essentially
means that for legacy versioned notifications, the instance payload
will always contain the image_meta for the instance before it's
rebuilt with the new image, which is something we don't do for
versioned notifications.
Test test_local_delete_without_info_cache is removed since it's
(1) weird in that it is doing mox-like stuff in a mock-based
test and (2) the code it was meant to test from change
Ie0bba032615d3da06cdd95b221beb37a9b8a377d no longer exists.
Change-Id: Ia1820334dcaceca9d7fa874dd7c553fa1c5befec
Closes-Bug: #1764390
The instance action event is recorded in API/Conductor/Compute using
EventReporter. We already had self.host in API/Conductor/Compute
(where the EventReporter would be used) in change
Idf57fb5fbc611abb83943bd7e36d3cebf03b3977.
In this patch, the self.host param would be passed in EventReporter,
and then the host would be recorded in instance action event.
Part of blueprint: add-host-to-instance-action-events
Change-Id: I4436f5c1cc819c55ca9186bda0362bb74555c95a
The utility method will be needed in the compute manager in
an upcoming change, so this moves it from a private method
in the compute.api to compute.utils, makes it public and adjusts
tests and such.
As a result, may_have_ports_or_volumes also has to move.
Change-Id: Iedd6be5ef473dcb4f2c465709f3e070ff529f456
There are a bunch of files that define CONF but don't ever reference
it. This cleans that up, removing unnecessary imports.
In order to get flake8 to pass I also had to remove some (unrelated)
unused imports from touched files.
Change-Id: Ib96ebeeb5e4bfedb20afb02c56da1521df9fac4f
The servergroup.addmember notification has been transformed
to the versioned notification framework.
Change-Id: I5291e453ba06f0f26c4928e4f7482c40e725a781
Implements: bp versioned-notification-transformation-rocky
This moves us to version 5.0, with a proxy for 4.x for Queens. This includes
a lot of churn due to a lot of kwarg cruft. This is all just refactor, as it
doesn't include the client side change. So if this passes, we know we're
compatible, and subsequent patches will add client-side support.
Change-Id: I539b0a439030be57fd03012526ff0236f2b88a4e
There are methods for versioned notifications
in nova/compute/utils.py.
Remove unnecessary arguments in them.
* notify_about_instance_create
Versioned notifications when creating a server are emitted
in the compute manager only.
The 'source' argument is not necessary.
So remove it.
* notify_about_volume_attach_detach
Versioned notifications when attaching/detaching a volume
to/from a server are emitted in the compute manager only.
The 'source' argument is not necessary.
So remove it.
* notify_about_instance_rescue_action
This method is only utilized in the 'rescue' action.
The 'action' argument is not necessary.
Versioned notifications for the rescue action are emitted
in the compute manager only.
The 'source' argument is not necessary.
So remove them.
* notify_about_volume_swap
This method is only utilized in the 'volume_swap' action.
The 'action' argument is not necessary.
So remove it.
Change-Id: I678d4b634f74051eb069c792ec32b2fc317b73e8
The instance.resize_prep.start and instance.resize_prep.end
notifications are transformed to the versioned framework.
Co-Authored-By: Takashi Natsume <natsume.takashi@lab.ntt.co.jp>
Change-Id: If33d092e8688ed4d4864b2f7130104ab85ce1ea4
Implements: bp versioned-notification-transformation-queens
The rescue (instance.rescue.start and instance.rescue.end) and unrescue
(instance.unrescue.start and instance.unrescue.end) notifications are
transformed to the versioned framework.
This patch also fixes the power state of the server
when unrescuing it with the fake compute driver.
Co-Authored-By: Takashi Natsume <natsume.takashi@lab.ntt.co.jp>
Change-Id: Ib1d03c6d693e3b04886c638c956e35809fed8fc2
Implements: bp versioned-notification-transformation-queens
Closes-Bug: #1742133
Change the Adapter loading for glance to use the auth from the user
context instead of exposing and requiring it in the conf. With this
change, it is possible to leave the [glance] conf section empty and
still be able to discover the image API endpoint from the service
catalog.
Note that, when we do this, we often end up with the user auth being a
_ContextAuthPlugin, which doesn't conform to the characteristics of
keystoneauth1.identity.base.BaseIdentityPlugin as augmented in
keystoneauth1 3.1.0. This requires a series of workarounds until bug
1709118 is fixed. These, along with workarounds for bugs 1707993 and
1707995, are subsumed with this change set in a (hopefully temporary)
helper method nova.utils.get_endpoint.
This lays the foundation for other services that should use user
context for authentication - those via which Nova is acting on behalf
of the user, i.e. cinder, keystone, and (sometimes) neutron[1].
(Services such as placement and ironic (and sometimes neutron) should
continue to use admin auth context loaded from the conf.)
[1] bb4faf40df/nova/network/neutronv2/api.py (L149-L160)
Co-Authored-By: Eric Fried <efried@us.ibm.com>
Partial-Implements: bp use-ksa-adapter-for-endpoints
Change-Id: I4e755b9c66ec8bc3af0393e81cffd91c56064717
Stephen Finucane