If an image is encrypted, we will need to retrieve the passphrase from
the key manager service in order to create an instance from it.
This adds an image property to store the secret UUID that belongs to
the image. It will only be used to decrypt the image and will not be
used to encrypt or decrypt any other image. Nova will create a new
secret for each disk image it creates, including snapshots.
Related to blueprint ephemeral-storage-encryption
Change-Id: I01eef6adc2c8feb64e86b33392b8b4b483041e27
This change adds a new hw:locked_memory extra spec and hw_locked_memory
image property to contol preventing guest memory from swapping.
This change adds docs and extend the flavor
validators for the new extra spec.
Also add new image property.
Blueprint: libvirt-viommu-device
Change-Id: Id3779594f0078a5045031aded2ed68ee4301abbd
arch in libvirt driver support
This is split 1 of 3 for the architecture emulation feature.
This adds the 'hw_emulation_architecture' property to the image meta
properties, allowing for operator to define whether they will use
emulation or not.
This adds the capability as a feature to ensure no impact to normal
operations or functionality.
Account for object versioning has been added to raise exceptions and
handle proper
Implements: blueprint pick-guest-arch-based-on-host-arch-in-libvirt-driver
Signed-off-by: Jonathan Race <jrace@augusta.edu>
Change-Id: If4f598c0d3f9e64617beb54450faa04e7d20dd20
Finish up removing these entries from the versioned instance
notifications. They're useless since we dropped support for the XenAPI
virt driver. The underlying model is retained for now: that will be
handled separately.
Change-Id: I774c50fca99bc655ca5010e3b9d8247b739293b3
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
The 'bochs' display device is a recommended[1] safe option for UEFI
guests. This is supported in libvirt from version 5.6.0; Nova's current
MIN_LIBVIRT_VERSION (6.0.0) satisfies this requirement.
Partially quoting the "bochs display device" section from a post written
by a QEMU graphics maintainer[1]:
[...] Main advantage over standard VGA is that this device is alot
simpler. The code size and complexity needed to emulate this device
is an order of magnitude smaller, resulting in a reduced attack
surface. Another nice feature is that you can place this device in a
PCI Express slot.
For UEFI guests it is safe to use the bochs display device instead
of the standard VGA device. The firmware will setup a linear
framebuffer as GOP anyway and never use any legacy VGA features.
For BIOS guests this device might be useable as well, depending on
whenever they depend on direct VGA hardware access or not. There is
a vgabios which supports text rendering on a linear framebuffer, so
software which uses the vgabios services for text output will
continue to work. Linux bootloaders typically fall into this
category. The linux text mode console (vgacon) uses direct hardware
access and does not work. The framebuffer console (fbcon running on
vesafb or bochs-drm) works.
[1] https://www.kraxel.org/blog/2019/09/display-devices-in-qemu/
Implements: blueprint add-bochs-display-device
Change-Id: Id194028f5d2cbaac1c5e699b71fd9751f87f5da5
Signed-off-by: Kashyap Chamarthy <kchamart@redhat.com>
This continues on from I81fec10535034f3a81d46713a6eda813f90561cf and
removes all other references to 'instance_type' where it's possible to
do so. The only things left are DB columns, o.vo fields, some
unversioned objects, and RPC API methods. If we want to remove these, we
can but it's a lot more work.
Change-Id: I264d6df1809d7283415e69a66a9153829b8df537
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
The USB-based tablet is often the only USB device in an x86 instance,
while the USB-based keyboard is often the only such device in an AArch64
instance (x86 have PS2 keyboards and mice). Replacing these with
virtio-based devices can eliminate the need to have a USB host adapter
in the instance. Enable just that possibility by adding a new value
image metadata property, 'hw_input_bus'. This allows us to specify not
only virtio-based pointer and keyboard input devices but also USB
equivalents.
Note that this also fixes one instance of a particular class of bugs,
whereby we have checks for *guest* architecture-specific behavior that
are being toggled based on the *host* architecture. In this instance,
we were attempting to add a keyboard device on AArch64 guests since they
don't have one by default, but we were determining the architecture by
looking at the CPU architecture reported in the host capabilities. By
replacing this check of the host capabilities with a call to the
'nova.virt.libvirt.utils.get_arch' helper, we correctly handle requests
to create non-host architecture guests via the 'hw_architecture' image
metadata property. There are many other instances of this bug and those
can be resolved separately.
Change-Id: If9f3ede3e8449f9a6c8d1da927974c0a73923d51
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
This patch adds the 'socket' value to the allowed PCI NUMA affinity
policies, both to the 'hw:pci_numa_affinity_policy' flavor extra spec,
and the 'hw_pci_numa_affinity_policy' image property.
For now the new value is a no-op and remains undocumented. It will be
wired-in in a subsequent patch.
Implements: blueprint pci-socket-affinity
Change-Id: I0680d4e21f3e317ac702b55afef4c87e8acbfc3a
This one is a little more involved because there are persistent objects
and versioned notifications to worry about, neither of which we can
remove right now.
Change-Id: Ic7c330ee1cccdc44a2a555c16cb6090eecbf6ce1
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Add support for the 'hw:tpm_version' and 'hw:tpm_model' flavor extra
specs along with the equivalent image metadata properties. These are
picked up by the scheduler and transformed into trait requests. This is
effectively a no-op for now since we don't yet have a driver that
reports these traits.
Part of blueprint add-emulated-virtual-tpm
Change-Id: I8645c31b4ecb18afea592b2a5b360b0165626009
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Introduce a 'mixed' instance CPU allocation policy and
will be worked with upcoming patches, for purpose of
creating an instance combined shared CPUs with dedicated
or realtime CPUs.
In an instance mixed with different type of CPUs, the shared CPU
shared CPU time slots with other instances, and also might be a
CPU with less or un-guaranteed hardware resources, which implies
to have no guarantee for the behavior of the workload running on
it. If we call the shared CPU as 'low priority' CPU, then the
realtime or dedicated CPU could be called as 'high priority' CPU,
user could assign more hardware CPU resources or place some
guaranteed resource to it to let the workload to entail high
performance or stable service quality.
Based on https://review.opendev.org/714704
Part of blueprint use-pcpu-and-vcpu-in-one-instance
Change-Id: I99cfee14bb105a8792651129426c0c5a3749796d
Signed-off-by: Wang Huaqiang <huaqiang.wang@intel.com>
Update the versioned object notification for 'InstanceNUMACell'
object to reflect the change of a new 'pcpuset' field.
Part of blueprint use-pcpu-and-vcpu-in-one-instance
Change-Id: Ifeb21b9eff828952f67002172c8500f738e9984c
Signed-off-by: Wang Huaqiang <huaqiang.wang@intel.com>
The 'nova.block_device' module is essentially a catchall utils-like
module for all things BDM. The 'get_bdm_image_metadata' module, and
closely related 'get_image_metadata_from_volume' both fall into the
category of functions that belong here so move them. This allows us to
clean up tests and, crucially, avoid a circular reference seen when we
want to use proper type hints in the 'nova.virt.driver' module.
nova.context imports...
nova.utils, which imports...
nova.block_device, which imports...
nova.virt.driver, which tries to import...
nova.context, causing a circular dependency
Change-Id: I48177d6e93f2ff132d26b53cd682fd24a43a4b31
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
The 'inspect.trace()' function is expected to be called within the
context of an exception handler. The 'from_exc_and_traceback' class
method of the 'nova.notification.objects.exception.ExceptionPayload'
class uses this to get information about a provided exception, however,
there are cases where this is called from outside of an exception
handler. In these cases, we see an 'IndexError' since we can't get the
last frame of a non-existent stacktrace. The solution to this is to
fallback to using the traceback embedded in the exception. This is a bit
lossy when decorators are involved but for all other cases this will
give us the same information. This also allows us to avoid passing a
traceback argument to the function since we have it to hand already.
Change-Id: I404ca316b1bf2a963106cd34e927934befbd9b12
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Closes-Bug: #1881455
This doesn't exist for 'nova.volume' and no longer exists for
'nova.network'. There's only one image backend we support, so do like
we've done elsewhere and just use 'nova.image.glance'.
Change-Id: I7ca7d8a92dfbc7c8d0ee2f9e660eabaa7e220e2a
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
At some point in the past, there was only nova-network and its code
could be found in 'nova.network'. Neutron was added and eventually found
itself (mostly!) in the 'nova.network.neutronv2' submodule. With
nova-network now gone, we can remove one layer of indirection and move
the code from 'nova.network.neutronv2' back up to 'nova.network',
mirroring what we did with the old nova-volume code way back in 2012
[1]. To ensure people don't get nova-network and 'nova.network'
confused, 'neutron' is retained in filenames.
[1] https://review.opendev.org/#/c/14731/
Change-Id: I329f0fd589a4b2e0426485f09f6782f94275cc07
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
ImageMetaPropsPayload (for notifications) was supposed to contain
exactly the same fields as ImageMetaProps itself; and the SCHEMA member
was those same keys in a trivially different shape. And we were having
trouble remembering to keep these two objects in sync before. So this
commit eliminates the repetition and just generates
ImageMetaPropsPayload's `fields` and `SCHEMA` from
ImageMetaProps.fields.
Change-Id: I7dc38f7f5a62098fb0c366736f27e23bd7b7ee86
This change adds missing fields to the image metadata
notification object and adds a unit tests to assert
that the notification object and nova object stay in sync.
This change also adds unit tests to assert the notification
schema and fields are in sync and that the schema is
valid.
Change-Id: I36ea5d5e677ab3e6c88223b20f5377e9471c55db
Closes-Bug: #1856100
This addresses bug #1795920 by adding support for
defining a pci numa affinity policy via the flavor
extra specs or image metadata properties enabling
the policies to be applied to neutron sriov port
including hardware offloaded ovs.
Closes-Bug: #1795920
Related-Bug: #1805891
Implements: blueprint vm-scoped-sriov-numa-affinity
Change-Id: Ibd62b24c2bd2dd208d0f804378d4e4f2bbfdaed6
This adds AggregateCacheNotification, related payload, and code in
conductor to emit this per-compute with progress information. This
also adds a "progress" phase to NotificationPhase, which allows for
start..progress..progress..end information for a single operation
(cache_images in this case).
Related to blueprint image-precache-support
Change-Id: I69ae26d4caf4b56ab2c4864455bfe9b5b736dbf3
This adds the functional notification sample test for the
aggregate.cache_images.start and aggregate.cache_images.end
versioned notifications.
I also added a comment to the docs builder code since it took
me a bit to figure out how to get the notification sample
linked into the docs, and for whatever reason figured that out
by looking through code rather than our nicely detailed docs
that already explain it.
Part of blueprint image-precache-support
Change-Id: I0869979a1b8a0966f0e7b49e5a5984f76d7d67cd
This adds the bulk of the image pre-caching logic to the conductor
task manager. It takes an aggregate and list of image ids from the
API service and handles the process of calling to the relevant compute
nodes to initiate the image downloads, honoring the (new) config knob
for overall task parallelism.
Related to blueprint image-precache-support
Change-Id: Id7c0ab7ae0586d49d88ff2afae149e25e59a3489
The api documentation is now published on docs.openstack.org instead
of developer.openstack.org. Update all links that are changed to the
new location.
Note that Neutron publishes to api-ref/network, not networking anymore.
Note that redirects will be set up as well but let's point now to the
new location.
For details, see:
http://lists.openstack.org/pipermail/openstack-discuss/2019-July/007828.html
Change-Id: Id2cf3aa252df6db46575b5988e4937ecfc6792bb
- This change extends the VideoModel field object to allow 3 new values
(virtio, gop, none)
- This change makes the libvirt driver use ALL tuple from the
nova.fields.VideoModel object instead of declaring a second
tuple inline for validation.
- This change allows the virtio video model to now be used
for all architectures when explicitly requested via the
hw_video_model image metadata property
- This change introduces unit tests and a release note
for the new capablities.
Change-Id: I2830ccfc81cfa9654cfeac7ad5effc294f523552
Implements: blueprint libvirt-video-device-models
Thankfully the bulk of this is neatly organized in a single directory
and can be removed, now that the bulk of the references to it have been
removed. The only complicated area is the tests, though effort has been
taken to minimise the diff here wherever possible.
Part of blueprint remove-cells-v1
Change-Id: Ib0e0b708c46e4330e51f8f8fdfbb02d45aaf0f44
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
This patch adds a new parameter ``locked_reason`` to
``POST /servers/{server_id}/action`` request where the
action is lock. It enables the user to specify a reason when locking
a server.
The locked_reason will be exposed through ``GET servers/{server_id}``,
``GET /servers/detail``, ``POST /servers/{server_id}/action`` where
the action is rebuild and ``PUT servers/{server_id}`` requests' responses.
The InstanceActionNotification will emit the locked_reason
along with the other instance details. This patch hence changes the
payload object to include the "locked_reason" field.
Note that "locked" will be allowed as a valid filtering/sorting parameter
for ``GET /servers/detail`` and ``GET /servers`` from this new microversion.
Implements blueprint add-locked-reason
Change-Id: I46edd595e7417c584106487123774a73c6dbe65e
Change I019e88fabd1d386c0d6395a7b1969315873485fd in Stein, which
is not yet officially released, exposes the unencrypted
database_connection URL and MQ transport_url to a CellMapping in
the select_destinations versioned notification CellMappingPayload.
While notifications are not meant to be consumed by end users of
the cloud but only internal services of the deployment, it still
seems like a bad idea to give the keys to the nova cell DB and MQ
to an external-to-nova service like ceilometer.
This change removes the fields from the CellMappingPayload and
bumps the major version to 2.0 to signal the change to consumers,
although I don't expect anything is consuming this yet but we should
follow standard versioning procedure anyway.
Note that notification consumers do not request a specific payload
version nor do they get a schema to perform their own backporting,
they just get what they get, so after this there should be no worry
about needing to support the 1.0 format for this payload.
Change-Id: Ib5edea32d15db01000e6730aebceaf119daf8c5c
Closes-Bug: #1823104
Transform the scheduler.select_destinations.start and
scheduler.select_destinations.end notifications to the
versioned notification framework.
Change-Id: I019e88fabd1d386c0d6395a7b1969315873485fd
Implements: bp versioned-notification-transformation-stein
The following notifications have been transformed to
the versioned notification framework.
* compute_task.build_instances
* compute_task.migrate_server
* compute_task.rebuild_server
Co-Authored-By: Takashi Natsume <natsume.takashi@lab.ntt.co.jp>
Change-Id: Ibfb0a6db5920d921c4fc7cabf3f4d2838ea7f421
Implements: bp versioned-notification-transformation-stein
The volume.usage notification has been transformed to
the versioned notification framework.
Change-Id: Ica45a95d26b602f9a149d42516baf4b84fc01cec
Implements: bp versioned-notification-transformation-stein
The libvirt.error notification has been transformed to
the versioned notification framework.
Co-Authored-By: Takashi Natsume <natsume.takashi@lab.ntt.co.jp>
Change-Id: I7d2287ce06d77c0afdef0ea8bdfb70f6c52d3c50
Implements: bp versioned-notification-transformation-stein
There is a DEBUG log coming from the notification sending code path
that tries to populate the notification payload from a nova OVO and
encounters an unset / non-lazy loadable field in the source OVO.
As there are fields like MonitorMetric.numa_membw_values that are
notorious unset in many circumstances but used in the MetricPayload
we got excessive DEBUG logging from this code path.
This patch removes the offending logging statement as soo far we failed
to found cases based on this log where anything else we could do than
what the code anyhow does, defaulting the payload field to None.
Change-Id: I680e1d9860d839a76e229c4891933872388eb050
Closes-Bug: #1788634
The metrics.update notification has been transformed to
the versioned notification framework.
Change-Id: I2f6aba0c032baf181b103d2a16b8e768815170c9
Implements: bp versioned-notification-transformation-rocky
The legacy instance.exists notification includes the full url of the glance
image of the given instance. But the versioned notification only includes
the image uuid. Generating the full url can be a costly operation as it
needs to talk to Keystone.
So this patch makes sure that generate_image_url only called when the
generated information will be used.
Change-Id: I78c2a34b3d03438457cc968cd0a38b8131e4f6e6
Closes-Bug: #1757407
The instance.live_migration_force_complete.start and
instance.live_migration_force_complete.end notifications have been
transformed to the versioned notification framework.
Co-Authored-By: Takashi Natsume <natsume.takashi@lab.ntt.co.jp>
Change-Id: I2e9a474457a7597a9febc111b13a67ddcf45f29e
Implements: bp versioned-notification-transformation-rocky
The aggregate.update_prop.start and aggregate.update_prop.end
notifications has been transformed to the versioned notification
framework.
Co-Authored-By: Takashi Natsume <natsume.takashi@lab.ntt.co.jp>
Change-Id: I37b19573b6d0e1131c446fcec361f01fa2560f82
Implements: bp versioned-notification-transformation-rocky
In this patch, the ServerGroupPayload is updated to include
the new ``policy`` field; the ``policies`` field is deprecated
for removal but still put into the notification payload for
backward compatibility.
Related to blueprint complex-anti-affinity-policies
Change-Id: Ie739ee8dec4685cd70e735ff83f7f30bc7e95a57
melanie witt