RDP console was only for HyperV driver so removing the
API. As API url stay same (because same used for other
console types API), RDP console API will return 400.
Cleaning up the related config options as well as moving its
API ref to obsolete seciton.
Keeping RPC method to avoid error when old controller is used
with new compute. It can be removed in next RPC version bump.
Change-Id: I8f5755009da4af0d12bda096d7a8e85fd41e1a8c
This chnage adds the pre-commit config and
tox targets to run codespell both indepenetly
and via the pep8 target.
This change correct all the final typos in the
codebase as detected by codespell.
Change-Id: Ic4fb5b3a5559bc3c43aca0a39edc0885da58eaa2
Based on the Libvirt and QEMU versions, two traits,
COMPUTE_ADDRESS_SPACE_PASSTHROUGH and COMPUTE_ADDRESS_SPACE_EMULATED,
are controlled. Since the two are supported from the same version on
the Libvirt and QEMU, Nova handles them in the same way.
Blueprint: libvirt-maxphysaddr-support
Depends-On: https://review.opendev.org/c/openstack/os-traits/+/871226
Signed-off-by: Nobuhiro MIKI <nmiki@yahoo-corp.jp>
Change-Id: If6c7169b7b8f43ad15a8992831824fb546e85aab
This adds a number of functional test cases for the stable-compute-uuid
error cases. Specifically around checks and aborted startups to make
sure we're catching what we expect, and failing in the appropriate
ways.
Related to blueprint stable-compute-uuid
Change-Id: I8bcb93a6887ed06dbd4b7c28c93a20a3705a6077
When we are loading our ComputeNode objects by UUID according to what
the virt driver reported, we can sanity check the DB records against
the virt driver's hostname. This covers the case where our CONF.host
has not changed but the hostname reported by the virt driver has,
assuming we can find the ComputeNode object(s) that match our
persistent node uuid.
Related to blueprint stable-compute-uuid
Change-Id: I41635210d7d6f46b437b06d2570a26a80ed8676a
This makes the resource tracker look up and create ComputeNode objects
by uuid instead of nodename. For drivers like ironic that already
provide 'uuid' in the resources dict, we can use that. For those
that do not, we force the uuid to be the locally-persisted node
uuid, and use that to find/create the ComputeNode object.
A (happy) side-effect of this is that if we find a deleted compute
node object that matches that of our hypervisor, we undelete it
instead of re-creating one with a new uuid, which may clash with our
old one. This means we remove some of the special-casing of ironic
rebalance, although the tests for that still largely stay the same.
Change-Id: I6a582a38c302fd1554a49abc38cfeda7c324d911
This adds a get_available_node_uuids() method to the virt driver
interface. This aims to eventually replace the nodename-based
interface, but currently provides an implementation that will work
for most drivers. Any driver that does not override this method
will get the locally-persistent UUID from nova.virt.node.
Ironic obviously needs to override this (which is easy), as well as
the fake driver because it supports multiple nodes for testing. The
libvirt driver overrides it only because we test multiple libvirt
driver instances on a single host and we need each instantiation
of it to "capture" the UUID we have mocked out at the time it is
started.
Change-Id: Ibe14d2b223c737d82c217a74bc94e41603271a9d
A later patch in the pci-device-tracking-in-placement work
will extend the existing [pci]passthrough_whitelist config syntax.
So we take the opportunity here to deprecate the old non inclusive
passthrough_whitelist name and introduce a better one.
All the usage of CONF.pci.passthrough_whitelist is now changed over to
the new device_spec config. Also the in tree documentation is updated
accordinly.
However the nova code still has a bunch of references to the
"whitelist" terminology. That will be handled in subsequent patches.
blueprint: pci-device-tracking-in-placement
Change-Id: I843032e113642416114f169069eebf6a56ed78dd
Today Nova updates the mac_address of a direct-physical port to reflect
the MAC address of the physical device the port is bound to. But this
can only be done before the port is bound. However during migration Nova
does not update the MAC when the port is bound to a different physical
device on the destination host.
This patch extends the libvirt virt driver to provide the MAC address of
the PF in the pci_info returned to the resource tracker. This
information will be then persisted in the extra_info field of the
PciDevice object.
Then the port update logic during migration, resize, live
migration, evacuation and unshelve is also extended to record the MAC of
physical device in the port binding profile according to the device on
the destination host.
The related neutron change Ib0638f5db69cb92daf6932890cb89e83cf84f295
uses this info from the binding profile to update the mac_address field
of the port when the binding is activated.
Closes-Bug: #1942329
Change-Id: Iad5e70b43a65c076134e1874cb8e75d1ba214fde
In order to support remote-managed ports the following is needed:
* Nova compute driver needs to support this feature;
* For the Libvirt compute driver, a given host needs to have the right
version of Libvirt - the one which supports PCI VPD (7.9.0
https://libvirt.org/news.html#v7-9-0-2021-11-01).
Therefore, this change introduces a new capability to track driver
support for remote-managed ports.
Change-Id: I7ea96fd85d2607e0af0f6918b0b45c58e8bec058
Retrieve PF mac and VF logical number at runtime for
a given VF PCI address and include them in port updates to Neutron.
Implements: blueprint integration-with-off-path-network-backends
Change-Id: I83a128a260acdd8bf78fede566af6881b8b82a9c
autopep8 is a code formating tool that makes python code pep8
compliant without changing everything. Unlike black it will
not radically change all code and the primary change to the
existing codebase is adding a new line after class level doc strings.
This change adds a new tox autopep8 env to manually run it on your
code before you submit a patch, it also adds autopep8 to pre-commit
so if you use pre-commit it will do it for you automatically.
This change runs autopep8 in diff mode with --exit-code in the pep8
tox env so it will fail if autopep8 would modify your code if run
in in-place mode. This allows use to gate on autopep8 not modifying
patches that are submited. This will ensure authorship of patches is
maintianed.
The intent of this change is to save the large amount of time we spend
on ensuring style guidlines are followed automatically to make it
simpler for both new and old contibutors to work on nova and save
time and effort for all involved.
Change-Id: Idd618d634cc70ae8d58fab32f322e75bfabefb9d
The legacy block_device_info format has not been used for a while in the
virt drivers and the Fake virt drivers shouldn't be using it anymore.
Some compute tests have been updated as they were previously asserting
the legacy format incorrectly. TODOs are also left to remove support for
the legacy format in a follow up change.
Change-Id: I066fb5617bae944a03273049117fc2b5bf329283
This change adds a simple optional kwarg to the virt driver signature of
destroy and cleanup to allow for callers to control when secrets should
be removed.
Change-Id: I856268b371f7ba712b02189db3c927cd762a4dc3
This one is a little more involved because there are persistent objects
and versioned notifications to worry about, neither of which we can
remove right now.
Change-Id: Ic7c330ee1cccdc44a2a555c16cb6090eecbf6ce1
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
This is no longer used by any in-tree virt drivers and can be removed.
The equivalent RPC API now always raises 'NotImplementedError', which
was the behavior with virt drivers that didn't support the feature.
Change-Id: Iab881ef4f52eff4815e781f10204497968f8a06b
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
This series implements the referenced blueprint to allow for specifying
custom resource provider traits and inventories via yaml config files.
This fourth commit adds the config option, release notes, documentation,
functional tests, and calls to the previously implemented functions in
order to load provider config files and merge them to the provider tree.
Change-Id: I59c5758c570acccb629f7010d3104e00d79976e4
Blueprint: provider-config-file
In a future change, we'll want access to this so that it's possible to
retrieve vTPM data during the unrescue operation. While we're here, it
seems nothing is using the 'network_info' argument anymore, presumably
since the demise of nova-network, and this can and should be dropped.
Resolve both issues in one go, adding the 'context' parameter, dropping
the 'network_info' one, and updating the various callers and tests for
same.
Maintainers of out-of-tree drivers have been notified of these changes
[1].
[1] http://lists.openstack.org/pipermail/openstack-discuss/2020-July/015824.html
Part of blueprint add-emulated-virtual-tpm
Change-Id: Id5e4b0f26d5a2a93db6a7d96555a2cff29d9a2cf
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
The inject_file call was removed from the compute RPC at 3.18.
As we have RCP 5.x now it is safe to remove the manager method
handing that call. This removes the last user of the inject_file virt
diver method so that is deleted too.
Change-Id: If257fa301e8c12bb0c5e4d396ccc2c7277b25233
Some features like vGPUs require to have allocations to be checked locally
by the virt driver but resize (and cold migrate) was not passing them.
This change just gives the allocations to the target node when the resize
is done by adding the allocations to the finish_migration() interface.
Related-Bug: #1778563
Change-Id: Ifd76289964d513e9544544e5787f5f8999589475
Building on Ic2ad1468d31b7707b7f8f2b845a9cf47d9d076d5, when requested
this microversion will allow boot from volume requests to proceed when
the COMPUTE_RESCUE_BFV trait is also reported by the compute the
instance currently resides on.
Implements: blueprint virt-bfv-instance-rescue
Change-Id: I3242fec1547693078cf36c3637116f8c41f1d0bc
This change adds block_device_info as an argument to the rescue method
signature for all virt divers currently implementing instance rescue.
This will allow future changes to provide all disks during a stable
device rescue attempt.
Change-Id: I9c2d9013d741774e521021913ec04f37b310e9d4
. Do not delete accelerator requests in stop code paths.
. In the start code path, get the list of accelerator requests from
Cyborg in the compute manager 'power_on'.
. Pass accel_info (said list) to the virt driver power_on.
. In libvirt driver, pass that accel_info to driver power_on.
Change-Id: I8c94504b87aa4450d163fe2b33f6aa0eb5dae5ff
Blueprint: nova-cyborg-interaction
Update the signature of the spawn() API for each virt driver
to include accel_info, which is a list of accelerator requests.
Change-Id: I4aac66c125a162bf35991a7d0c2638c7475ec0e7
Blueprint: nova-cyborg-interaction
Required by an upcoming bugfix to the Libvirt driver's implementation of
extend_volume that will require external authenticated calls to Cinder
and Barbican.
Change-Id: I0ef84dd0a6c3f1788caf7d3a8e3837203f6d5429
This uses the COMPUTE_SAME_HOST_COLD_MIGRATE trait in the API during a
cold migration to filter out hosts that cannot support same-host cold
migration, which is all of them except for the hosts using the vCenter
driver.
For any nodes that do not report the trait, we won't know if they don't
because they don't support it or if they are not new enough to report
it, so the API has a service version check and will fallback to old
behavior using the config if the node is old. That compat code can be
removed in the next release.
As a result of this the FakeDriver capabilities are updated so the
FakeDriver no longer supports same-host cold migration and a new fake
driver is added to support that scenario for any tests that need it.
Change-Id: I7a4b951f3ab324c666ab924e6003d24cc8e539f5
Closes-Bug: #1748697
Related-Bug: #1811235
The failed case covers the situation when the IntancePCIRequest cannot be
updated with the PF device names on the target host due to incorrect
naming of the device RPs in placement.
The abort case covers when the API user cancels the running live
migration.
Change-Id: I1222fc21bde4158df1db70370c7f3bd319ec081f
blueprint: support-move-ops-with-qos-ports-ussuri
Firewall support is not needed with neutron, which supports both
security groups for per-port filtering and FWaaS for per-network
filtering. Remove both the generic firewalls and the hypervisor-specific
implementations.
This change focuses on removing the firewall-related API calls from the
various virt drivers. The firewall drivers themselves are removed
separately.
Change-Id: I5a9e5532c46a5f7064441ae644125d21efe5fda1
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
There was no code in FakeDriver that updated the internal
dict `self.instances` when a FakeInstance was live migrated.
This commit fills this gap. As a result, a couple of versioned
notification samples get updated since we are now properly
tracking a live migrated instance on the destination host as
running vs pending power state.
Closes-Bug: 1426433
Change-Id: I9562e1bcbb18c7d543d5a6b36011fa28c13dfa79
This adds the finish_revert_snapshot_based_resize_at_source() method
to the compute service. This completes the cross-cell resize revert
on the source host by updating port bindings, re-connecting volumes
and spawning the guest and waiting for the network-vif-plugged
event from neutron.
If the resize started with a stopped instance, the revert ensures
the guest is powered off when it is spawned.
The instance record is updated with the old_flavor information,
allocations are reverted in placement, and the migration status in
the source cell database is updated to "reverted".
Notifications are not sent from this method like in the
finish_revert_resize() method because those will be sent from
conductor.
Note that this implementation tries to be a bit more graceful
about error handling than the old same-cell finish_revert_resize
method since there are a lot of moving parts and we want to try
to clean up as much as possible.
Part of blueprint cross-cell-resize
Change-Id: I7e7afddbd9e0f57dfb1175a0bb2b54f2ed5500f2
This adds the logic to the API confirmResize
operation such that if the migration is a cross-cell
resize the API will RPC cast to conductor to
confirm the resize rather than directly to the source
compute service like a traditional resize. Conductor
will then orchestrate the confirm process between the
source and target cell.
Now that the API has confirmResize plumbed this change
builds on the cross-cell resize functional tests by
confirming the resize for the image-backed server test.
To make that extra fun, a volume is attached to the
server while it is in VERIFY_RESIZE status to assert it
remains attached to the instance in the target cell
after the resize is confirmed.
In addition, the FakeDriver.cleanup() method is updated
to guard against calling it before the guest is destroyed
from the "hypervisor". This is to make sure the cleanup()
method is called properly from
confirm_snapshot_based_resize_at_source() in the compute
service on the source host.
The _confirm_resize_on_deleting scenario will be
dealt with in a later change.
Part of blueprint cross-cell-resize
Change-Id: Ia5892e1d2cb7c7685e104466f83df7bb00b168c0
In some situations, we may enter into a race-safe wait context for an
external event and then determine within the context that the event
would have already fired or for some other reason waiting is not required.
In that case, this introduces a VirtAPI method to abort the wait for
specific events in a clean way.
This should be used sparingly and only in certain circumstances where
it is clear that the event has already occurred. For example, if the
action triggering the event was initiated outside of the wait context,
we can enter the wait context (establishing a waiting event to capture),
and then poll once for completion of that thing. If incomplete, we will
block and wait on context exit as usual. If the thing we are waiting for
has already completed, then we can avoid the long wait by calling the
early-exit helper and avoid what would ultimately be a timeout and
failure.
We are specifically adding this to address the cyborg scenario as follows:
Once we select a host in the conductor for an instance build, we can call
to cyborg to start the preparation (i.e. programming) of the device on the
destination host. This takes time, and by kicking it off early, we can
overlap that process with other work we have to do. Because that will send
an event upon completion, which will be routed to the compute, we will race
to start the build process on the compute and begin the race-safe event
wait in the compute manager. Instead of collapsing the window and waiting
synchronously, we can trigger it early, then start the wait in the compute
later, check for early completion *after* the wait has been set up, and then
exit the wait early if we determine that the event had already been sent.
This allows us to be async with cyborg across two services while still being
safe in the critical region where we need to setup the wait before we rely
on the event being sent.
Change-Id: I8e5a0683069b2f322cb059d6eb501d732d1bd851
This adds a functional test where the finish resize on the
dest compute fails triggering a rollback in conductor. The
test also hard reboots the server in the source cell to
recover it from ERROR to ACTIVE status.
Finally, the server is resized again without the fault
injected to make sure the target cell DB was properly
cleaned up on rollback from the initial resize failure.
Because prep_snapshot_based_resize_at_source will destroy
the guest from the source hypervisor, this change needs to
implement some reboot logic in the fake driver such that
if a hard reboot is being attempted on a guest that is not
being tracked, the fake driver will (re)spawn the guest
similar to how hard reboot works for the libvirt driver.
Part of blueprint cross-cell-resize
Change-Id: If6a8cf58944366917a729527cb3b87494de38fd0
This adds the beginning of functional tests for
cross-cell resize which get the instance to VERIFY_RESIZE
state and assert various aspects of the instance both in
the source cell and target cell.
The fake virt driver is updated to support creating a
server with device tags and also to have a host-specific
volume connector dict to aid in debugging test failures.
Confirm/revert resize flows will build on these tests.
Part of blueprint cross-cell-resize
Change-Id: I398aeb73da6bf09e177e5f1fdc668f59f06de8dd
This adds the bulk of the image pre-caching logic to the conductor
task manager. It takes an aggregate and list of image ids from the
API service and handles the process of calling to the relevant compute
nodes to initiate the image downloads, honoring the (new) config knob
for overall task parallelism.
Related to blueprint image-precache-support
Change-Id: Id7c0ab7ae0586d49d88ff2afae149e25e59a3489
We want to make sure people who have both pinned and unpinned instances
on a host (yes, it's dumb, but it happens) won't do a straightforward
'vcpu_pin_set' -> '[compute] cpu_dedicated_set' or '[compute]
cpu_shared_set' transition, breaking their ability to restart or rebuild
instances. We add a new 'supports_pcpus' driver capability to avoid
needing to do this on drivers that don't support PCPUs and CPU pinning
in general.
Change-Id: Ia1b3ab0b66fdaf569f6c7a09510f208ee28725b2
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
The FakeDriverWithPciResources creates 3 PFs with VFs for each compute
in a symmetric way. For example PF1 in every compute has a device name
'ens1'. When a server with SRIOV ports with resource request is booted
the PCI request of the server is updated to point to the PF dev name
that provides the bandwidth for the port. When such a server is migrated
this reference needs to be updated according to the destination host
resource allocation. As the compute were symmetric the update was a
no-op (e.g. from ens1 on the source to the ens1 on the destination
host). This prevent us to properly assert that the migration did the
update.
So this patch breaks the symmetry by appending the hostname to the PF
device names.
Change-Id: Id801d36f96b16708458db8549ce03abd308bc119
blueprint: support-move-ops-with-qos-ports
For a cross-cell resize, the equivalent of the "finish_resize"
method on the destination compute is going to call the
driver spawn() method rather than the finish_migration()
method and needs to pass through the power_on value, similar
to finish_migration, so that when resizing a stopped server
it remains stopped once it is resized.
The finish_migration method in the driver behaves very similar
to spawn so the semantics are the same.
This change updates the spawn method signature for all in-tree
compute drivers but only implements the logic for the libvirt
driver as that is the only driver (currently) which supports
cross-cell resize (note the can_connect_volume method is also
necessary for cross-cell resize implementation in the driver).
Part of blueprint cross-cell-resize
Change-Id: I6929c588dd2e0e805f2e30b2e30d29967469d756
This patch adds a new external event called "power-update"
through which ironic will convey all (power_off and power_on)
power state changes (running -> shutdown or shutdown -> running
will be the only ones handled by nova and the rest will be ignored)
on a physical instance to nova. The database will be updated
accordingly to reflect the real vm_state and power_state of the
instance. This way nova will not be able to enforce
an incorrect power state on the physical instance during
the periodic "sync_power_states" task.
Implements blueprint nova-support-instance-power-update
Story: 2004969
Task: 29423
Change-Id: I2b292050cc3ce5ef625659f5a1fe56bb76072496
This adds a functional test which recreates bug 1839560
where the driver reports a node, then no longer reports
it so the compute manager deletes it, and then the driver
reports it again later (this can be common with ironic
nodes as they undergo maintenance). The issue is that since
Ia69fabce8e7fd7de101e291fe133c6f5f5f7056a in Rocky, the
ironic node uuid is re-used for the compute node uuid but
there is a unique constraint on the compute node uuid so
when trying to create the compute node once the ironic node
is available again, the compute node create fails with a
duplicate entry error due to the duplicate uuid. To recreate
this in the functional test, a new fake virt driver is added
which provides a predictable uuid per node like the ironic
driver. The test also shows that archiving the database is
a way to workaround the bug until it's properly fixed.
Change-Id: If822509e906d5094f13a8700b2b9ed3c40580431
Related-Bug: #1839560
In order to obtain plug-time events, the libvirt driver needs to pass
a Migration object to the network model. Previously [1], this object
was obtained with a database lookup. This was done to avoid modifying
the virt driver interface, thus making the fix backportable. This
patch makes the compute manager pass the migration object to the virt
driver in order to avoid a needless database query.
[1] https://review.opendev.org/#/c/667177/
Change-Id: I9095c1be65b127d40f5bbe7af5d63fefedb73d33
Ghanshyam Mann