Commit Graph

172 Commits

Author SHA1 Message Date
Stephen Finucane 33be6d4b56 tests: Move libvirt-specific fixtures
These were left to last since there's a bit of cleanup necessary to move
everything across.

Change-Id: I921c812ac03f7d32eec31200772020c17f292851
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2021-05-24 11:00:29 +01:00
Stephen Finucane a62dd42c0d libvirt: Delegate OVS plug to os-vif
os-vif 1.15.0 added the ability to create an OVS port during plugging
by specifying the 'create_port' attribute in the 'port_profile' field.
By delegating port creation to os-vif, we can rely on it's 'isolate_vif'
config option [1] that will temporarily configure the VLAN to 4095
(0xfff), which is reserved for implementation use [2] and is used by
neutron to as a dead VLAN [3]. By doing this, we ensure VIFs are plugged
securely, preventing guests from accessing other tenants' networks
before the neutron OVS agent can wire up the port.

This change requires a little dance as part of the live migration flow.
Since we can't be certain the destination host has a version of os-vif
that supports this feature, we need to use a sentinel to indicate when
it does. Typically we would do so with a field in
'LibvirtLiveMigrateData', such as the 'src_supports_numa_live_migration'
and 'dst_supports_numa_live_migration' fields used to indicate support
for NUMA-aware live migration. However, doing this prevents us
backporting this important fix since o.vo changes are not backportable.
Instead, we (somewhat evilly) rely on the free-form nature of the
'VIFMigrateData.profile_json' string field, which stores JSON blobs and
is included in 'LibvirtLiveMigrateData' via the 'vifs' attribute, to
transport this sentinel. This is a hack but is necessary to work around
the lack of a free-form "capabilities" style dict that would allow us do
backportable fixes to live migration features.

Note that this change has the knock on effect of modifying the XML
generated for OVS ports: when hybrid plug is false will now be of type
'ethernet' rather than 'bridge' as before. This explains the larger than
expected test damage but should not affect users.

[1] https://opendev.org/openstack/os-vif/src/tag/2.4.0/vif_plug_ovs/ovs.py#L90-L93
[2] https://en.wikipedia.org/wiki/IEEE_802.1Q#Frame_format
[3] https://answers.launchpad.net/neutron/+question/231806

Change-Id: I11fb5d3ada7f27b39c183157ea73c8b72b4e672e
Depends-On: Id12486b3127ab4ac8ad9ef2b3641da1b79a25a50
Closes-Bug: #1734320
Closes-Bug: #1815989
2021-04-30 12:51:35 +01:00
Balazs Gibizer e56cc4f439 Replace blind retry with libvirt event waiting in detach
Nova so far applied a retry loop that tried to periodically detach the
device from libvirt while the device was visible in the domain xml. This
could lead to an issue where an already progressing detach on the
libvirt side is interrupted by nova re-sending the detach request for
the same device. See bug #1882521 for more information.

Also if there was both a persistent and a live domain the nova tried the
detach from both at the same call. This lead to confusion about the
result when such call failed. Was the detach failed partially?

We can do better, at least for the live detach case. Based on the
libvirt developers detaching from the persistent domain always
succeeds and it is a synchronous process. Detaching from the live
domain can be both synchronous or asynchronous depending on the guest
OS and the load on the hypervisor. But for live detach libvirt always
sends an event [1] nova can wait for.

So this patch does two things.

1) Separates the detach from the persistent domain from the detach from
   the live domain to make the error cases clearer.

2) Changes the retry mechanism.

   Detaching from the persistent domain is not retried. If libvirt
   reports device not found, while both persistent and live detach
   is needed, the error is ignored, and the process continues with
   the live detach. In any other case the error considered as fatal.

   Detaching from the live domain is changed to always wait for the
   libvirt event. In case of timeout, the live detach is retried.
   But a failure event from libvirt considered fatal, based on the
   information from the libvirt developers, so in this case the
   detach is not retried.

Related-Bug: #1882521

[1]https://libvirt.org/html/libvirt-libvirt-domain.html#virConnectDomainEventDeviceRemovedCallback

Change-Id: I7f2b6330decb92e2838aa7cee47fb228f00f47da
2021-04-18 10:24:08 +02:00
Stephen Finucane 22fcfcda23 tests: Add functional test for vDPA device
Add a simple test to validate behavior with vDPA devices. Most of this
is simply fleshing out the fixtures we use to fake out vDPA devices and
generally tweaking things to make them better.

Change-Id: I1423d8a9652751b667463f90c69eae1a054dd776
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Co-authored-by: Sean Mooney <work@seanmooney.info>
2021-03-16 20:39:27 +00:00
Zuul bf4d4c16fd Merge "libvirt: Wire up 'os_secure_boot' property" 2021-03-16 11:47:12 +00:00
Zuul 7422eb00bf Merge "libvirt: Use firmware metadata files to configure instance" 2021-03-15 14:56:16 +00:00
Zuul 896bf25e74 Merge "libvirt: Add vDPA nodedev parsing" 2021-03-15 13:38:20 +00:00
Zuul 1e5b731005 Merge "Add device event constants to fakelibvirt" 2021-03-15 11:08:16 +00:00
Zuul 29c841b9bc Merge "fakelibvirt: make kB_mem default not laughable" 2021-03-15 09:44:16 +00:00
Sean Mooney 9a673a8faa libvirt: Add vDPA nodedev parsing
Add the ability to look up vDPA nodedevs via their parent VF and
marshal the result into a nodedev object. This requires a new version of
libvirt and QEMU so we must add constants to protect that.

Blueprint: libvirt-vdpa-support
Change-Id: I043880cb81b02488d13c3387d696142545c13395
2021-03-12 11:48:35 +00:00
Artom Lifshitz d79f366cf5 fakelibvirt: make kB_mem default not laughable
Previously, the default for kB_mem was 2048. That's 2MB of RAM. Change
this to 16GB, and cleanup tests that were forced to explicitly set
sane value by removing the kB_mem argument. In cases where tests were
using the default CPU configuration, arguments to HostInfo are removed
entirely.

Change-Id: I1c97776583f9e43c4a284fb9ef59bd6293730a28
2021-03-09 15:38:57 -05:00
Stephen Finucane e7d704899f libvirt: Wire up 'os_secure_boot' property
The penultimate step in our journey from the secure boot wilderness.
Start configuring the relevant attribute of the guest if and when secure
boot is enabled.

Blueprint: allow-secure-boot-for-qemu-kvm-guests
Change-Id: Ic38ab840f59619bf921e5387cd7a11c88a77b2a5
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2021-03-09 15:56:18 +00:00
Stephen Finucane 9fff6893ce libvirt: Use firmware metadata files to configure instance
We now have the machinery in place to parse QEMU's firmware metadata
files, which means we no longer need to store a hardcoded list of
possible firmware locations nor build upon this to include additional
information like "does this support secure boot". Start using this and
cut out the legacy stuff.

Eventually all of this will be changed yet again in favour of libvirt's
firmware auto-selection functionality, but that needs a little more work
before it's suitable for us [1].

[1] https://bugzilla.redhat.com/show_bug.cgi?id=1906500

Blueprint: allow-secure-boot-for-qemu-kvm-guests
Change-Id: Ie99e43cb0408eae4034d410b9dd204cd39984fd1
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2021-03-09 15:55:47 +00:00
Stephen Finucane 452d2fb3a0 libvirt: Move some host arch checks to guest checks
We have a load of tests to enable specific behavior depending on the
guest architecture, but in many cases we're actually testing the host
architecture. This is wrong and breaks our ability to create guests with
non-host architectures plus our ability to rely on mocking of these APIs
by 'FakeLibvirt'. We fix this by switching checks against the CPU
architecture reported by libvirt's host capabilities, to checks against
the guest image metadata (via 'nova.virt.libvirt.utils.get_arch') with a
fallback to the host architecture (via 'os.uname'). This seemingly
simple change subsequently highlights *a lot* of gaps in the mocking
done by 'FakeLibvirtFixture'. Pretty much all of these stem from the
fact that by previously extracting the CPU host architecture from host
capabilities, we were in practice almost always checking against x86-64.
Now that we're doing checks against image metadata with a fallback to
'os.uname()', we suddenly need to be able to provide host and domain
capabilities for other architectures and doing additional mocks for some
of these architecture-specific code paths.

Note that this change effectively removes test coverage for the
'LibvirtDriver._has_uefi_support' helper because we now mock it in the
'FakeLibvirtFixture' to avoid checks in AArch64 tests (AArch64 defaults
to a UEFI bootloader). This is okay since we're going to remove this
helper entirely in a future change.

Also note that this _still_ isn't as complete as I'd like it to be,
owing to time constraints and the sheer depth of the problem here. The
remaining issues will be addressed in follow-ups, potentially alongside
patches to drop support for legacy architectures like MIPS and I686 (the
former has been EOL'd in favour of RISC-V as of March 2021 and both
Intel and AMD stopped manufacturing products using the latter ISA for
anything other than embedded use cases many moons ago).

Change-Id: I158dd70076ecdbef422ef08c11ce6a140c991537
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2021-03-09 12:37:21 +00:00
Balazs Gibizer aeb8de892b Add device event constants to fakelibvirt
The patch I92eb27b710f16d69cf003712431fe225a014c3a8 added the usage of
two new libvirt event constants but missed to add them also to the
fakelibvirt. This caused an extra warning to appear in a lot of unit
tests.

URI qemu:///system does not support events: module
'nova.tests.unit.virt.libvirt.fakelibvirt' has no attribute
'VIR_DOMAIN_EVENT_ID_DEVICE_REMOVED'

The missing constants now added.

This change showed that the above patch also missed to adapt some of the
unit test to the registration of the two new events but this was masked
by the missing fakelibvirt change. So now the unit tests also adapted.

Change-Id: I926553e057ff4838d84244e6f4ef4f39e418788e
2021-03-05 18:01:52 +01:00
Stephen Finucane a65e4201cc tests: Stop mocking 'nova.virt.libvirt.utils.get_arch'
In change I03febf6ad7d76c7eec818d3b16a3ef8b26dcd84c we introduced a mock
of nova.virt.libvirt.utils.get_arch' to the 'FakeLibvirtFixture'. This
was a follow-up to change Icf55488e61793d5b055d230c8524df8210cd00fd
which introduced this mock for many individual libvirt-related tests. In
change I458b1db091e33c0b835e44b5a86de6c0a08f99a3, we decided to poison
'os.uname' rather than 'get_arch' since that was the underlying cause of
test differences, but we never extended this reasoning to the existing
mocks of 'get_arch'. The mocking of 'get_arch' is now a hindrance to any
test that wants to validate behavior where the architecture requested
via image metadata (the 'hw_architecture' property) differs from the
host architecture, so it is time to make this switch and stop mocking
'get_arch' (almost) entirely.

Change-Id: Iff5ca9f4a51fe30435e57f0a9ca3d43a2065978c
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2021-03-05 10:59:58 +00:00
Zuul 5eceb4acd0 Merge "tests: Add 'nova.virt.libvirt.utils.get_arch' stub to fixture" 2021-03-01 19:43:58 +00:00
Zuul 44611935b8 Merge "libvirt: add IP address to libvirt guest metadata" 2021-02-25 11:14:45 +00:00
Artom Lifshitz 95b9481aa4 libvirt: start tracking NUMACell.socket for hosts
This patch adds a `socket` field to NUMACell, and the libvirt driver
starts populating it. For testing, we need to fix how fakelibvirt's
HostInfo handled sockets: it previously assumed one or more sockets
within a NUMA node, but we want the reverse - one or more NUMA nodes
within a socket.

Implements: blueprint pci-socket-affinity
Change-Id: Ie4deb265f6093558ab86dc69f6ffab9da62ca15d
2021-02-21 11:30:07 -05:00
Stephen Finucane 1053ae0b61 tests: Add 'nova.virt.libvirt.utils.get_arch' stub to fixture
By add this to the 'LibvirtFixture', we ensure virtually every test
that validates a libvirt thing will have this stubbed automatically.

Change-Id: I03febf6ad7d76c7eec818d3b16a3ef8b26dcd84c
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2021-02-20 15:32:14 +00:00
Nobuhiro MIKI 838370a490
libvirt: add IP address to libvirt guest metadata
Libvirt XML contains useful configuration information such as instance names,
flavors and images as metadata. This change extends this metadata to include
the IP addresses of the instances.

Example:
<metadata>
  <nova:instance xmlns:nova="http://openstack.org/xmlns/libvirt/nova/1.1">
    ...
    <nova:ports>
      <nova:port uuid="567a4527-b0e4-4d0a-bcc2-71fda37897f7">
        <nova:ip type="fixed" address="192.168.1.1" ipVersion="4"/>
        <nova:ip type="fixed" address="fe80::f95c:b030:7094" ipVersion="6"/>
        <nova:ip type="floating" address="11.22.33.44" ipVersion="4"/>
      </nova:port>
    </nova:ports>
    ...
  </nova:instance>
</metadata>

Change-Id: I45f1df4935905170957c2ea2496c8a698a7464a2
blueprint: libvirt-driver-ip-metadata
Signed-off-by: Nobuhiro MIKI <nmiki@yahoo-corp.jp>
2021-02-18 16:05:01 +09:00
Lee Yarwood 95724bbaef libvirt: Bump MIN_{LIBVIRT,QEMU}_VERSION and NEXT_MIN_{LIBVIRT,QEMU}_VERSION
I8e349849db0b1a540d295c903f1470917b82fd97 bumped these versions late in
the Victoria cycle and it's time to do the same again during Wallaby.

The new MIN_{LIBVIRT,QEMU}_VERSIONs are:

MIN_LIBVIRT_VERSION = (6, 0, 0)
MIN_QEMU_VERSION = (4, 2, 0)

These versions are met by the three defined LTS distros supported by the
Wallaby release [1][2] of Ubuntu 20.04, CentOS 8 and openSUSE Leap 15.2.

The following constants are removed as part of this patch as since
I864494e11ff697788167c996a39f38d3d833d0d7 we now use these new minimum
versions in tests straight away, thus breaking many tests exercising
these now obsolete constants.

- MIN_LIBVIRT_VTPM
- MIN_LIBVIRT_S390X_CPU_COMPARE
- MIN_{LIBVIRT,QEMU}_BLOCKDEV

The removal of MIN_{LIBVIRT,QEMU}_BLOCKDEV means that the swap_volume
will always use the blockCopy libvirt method to copy contents between
disks. This in turn requires that fakelibvirt correctly model the
required flags and blockCopy method.

A future change will look into switching the remaining blockRebase calls
in the live snapshot flow to blockCopy.

Finally, NEXT_MIN_{LIBVIRT,QEMU}_VERSION are also updated to:

NEXT_MIN_LIBVIRT_VERSION = (7, 0, 0)
NEXT_MIN_QEMU_VERSION = (5, 2, 0)

[1] https://governance.openstack.org/tc/reference/runtimes/wallaby.html
[2] https://wiki.openstack.org/wiki/LibvirtDistroSupportMatrix

Change-Id: I017083b27cd9d145eecb01106388d4ce880ba823
2021-01-22 10:06:41 +00:00
Stephen Finucane 3a390c2c82 libvirt: Drop support for Xen
This hasn't been validated upstream and there doesn't appear to be
anyone using it. It's time to drop support for this. This is mostly test
and documentation damage, though there is some other cleanup going on,
like the removal of the essentially noop 'pick_disk_driver_name' helper.

Change-Id: I73305e82da5d8da548961b801a8e75fb0e8c4cf1
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2021-01-22 10:06:40 +00:00
Stephen Finucane d02ce3c4f0 libvirt: Drop support for UML
This has not been tested in the gate for a long time and was only added
to enable CI in the early days of OpenStack. Time to bid adieu.

Change-Id: I7a157f37d2a67e1174a1725fd579c761d81a09b1
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2021-01-22 10:05:57 +00:00
Stephen Finucane ec8732a4b8 functional: Add live migration tests for PCI, SR-IOV servers
There are a few holes here, mostly to do with dodgy mocking from
fakelibvirt, but it's a start that we can build upon in future changes.

Change-Id: Ic641d9b975710538beb6dd9af55cc940501623da
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2020-11-09 16:25:32 +00:00
Zuul 7134cbdc7a Merge "Update pci stat pools based on PCI device changes" 2020-11-06 05:23:46 +00:00
Lee Yarwood 0d2ca53bb8 libvirt: Only ask tpool.Proxy to autowrap vir* classes
I668643c836d46a25df46d4c99a973af5e50a39db attempted to fix service wide
pauses by providing a more complete list of classes to tpool.Proxy.

While this excluded libvirtError it can include internal libvirt-python
classes pointed to by private globals that have been introduced with the
use of type checking within the module.

Any attempt to wrap these internal classes will result in the failure
seen in bug #1901383. As a result this change simply ignores any class
found during inspection that doesn't start with the `vir` string, used
by libvirt to denote public methods and classes.

Closes-Bug: #1901383
Co-Authored-By: Daniel Berrange <berrange@redhat.com>
Change-Id: I568b0c4fd6069b9118ff116532f14abb46cc42ab
2020-10-27 09:38:18 +00:00
Hemanth Nakkina b8695de6da Update pci stat pools based on PCI device changes
At start up of nova-compute service, the PCI stat pools are
populated based on information in pci_devices table in Nova
database. The pools are updated only when new device is added
or removed but not on any device changes like device type.

If an existing device is configured as SRIOV and nova-compute
is restarted, the pci_devices table gets updated but the device
is still listed under the old pool in pci_tracker.stats.pool
(in-memory object).

This patch looks for device type updates in existing devices
and updates the pools accordingly.

Change-Id: Id4ebb06e634a612c8be4be6c678d8265e0b99730
Closes-Bug: #1892361
2020-10-15 13:10:36 +05:30
Lee Yarwood 75250d6b17 fakelibvirt: Use versionutils to set min versions found in the driver
These were previously manually converted ints based on the version
tuples present in the driver. To avoid ever needing to worry about these
again this change simply uses versionutils to covert the tuples found in
the driver into ints used by fakelibvirt.

Change-Id: I864494e11ff697788167c996a39f38d3d833d0d7
2020-09-17 21:21:28 +01:00
Balazs Gibizer 1361ea5ad1 Support SRIOV interface attach and detach
For attach:
* Generates InstancePciRequest for SRIOV interfaces attach requests
* Claims and allocates a PciDevice for such request

For detach:
* Frees PciDevice and deletes the InstancePciRequests

On the libvirt driver side the following small fixes was necessar:
* Fixes PCI address generation to avoid double 0x prefixes in LibvirtConfigGuestHostdevPCI
* Adds support for comparing LibvirtConfigGuestHostdevPCI objects
* Extends the comparison of LibvirtConfigGuestInterface to support
  macvtap interfaces where target_dev is only known by libvirt but not
  nova
* generalize guest.get_interface_by_cfg() to work with both
  LibvirtConfigGuest[Inteface|HostdevPCI] objects

Implements: blueprint sriov-interface-attach-detach

Change-Id: I67504a37b0fe2ae5da3cba2f3122d9d0e18b9481
2020-09-10 18:44:53 +01:00
Zuul 662398a1a4 Merge "fakelibvirt: Remove nova-network remnants" 2020-09-01 09:53:05 +00:00
Chris Friesen e0ca2652ed libvirt: Add emulated TPM support to Nova
Previous patches added support for parsing the vTPM-related flavor extra
specs and image metadata properties, the necessary integrations with the
Castellan key manager API etc. This change adds the ability to enable
support in the libvirt driver and create guests with vTPM functionality
enabled. Cold migration and resize are not yet supported. These will be
addressed in follow-on changes.

Functional tests are included. These require expansion of the
fakelibvirt stubs to implement basic secret management

Part of blueprint add-emulated-virtual-tpm

[1] https://review.opendev.org/686804

Change-Id: I1ff51f608b85dbb621814e70079ecfdd3d1a1d22
Co-Authored-By: Eric Fried <openstack@fried.cc>
Co-Authored-By: Stephen Finucane <stephenfin@redhat.com>
2020-08-25 17:55:33 +01:00
Stephen Finucane 9b6a93ae3e fakelibvirt: Remove nova-network remnants
We haven't used the nwfilter stuff in libvirt since we removed support
for firewalls in change I2dccf1610d6cbbb076fda393f1ef695d0be84b13.
There's no reason to keep the libvirt stubs around.

Change-Id: I552743286c70aa69f75880e9ccc3b896daa9115e
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
2020-08-10 15:55:41 +01:00
Sean Mooney efc27ff84c Lookup nic feature by PCI address
In some environments the libvirt nodedev list can become out of sync
with the current MAC address assigned to a netdev, As a result the
nodedev lookup can fail. This results in an uncaught libvirt exception
which breaks the update_available_resource function resultingin an
incorrect resource view in the database.

e.g. libvirt.libvirtError: Node device not found:
no node device with matching name 'net_enp7s0f3v1_ea_60_77_1f_21_50'

This change removes the dependency on the nodedev name when looking up
nic feature flags.

Change-Id: Ibf8dca4bd57b3bddb39955b53cc03564506f5754
Closes-Bug: #1883671
2020-08-04 22:25:02 +00:00
Lee Yarwood 902f09af25 libvirt: Handle VIR_ERR_DEVICE_MISSING when detaching devices
Introduced in libvirt v4.1.0 [1] this error code replaces the previously
raised VIR_ERR_INVALID_ARG, VIR_ERR_OPERATION_FAILED and
VIR_ERR_INVALID_ARG codes [2][3].

VIR_ERR_OPERATION_FAILED was introduced and tested as an
active/live/hot unplug config device detach error code in
I131aaf28d2f5d5d964d4045e3d7d62207079cfb0.

VIR_ERR_INTERNAL_ERROR was introduced and tested as an
active/live/hot unplug config device detach error code in
I3055cd7641de92ab188de73733ca9288a9ca730a.

VIR_ERR_INVALID_ARG was introduced and tested as an
inactive/persistent/cold unplug config device detach error code in
I09230fc47b0950aa5a3db839a070613c9c817576.

This change introduces support for the new VIR_ERR_DEVICE_MISSING error
code while also retaining coverage for these codes until
MIN_LIBVIRT_VERSION is bumped past v4.1.0.

The majority of this change is test code motion with the existing tests
being modified to run against either the active or inactive versions of
the above  error codes for the time being.

test_detach_device_with_retry_operation_internal and
test_detach_device_with_retry_invalid_argument_no_live have been removed
as they duplicate the logic within the now refactored
_test_detach_device_with_retry_second_detach_failure.

[1] https://libvirt.org/git/?p=libvirt.git;a=commit;h=bb189c8e8c93f115c13fa3bfffdf64498f3f0ce1
[2] https://libvirt.org/git/?p=libvirt.git;a=commit;h=126db34a81bc9f9f9710408f88cceaa1e34bbbd7
[3] https://libvirt.org/git/?p=libvirt.git;a=commit;h=2f54eab7c7c618811de23c60a51e910274cf30de

Closes-Bug: #1887946
Change-Id: I7eb86edc130d186a66c04b229d46347ec5c0b625
2020-07-22 08:33:19 +00:00
Zuul 0b6ae371d8 Merge "Functional tests for NUMA live migration" 2020-04-28 20:46:37 +00:00
Zuul 966d21fa75 Merge "libvirt: Remove VIR_DOMAIN_BLOCK_REBASE_RELATIVE flag check" 2020-04-16 21:40:57 +00:00
Zuul d695b7dfbc Merge "Functional test with pGPUs" 2020-04-09 23:29:21 +00:00
Sylvain Bauza 9dcc0941f1 Functional test with pGPUs
We provide two testclasses :
- one will only focus about a single supported type
- the other one will focus on multiple types

The idea is that we check that inventories are created at compute
startup accordingly with no allocations and then we verify when we boot
a server that both allocations are correctly made against the right
pGPU Resource Provider but also that the mdev is created.

Change-Id: Ib5dbcee295d8a78b9a55806054de6265c3663343
2020-04-09 17:54:31 +02:00
Zuul 7520676b28 Merge "tests: work around malformed serial XML" 2020-04-02 18:39:25 +00:00
Lee Yarwood b1b0f1be18 libvirt: Always provide the size in bytes when calling virDomainBlockResize
Since 0.9.11 virDomainBlockResize has accepted the size argument in
bytes when the VIR_DOMAIN_BLOCK_RESIZE_BYTES flag is provided.

This change switches all callers over to using bytes to simplify the
required call, avoiding the need to divide by units.Ki etc.

Change-Id: Ib8d9318596186acd86a738ceea187420698645e6
2020-04-01 15:28:32 +01:00
Artom Lifshitz 06aaa08dc4 Functional tests for NUMA live migration
This patch add a functional tests for NUMA live migration. It also
adds all the previously missing scaffolding, specifically:

* Make the fakelibvirt Domain support the vcpupin, memnode and
  emulatorpin XML elements.

* To support testing RPC pinning, a new StubComputeRPCAPI is added to
  the integrated helpers. It replaces the real ComputeAPI's router()
  method with a stub that uses the RPC version set by the object's
  __init__. Tests can replace a real ComputeAPI object with the Stub
  to simulate compute services and/or conductors being pinned to
  different RPC versions.

* In the fake_imagebackend, is_shared_block_storage() gets stubbed.
  It's called indirectly from the
  _check_can_live_migrate_(source|destination) checks.

Implements blueprint numa-aware-live-migration
Change-Id: Ia3d7351c1805d98bcb799ab0375673c7f1cb8848
2020-03-24 10:10:57 -04:00
Artom Lifshitz 4ba00c803b tests: work around malformed serial XML
The serial elements in fakelibvirt's XMLDesc() are malformed. They are
only enough to pass existing unit tests, but are not future-proof, and
would not work with something like migration._update_serial_xml(). In
addition, two of them (type=file and type=pty) are not actually used by
anything.

This patch removes the unused types and starts checking the
CONF.serial_console.enabled flag before including any serial XML in
the fake domain. This allows tests that don't set
serial_console_enabled (like the subsequent NUMA LM func test) to not
have to deal with the broken serial XML, while tests that relied on it
in the past now just set serial_console_enabled = True.

Change-Id: Ie74fcd689a627c178f80ec60df2dfef40576f182
2020-03-24 10:10:57 -04:00
Lee Yarwood 90032a7e01 libvirt: Remove VIR_DOMAIN_BLOCK_REBASE_RELATIVE flag check
_volume_snapshot_delete would previously check if the
VIR_DOMAIN_BLOCK_REBASE_RELATIVE flag, introduced in 1.2.7, was
available with the installed version of Libvirt. This check can be
removed as MIN_LIBVIRT_VERSION now sit at 4.0.0 ensuring this flag is
always present.

The value of VIR_DOMAIN_BLOCK_REBASE_RELATIVE used by fakelibvirt is
also updated to reflect what is actually used by Libvirt [1].

[1] 26badd13e8/include/libvirt/libvirt-domain.h (L2553)

Change-Id: I53e637387881a8919dcc7b425a1cb96102c8d610
2020-03-22 13:21:43 +00:00
Stephen Finucane 30141e6b52 nova-net: Remove firewall support (pt. 2)
Firewall support is not needed with neutron, which supports both
security groups for per-port filtering and FWaaS for per-network
filtering. Remove both the generic firewalls and the hypervisor-specific
implementations.

This change focuses on removing the firewall-related API calls from the
various virt drivers. The firewall drivers themselves are removed
separately.

Change-Id: I5a9e5532c46a5f7064441ae644125d21efe5fda1
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
2020-01-06 12:46:30 +00:00
Sean Mooney 8c72241726 support pci numa affinity policies in flavor and image
This addresses bug #1795920 by adding support for
defining a pci numa affinity policy via the flavor
extra specs or image metadata properties enabling
the policies to be applied to neutron sriov port
including hardware offloaded ovs.

Closes-Bug: #1795920
Related-Bug: #1805891
Implements: blueprint vm-scoped-sriov-numa-affinity
Change-Id: Ibd62b24c2bd2dd208d0f804378d4e4f2bbfdaed6
2019-12-11 14:39:12 +00:00
Zuul 2ee15de070 Merge "libvirt: Bump MIN_{LIBVIRT,QEMU}_VERSION for "Ussuri"" 2019-11-26 12:08:23 +00:00
Kashyap Chamarthy 22c1916b49 libvirt: Bump MIN_{LIBVIRT,QEMU}_VERSION for "Ussuri"
We didn't bump the MIN_{LIBVIRT_QEMU} For the "Train" release, although
we advertized it[1].  Let's actually bump it for "Ussuri", while we're
early on in the development cycle to  give it enough "soaking time".

As part of this bump, fix the small fallout:

  - Remove the MIN_LIBVIRT_MDEV_SUPPORT version constant; fix tests.
  - Remove the "GOP" (Graphic Output Protocol) video model version
    constant from MIN_LIBVIRT_VIDEO_MODEL_VERSIONS; fix tests.
  - Fix the fucntional test:
    VGPUReshapeTests.test_create_servers_with_vgpu

The following version constants and related tests, which are now no
longer required, will be removed in separate patches:

    MIN_LIBVIRT_KVM_AARCH64_VERSION
    MIN_LIBVIRT_MULTIATTACH
    MIN_LIBVIRT_FILE_BACKED_VERSION
    MIN_QEMU_FILE_BACKED_VERSION
    MIN_QEMU_FILE_BACKED_DISCARD_VERSION

And here[2] is the discussion for picking future libvirt/QEMU versions
for "V" release on the list.

[1] https://opendev.org/openstack/nova/commit/489b5f762e
    -- Pick next minimum libvirt / QEMU versions for "T" release,
    2018-09-25)
[2] http://lists.openstack.org/pipermail/openstack-discuss/2019-November/010907.html
    -- On next minimum libvirt / QEMU versions for "V" release

Change-Id: Ia18e9be4d07b732afd9db0007c541253f3cdaf82
Signed-off-by: Kashyap Chamarthy <kchamart@redhat.com>
2019-11-21 15:07:25 +01:00
Matt Riedemann aa87b9c288 libvirt: check job status for VIR_DOMAIN_EVENT_SUSPENDED_MIGRATED event
Change Ic5cab99944df9e501ba2032eb96911c36304494d added handling for
the VIR_DOMAIN_EVENT_SUSPENDED_MIGRATED event during live migration
but failed to distinguish between the live migration actually succeeding
or failing before queueing the EVENT_LIFECYCLE_MIGRATION_COMPLETED
up into the ComputeManager.handle_lifecycle_event method.

As a result, failed live migrations will inadvertantly trigger
activation of the port bindings on the destination host, which
deactivates the source host port bindings, and then
_rollback_live_migration will delete those activated dest host port
bindings and leave the source host port bindings deactivated.

In this change, if we get the VIR_DOMAIN_EVENT_SUSPENDED_MIGRATED
event, we attempt to get the job status to determine the course to
take and only queue the EVENT_LIFECYCLE_MIGRATION_COMPLETED event,
which triggers the dest host port activation, if we can determine
the live migration job completed successfully. Otherwise we simply
report the guest as paused, the same as before Ic5cab9994.

Change-Id: I6a4252b0c12c41c233299f30ce8294fef21c7b40
Closes-Bug: #1788014
2019-11-05 16:05:02 +00:00
ya.wang af45b6e981 libvirt: Get the CPU model, not 'arch' from get_capabilities()
This fixes a regression introduced with change
Ib37ff6fc2e1f9de0e60adca54e87c74f45f12ffa.

Without this, setting [libvirt]/cpu_model_extra_flags
to some value results in libvirt failing to start with
an error like this:

  Configured extra flag: aes it not correct, or the host CPU
  does not support this flag. Please correct the config and
  try again. Migration pre-check error: CPU doesn't have
  compatibility.

Closes-Bug: 1844211
Change-Id: I9c7e842c4ef29cc78057a7ed9701f28ecb5e1a8b
2019-09-20 16:04:46 +00:00