summaryrefslogtreecommitdiff
diff options
context:
space:
mode:
authorKashyap Chamarthy <kchamart@redhat.com>2018-04-28 20:04:36 +0200
committerMatt Riedemann <mriedem.os@gmail.com>2018-05-01 11:47:18 -0400
commitcc27a2007f314df18812d54a087f6192d6aed3fb (patch)
tree888df29980d3166795f487c0c4289a14b2e21a6a
parent8e438eda9bb16cdd3b627b93da2435572275b921 (diff)
libvirt: Lift the restriction of choices for `cpu_model_extra_flags`
Commit 6b601b7 (libvirt: Allow to specify granular CPU feature flags) added support for allowing to specify individual CPU feature flags, but restricted the options only to "PCID" (refer to its commit message for why). In this change we lift the restriction of choices, and allow to specify multiple CPU feature flags for all three CPU modes for the libvirt driver: 'custom', 'host-model', and 'host-passthrough'. For example: [libvirt] cpu_mode = custom cpu_model = IvyBridge cpu_model_extra_flags = pcid, vmx, pdpe1gb This will allow additional use cases such as: - Ability to use 1GB huge pages with models that don't provide it (such as Intel "Haswell" variants) as one use case for extra flags: cpu_mode = custom cpu_model = Haswell-noTSX-IBRS cpu_model_extra_flags = pdpe1gb - Nested Virtualization -- an operator can specify the Intel 'vmx' (or AMD 'svm') flags for the level-1 Nova guest CPU models. (Assuming the 'nested' flag is enabled on the level-0 / bare-metal kernel.) (A future Nova patch will also allow ability to remove CPU flags.) Change-Id: I9a862619f379057bb48cb85a84dfc50d763030a6 Signed-off-by: Kashyap Chamarthy <kchamart@redhat.com> BluePrint: libvirt-cpu-model-extra-flags
Notes
Notes (review): Code-Review+2: Matt Riedemann <mriedem.os@gmail.com> Verified+1: IBM PowerKVM CI <kvmpower@linux.vnet.ibm.com> Code-Review+2: Dan Smith <dms@danplanet.com> Workflow+1: Dan Smith <dms@danplanet.com> Verified+2: Zuul Submitted-by: Zuul Submitted-at: Wed, 02 May 2018 00:26:11 +0000 Reviewed-on: https://review.openstack.org/563926 Project: openstack/nova Branch: refs/heads/master
-rw-r--r--nova/conf/libvirt.py61
-rw-r--r--nova/tests/unit/virt/libvirt/test_driver.py47
-rw-r--r--nova/virt/libvirt/driver.py19
-rw-r--r--releasenotes/notes/libvirt-cpu-model-extra-flags-a23085f58bd22d27.yaml8
4 files changed, 85 insertions, 50 deletions
diff --git a/nova/conf/libvirt.py b/nova/conf/libvirt.py
index c865325..0d21880 100644
--- a/nova/conf/libvirt.py
+++ b/nova/conf/libvirt.py
@@ -530,28 +530,49 @@ would result in an error and the instance launch will fail.
530 cfg.ListOpt( 530 cfg.ListOpt(
531 'cpu_model_extra_flags', 531 'cpu_model_extra_flags',
532 item_type=types.String( 532 item_type=types.String(
533 choices=['pcid'],
534 ignore_case=True, 533 ignore_case=True,
535 ), 534 ),
536 default=[], 535 default=[],
537 help=""" 536 help="""
538This allows specifying granular CPU feature flags when specifying CPU 537This allows specifying granular CPU feature flags when configuring CPU
539models. For example, to explicitly specify the ``pcid`` 538models. For example, to explicitly specify the ``pcid``
540(Process-Context ID, an Intel processor feature) flag to the "IvyBridge" 539(Process-Context ID, an Intel processor feature -- which is now required
541virtual CPU model:: 540to address the guest performance degradation as a result of applying the
541"Meltdown" CVE fixes to certain Intel CPU models) flag to the
542"IvyBridge" virtual CPU model::
542 543
543 [libvirt] 544 [libvirt]
544 cpu_mode = custom 545 cpu_mode = custom
545 cpu_model = IvyBridge 546 cpu_model = IvyBridge
546 cpu_model_extra_flags = pcid 547 cpu_model_extra_flags = pcid
547 548
548Currently, the choice is restricted to only one option: ``pcid`` (the 549To specify multiple CPU flags (e.g. the Intel ``VMX`` to expose the
549option is case-insensitive, so ``PCID`` is also valid). This flag is 550virtualization extensions to the guest, or ``pdpe1gb`` to configure 1GB
550now required to address the guest performance degradation as a result of 551huge pages for CPU models that do not provide it):
551applying the "Meltdown" CVE fixes on certain Intel CPU models.
552 552
553Note that when using this config attribute to set the 'PCID' CPU flag, 553 [libvirt]
554not all virtual (i.e. libvirt / QEMU) CPU models need it: 554 cpu_mode = custom
555 cpu_model = Haswell-noTSX-IBRS
556 cpu_model_extra_flags = PCID, VMX, pdpe1gb
557
558As it can be noticed from above, the ``cpu_model_extra_flags`` config
559attribute is case insensitive. And specifying extra flags is valid in
560combination with all the three possible values for ``cpu_mode``:
561``custom`` (this also requires an explicit ``cpu_model`` to be
562specified), ``host-model``, or ``host-passthrough``. A valid example
563for allowing extra CPU flags even for ``host-passthrough`` mode is that
564sometimes QEMU may disable certain CPU features -- e.g. Intel's
565"invtsc", Invariable Time Stamp Counter, CPU flag. And if you need to
566expose that CPU flag to the Nova instance, the you need to explicitly
567ask for it.
568
569The possible values for ``cpu_model_extra_flags`` depends on the CPU
570model in use. Refer to ``/usr/share/libvirt/cpu_map.xml`` possible CPU
571feature flags for a given CPU model.
572
573Note that when using this config attribute to set the 'PCID' CPU flag
574with the ``custom`` CPU mode, not all virtual (i.e. libvirt / QEMU) CPU
575models need it:
555 576
556* The only virtual CPU models that include the 'PCID' capability are 577* The only virtual CPU models that include the 'PCID' capability are
557 Intel "Haswell", "Broadwell", and "Skylake" variants. 578 Intel "Haswell", "Broadwell", and "Skylake" variants.
@@ -561,18 +582,14 @@ not all virtual (i.e. libvirt / QEMU) CPU models need it:
561 even if the host CPUs by the same name include it. I.e. 'PCID' needs 582 even if the host CPUs by the same name include it. I.e. 'PCID' needs
562 to be explicitly specified when using the said virtual CPU models. 583 to be explicitly specified when using the said virtual CPU models.
563 584
564For now, the ``cpu_model_extra_flags`` config attribute is valid only in 585The libvirt driver's default CPU mode, ``host-model``, will do the right
565combination with ``cpu_mode`` + ``cpu_model`` options. 586thing with respect to handling 'PCID' CPU flag for the guest --
566 587*assuming* you are running updated processor microcode, host and guest
567Besides ``custom``, the libvirt driver has two other CPU modes: The 588kernel, libvirt, and QEMU. The other mode, ``host-passthrough``, checks
568default, ``host-model``, tells it to do the right thing with respect to 589if 'PCID' is available in the hardware, and if so directly passes it
569handling 'PCID' CPU flag for the guest -- *assuming* you are running 590through to the Nova guests. Thus, in context of 'PCID', with either of
570updated processor microcode, host and guest kernel, libvirt, and QEMU. 591these CPU modes (``host-model`` or ``host-passthrough``), there is no
571The other mode, ``host-passthrough``, checks if 'PCID' is available in 592need to use the ``cpu_model_extra_flags``.
572the hardware, and if so directly passes it through to the Nova guests.
573Thus, in context of 'PCID', with either of these CPU modes
574(``host-model`` or ``host-passthrough``), there is no need to use the
575``cpu_model_extra_flags``.
576 593
577Related options: 594Related options:
578 595
diff --git a/nova/tests/unit/virt/libvirt/test_driver.py b/nova/tests/unit/virt/libvirt/test_driver.py
index f83868c..23280df 100644
--- a/nova/tests/unit/virt/libvirt/test_driver.py
+++ b/nova/tests/unit/virt/libvirt/test_driver.py
@@ -6372,6 +6372,35 @@ class LibvirtConnTestCase(test.NoDBTestCase,
6372 mock_warn.assert_not_called() 6372 mock_warn.assert_not_called()
6373 6373
6374 @mock.patch.object(libvirt_driver.LOG, 'warning') 6374 @mock.patch.object(libvirt_driver.LOG, 'warning')
6375 def test_get_guest_cpu_config_custom_with_multiple_extra_flags(self,
6376 mock_warn):
6377 drvr = libvirt_driver.LibvirtDriver(fake.FakeVirtAPI(), True)
6378 instance_ref = objects.Instance(**self.test_instance)
6379 image_meta = objects.ImageMeta.from_dict(self.test_image_meta)
6380
6381 self.flags(cpu_mode="custom",
6382 cpu_model="IvyBridge",
6383 cpu_model_extra_flags=['pcid', 'vmx'],
6384 group='libvirt')
6385 disk_info = blockinfo.get_disk_info(CONF.libvirt.virt_type,
6386 instance_ref,
6387 image_meta)
6388 conf = drvr._get_guest_config(instance_ref,
6389 _fake_network_info(self, 1),
6390 image_meta, disk_info)
6391 features = [feature.name for feature in conf.cpu.features]
6392 self.assertIsInstance(conf.cpu,
6393 vconfig.LibvirtConfigGuestCPU)
6394 self.assertEqual(conf.cpu.mode, "custom")
6395 self.assertEqual(conf.cpu.model, "IvyBridge")
6396 self.assertIn("pcid", features)
6397 self.assertIn("vmx", features)
6398 self.assertEqual(conf.cpu.sockets, instance_ref.flavor.vcpus)
6399 self.assertEqual(conf.cpu.cores, 1)
6400 self.assertEqual(conf.cpu.threads, 1)
6401 mock_warn.assert_not_called()
6402
6403 @mock.patch.object(libvirt_driver.LOG, 'warning')
6375 def test_get_guest_cpu_config_host_model_with_extra_flags(self, 6404 def test_get_guest_cpu_config_host_model_with_extra_flags(self,
6376 mock_warn): 6405 mock_warn):
6377 drvr = libvirt_driver.LibvirtDriver(fake.FakeVirtAPI(), True) 6406 drvr = libvirt_driver.LibvirtDriver(fake.FakeVirtAPI(), True)
@@ -6379,7 +6408,7 @@ class LibvirtConnTestCase(test.NoDBTestCase,
6379 image_meta = objects.ImageMeta.from_dict(self.test_image_meta) 6408 image_meta = objects.ImageMeta.from_dict(self.test_image_meta)
6380 6409
6381 self.flags(cpu_mode="host-model", 6410 self.flags(cpu_mode="host-model",
6382 cpu_model_extra_flags="pcid", 6411 cpu_model_extra_flags="pdpe1gb",
6383 group='libvirt') 6412 group='libvirt')
6384 disk_info = blockinfo.get_disk_info(CONF.libvirt.virt_type, 6413 disk_info = blockinfo.get_disk_info(CONF.libvirt.virt_type,
6385 instance_ref, 6414 instance_ref,
@@ -6387,14 +6416,17 @@ class LibvirtConnTestCase(test.NoDBTestCase,
6387 conf = drvr._get_guest_config(instance_ref, 6416 conf = drvr._get_guest_config(instance_ref,
6388 _fake_network_info(self, 1), 6417 _fake_network_info(self, 1),
6389 image_meta, disk_info) 6418 image_meta, disk_info)
6419 features = [feature.name for feature in conf.cpu.features]
6390 self.assertIsInstance(conf.cpu, 6420 self.assertIsInstance(conf.cpu,
6391 vconfig.LibvirtConfigGuestCPU) 6421 vconfig.LibvirtConfigGuestCPU)
6392 self.assertEqual(conf.cpu.mode, "host-model") 6422 self.assertEqual(conf.cpu.mode, "host-model")
6393 self.assertEqual(len(conf.cpu.features), 0) 6423 self.assertIn("pdpe1gb", features)
6394 self.assertEqual(conf.cpu.sockets, instance_ref.flavor.vcpus) 6424 self.assertEqual(conf.cpu.sockets, instance_ref.flavor.vcpus)
6395 self.assertEqual(conf.cpu.cores, 1) 6425 self.assertEqual(conf.cpu.cores, 1)
6396 self.assertEqual(conf.cpu.threads, 1) 6426 self.assertEqual(conf.cpu.threads, 1)
6397 self.assertTrue(mock_warn.called) 6427 # For 'host-model', it is now valid to use 'extra_flags';
6428 # assert that no warning is thrown
6429 mock_warn.assert_not_called()
6398 6430
6399 @mock.patch.object(libvirt_driver.LOG, 'warning') 6431 @mock.patch.object(libvirt_driver.LOG, 'warning')
6400 def test_get_guest_cpu_config_host_passthrough_with_extra_flags(self, 6432 def test_get_guest_cpu_config_host_passthrough_with_extra_flags(self,
@@ -6404,7 +6436,7 @@ class LibvirtConnTestCase(test.NoDBTestCase,
6404 image_meta = objects.ImageMeta.from_dict(self.test_image_meta) 6436 image_meta = objects.ImageMeta.from_dict(self.test_image_meta)
6405 6437
6406 self.flags(cpu_mode="host-passthrough", 6438 self.flags(cpu_mode="host-passthrough",
6407 cpu_model_extra_flags="pcid", 6439 cpu_model_extra_flags="invtsc",
6408 group='libvirt') 6440 group='libvirt')
6409 disk_info = blockinfo.get_disk_info(CONF.libvirt.virt_type, 6441 disk_info = blockinfo.get_disk_info(CONF.libvirt.virt_type,
6410 instance_ref, 6442 instance_ref,
@@ -6412,14 +6444,17 @@ class LibvirtConnTestCase(test.NoDBTestCase,
6412 conf = drvr._get_guest_config(instance_ref, 6444 conf = drvr._get_guest_config(instance_ref,
6413 _fake_network_info(self, 1), 6445 _fake_network_info(self, 1),
6414 image_meta, disk_info) 6446 image_meta, disk_info)
6447 features = [feature.name for feature in conf.cpu.features]
6415 self.assertIsInstance(conf.cpu, 6448 self.assertIsInstance(conf.cpu,
6416 vconfig.LibvirtConfigGuestCPU) 6449 vconfig.LibvirtConfigGuestCPU)
6417 self.assertEqual(conf.cpu.mode, "host-passthrough") 6450 self.assertEqual(conf.cpu.mode, "host-passthrough")
6418 self.assertEqual(len(conf.cpu.features), 0) 6451 self.assertIn("invtsc", features)
6419 self.assertEqual(conf.cpu.sockets, instance_ref.flavor.vcpus) 6452 self.assertEqual(conf.cpu.sockets, instance_ref.flavor.vcpus)
6420 self.assertEqual(conf.cpu.cores, 1) 6453 self.assertEqual(conf.cpu.cores, 1)
6421 self.assertEqual(conf.cpu.threads, 1) 6454 self.assertEqual(conf.cpu.threads, 1)
6422 self.assertTrue(mock_warn.called) 6455 # We have lifted the restriction for 'host-passthrough' as well;
6456 # so here too, assert that no warning is thrown
6457 mock_warn.assert_not_called()
6423 6458
6424 def test_get_guest_cpu_topology(self): 6459 def test_get_guest_cpu_topology(self):
6425 instance_ref = objects.Instance(**self.test_instance) 6460 instance_ref = objects.Instance(**self.test_instance)
diff --git a/nova/virt/libvirt/driver.py b/nova/virt/libvirt/driver.py
index a494c78..d9eab43 100644
--- a/nova/virt/libvirt/driver.py
+++ b/nova/virt/libvirt/driver.py
@@ -3815,25 +3815,6 @@ class LibvirtDriver(driver.ComputeDriver):
3815 msg = _("A CPU model name should not be set when a " 3815 msg = _("A CPU model name should not be set when a "
3816 "host CPU model is requested") 3816 "host CPU model is requested")
3817 raise exception.Invalid(msg) 3817 raise exception.Invalid(msg)
3818 # FIXME (kchamart): We're intentionally restricting the choices
3819 # (in the conf/libvirt.py) for 'extra_flags` to just 'PCID', to
3820 # address the immediate guest performance degradation caused by
3821 # "Meltdown" CVE fixes on certain Intel CPU models. In a future
3822 # patch, we will:
3823 # (a) Remove the restriction of choices for 'extra_flags',
3824 # allowing to add / remove additional CPU flags, as it will
3825 # make way for other useful features.
3826 # (b) Remove the below check for "host-model", as it is a
3827 # valid configuration to supply additional CPU flags to it.
3828 # (c) Revisit and fix the warnings / exception handling for
3829 # different combinations of CPU modes and 'extra_flags'.
3830 elif ((mode == "host-model" or mode == "host-passthrough") and
3831 extra_flags):
3832 extra_flags = []
3833 LOG.warning("Setting extra CPU flags is only valid in "
3834 "combination with a custom CPU model. Refer "
3835 "to the 'nova.conf' documentation for "
3836 "'[libvirt]/cpu_model_extra_flags'")
3837 3818
3838 LOG.debug("CPU mode '%(mode)s' model '%(model)s' was chosen, " 3819 LOG.debug("CPU mode '%(mode)s' model '%(model)s' was chosen, "
3839 "with extra flags: '%(extra_flags)s'", 3820 "with extra flags: '%(extra_flags)s'",
diff --git a/releasenotes/notes/libvirt-cpu-model-extra-flags-a23085f58bd22d27.yaml b/releasenotes/notes/libvirt-cpu-model-extra-flags-a23085f58bd22d27.yaml
index 14156df..9e69275 100644
--- a/releasenotes/notes/libvirt-cpu-model-extra-flags-a23085f58bd22d27.yaml
+++ b/releasenotes/notes/libvirt-cpu-model-extra-flags-a23085f58bd22d27.yaml
@@ -3,9 +3,11 @@ features:
3 - | 3 - |
4 The libvirt driver now allows specifying individual CPU feature 4 The libvirt driver now allows specifying individual CPU feature
5 flags for guests, via a new configuration attribute 5 flags for guests, via a new configuration attribute
6 ``[libvirt]/cpu_model_extra_flags`` -- only with ``custom`` as the 6 ``[libvirt]/cpu_model_extra_flags`` -- this is valid in combination
7 ``[libvirt]/cpu_model``. Refer to its documentation in 7 with all the three possible values for ``[libvirt]/cpu_mode``:
8 ``nova.conf`` for usage details. 8 ``custom``, ``host-model``, or ``host-passthrough``. The
9 ``cpu_model_extra_flags`` also allows specifying multiple CPU flags.
10 Refer to its documentation in ``nova.conf`` for usage details.
9 11
10 One of the motivations for this is to alleviate the performance 12 One of the motivations for this is to alleviate the performance
11 degradation (caused as a result of applying the "Meltdown" CVE 13 degradation (caused as a result of applying the "Meltdown" CVE