Provide original fault message when BFV fails

When booting from volume and Nova is creating the volume,
it can fail (timeout, invalid AZ in Cinder, etc) and the
generic Exception handling in _prep_block_device will log
the original exception trace but then raise a generic
InvalidBDM exception, which is handled higher up and converted
to a BuildAbortException, which is recorded as an instance
fault, but the original error message is lost from the fault.

It would be better to include the original exception message that
triggered the failure so that goes into the fault for debug.

For example, this is a difference of getting an error like this:

  BuildAbortException: Build of instance
  9484f5a7-3198-47ff-b728-178515a26277 aborted:
  Block Device Mapping is Invalid.

To something more useful like this:

  BuildAbortException: Build of instance
  9484f5a7-3198-47ff-b728-178515a26277 aborted:
  Volume da947c97-66c6-4b7e-9ae6-54eb8128bb75 did not finish
  being created even after we waited 3 seconds or 2 attempts.
  And its status is error.

Change-Id: I20a5e8e5e10dd505c1b24c208f919c6550e9d1a4
Closes-Bug: #1693315
(cherry picked from commit 20c4715a49)
(cherry picked from commit 0d5fd4356a)
This commit is contained in:
Matt Riedemann 2017-06-15 11:46:44 -04:00
parent ae45c4a72a
commit 89b6da0c28
2 changed files with 28 additions and 2 deletions

View File

@ -1594,10 +1594,14 @@ class ComputeManager(manager.Manager):
LOG.warning(msg, instance=instance)
raise exception.VolumeLimitExceeded()
except Exception:
except Exception as ex:
LOG.exception(_LE('Instance failed block device setup'),
instance=instance)
raise exception.InvalidBDM()
# InvalidBDM will eventually result in a BuildAbortException when
# booting from volume, and will be recorded as an instance fault.
# Maintain the original exception message which most likely has
# useful details which the standard InvalidBDM error message lacks.
raise exception.InvalidBDM(six.text_type(ex))
def _update_instance_after_spawn(self, context, instance):
instance.power_state = self._get_power_state(context, instance)

View File

@ -4136,6 +4136,28 @@ class ComputeManagerBuildInstanceTestCase(test.NoDBTestCase):
mock_prep.assert_called_once_with(self.context, self.instance,
self.block_device_mapping)
@mock.patch('nova.virt.block_device.attach_block_devices',
side_effect=exception.VolumeNotCreated('oops!'))
def test_prep_block_device_maintain_original_error_message(self,
mock_attach):
"""Tests that when attach_block_devices raises an Exception, the
re-raised InvalidBDM has the original error message which contains
the actual details of the failure.
"""
bdms = objects.BlockDeviceMappingList(
objects=[fake_block_device.fake_bdm_object(
self.context,
dict(source_type='image',
destination_type='volume',
boot_index=0,
image_id=uuids.image_id,
device_name='/dev/vda',
volume_size=1))])
ex = self.assertRaises(exception.InvalidBDM,
self.compute._prep_block_device,
self.context, self.instance, bdms)
self.assertEqual('oops!', six.text_type(ex))
def test_failed_bdm_prep_from_delete_raises_unexpected(self):
with test.nested(
mock.patch.object(self.compute,