This chnage adds the pre-commit config and
tox targets to run codespell both indepenetly
and via the pep8 target.
This change correct all the final typos in the
codebase as detected by codespell.
Change-Id: Ic4fb5b3a5559bc3c43aca0a39edc0885da58eaa2
Based on the Libvirt and QEMU versions, two traits,
COMPUTE_ADDRESS_SPACE_PASSTHROUGH and COMPUTE_ADDRESS_SPACE_EMULATED,
are controlled. Since the two are supported from the same version on
the Libvirt and QEMU, Nova handles them in the same way.
Blueprint: libvirt-maxphysaddr-support
Depends-On: https://review.opendev.org/c/openstack/os-traits/+/871226
Signed-off-by: Nobuhiro MIKI <nmiki@yahoo-corp.jp>
Change-Id: If6c7169b7b8f43ad15a8992831824fb546e85aab
Once again, the CI has disappeared for this. It doesn't appear to be a
priority for VMWare and we'd be lying to users if we suggested
otherwise. Mark the driver as experimental. We can explore deprecating
it down the line once it starts costing us.
Change-Id: I48d9812851449768797bbe799b3fe84fa2ce6d02
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
The vmwareapi driver uses Managed-Object references throughout the code
with the assumption that they are stable. It is however a database id,
which may change during the runtime of the compute node. e.g. If an
instance is unregistered and re-registerd in the vcenter, the moref will
change. By wrapping a moref in a proxy object, with an additional method
to resolve the openstack object to a moref, we can hide those changes
from a caller.
MoRef implementation with closure - should ease the transition to stable
mo-refs One simply has to pass the search function as a closure to the
MoRef instance, and the very same method will be called when an
exception is raised for the stored reference.
Stable Volume refs - The connection_info['data'] contains the
managed-object reference (moref) as well as the uuid of the volume.
When the moref become invalid for some reason, we can recover it by
searching for the volume-uuid as the `config.instanceUuid` attribute
of the shadow-vm.
Stable VM Ref - By encapsulating all the parameters for searching for
the vm-ref again, we can move the retry logic to the session object,
where we can try to recover the vm-ref should it result in a
ManagedObjectNotFound exception.
Use refs as index for fakedb - It was previously using the object-id
to lookup an object, meaning that you couldn't pass a newly created
Managed-object-reference like you could over the vmware-api. Now the
lookup happens over the ref-id string, and in turn some functions
were refactored to take that into account.
Partial-Bug: #1962771
Change-Id: I2a3ddf95b7fe07630855b06e732f8764efb13e91
The VMwareAPISession object is not only used by the driver, but in
practically all modules of vmwareapi. It reduces a bit the scope of
the driver module itself.
Partial-Bug: #1962771
Change-Id: I4094b6031872bd3b5c871b9a82c7e01280a3352d
If instance memory is not multiple of 4, creating instance on ESXi
fails with error "['GenericVmConfigFault'] VimFaultException: Memory
(RAM) size is invalid.". However this is after instance is built and
tried to launch on ESXi. Add check in prepare_for_spawn to trigger
failure early and avoid further steps i.e. build as well as launch.
Closes-Bug: #1966987
Change-Id: I7ed8ac986283cd455e54e3f18ab955f43b3248d0
vSphere 6.5 introduced APIs to manage virtual disks (volumes)
as first class objects. The new managed disk entity is called
VStorageObject aka First Class Disk (FCD). Adding support for
volumes backed by VStorageObject.
Change-Id: I4a5a9d3537dc175508f0a0fd82507c498737d1a5
In order to support remote-managed ports the following is needed:
* Nova compute driver needs to support this feature;
* For the Libvirt compute driver, a given host needs to have the right
version of Libvirt - the one which supports PCI VPD (7.9.0
https://libvirt.org/news.html#v7-9-0-2021-11-01).
Therefore, this change introduces a new capability to track driver
support for remote-managed ports.
Change-Id: I7ea96fd85d2607e0af0f6918b0b45c58e8bec058
The regex isn't used anywhere, and uses an unescaped format
The last use was removed in 2f7403b
(Change-Id: Ifc17c5049e3ed29c8dd130339207907b00433960).
Change-Id: I76aaf133af517eb70fcaf3783953625c63141083
autopep8 is a code formating tool that makes python code pep8
compliant without changing everything. Unlike black it will
not radically change all code and the primary change to the
existing codebase is adding a new line after class level doc strings.
This change adds a new tox autopep8 env to manually run it on your
code before you submit a patch, it also adds autopep8 to pre-commit
so if you use pre-commit it will do it for you automatically.
This change runs autopep8 in diff mode with --exit-code in the pep8
tox env so it will fail if autopep8 would modify your code if run
in in-place mode. This allows use to gate on autopep8 not modifying
patches that are submited. This will ensure authorship of patches is
maintianed.
The intent of this change is to save the large amount of time we spend
on ensuring style guidlines are followed automatically to make it
simpler for both new and old contibutors to work on nova and save
time and effort for all involved.
Change-Id: Idd618d634cc70ae8d58fab32f322e75bfabefb9d
Instead of retrieving the hardware devices
through a get_object_property directly,
and then inconsistently handling the normalisation
in the called function or in the caller,
use get_hardware_devices everywhere and handle
the normalisation there.
Change-Id: I221013216c4bdd241a894fb0a68c95d98d649c33
In various places, own version of iterating over the results are implemented,
sometimes even faulty.
The following functions where only getting up to vmware.maximum_objects objects (100 by default)
vm_util.get_all_cluster_mors, vm_util.get_stats_from_cluster.
Previously, the results were fetched in batches of up to vmware.maximum_objects items.
Using WithRetrieval yields an iterator to the results, which pages transparently to
the next request.
Consumers of the output of the results where changed to work on an iterator, where easily
possible.
Replaced the quadratic algorithm in `ds_util._filter_datastores_matching_storage_policy`
with one of O(n log(n)) runtime
Closes-Bug: #1940399
Change-Id: I8283c3e76c595cb32527d1b8745933d044e22734
This change adds a simple optional kwarg to the virt driver signature of
destroy and cleanup to allow for callers to control when secrets should
be removed.
Change-Id: I856268b371f7ba712b02189db3c927cd762a4dc3
With switching the SOAP library backing oslo.vmware [1], the internal
representation of ManagedObjectReference's attributes changes. To be able
to make the switch without interruption, we introduced helper functions
in oslo.vmware. This commit uses one of those - get_moref_value()
- to make the access to the "value" attribute compatible with both
backing libraries.
Bump oslo.vmware lower-constraint to a version containing the helper
function. We bump one version higher, because we already merged a patch
(I3c769d3499f906b33725171a57313c8ae35b6a1d) that uses the cookiejar
access contained in that version.
[1] https://specs.openstack.org/openstack/oslo-specs/specs/victoria/oslo-vmware-soap-library-switch.html
Change-Id: I48841a777aaa2e515c141857099c16483ab619e0
We want to switch the SOAP library backing oslo.vmware [1] and the new
library differs in how empty lists are returned. The old library SUDS
didn't set attributes if they were empty lists. The new library sets
those attributes to empty lists.
Therefore, every check with `hasattr()` that later on accesses a single
element instead of iterating over the whole list needs to be changed to
also check that the list contains elements. This ensures compatiblity
between the backend library versions.
[1] https://specs.openstack.org/openstack/oslo-specs/specs/victoria/oslo-vmware-soap-library-switch.html
Change-Id: I9322037cd47ace83fbcb19bbbe051e2feb9af2c7
With changing the SOAP library backing oslo.vmware [1], boolean types
will be represented as such in the generated XML while the previous SOAP
library always generated strings. For OptionValue objects, the vCenter
complains about an InvalidArgument when we send a boolean instead of a
string for the value attribute. Therefore, we change all occurrences to
strings.
[1] https://specs.openstack.org/openstack/oslo-specs/specs/victoria/oslo-vmware-soap-library-switch.html
Change-Id: I7659a48f0eeb28cb396e8476a3e2c4b7f027c0bb
In some cases the logical switch is not ready when the VM is booted,
so retry can help avoid the error.
Closes-Bug: #1908408
Change-Id: Id6e2be2b407eb08e269c35e0f89d235c67a1ab99
When a VM is storage-vMotioned, the name of the folder its files are in
can change. Then, a rescue command trying to put the rescue disk into a
folder named after the instance's UUID cannot work anymore and actually
raises a FileNotFoundException for the directory path.
To fix this, we now take the root VMDK's folder and copy the rescue
image into that.
Change-Id: Icef785b96e51942e7bac2df10c116078c77fedc4
Closes-Bug: #1915815
There are a bunch of unused parameters in DynamicVendorData constructor
and a comment that they cannot be removed due to JsonFileVendorData. But
JsonFileVendorData does not depends on those paramters and both the
base class and JsonFileVendorData uses *args **kwargs. So it is safe to
remove the unused params.
The context field of DynamicVendorData is also removed as it is unused.
This makes the request_context parameter of the InstanceMeta
constructor also unused so that is removed.
Change-Id: Ie27fd6a5513e53903b9acd5d63038b3b484acbde
With changing the SOAP library backing oslo.vmware [1], the cookiejar of
a session must be accessed differently. Therefore, oslo.vmware
introduced a property on the client to abstract this change away. This
commit uses the new place to access the attribute.
[1] https://specs.openstack.org/openstack/oslo-specs/specs/victoria/oslo-vmware-soap-library-switch.html
Change-Id: I3c769d3499f906b33725171a57313c8ae35b6a1d
These options were deprecated way back in Rocky due to buggy behavior
they introduced. We can remove them now.
Change-Id: I9266edfd4ea6315239c54ff8d91e37d197c760c0
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
When an attempt to delete an instance doesn't succeed and nova retries
on the next nova-compute restart, an instance not existing in the back
end anymore can lead to an uncatched exception in the vmware driver
prohibiting instance deletion. This is the case, if the instance had
volumes attached, because `_detach_instance_volumes()` always powers off
the instance - which cannot work if the instance doesn't exist anymore.
While the code already catched `ManagedObjectNotFoundException`, it also
needs to catch `InstanceNotFound` raised by `vm_util.get_vm_ref()` to
complete the deletion as seen in the traceback below (which comes from a
queens codebase):
Traceback (most recent call last):
File "/nova/compute/manager.py", line 874, in _init_instance
self._delete_instance(context, instance, bdms)
File "/nova/hooks.py", line 154, in inner
rv = f(*args, **kwargs)
File "/nova/compute/manager.py", line 2500, in _delete_instance
self._shutdown_instance(context, instance, bdms)
File "/nova/compute/manager.py", line 2392, in _shutdown_instance
requested_networks)
File "/lib/python2.7/site-packages/oslo_utils/excutils.py", line 220, in __exit__
self.force_reraise()
File "/lib/python2.7/site-packages/oslo_utils/excutils.py", line 196, in force_reraise
six.reraise(self.type_, self.value, self.tb)
File "/nova/compute/manager.py", line 2379, in _shutdown_instance
block_device_info)
File "/nova/virt/vmwareapi/driver.py", line 574, in destroy
self._detach_instance_volumes(instance, block_device_info)
File "/nova/virt/vmwareapi/driver.py", line 536, in _detach_instance_volumes
self._vmops.power_off(instance)
File "/nova/virt/vmwareapi/vmops.py", line 1762, in power_off
vm_util.power_off_instance(self._session, instance)
File "/nova/virt/vmwareapi/vm_util.py", line 1732, in power_off_instance
vm_ref = get_vm_ref(session, instance)
File "/nova/virt/vmwareapi/vm_util.py", line 171, in wrapper
return _vm_ref_cache(id, func, session, instance)
File "/nova/virt/vmwareapi/vm_util.py", line 162, in _vm_ref_cache
vm_ref = func(session, data)
File "/nova/virt/vmwareapi/vm_util.py", line 1214, in get_vm_ref
raise exception.InstanceNotFound(instance_id=uuid)
InstanceNotFound: Instance 2af34cc5-22e0-400c-8b80-f130e86027fd could not be found.
Change-Id: I65d2f76068e4b033ffd20959c9e74c870c8aa8e0
In vSphere 7.0, the VirtualDevice.key cannot be the same any more.
So set different values to VirtualDevice.key
Change-Id: I574ed88729d2f0760ea4065cc0e542eea8d20cc2
Closes-Bug: #1892961
This is caused by the "/" operation difference between python 2.x and python 3.x
In python 2.x "/" int / int returns int while
in python 3.x int / int return the real result.
Change-Id: I1aa96bdc2fdb2ca856b73b91e3120c75ec104537
In a future change, we'll want access to this so that it's possible to
retrieve vTPM data during the unrescue operation. While we're here, it
seems nothing is using the 'network_info' argument anymore, presumably
since the demise of nova-network, and this can and should be dropped.
Resolve both issues in one go, adding the 'context' parameter, dropping
the 'network_info' one, and updating the various callers and tests for
same.
Maintainers of out-of-tree drivers have been notified of these changes
[1].
[1] http://lists.openstack.org/pipermail/openstack-discuss/2020-July/015824.html
Part of blueprint add-emulated-virtual-tpm
Change-Id: Id5e4b0f26d5a2a93db6a7d96555a2cff29d9a2cf
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
We neither need nor want these translated so remove them. A couple of
logs with unnecessary brackets or weird indentation were identified in
the process, and these are also modified. Note that exceptions must
still be translated. Refer to [1] for more details.
[1] https://docs.openstack.org/oslo.i18n/latest/user/guidelines.html
Change-Id: I4573a7c5f3a7b5716a15fbd15ad9336807843a03
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Some features like vGPUs require to have allocations to be checked locally
by the virt driver but resize (and cold migrate) was not passing them.
This change just gives the allocations to the target node when the resize
is done by adding the allocations to the finish_migration() interface.
Related-Bug: #1778563
Change-Id: Ifd76289964d513e9544544e5787f5f8999589475
This change adds block_device_info as an argument to the rescue method
signature for all virt divers currently implementing instance rescue.
This will allow future changes to provide all disks during a stable
device rescue attempt.
Change-Id: I9c2d9013d741774e521021913ec04f37b310e9d4
. Do not delete accelerator requests in stop code paths.
. In the start code path, get the list of accelerator requests from
Cyborg in the compute manager 'power_on'.
. Pass accel_info (said list) to the virt driver power_on.
. In libvirt driver, pass that accel_info to driver power_on.
Change-Id: I8c94504b87aa4450d163fe2b33f6aa0eb5dae5ff
Blueprint: nova-cyborg-interaction
Update the signature of the spawn() API for each virt driver
to include accel_info, which is a list of accelerator requests.
Change-Id: I4aac66c125a162bf35991a7d0c2638c7475ec0e7
Blueprint: nova-cyborg-interaction
As of now, questions on the mailing list are going unanswered, and the
Nova team does not have a clear representative owner for the driver
to which bugs and other reports can be directed. There does not appear
to be a CI system running tests for the driver anymore, and the latest
indication from the community[1] points to it being potentially broken with
devstack.
This patch starts the deprecation timer for the driver and/or serves
as a flare to gauge interest (or lack thereof) in continuing to maintain
the driver.
1: http://lists.openstack.org/pipermail/openstack-discuss/2020-March/013066.html
Change-Id: Ie39e9605dc8cebff3795a29ea91dc08ee64a21eb
This doesn't exist for 'nova.volume' and no longer exists for
'nova.network'. There's only one image backend we support, so do like
we've done elsewhere and just use 'nova.image.glance'.
Change-Id: I7ca7d8a92dfbc7c8d0ee2f9e660eabaa7e220e2a
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
At some point in the past, there was only nova-network and its code
could be found in 'nova.network'. Neutron was added and eventually found
itself (mostly!) in the 'nova.network.neutronv2' submodule. With
nova-network now gone, we can remove one layer of indirection and move
the code from 'nova.network.neutronv2' back up to 'nova.network',
mirroring what we did with the old nova-volume code way back in 2012
[1]. To ensure people don't get nova-network and 'nova.network'
confused, 'neutron' is retained in filenames.
[1] https://review.opendev.org/#/c/14731/
Change-Id: I329f0fd589a4b2e0426485f09f6782f94275cc07
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
Sean Mooney