In Python 3.11, inspect.getargspec. We shall use getfullargspec from now
on instead. This patch was added to Debian. See Debian bug:
https://bugs.debian.org/1025112
Change-Id: Ib7982470958377d9454446dab326aaa9e782ad69
By default, Hyper-V filters some of the SCSI commands received from
the VMs. This change allows disabling the filter.
Signed-off-by: Stefan Chivu <schivu@cloudbasesolutions.com>
Change-Id: I1b0fc8e7e75a6c5395aedd5b8e94b3bd000e1843
The setDaemon method of the threading.Thread was deprecated
in Python 3.10 (*).
Replace the setDaemon method with the daemon property.
*: https://docs.python.org/3.10/library/threading.html#threading.Thread.setDaemon
Change-Id: I34c7753ae1ba3704a1c2ec1a9b5a9905b6bcb8f5
Signed-off-by: Takashi Natsume <takanattie@gmail.com>
Python3 has deprecated (and recently dropped) collections.Iterable in
favor of using the collections.abc module. This change fixes the import.
Closes-Bug: #1966442
Change-Id: I76ed972890a2fe0bedebd0364276590d5aa90f29
Hyper-V exposes the VHD GUIDs to guests, which can be used to
easily identify attached disks.
This change will allow setting the VHD GUID for new as well as
existing images.
Worth mentioning that the Kubernetes Openstack provider will
benefit from this. Without it, k8s Persistent Volumes can't be
identified by some guests.
http://paste.openstack.org/raw/801011/
Change-Id: Ied73997e6f5f3ded9827703867f059ef3dfca159
The soft shutdown operation is expected to fail while the VM is
still booting. The issue is that in some cases, os-win will
propagate a wmi exception that Nova cannot handle:
http://paste.openstack.org/raw/800833/
That's because of Msvm_ShutdownComponent.InitiateShutdown, an
asynchronous WMI method that can sometimes throw a wmi exception
right away.
For now, we're taking care of this specific WMI call only. If needed,
we may consider handling all async WMI methods by introducing a new
decorator or helper function.
Change-Id: I9263b14c3c343db6453645152cb9274e38864b48
Closes-Bug: #1907270
The Msvm_VirtualSystemSettingData object associated with a VM has a
field named "ChassisAssetTag". The value set in this field is reflected
in Linux VMs in /sys/class/dmi/id/chassis-asset-tag. This value is
checked by cloud-init in order to figure out if it is currently running
inside an OpenStack VM.
The verification above has been introduced in cloud-init [1] in order to
avoid costly metadata probes that aren't needed in non-OpenStack VMs.
Setting the ChassisAssetTag will allow us to pass these checks.
Partially-Fixes: #1895976
[1] 1efa8a0a03
Change-Id: I50dd761b3df9f2d243ca295f02d20422f6af965e
With the removal of Python 2.x we can remove the unittest2 compat
wrappers and switch to assertCountEqual instead of assertItemsEqual
We have been able to use them since then, because
testtools required unittest2, which still included it. With testtools
removing Python 2.7 support [3][4], we will lose support for
assertItemsEqual, so we should switch to use assertCountEqual.
[1] - https://bugs.python.org/issue17866
[2] - https://hg.python.org/cpython/rev/d9921cb6e3cd
[3] - testing-cabal/testtools#286
[4] - testing-cabal/testtools#277^
Change-Id: I4fd20ef857846117b50e8d17fdf98fa4aa117262
While Windows paths are case insensitive, a few os-win checks are
case sensitive and are addressed by this patch.
Change-Id: I60c0d3e12501d7b2f62df8f95a3d3f328de83817
Closes-Bug: #1878588
The monotonic package was needed for monotonic time operations when
running under Python runtimes older than 3.3. Since we now only support
versions higher than this, this third party package requirement can now
be removed.
Change-Id: I809de2f6fe257c80de7c81f9e08499a919f52ff8
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
Now that we no longer support py27, we can use the standard library
unittest.mock module instead of the third party mock lib.
Change-Id: I24c8926bc7a571b45af77c497f0791d101ebbc12
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
This repo is now testing only with Python 3, so let's make
a few cleanups:
- Remove python 2.7 stanza from setup.py
- Add requires on python >= 3.6 to setup.cfg so that pypi and pip
know about the requirement
- Remove obsolete sections from setup.cfg
- Update classifiers
- Switch to using sphinx-build
- Cleanup doc/source/conf.py to remove now obsolete content.
- Use newer openstackdocstheme version
- Remove install_command from tox.ini, the default is fine
- Remove old python compatiblity test code
Change-Id: If7296703fc4848a16dd10d904ba3f49f19054546
The repo is Python 3 now, so update hacking to version 3.0 which
supports Python 3.
Fix problems found.
Update local hacking checks for new flake8.
Change-Id: I7c6896dd25d76dc02145f4d09a4d09de35a4afb1
We've been running the unit tests mostly on Linux as part of the
CI test jobs.
On Windows, some of the tests fail mostly due to the WMI lib not
being mocked correctly. This issue is addressed by this change.
Change-Id: Ib4de3d262a7cf98a552f6234f6561d6c87b2067d
One unit test is using the wrong type, unsigned long long instead of
long, for which reason it will fail on Linux x86 or Windows, where
the size of those types do not match.
Change-Id: If49e57ada8d367f537710974ea59728d20720275
Closes-Bug: 1850770
In some cases, Hyper-V locks on VHD images persist for a few seconds
even after the instances are deleted (not sure if that's due to
Hyper-V or the filesystem). In order to mitigate this, we're
already performing some retries when deleting instance files.
We've encountered similar issues with 3rd party backup tools
that kept open file handles.
This change adds a configurable timeout that will be used when waiting
for in-use fles. The default value is 15 seconds, slightly more than
the current hardcoded value. At the same time, this timeout will
also be used when deleting individual files or renaming folders or
files.
Change-Id: I49307146b79a20089190e1ce8bde73d656d37c6d
After a reboot, the Failover Cluster WMI provider may be unavailable.
This change will wait for it when initializing clusterutils, having
a configurable timeout defaulting to infinite.
Closes-Bug: #1841779
Change-Id: Id7f2d65cb0307bc952f1e6a2724295ea524835f3
Attempting to disconnect a VHD from the local host may fail if the
image is in-use by a different host, even though it's not actually
mounted on the local host.
The reason is that the detach API requires a handle, which we may
fail to obtain if the file is in-use.
This change will try to figure out if the image is actually attached
to the local host before erroring out. This check is a bit more
complicated than it should, again due to file locks. We'll use
two approaches:
* the preffered way, using virtdisk.dll:GetVirtualDiskInformation,
which may fail due to file locks
* the slower way, used as a fallback, which involves getting a full
list of attached images and comparing file and volume ids as
the file may be accessible through multiple paths
Closes-Bug: #1840954
Change-Id: Ib5bf0070c0ba4cb18e98c1cb983c641eaae582bf
Use latest release 1.1.0 and compatible changes w.r.t pep8
It's worth noting that the pep8 package has been renamed to
pycodestyle. New hacking packages no longer install the old pep8
package automatically.
Change-Id: Ibbad8b42188ade597d9a3032389e930c2f5fe2d0
This will expose Windows mutexes. We usually rely on file locks
when synchronizing multiple processes, mostly for portability
reasons.
Some times, file locks may not be an option, in which case a
named mutex can be useful. We intend to use this in Glance when
ensuring that only one scrubber can run at a time.
Change-Id: I4a2a51e2164c6f5816f2af2641d41d1db7870600
This change cleans up the utilsfactory map, making it easier to add
new classes.
Most classes have 6.2 as a minimum version, without having a maximum
version cap. For this reason, this change sets those versions as
defaults so that we don't have to be explicit every time.
At the same time, we're now using a list for each class type, instead
of a dict that uses the class name as a key (which is redundant). So,
something like this:
'networkutils': {
'NetworkUtils': {
'min_version': 6.2,
'max_version': 6.3,
'path': 'os_win.utils.network.networkutils.NetworkUtils'},
'NetworkUtilsR2': {
'min_version': 6.3,
'max_version': None,
'path': 'os_win.utils.network.networkutils.NetworkUtilsR2'}},
... becomes
'networkutils': [
{'max_version': 6.3,
'path': 'os_win.utils.network.networkutils.NetworkUtils'},
{'min_version': 6.3,
'path': 'os_win.utils.network.networkutils.NetworkUtilsR2'},
],
Change-Id: I7919617166b2e872f1b50c0a0f40196fc90e9a82
This change adds a few helpers that will allow creating and consuming
pipes. While similar functions are already provided by the built-in
os module, the handle inherit flags do not work on Windows.
We intend to use those helpers in Glance when dealing with
subprocesses.
Change-Id: I6ed456b5f95b5691da555e36846de4286ad96f15
os-win should silently pass when requested to disconnect a vhd that's missing.
At the moment, it's erroring out, which prevents some Nova instances from being
cleaned up.
This change checks if the image to be disconnected actually exists before
attempting to open it.
Change-Id: Ic734d38e1939165a89a79842d37e98b4cbbcb3bf
Closes-Bug: #1810927
WMI jobs may return multiple errors, but only one of them
would be logged.
This change ensures that we're fetching all of them, calling the
Msvm_ConcreteJob.GetErrorEx method.
Change-Id: I7311e63e4f95c4e1d17357d28fdd1c7425362270
Closes-Bug: 1748208
Due to the known issues related to WMI object caching, it's safer
if we just disable it by default.
Change-Id: If859917af9a185fc612231002d252c9e0ac6ddd3
Related-Bug: #1801701
One cluster listener daemon isn't properly mocked, for which reason
it's running in a separate thread and calling mocked methods,
affecting other test cases and their assertions.
http://paste.openstack.org/raw/734339/
This change mocks it properly, ensuring that other test cases will
not be affected by it.
Change-Id: I2e9d2b98457bac280bf7a19afc12dc822a83eb0f
The Failover Cluster WMI provider is quite unreliable, especially
in recent Windows Server 2016 versions.
After failovers, some valid WMI queries fail with "Invalid Property"
errors, which really looks like a WMI provider bug. Event listeners
are affected as well, for which reason we're missing VM failover events.
This change refactors "clusterutils", switching to the underlying
C library.
The new failover event listener cannot provide the source of the
migrated instances, so we're keeping the old one as well for backwards
compatibility, for now.
The only other places in which we're still using WMI are the methods
that add or destroy VM cluster roles.
The reason is that we'd have to explicitly create the the cluster
resources and group, probably set some resource dependencies and
properties. For now, it's easier/safer to just stick with the WMI
method (which really shouldn't fail).
Also, destroying VM cluster groups using clusapi's DestroyClusterGroup
function behaves strange. VMs get recreated asyncronuously and put in
suspended state, breaking everything. We're avoiding it for now.
Closes-Bug: #1798069
Change-Id: I63d1aa3a6f9fb12d08478eb41fe973b1582b540c
This will allow fetching various VM memory information, such as
dynamic memory limits and reserved amount.
Change-Id: Ifa9649e490a851090cdf3f8a0390e4ab7d5a6373
This change will allow enabling metrics for individual virtual disks.
The Nova Hyper-V driver will use it when attaching Cinder volumes.
Note that Hyper-V does not support collecting metrics for passthrough
disks at the moment.
Change-Id: Iaba32ca94b2c09661f150fa4b12b2c7eb83ff6d9
Partial-Bug: #1794456
We need a way to retrieve VM nic names. This will allow the Nova
Hyper-V driver to enable port metrics when receiving instance power
on events.
It's safer to query Hyper-V directly rather than relying on Nova's port
cache.
Change-Id: Ibe7409b392b91204f0cea2bdd32c86ec03d06b67
Partial-Bug: #1794456
At the moment, port metrics are not enabled. The reason is that we're
using the wrong switch port WMI class.
This issue went more or less unnoticed, especially due to the fact that
we're currently ignoring the according WMI method return code.
This change ensures that port metrics are properly enabled and that an
exception will be raised when failing to do so.
Closes-Bug: #1794291
Change-Id: I8086e7dd9e134749760b05469f5b2c8bcfdcdbde
In a few places, we're using "is" or "is not", when comparing
numeric variables.
This is incorrect and will give unexpected results when the types
do not match (for example, if one of the variables is a long int):
http://paste.openstack.org/raw/730618/
This change fixes those comparisons.
Closes-Bug: #1794068
Change-Id: Ibf28778879050cfdfa82ef4f9b587170dde6203f
We're using a hardcoded 15 seconds timeout when canceling WMI jobs.
Some jobs may not be cancelable, some may take longer to stop. For
example, Nova instances may end up in error state after attempting
to delete them as this hardcoded timeout gets exceeded.
That being considered, we're now adding a config option for this,
also increasing the default timeout to 2 minutes.
Ideally, Nova should pass a timeout here. Still, by adding a config
option at the os-win level, it's easier to backport this to older
releases, also providing a default for any kind of WMI job.
Change-Id: I797366c40a3a4bb74fe925bd10697f227e22915c
Closes-Bug: #1790890
Ghanshyam Mann