Hacking has bumped the version of flake8 that it's using to 5.0 in its
6.0.1 release. This turns up quite a few pep8 errors lurking in our
code. Fix them.
Needed-by: https://review.opendev.org/c/openstack/hacking/+/874516
Change-Id: I3b9c7f9f5de757f818ec358c992ffb0e5f3e310f
Replace six.text_type with str.
A subsequent patch will replace other six.text_type.
Change-Id: I23bb9e539d08f5c6202909054c2dd49b6c7a7a0e
Implements: blueprint six-removal
Signed-off-by: Takashi Natsume <takanattie@gmail.com>
_detect_nbd_devices uses the filter
builtin internally to filter valid devices.
In python 2, filter returns a list. In python 3,
filter returns an iterable or generator function.
This change eagerly converts the result of calling filter
to a list to preserve the python 2 behaviour under python 3.
Closes-Bug: #1840068
Change-Id: I25616c5761ea625a15d725777ae58175651558f8
The call in nova.virt.disk.mount.api.Mount.mnt_dev() to
nova.privsep.fs.mount() should include the `options` argument to
fulfill with the method signature.
The test test_do_mount_need_to_specify_fs_type has been modified to
check that the caller use the correct signature.
Closes-Bug: 1829506
Change-Id: Id14993db6ea33b2da14caa4b58671fc57c182706
Signed-off-by: Miguel Herranz <miguel@midokura.com>
A similar pattern to the others. Maybe we should write a more fully fledged
wrapper around blockdev at some point, but I think its not required yet.
Change-Id: Ie65d7c90abd0f8448500089fa48e831aef7b7abf
blueprint: hurrah-for-privsep
The same pattern as previous patches. Some of these unit tests
are starting to be a bit simpler as we finish the transition.
Change-Id: If0e1fe4c0466f2f88525dc575af2ef366d4bb59d
blueprint: hurrah-for-privsep
Add a new filesytem mounting helper in privsep, and then start
moving things across to it. This currently implements mount and
unmount. We get to cleanup some rmdir calls while we're at it
which is nice as well.
I've added an upgrade note mentioning that we no longer ignore
the value of stderr from mount calls, as requesed in code review.
Change-Id: Ib5e585fa4bfb99617cd3ca983674114d323a3cce
blueprint: hurrah-for-privsep
The i18n team has decided not to translate the logs because it
seems like it not very useful; operators prefer to have them in
English so that they can search for those strings on the internet.
Partially fix on nova/virt other paths will be fixed on next commits
Change-Id: Ie7821aa4a5147cdb0616741bd1a1b1fc22080440
There's a race condition when trying to perform file injection without
libguestfs, which causes a fallback to nbd device. Although the kpartx
command succeeds, it does so after the code has tested for success, so
Nova thinks it failed.
Retry a few times to avoid this.
Co-Authored-By: Paul Carlton <paul.carlton2@hp.com>
Change-Id: Ie5c186562475cd56c55520ad7123f47a0130b2a4
Closes-Bug: #1428639
Closes-Bug: #1484586
Add options from 'virt.disk'. These options are part of the
'DEFAULT' group but are included in the "nova.conf.virt" file in hope
that they can eventually be moved to their own group.
Change-Id: Ie69f5bef5f65c7ffe42adeff1defc8e86649bdba
Implements: bp centralize-config-options
In some modules the global LOG is not used any more. And the import
of logging is not used. This patch removes the unused logging import
and LOG vars.
Change-Id: I28572c325f8c31ff38161010047bba00c5d5b833
Block backed images, are already presented as block devices, so
LocalBlockImages can be mounted directly instead of being linked
with loopback or nbd to another block device.
Change-Id: I5aae1e293832f7cff7bc00d793bcc86ededf5eec
Closes-bug: #1287754
Convert the nova.virt.disk.mount classes to use the
nova.virt.image.model classes instead of manually
passing a filename and image format as parameters.
Related-Bug: #1257674
Change-Id: Ibabcf48c95185ba443ef7575590eae6a63e0f8e5
Convert the use of the incubated version of the log module
to the new oslo.log library.
Sync oslo-incubator modules to update their imports as well.
Co-Authored-By: Doug Hellmann <doug@doughellmann.com>
Change-Id: Ic4932e3f58191869c30bd07a010a6e9fdcb2a12c
The oslo team is recommending everyone to switch to the
non-namespaced versions of libraries. Updating the hacking
rule to include a check to prevent oslo.* import from
creeping back in.
This commit includes:
- using oslo_utils instead of oslo.utils
- using oslo_serialization instead of oslo.serialization
- using oslo_db instead of oslo.db
- using oslo_i18n instead of oslo.i18n
- using oslo_middleware instead of oslo.middleware
- using oslo_config instead of oslo.config
- using oslo_messaging instead of "from oslo import messaging"
- using oslo_vmware instead of oslo.vmware
Change-Id: I3e2eb147b321ce3e928817b62abcb7d023c5f13f
oslo.i18n uses different marker functions to separate the
translatable messages into different catalogs, which the translation
teams can prioritize translating. For details, please refer to:
http://docs.openstack.org/developer/oslo.i18n/guidelines.html#guidelines-for-use-in-openstack
There were not marker fuctions some places in directory network.
This commit makes changes:
* Add missing marker functions
* Use ',' instead of '%' while adding variables to log messages
Change-Id: I913077d3b0fdee78e423c35b3a48137a17946a7b
oslo.i18n uses different marker functions to separate the
translatable messages into different catalogs, which the translation
teams can prioritize translating. For details, please refer to:
http://docs.openstack.org/developer/oslo.i18n/guidelines.html#guidelines-for-use-in-openstack
There were not marker fuctions some places in directory network.
This commit makes changes:
* Add missing marker functions
* Use ',' instead of '%' while adding variables to log messages
Added a hacking rule for the log info about checking
translation for it.
Change-Id: I96766d723b01082339876ed94bbaa77783322b8c
oslo.i18n uses different marker functions to separate the
translatable messages into different catalogs, which the translation
teams can prioritize translating. For details, please refer to:
http://docs.openstack.org/developer/oslo.i18n/guidelines.html#guidelines-for-use-in-openstack
There were not marker fuctions some places in directory network.
This commit makes changes:
* Add missing marker functions
* Use ',' instead of '%' while adding variables to log messages
Change-Id: I5a8f381b6f8fdb4e8febe9e6a901f7cdc6846646
oslo.utils library now provides the functionality previously in
oslo-incubator's excutils, importutils, network_utils, strutils
timeutils, units etc. Some modules already moved to oslo.utils
will still be around since other code in nova/openstack/common/
are using it and will be removed in a subsequent commit.
Change-Id: Idc716342535fdfa680963e0e073ddb46f5f1eb34
oslo.i18n provides the i18n functions that were provided by
oslo-incubator's gettextutils module. Some tests that were
using internal details of the library were removed.
Change-Id: I44cfd5552e0dd86af21073419d31622f5fdb28e0
Our translation policy
(https://wiki.openstack.org/wiki/LoggingStandards#Log_Translation) calls
for not translating debug level logs. This is to help prioritize log
translation. Furthermore translation has a performance overhead, even if
the log isn't used (since nova doesn't support lazy translation yet).
Change-Id: I524b48f530d8afd59a067074332e3964426e4d70
o nova/cells/messaging.py
- def _at_max_hop_count(self, do_raise=True):
This method was added in the following commit.
commit f9a868e86c
Author: Chris Behrens <cbehrens@codestud.com>
Date: Fri Apr 13 05:54:48 2012 +0000
Cells: Add the main code.
But there is no evidence that was used.
o nova/virt/disk/mount/nbd.py
- def _read_pid_file(self, pidfile):
This method was added in the following commit.
commit e4377fdb0e
Author: Michael Still <michael.still@canonical.com>
Date: Wed Dec 12 17:13:53 2012 +1100
Stop nbd leaks, remove pid race.
But there is no evidence that was used.
o nova/virt/hyperv/imagecache.py
- def _validate_vhd_image(self, vhd_path):
This method was removed in the following commit.
commit 689e9e3e3d
Author: Alessandro Pilotti <ap@pilotti.it>
Date: Mon Mar 4 11:34:42 2013 +0200
Fixes disk size issue during image boot on Hyper-V
o nova/virt/hyperv/vmutils.py
- def _clone_wmi_obj(self, wmi_class, wmi_obj):
This method was removed in the following commit.
commit d1a2e10869
Author: Alessandro Pilotti <apilotti@cloudbasesolutions.com>
Date: Tue Jul 30 03:58:11 2013 +0300
Improves Hyper-V vmutils module for subclassing
o nova/virt/libvirt/firewall.py
- def _define_filters(self, filter_name, filter_children):
This method was removed in the following commit.
commit b4fae4821f
Author: Yaguang Tang <heut2008@gmail.com>
Date: Tue Feb 14 10:43:39 2012 +0800
remove unused nwfilter methods and tests.
o nova/virt/xenapi/volume_utils.py
- def _get_volume_id(path_or_id):
This method was removed in the following commit.
commit 503d572954
Author: Armando Migliaccio <amigliaccio@internap.com>
Date: Fri Dec 7 21:06:43 2012 +0000
Fixes KeyError: 'sr_uuid' when booting from volume on xenapi
Change-Id: I216f7c4ff3546c41e1fb0813a71a6c7ff9498584
We don't need to have the vi modelines in each source file,
it can be set in a user's vimrc if required.
Also a check is added to hacking to detect if they are re-added.
Change-Id: I347307a5145b2760c69085b6ca850d6a9137ffc6
Closes-Bug: #1229324
/sys/block/*device*/pid check is not enough. I see that the unix
socket used by the device my be stuck as well, so let's add another
check for the path to the unix socket for the device as well to
figure out if the device is free. Complain loud and clear that the
qemu-nbd is leaking resources.
Change-Id: I28cedffba7a9915ef6f7888989e40e4d0de475c6
Closes-Bug: #973413
Partial-Bug: #1254890
These were initially noted during import into the manuals at
https://review.openstack.org/67502
Fixes include rework of text, missing or extra spaces, wrong
capitalization and no sentence style capitalization.
A hacking check has been added that tests that all config options
start with a capital letter.
Change-Id: I0a1c333bbb3a812932d3972a7403f94da0bff341
Co-Authored-By: Daniel Berrange <berrange@redhat.com>
Co-Authored-By: Diane Fleming <diane.fleming@rackspace.com>
NBD device once used seem to run into intermittent trouble when
used with mount repeatedly. Adding code to explicitly flush the
device buffers using 'blockdev --flushbufs'.
Closes-Bug: #973413
Partial-Bug: #1254890
Change-Id: I2b7053b9a069d6e82f6f6baf9ad480efa4388d91
Avoid the situation where two local threads choose the same nbd number
for injecting files into the instance. If this happened quickly enough
nova was left with one qemu-nbd hanging and blocking the device forever.
Fixes bug 1207422
Change-Id: I18864b19ebd30669534e45ca7a50e12f61207302
Previous _ was monkey patched into builtins whenever
certain modules were imported. This removes that and
simply imports it when it is needed.
Change-Id: I0af2c6d8a230e94440d655d13cab9107ac20d13c
When the 'mount' operation is successful but stderr is not empty during
injecting key/file into image, there may be some warning messages output to
stderr, so we need to discard them and continue to inject files into image.
Fixes bug #1152519
Change-Id: Icc7d15196f3ee67674f9d815ddeef4378f363e2c
The cfg API is now available via the oslo-config library, so switch to
it and remove the copied-and-pasted version.
Add the 2013.1b4 tarball to tools/pip-requires - this will be changed
to 'oslo-config>=2013.1' when oslo-config is published to pypi. This
will happen in time for grizzly final.
Add dependency_links to setup.py so that oslo-config can be installed
from the tarball URL specified in pip-requires.
Remove the 'deps = pep8==1.3.3' from tox.ini as it means all the other
deps get installed with easy_install which can't install oslo-config
from the URL.
Make tools/hacking.py include oslo in IMPORT_EXCEPTIONS like it already
does for paste. It turns out imp.find_module() doesn't correct handle
namespace packages.
Retain dummy cfg.py file until keystoneclient middleware has been
updated (I18c450174277c8e2d15ed93879da6cd92074c27a).
Change-Id: I4815aeb8a9341a31a250e920157f15ee15cfc5bc
5f697f64e5 introduced a regression
when using lxc containers with qcow2 and qemu-nbd.
When removing the rootfs from the host namespace, it would
terminate the qemu-nbd process as well. This would cause
the kernel too oops under Ubuntu 13.04. Since the underlying
device thtat the libvirt_lxc process disapears.
To get around this we just clean up the host namespace if the
instance is powered-on. When the instance terminates it will
teardown the whole container.
This fixes LP: #1115786
Change-Id: I98bec2338cb455dbd277295ab36767149e05634c
Signed-off-by: Chuck Short <chuck.short@canonical.com>
Fix N402 errors (single line docstring should end in a period)
for nova/virt, part of a larger attempt to stop ignoring our own
hacking.py tests.
Change-Id: I523ce41bd2b38c73cf3fdb031101ccc0695f2488
Other device implementations require retry logic as well. This
change pushes the retry logic up one layer so that its easy to
implement for the other drivers that will benefit from it, and
then adds retry to the loop driver.
This change also adds some unit test coverage for the loop
driver.
Change-Id: Iab0d42d5075e9d50b4e7eb8c7fcef12cae281b40
With the previous implementation, if a nbd device was found to be in
use by something other than nova it was removed from the class scoped
list of devices, but never re-added. This meant we "leaked" devices
away over time if we were competing with other nbd users on the
machine.
Instead of tracking our use at all, we should rely on the presence of
a user pid in /sys.
Resolves bug 1088083.
Change-Id: If777e270a0dda12034ea7ef1bc7fd688cadde8a9
This means we don't need to the max_nbd_devices flag any more at all.
This patch also moves the internal representation of the list of
available nbd devices to not including the leading /dev/, but that is
not exposed as a change in the interface to the class.
Resolves bug 861504 and its duplicate 1088339.
DocImpact.
Change-Id: I5c6218122ab09781d8a2ecd6d8b76a74be4a6e91
We need to wait for the config parser to have been run before we
query the value of the max_nbd_devices flag. We will always get
the default value with the previous implementation.
Based on a conversation with Robert Collins. Resolves bug 1088073.
Change-Id: If7b5d6a5c7d8153530e97532aad67c42e744ffec
To assist in troubleshooting file injection code, add some
LOG.debug() statements to the nova/virt/disk/mount/*.py
modules
blueprint: virt-disk-api-refactoring
Change-Id: I75214145ef489a29a58f69abe6195ff01f36153c
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
The nova.virt.disk.mount.guestfs.API impl is now unused,
since there is a dedicated libguestfs impl of the VFS
APIs. It can thus be removed
blueprint: virt-disk-api-refactoring
Change-Id: I6c19e38f23b056d770bfb93e21fa3e64ac179a42
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Now that the only disk code requiring host filesystem
mounts is the LXC filesystem setup, the img_handlers
config parameter is an solution looking for a purpose.
For any disk image format we can categorically say
whether it should be mounted using loop or qemu-nbd
mount impls. There is no desire to use libguestfs FUSE
for mounting LXC disk images, since users of LXC do
not want a huge KVM process alongside every container!
Move the mount class choice logic into the mount API,
by creating new methods
nova.virt.disk.mount.API.instance_for_format()
nova.virt.disk.mount.API.instance_for_device()
and then call these from nova.virt.disk.API code
blueprint: virt-disk-api-refactoring
Change-Id: I088b5debdf62d4a9a6b4521eee68cd4757f2ff42
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Modules import nova.config for two reasons right now - firstly, to
reference nova.config.CONF and, secondly, if they use one of the
options defined in nova.config.
Often modules import nova.openstack.common.cfg and nova.config
which is a bit pointless since they could just use cfg.CONF if
they just want to nova.config in order to reference CONF.
Let's just use cfg.CONF everywhere and we can explicitly state
where we actually require options defined in nova.config.
Change-Id: Ie4184a74e3e78c99658becb18dce1c2087e450bb
Artom Lifshitz