As part of the refactor for the libvirt volume drivers, move the
volume-specific modules under a new 'volume' subdirectory.
The next step will be breaking the volume drivers out into their
own specific modules.
Part of blueprint consolidate-libvirt-fs-volume-drivers
Change-Id: I5fc2425d2c25076ea87686b2e41be35f66ebb923
This patch changes the internals of some
of the libvirt volume drivers to use the
os-brick Connector objects. Cinder already
uses os-brick for volume discovery and removal
for copy volume to image and image to volume
operations.
This patch changes the following libvirt volume drivers:
LibvirtISCSIVolumeDriver
LibvirtISERVolumeDriver
LibvirtAOEVolumeDriver
LibvirtFibreChannelVolumeDriver
This patch also removes the need to have the
nova/storage module that was used by the above listed
libvirt volume drivers.
This patch also fetches the initiator side information
from os-brick. This replaces the internals of the
libvirt driver's get_volume_connector
Also updated the rootwrap filters to consolidate them under
a single comment, and added a new os-brick needed command.
blueprint use-os-brick-library
Change-Id: I400db60fcc29c2d5e2d3b9dabc055649138468eb
Depends-On: Id36f9665c8ff2a720713ceaaa5b05f9b03706681
Changeset ed9f1b8b restricted the allowed options for iscsi_iface
to the allowed transports mentioned in the help text. iface &
transport are distinct entities for open-iscsi, and an iface is
generally a mix of transport plus an underlying interface, hence
this change was incorrect. This breaks transport support and is
generally incorrect, hence reverting
Closes-Bug:1466730
Change-Id: I687fb9c95133dbbb193004df44794404b58ca692
Modules eventlet_backdoor, loopingcall, periodic_task,
service, sslutils, systemd, threadgroup were removed
from nova. These modules were imported from oslo.service
library.
Co-Authored-By: Marian Horban <mhorban@mirantis.com>
Depends-On: I305cf53bad6213c151395e93d656b53a8a28e1db
Change-Id: Iaef67e16af3d69f845742f7bdcb43667bf1576ee
The oslo_config library supports the choices keyword argument. This commit
leverages the use of choices for StrOpts in Nova's configuration.
This change also adds 'ploop' as a valid value for the images_type option.
This was previously a valid value, but not documented.
Closes-Bug: #1423973
Change-Id: I4c0eb1366cc8a1b001fd403b67c3eec11f831541
With this change, "tox -e py34" now pass.
Changes:
* Replace unichr() with six.unichr()
* Replace StringIO.StringIO() with six.Bytes() in
nova.crypto.generate_key_pair()
* On Python 3, replace UserDict.IterableUserDict with
collections.UserDict
* Replace __builtin__.open with six.moves.builtins.open
* Replace ConfigParser import with six.moves.configparser
* On Python 3, get the original "queue" module instead of the "Queue"
module in nova/virt/libvirt/host.py
* Replace urllib2 with six.moves.urllib
* tox.ini: "tox -e py34" now only runs nova.tests.unit.test_versions
Blueprint nova-python3
Change-Id: I056769c7c5b32276894f7aade8c0a27af81c42ae
This patch was generated by the sixer tool version 0.2 using the
"iteritems" operation:
https://pypi.python.org/pypi/sixer
Manual changes:
- Don't change get_instance_metadata() in nova/compute/api.py:
fixed by the change Ifd455e70002eb9636b87f83788384127ba6edeeb.
- Don't change sqlalchemy code and
nova/tests/unit/db/test_db_api.py. sqlalchemy objects cannot be
converted to a dictionary using dict(obj) directly yet. It will be
possible with the change I702be362a58155a28482e733e60539d36c039509.
- Revert change in a comment in nova/objects/instance.py; the sixer tool
is limited and don't understand comments
- Reformat nova/virt/vmwareapi/driver.py to respect the 80 columns
contraint
Blueprint nova-python3
Change-Id: I81465661cb8a74778d70ba9b6641073f1effa49b
terminating instance that attached more than one volume, disconnect
the first volume is ok, but the first volume is not removed, then
disconnect the second volume, disconnect_volume_multipath_iscsi
will call _rescan_iscsi so that rescan the first device, although
the instance is destroyed, the first device is residual, therefor
we don't need rescan when disconnect volume.
Change-Id: I7f2c688aba9e69afaf370b2badc86a2bb3ee899d
Closes-Bug:#1402535
During disconnect_volume in LibvirtFibreChannelVolumeDriver there is
a chance that find_multipath_device behaves badly and returns None.
When this happens we end up with a failure as we don't check the
return value from find_multipath_device.
Co-Authored-By: Melanie Witt <melwitt@yahoo-inc.com>
Closes-Bug: #1424968
Change-Id: I5a5c7e26b70df237c7446efe8f99ce2304c41ab4
The Libvirt SMB volume driver fails to attach volumes if the
connection info does not contain mount options.
The reason is that the mount options are not parsed correctly.
It attempts to join the connection info mount options (which in
this case are set to None) with the locally configured mount
options.
This patch handles the case when the mount options are set to None.
Change-Id: I3166ef5420409ad45cf64f04e6c1d30b6c674d20
Closes-Bug: #1437389
Convert the use of the incubated version of the log module
to the new oslo.log library.
Sync oslo-incubator modules to update their imports as well.
Co-Authored-By: Doug Hellmann <doug@doughellmann.com>
Change-Id: Ic4932e3f58191869c30bd07a010a6e9fdcb2a12c
There are some platform specific changes needed to allow Nova to support
System z: the System z specific format of the device file paths, adding
and removing devices explicitely, and ignoring vHBAs which are offline.
Partial-Implements blueprint libvirt-kvm-systemz
Change-Id: Ica4938d9962bbb402e26dd47f0aea502ea0c4245
Looking at the s390 fc support patch, this function seemed overly
nested. I think it's better if we isolate the matrix computation
first, so that when we are doing real work we're looping over 1 pre
computed set of device tuples, instead of multiple nested for loops.
Change-Id: Ic5111c76cc976a873a6a816ac7163ab8dd4579a1
We actually need transport_iface, not transport as the param
(iscsi_tcp & iser are exceptions to this, where transport and
transport_iface are one and the same), so change it so that user is
not confused as to what must be provided. Also fixes a typo in
param help section.
Also added code to ensure that non-existing transport_iface is not
provided, in which case we we log a warning and fall back to the
default instead of failing as would have previously happened.
There is no change in functionality, this will just ensure that
documentation is not misleading.
Originally added in commit 554647a4de.
DocImpact - due to the renamed config option
Closes-Bug: #1420971
Change-Id: I6c190328803e8efcde34d522e2c1814ef6afc220
Driver for Quobyte storage product.
Mounts respective Quobyte volume of VM images.
This is the Nova part of integrating Quobyte.
Corresponding Cinder Driver Blueprint:
https://blueprints.launchpad.net/cinder/+spec/quobyte-usp-driver
The Cinder driver has been merged into Cinder,
it's review can be found at:
https://review.openstack.org/#/c/94186/
blueprint quobyte-nova-driver
DocImpact: Adding option 'quobyte_mount_point_base' and 'quobyte_client_cfg'
Change-Id: Ica1820031f1fc8b66d7ed7fe76ffeb985cf0ef35
The oslo team is recommending everyone to switch to the
non-namespaced versions of libraries. Updating the hacking
rule to include a check to prevent oslo.* import from
creeping back in.
This commit includes:
- using oslo_utils instead of oslo.utils
- using oslo_serialization instead of oslo.serialization
- using oslo_db instead of oslo.db
- using oslo_i18n instead of oslo.i18n
- using oslo_middleware instead of oslo.middleware
- using oslo_config instead of oslo.config
- using oslo_messaging instead of "from oslo import messaging"
- using oslo_vmware instead of oslo.vmware
Change-Id: I3e2eb147b321ce3e928817b62abcb7d023c5f13f
Storage server may be configured to protect target discovering phase with CHAP
authentication, in this case existing discovery command will be failed.
The authentication properties are:
"discovery.sendtargets.auth.authmethod",
"discovery.sendtargets.auth.username",
"discovery.sendtargets.auth.password"
Cinder Storage driver need to send discovery auth properties in this case,
and the properties are:
iscsi_properties['discovery_auth_method']
iscsi_properties['discovery_auth_username']
iscsi_properties['discovery_auth_password']
Change-Id: Ic70426d7d0fd8126879840f05341731ed92d6392
Closes-Bug: #1367189
Adds the new libvirt parameter iscsi_transport that can be used to
specify an iscsi transport, which used in conjuction with the
--interface parameter provides offloaded iscsi support.
Also happens to implement code that was originally supposed to be
covered by hw-iscsi-device-name-support as this is a
requirement for transport support.
DocImpact
Closes-Bug: #1370226
Implements: blueprint hw-iscsi-device-name-support
Implements: blueprint add-open-iscsi-transport-support
Change-Id: I1034f1e26e0b00e64430e6347d232793c3401ba8
This patch allows nova-compute to use QEMU iSCSI built-in
initiator for Cinder iSCSI volume drivers. It doesn't provide
iSCSI multipath capability, but host OS doesn't have to handle
iSCSI connection for volume because QEMU does it directly with
libiscsi.
To use this, you have to write a parameter at nova.conf:
volume_drivers = iscsi=nova.virt.libvirt.volume.LibvirtNetVolumeDriver,iser=nova.virt.libvirt.volume.LibvirtISERVolumeDriver,...
or just
volume_drivers = iscsi=nova.virt.libvirt.volume.LibvirtNetVolumeDriver
Note that qemu-system-x86 in Ubuntu has no iSCSI built-in
initiator support because libiscsi isn't in main repository
but universe. I've tested qemu-system-x86 built with libiscsi2
package of Debian on Ubuntu 14.04.
Change-Id: Ieb9a03d308495be4e8c54b5c6c0ff781ea7f0559
Implements: blueprint qemu-built-in-iscsi-initiator
Move _get_hypervisor_type, _get_hypervisor_hostname and
_get_hypervisor_version into the Host class
Blueprint: libvirt-driver-class-refactor
Change-Id: I2241c831d567571ace59244fea91e338b60a695c
Currently, GPFS use LibvirtVolumeDriver to generate the libvirt
xml configuration when attaching a gpfs volume. But the driver
always set the disk_type to 'block' which should be 'file' for
GPFS volume.
This patch create a new volume driver class for GPFS, the class
can generate the right xml configuration for GPFS volume.
Change-Id: Ica1cf3558baeea12b519bda50c61cea446429364
Closes-Bug:#1405044
oslo.concurrency-0.3.0 has moved its path to oslo_concurrency,
the old path oslo.concurrency can still work but is deprecated now.
Change-Id: I1538dad5cbacf1e6041c571d5bc55994689c08b0
Closes-Bug: #1398656
'options' object type is 'unicode',it might be None or empty in here.
So, useless blank be added to the argument of the mount command.
After all, execute/processutils process will fail in the current situation.
Change-Id: I45dcacd6499c3d4e0d26c0e2ee46b7f7d149f803
Closes-Bug: #1374915
In Icehouse with "iscsi_use_multipath=true", detaching a multipath
iSCSI volume kills all iSCSI volumes visible from the nova compute node.
When we use different targets(IQNs) associated with same portal for
each different multipath device, all of the targets will be deleted
via disconnect_volume().
This patch fixes the behavior of detaching volume:
1. Extract the targets for the detached multipath device.
2. Delete/disconnect the targets for the detached multipath device.
Closes-Bug: #1382440
Change-Id: I38eafdaee03d136282cfde1fd013e322a4256cc4
Currently, there are Libvirt volume drivers that support
network-attached file systems such as Gluster or NFS. This patch
adds a new volume driver in order to support attaching volumes
hosted on SMB shares.
Co-Authored-By: Gabriel Samfira <gsamfira@cloudbasesolutions.com>
DocImpact
Change-Id: I1db3d2a6d8ee94932348c63cc03698fdefff0b5c
Implements: blueprint libvirt-smbfs-volume-support
In Icehouse with "iscsi_use_multipath=true", attaching a multipath
iSCSI volume may create unnecessary iSCSI sessions.
The iscsiadm discovery command in connect_volume() returns all of the
targets in the Cinder node, not just the ones related to the multipath
volume which is specified by iqn. If the storage has many targets,
connecting to all these volumes will also result in many unnecessary
connections.
There are two types of iSCSI multipath devices. One which shares
the same iqn between multiple portals, and the other which use
different iqns on different portals. connect_volume() needs to
identify the type by checking iscsiadm the output if the iqn is
used by multiple portals.
This patch changes the behavior of attaching volume:
1. Identify the type by checking the iscsiadm output.
2. Connect to the correct targets by connect_to_iscsi_portal().
Closes-Bug: #1382440
Change-Id: I488ad0c09bf26a609e27d67b9ef60b65bb45e0ad
Let's switch to the newly released oslo library for the
processutils and lockutils. We use the config fixture(s) to
specify disable_process_locking and lock_path in the CONF
variable of oslo.concurrency library for correctly setting the
flags.
The previous attempt at updating nova to use oslo.concurrency
was reverted as it stopped running any tests. lockutils-wrapper
should have been used in ini/shell scripts
instead of lockutils.py module as the the module is
no longer runnable (__main__ has been removed). In this
review we use lockutils-wrapper correctly. The test harness
has since been updated to bail out if 0 tests are run as
well for additional insurance.
Change-Id: I14c75ef2196fd08c9465d648e5cd53777bd9c7d1
Co-Authored-By: Nikola Dipanov <ndipanov@redhat.com>
Co-Authored-By: Corey Wright <corey.wright@rackspace.com>
This patch made unit tests stop running (always runs 0 unit tests) we
should revert and put a check in place to prevent that from happening
so that we don't regress the rest of Nova trying to figure out the
oslo.concurrency issue.
This reverts commit 382294762b.
Change-Id: I98b04ac2a0ba67ba8af44867799317fdf95b8aff
When a volume is attached to a VM in the source compute node through
multipath, the related files in /dev/disk/by-path/ are like this
stack@ubuntu-server12:~/devstack$ ls /dev/disk/by-path/*24
/dev/disk/by-path/ip-192.168.3.50:3260-iscsi-iqn.1992-04.com.emc:cx.
fnm00124500890.a5-lun-24
/dev/disk/by-path/ip-192.168.4.51:3260-iscsi-iqn.1992-04.com.emc:cx.
fnm00124500890.b4-lun-24
The information on its corresponding multipath device is like this
stack@ubuntu-server12:~/devstack$ sudo multipath -l 3600601602ba034
00921130967724e411
3600601602ba03400921130967724e411 dm-3 DGC,VRAID
size=1.0G features='1 queue_if_no_path' hwhandler='1 alua' wp=rw
|-+- policy='round-robin 0' prio=-1 status=active
| `- 19:0:0:24 sdl 8:176 active undef running
`-+- policy='round-robin 0' prio=-1 status=enabled
`- 18:0:0:24 sdj 8:144 active undef running
But when the VM is migrated to the destination, the related information is
like the following example since we CANNOT guarantee that all nodes are able
to access the same iSCSI portals and the same target LUN number. And the
information is used to overwrite connection_info in the DB before the post
live migration logic is executed.
stack@ubuntu-server13:~/devstack$ ls /dev/disk/by-path/*24
/dev/disk/by-path/ip-192.168.3.51:3260-iscsi-iqn.1992-04.com.emc:cx.
fnm00124500890.b5-lun-100
/dev/disk/by-path/ip-192.168.4.51:3260-iscsi-iqn.1992-04.com.emc:cx.
fnm00124500890.b4-lun-100
stack@ubuntu-server13:~/devstack$ sudo multipath -l 3600601602ba034
00921130967724e411
3600601602ba03400921130967724e411 dm-3 DGC,VRAID
size=1.0G features='1 queue_if_no_path' hwhandler='1 alua' wp=rw
|-+- policy='round-robin 0' prio=-1 status=active
| `- 19:0:0:100 sdf 8:176 active undef running
`-+- policy='round-robin 0' prio=-1 status=enabled
`- 18:0:0:100 sdg 8:144 active undef running
As a result, if post live migration in source side uses <IP>, <IQN> and
<TARGET LUN Number> to find the devices to clean up, it may use 192.168.3.51,
iqn.1992-04.com.emc:cx.fnm00124500890.a5 and 100.
However, the correct one should be 192.168.3.50, iqn.1992-04.com.emc:cx.
fnm00124500890.a5 and 24.
Similar philosophy in (https://bugs.launchpad.net/nova/+bug/1327497) can be
used to fix it: Leverage the unchanged multipath_id to find correct devices
to delete.
Change-Id: I875293c3ade9423caa2b8afe9eca25a74606d262
Closes-Bug: #1357368
Changes include:
55ca7c3 Split cliutils
1b44601 Allow dictionary lookup in credentials with dot notation
6c706c5 Delete graduated serialization files
3edbfb3 remove caching param from prettytable call
9ce1d96 Fix i18n import
5d40e14 Remove code that moved to oslo.i18n
a4be4ed Remove graduated config modules
6ff6b4b Switch oslo-incubator to use oslo.utils and remove old modules
70189c9 Remove graduated test and fixtures libraries
8b71f57 Delete graduated db files
6ede600 rpc, notifier: remove deprecated modules
6b65a41 Fix build break - switch from str to hash for lookup
f76f44c Delete the token and endpoint on expiry of token of client
aebb58f Fix typo to show correct log message
1131b56 Enabled mask_password to handle byte code strings
6b048e7 Let oslotest manage the six.move setting for mox
Change-Id: Ic588aec0fc0291440feaac3439836054bd3608cd
This is a follow up patch to
https://review.openstack.org/#/c/116998/, where libvirt
volume's connect_volume method were split into get_config and
connect_volume. The following patch changes libvirt driver
to call the appropriate volume method, get_config or
connect_volume, instead of the old monolithic call to
connect_volume. connect_volume will no longer need to return
the libvirt xml after this switch over. This will improve
performance and eliminate possible race conditions.
Change-Id: I0d0d7ba98c53ad7509c28bb4866c488c0a049d2f
Closes-Bug: #1359617
The connect_volume method in libvirt/volume.py does a couple of
things. It does the host OS integration/setup tasks, and it returns
the libvirt xml. In some cases, only the libvirt xml is required,
and the setup tasks are not necessary.
The following patch separates the setup tasks and libvirt xml return
in connect_volume. A new method, called get_config, is added to just
return the xml config, separate from the connect_volume method.
The connect_volume method will continue to return the libvirt xml,
so that existing functionality is not broken. This way, get_config
can be used to get the libvirt xml without any side-effects, e.g.
re-running shell commands to setup the volume.
There will be a follow up patch, where libvirt's driver.py will
switch over to use get_config and connect_volume will no longer
need to return the libvirt xml.
Change-Id: I5d300f8fe99f714d63d51197dc89f556c3a5d43b
Related-Bug: #1359617
Matt Riedemann