This change adds the `encrypted`, `encryption_secret_uuid`,
`encryption_format` and `encryption_options` to the BlockDeviceMapping
object and associated table in the database.
Change-Id: I6178c9bc249ef4d448de375dc16d82b2d087fc90
The 'nova.block_device' module is essentially a catchall utils-like
module for all things BDM. The 'get_bdm_image_metadata' module, and
closely related 'get_image_metadata_from_volume' both fall into the
category of functions that belong here so move them. This allows us to
clean up tests and, crucially, avoid a circular reference seen when we
want to use proper type hints in the 'nova.virt.driver' module.
nova.context imports...
nova.utils, which imports...
nova.block_device, which imports...
nova.virt.driver, which tries to import...
nova.context, causing a circular dependency
Change-Id: I48177d6e93f2ff132d26b53cd682fd24a43a4b31
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
This one's actually important since it will be an error in future
versions of Python.
Change-Id: Ib9f735216773224f91ac7f49fbe2eee119670872
Signed-off-by: Stephen Finucane <sfinucan@redhat.com>
When attaching block devices, we find the next device name to use by
looking at the used device letters in the block device mappings and
selecting the next available letter, as long as the letter is within
'a' - 'z' or 'aa' - 'zz'. This allows us to represent 702 devices (and
would interestingly hit an IndexError if all 'a' - 'z' and 'aa' - 'zz'
letters were in use already.
This adds a method for generating device names universally, not limited
to any particular maximum, based on a device index. For example:
vda(index=0), sdb(index=1), vdaa(index=26), sdabc(index=730), ...
This method and its helper method will be used in a subsequent patch.
Part of blueprint conf-max-attach-volumes
Change-Id: I2699f600a98222a4dc58903ec97ca70e57701c93
Add a new microversion 2.67 to support specify ``volume_type``
when boot instances.
Part of bp boot-instance-specific-storage-backend
Change-Id: I13102243f7ce36a5d44c1790f3a633703373ebf7
Add ``volume_type`` field to BlockDeviceMapping object to be used
in new instance block_device_mapping_v2.volume_type API extension.
Implements: blueprint boot-instance-specific-storage-backend
Change-Id: I66ff84585642ff8d40ecfefef3342349eb49f83d
This adds a uuid column to the block_device_mapping table,
and makes it semi-optional for graceful upgrade of existing rows
without a uuid.
Part of bp local-disk-serial-numbers
Co-Authored-By: Lee Yarwood <lyarwood@redhat.com>
Co-Authored-By: Matthew Booth <mbooth@redhat.com>
Partial-Bug: #1489581
Change-Id: Ibf0db6ad5b8367fc3267ac309516c08547d47e8c
Also remove translation marker because this is a programming error.
The output is not relevant to a user.
Change-Id: I8d613c844ada2de4909a30f56c4686b711dae698
This adds an attachment_id column to the block_device_mapping table as
required for Nova to begin interacting with cinder v3.
Implements: blueprint cinder-new-attach-apis
Change-Id: I5d6afbc22ee89b764440f3313229a29311883f52
BDM code had some input validation code, however that is not used at
all because json-schema has validated the input before passing the
input to BDM code.
For example, source_type and boot_index are validated with
'source_type': {
'type': 'string',
'enum': ['volume', 'image', 'snapshot', 'blank'],
},
'boot_index': {
'type': ['integer', 'string', 'null'],
'pattern': '^-?[0-9]+$',
},
So Nova doesn't need to have the BDM code and this patch removes it.
This patch adds the corresponding test.
Change-Id: I278baead330c32d39f3d6c42077e3563205eed35
The legacy v2 API allowed None for the boot_index [1]. It
allowed this implicitly because the API code would convert
the block_device_mapping_v2 dict from the request into a
BlockDeviceMapping object, which has a boot_index field that
is nullable (allows None).
The API reference documentation [2] also says:
"To disable a device from booting, set the boot index
to a negative value or use the default boot index value,
which is None."
It appears that with the move to v2.1 and request schema
validation, the boot_index schema was erroneously set to
not allow None for a value, which is not backward compatible
with the v2 API behavior.
This change fixes the schema to allow boot_index=None again
and adds a test to show it working.
This should not require a microversion bump since it's fixing
a regression in the v2.1 API which worked in the v2 API and
is already handled throughout Nova's block device code.
Closes-Bug: #1662699
[1] https://github.com/openstack/nova/blob/13.0.0/nova/compute/api.py#L1268
[2] http://developer.openstack.org/api-ref/compute/#create-server
Change-Id: Ice78a0982bcce491f0c9690903ed2c6b6aaab1be
1.As mentioned in [1], we should avoid using
six.iteritems to achieve iterators. We can
use dict.items instead, as it will return
iterators in PY3 as well. And dict.items/keys
will more readable. 2.In py2, the performance
about list should be negligible, see the link [2].
[1] https://wiki.openstack.org/wiki/Python3
[2] http://lists.openstack.org/pipermail/openstack-dev/2015-June/066391.html
The patch list:
1. cells.
2. compute api.
3. image.
4. network.
5. objects.
6. scheduler.
7. virt.
8. other resources.
Partial-Implements: blueprint replace-iteritems-with-items
Change-Id: Ic6e469eb80ee1774de1374bb36f38b5134b6b311
Fixes volume related unit tests.
Fixes non-sortable None items.
Fixes __getattr__ infinite recursion.
Fixes is_dict_like method. Dicts in python 3.4 do not
have the 'has_key' method.
Partially Implements: blueprint goal-python35
Change-Id: I97efc09f7657436f706b08e0b2795f0e59ac1dcd
This patch series adds the ability for a user to specify a tag to be
applied to a network interface or a disk when booting an instance.
This tag is then exposed through the metadata API. For example, a user
boots a VM with two network interfaces. One is connected to a private
network, the other to the public Internet. There is currently no
direct way to tell which interface is which. Specifying tags allows
the user to distinguish between the two interfaces.
This patch contains the database migration to add a 'tag' column to
the virtual_interfaces and block_device_mapping tables as well as the
corresponding change to the VirtualInterface and BlockDeviceMapping
objects.
Implements: blueprint virt-device-role-tagging
Co-authored-by: Vladik Romanovsky <vromanso@redhat.com>
Change-Id: Ic8be3de4e970116772f9b6ce01c55e26b829e6cb
We only use method compute_driver_matches for checking if compute_driver
equals 'xenapi.XenAPIDriver'. To be explicit, just rename it to is_xenapi.
Change-Id: I2fb61491c4a321ab430f39476127412d8ee17c0d
Add options from 'virt.driver'. These options are part of the
'DEFAULT' group but are included in the "nova.conf.virt" file in hope
that they can eventually be moved to their own group.
Change-Id: Icf6084c156c4990ac60c778c28a48ae5ab97273a
Implements: bp centralize-config-options
We use some helper methods to strip and extrapolate details from the
device name string, some of these methods are missing the check on None
values, if a None is passed the code raise a TypeError exception. This
change adds a check for managing None values correctly.
Change-Id: Iebfaa09e494dac085b78dbaffa867f8047c2106b
If we're going to log the block_device_list device names here
we should sort the list, since finding which device might be
missing in a big list, e.g. 25 entries, is not fun.
Note that we have to filter None entries from the list before
calling sorted because py34 will raise a TypeError if it is
asked to compare a str to None.
Change-Id: Id912666f3c435bd284ed40b3de29d01d070b68c4
Related-Bug: #1530847
33e3d4c6b9 drops device names - at the
time of writing, the idea was that Nova should collect all the block
device information and decide on device names with the help of the
hypervisor.
It turns out that as well meaning as that change was - it also removed
the ability to do some of the things that the EC2 API needs device names
for (see the linked bug for a more detailed explanation). An example is
overriding some of the attributes of a block device entry in the image.
The only "indexable" property of a block device from EC2 POV is device
name so we need to put it back even though they may be overriden by the
virt layer upon a successful intance boot.
Change-Id: Ib1ba130042aabbbe7bb8d60fc212c66e446c1d73
Related-bug: #1370177
Related-bug: #1370250
When use bdm to boot with image in local,we should check
whether boot_index is valid firstly.
For example,when use novaclient,boot_index is a string ,
it can't compare with integer directly.
Change-Id: I8c0c3793e4453b9a5d6daac7404c281c163ac474
Closes-bug:#1459917
The following commit: 33e3d4c6b9 wrongly
drops some of the block device mapping information that is needed to
fully describe the block device mapping. In addition - EC2 API needs
this information to be able to format the output.
Change-Id: I18a0aab1dd966f28973e1f3705f263cb46a42639
Partial-bug: #1370177
Make sure that 'delete_on_termination' is always a boolean, even if it
was not passed in and thus got defaulted to None up until now.
This patch is really more of a pedantic fix, as we use objects for BDMs
everywhere outside of the API service, and object fields are of the
correct type.
Related-bug: #1370177
Change-Id: I2724bcbe159490f3bdd85f833412f3b20c4e1e23
After preparing the block device mapping for creating new instance(s),
check if any ephemeral and/or a swap disks have been already defined. If
that is not the case and the instance type requires them, create the
needed ephemeral and/or swap BDMs.
Closes-Bug: #1297325
Related-Bug: #1457527
Change-Id: I44b30625cf1023d20ebec5e38f46b7b8dab325f6
This patch was generated by the sixer tool version 0.2 using the
"iteritems" operation:
https://pypi.python.org/pypi/sixer
Manual changes:
- Don't change get_instance_metadata() in nova/compute/api.py:
fixed by the change Ifd455e70002eb9636b87f83788384127ba6edeeb.
- Don't change sqlalchemy code and
nova/tests/unit/db/test_db_api.py. sqlalchemy objects cannot be
converted to a dictionary using dict(obj) directly yet. It will be
possible with the change I702be362a58155a28482e733e60539d36c039509.
- Revert change in a comment in nova/objects/instance.py; the sixer tool
is limited and don't understand comments
- Reformat nova/virt/vmwareapi/driver.py to respect the 80 columns
contraint
Blueprint nova-python3
Change-Id: I81465661cb8a74778d70ba9b6641073f1effa49b
On Python 3, iterators have a __next__() method, but no next() method.
Use the builtin next() function which works on Python 2 and Python 3.
This patch was generated by the sixer tool version 0.2:
https://pypi.python.org/pypi/sixer
Manual change:
* Fix indentation
* tarfile.TarFile is not an iterator and has a next() method on Python 2
and Python 3: revert changes in
nova/tests/unit/virt/xenapi/image/test_utils.py and changes on
self._tarfile in nova/virt/xenapi/image/utils.py.
Blueprint nova-python3
Change-Id: Ie691d6f236a5aeef049f0b191dd07167020443d7
https://bugs.launchpad.net/nova/+bug/1377958 fixed a problem
that source_type: image, destination_type: local is not
supported for boot instance, exception should be raised to
reject the param otherwise it will lead to instance become
ERROR state.
However the fix introduced a problem on nova client
https://bugs.launchpad.net/python-novaclient/+bug/1418484
The fix of the bug leads to following command become invalid
nova boot test-vm --flavor m1.medium --image centos-vm-32
--nic net-id=c3f40e33-d535-4217-916b-1450b8cd3987 --block-device
id=26b7b917-2794-452a-95e5-2efb2ca6e32d,bus=sata,source=volume,bootindex=1
So we need to release the original constraint to allow
the above special case pass the validation check then
we can revert the nova client exception
(https://review.openstack.org/#/c/165932/)
This patch checks the boot_index and whether image param is
given after we found the bdm has source_type: image,
destination_type: local, if this is the special case, then
no exception will be raised.
Closes-Bug: #1433609
Change-Id: If43faae95169bc3864449a8364975f5c887aac14
Convert the use of the incubated version of the log module
to the new oslo.log library.
Sync oslo-incubator modules to update their imports as well.
Co-Authored-By: Doug Hellmann <doug@doughellmann.com>
Change-Id: Ic4932e3f58191869c30bd07a010a6e9fdcb2a12c
The oslo team is recommending everyone to switch to the
non-namespaced versions of libraries. Updating the hacking
rule to include a check to prevent oslo.* import from
creeping back in.
This commit includes:
- using oslo_utils instead of oslo.utils
- using oslo_serialization instead of oslo.serialization
- using oslo_db instead of oslo.db
- using oslo_i18n instead of oslo.i18n
- using oslo_middleware instead of oslo.middleware
- using oslo_config instead of oslo.config
- using oslo_messaging instead of "from oslo import messaging"
- using oslo_vmware instead of oslo.vmware
Change-Id: I3e2eb147b321ce3e928817b62abcb7d023c5f13f
Attaching volume to iso instance results in failure like
this 'libvirtError: Requested operation is not valid: target
hda already exists'. The root device of iso instance is hda,
nova-compute should not assign hda to cinder volume again.
Note: this patch only address the issue of duplicate device
name 'hda', disk bus 'ide' cannot be hotplugged, so we should
stop iso instance first when attaching volume.
Change-Id: If9f848dc3a3788862d25692250746387c6b35286
Closes-Bug: #1379212
PEP-0274 introduced dict comprehensions to replace dict constructor
with a sequence of length-2 sequences, these are benefits copied
from [1]:
The dictionary constructor approach has two distinct disadvantages
from the proposed syntax though. First, it isn't as legible as a
dict comprehension. Second, it forces the programmer to create an
in-core list object first, which could be expensive.
Nova dropped python 2.6 support, we can leverage this now.
There is deep dive about PEP-0274[2] and basic tests about
performance[3].
Note: This commit doesn't handle dict constructor with kwagrs.
This commit also adds a hacking rule.
[1]http://legacy.python.org/dev/peps/pep-0274/
[2]http://doughellmann.com/2012/11/12/the-performance-impact-of-using-dict-instead-of-in-cpython-2-7-2.html
[3]http://paste.openstack.org/show/154798/
Change-Id: Ifb5cb05b9cc2b8758d5a8e34f7792470a73d7c40
By defining the API schema, it is possible to separate the validation
code from the API method. The API method can be more simple.
In addition, a response of API validation error can be consistent for
the whole Nova API.
Partially implements blueprint v2-on-v3-api
Change-Id: I6d53ef2e3dad5a4ac9b67a5838f6e07606a8372b
Specifying image to local BDM at nova boot is not supported.
This patch adds a validation to the API to reject this not supported
mapping.
Closes-bug: #1377958
Change-Id: I0c38134e2aacca83d98ed0b341f15c480afe14f1
oslo.utils library now provides the functionality previously in
oslo-incubator's excutils, importutils, network_utils, strutils
timeutils, units etc. Some modules already moved to oslo.utils
will still be around since other code in nova/openstack/common/
are using it and will be removed in a subsequent commit.
Change-Id: Idc716342535fdfa680963e0e073ddb46f5f1eb34
The following patch allows an empty volume to be created and
attached to an instance at boot time. Today, you can define a
bootable volume that is sourced from an image, volume, or
snapshot. However, you cannot define a volume that is sourced
from 'blank', even though it is one of the source options
available. For example, the following command will not work:
nova boot --flavor m1.tiny
--block-device source=blank,dest=volume,size=1,bootindex=0 test.
This is because the method used to identify ephemeral volumes
(new_format_is_ephemeral) believes any block device with
source_type=blank is an ephemeral, and there is no logic to
handle volumes where source_type=blank. The following patch
fixes these bugs by properly identifying ephemeral block
devices and creating blank/empty volumes when source_type=blank
and destination_type=volume.
Change-Id: I5aa9684bfad1749fadff3018b13a225ed8f16fe8
Closes-Bug: #1347028
Closes-Bug: #1347499
Adding a generator that would provide a non-root block device
mappings, when it's optional variable exclude_root_mapping is
set to true. Otherwise, all mappings will be returned.
The method will be used to handle LXC volumes, as it's root FS
should be handled differently.
Change-Id: I879916021c3b61f19dd69ff11838dbbac19f72d1
Related-Bug: #1269990
oslo.i18n provides the i18n functions that were provided by
oslo-incubator's gettextutils module. Some tests that were
using internal details of the library were removed.
Change-Id: I44cfd5552e0dd86af21073419d31622f5fdb28e0
Currently we can't pass values as kwargs to BlockDeviceDict classes.
This is a common way to call a dict constructor so it breaks the dict
abstraction a bit.
This patch fixes it in a backward compatible way.
Change-Id: I31117eecba4f060cb9c7fc3d329f70fd369b168d
Make sure that all user input device names end up in the database with
the '/dev/' prefix. This will make things more consistent and avoid
issues when attempting to parse device names.
This includes two main sources - block device mapping that can be passed
as part of a request or as image metadata, and root device name, passed
as image metadata.
Closes-bug: #1337821
Change-Id: Ibda82f511be99f1a68f2f77c72601a1b006be7a0
Lee Yarwood