On Xena we added the use_quota DB field to volumes and snapshots to
unify the tracking of temporary resources, but we still had to keep
compatibility code for the old mechanisms (due to rolling
upgrades).
This patch removes compatibility code with the old mechanism and adds
additional cleanup code to remove the tracking in the volume metadata.
Change-Id: I3f9ed65b0fe58f7b7a0867c0e5ebc0ac3c703b05
There were a number of discrepancies between the models and migrations
(both sqlalchemy-migrate and alembic-based - they're identical). These
need to be addressed. In this patch, we correct a number of
discrepancies related to ForeignKey and Index constraints. Most of these
take the form of adding the constraint to the model, but we also remove
a number of indexes defined on the models but not actually present in
the database (since they're not defined in migration).
Change-Id: I0b53e2ccc0b03e24b8b6f67e67ea065ab5b85d07
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
There were a number of discrepancies between the models and migrations
(both sqlalchemy-migrate and alembic-based - they're identical). These
need to be addressed. In this patch, we resolve discrepancies between
the 'nullable' values, where the models field indicated 'nullable=True'
but the migration had the default 'nullable=False', or vice versa. The
migrations are truth so the models are updated to reflect this.
Change-Id: Id4bbf46d55cdbdd5060436c6d81fad7927508739
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Cinder creates temporary resources, volumes and snapshots, during some
of its operations, and these resources aren't counted towards quota
usage.
Cinder currently has a problem to track quota usage is when deleting
temporary resources.
Determining which volumes are temporary is a bit inconvenient because we
have to check the migration status as well as the admin metadata, so
they have been the source of several bugs, though they should be
properly tracked now.
For snapshots we don't have any way to track which ones are temporary,
which creates some issues:
- Quota sync mechanism will count them as normal snapshots.
- Manually deleting temporary snapshots after an operation fails will
mess the quota.
- If we are using snapshots instead of clones for backups of in-use
volumes the quota will be messed on completion.
This patch proposes the introduction of a new field for those database
resource tables where we create temporary resources: volumes and
snaphots.
The field will be called "use_quota" and will be set to False for
temporary resources to indicate that we don't want them to be counted
towards quota on deletion.
Instead of using "temporary" as the field name "use_quota" was used to
allow other cases that should not do quota in the future.
Moving from our current mechanism to the new one is a multi-release
process because we need to have backward compatibility code for rolling
upgrades.
This patch adds everything needed to complete the multi-release process
so that anybody can submit next release patches. To do so the patch
adds backward compatible code adding the feature in this release and
TODO comments with the exact changes that need to be done for the next
2 releases.
The removal of the compatibility code will be done in the next release,
and in the one after that we'll remove the temporary metadata rows that
may still exist in the database.
With this new field we'll be able to make our DB queries more efficient
for quota usage calculations, reduce the chances of introducing new
quota usage bugs in the future, and allow users to filter in/out
temporary volumes on listings.
Closes-Bug: #1923828
Closes-Bug: #1923829
Closes-Bug: #1923830
Implements: blueprint temp-resources
Change-Id: I98bd4d7a54906b613daaf14233d749da1e1531d5
The Oslo Versioned Objects history is used to generate the manifests
required to do compatibility changes to OVOs on data serialization
between services running with different OVO history versions.
We haven't updated our OVO history since Train so all the history and
compatibility code (obj_make_compatible method) is no longer necessary.
This patch consolidates the OVO history into a single version reflecting
the current status of the OVO versions and removes the compatibility
code from the OVO classes.
Since we tend to forget to update the obj_make_compatible when we add a
field (like it happened with Volume in version 1.8 when we added
shared_targets) this patch also adds a note next to the "fields"
attribute (except for the list OVOs which are never updated).
Change-Id: Ibfacccfb7c7dc70bc8f8e5ab98cc9c8feae694fb
test.py was previously not in the tests
directory. This means that downstream packagers
of Cinder have to specifically exclude it from
the main Cinder package (which does not typically
include unit tests).
Move it under the cinder/tests/unit/ dir where it
should be, to clean this up.
Change-Id: I65c50722f5990f540d84fa361b997302bbc935c5
Ceph RBD backend ignores the `--incremental` option when creating a
volume backup. The first backup of a given volume is always a full
backup, and each subsequent backup is always an incremental backup.
This behavior makes it impossible to remove old backups while
keeping at least one recent backup.
Since Cinder will not find the latest_backup id as parent_id if
'--incremental=False', so we can use the parent_id to ensure
whether do the full backup in rbd driver or not.
If the incremental flag '--incremental' is not specified, this
patch will always create a new full backup for rbd volume.
Change-Id: I516b7c82b05b26e81195f7f106d43a9e0804082d
Closes-Bug: #1810270
Closes-Bug: #1790713
Co-Authored-By: Sofia Enriquez <lsofia.enriquez@gmail.com>
Now availability zone is highly integrated into
volume type's extra spec, it will be recognized
when creating and retyping, also we can filter
volume type by extra spec now.
Change-Id: I4e6aa7af707bd063e7edf2b0bf28e3071ad5c67a
Partial-Implements: bp support-az-in-volumetype
During Rocky PTG, we discussed the concept of
'sold out'. In order to fully utilize
current codes, we decided to achieve this via
scheduler filters, cloud vendors can write their own
scheduler filter plugin to disable new resource
creation actions on sold out pools. Therefore, the
only change on cinder framework side is to delivery
'operation' when asking scheduler to filter hosts.
For this first stage, the initial operations are:
1. create_group
2. manage_existing
3. extend_volume
4. create_volume
5. create_snapshot
6. migrate_volume
7. retype_volume
8. manage_existing_snapshot
Partial-Implements: bp support-mark-pool-sold-out
Change-Id: I4f0a14444675ebd0fe6397a5ff2ef9dca62b4453
As described in the launchpad bug [1], backup operations must take care
to ensure encryption key ID resources aren't lost, and that restored
volumes always have a unique encryption key ID.
[1] https://bugs.launchpad.net/cinder/+bug/1745180
This patch adds an 'encryption_key_id' column to the backups table. Now,
when a backup is created and the source volume's encryption key is
cloned, the cloned key ID is stored in the table. This makes it possible
to delete the cloned key ID when the backup is deleted. The code that
clones the volume's encryption key has been relocated from the common
backup driver layer to the backup manager. The backup manager now has
full responsibility for managing encryption key IDs.
When restoring a backup of an encrypted volume, the backup manager now
does this:
1) If the restored volume's encryption key ID has changed, delete the
key ID it had prior to the restore operation. This ensures no key IDs
are leaked.
2) If the 'encryption_key_id' field in the backup table is empty, glean
the backup's cloned key ID from the backup's "volume base metadata."
This helps populate the 'encryption_key_id' column for backup table
entries created prior to when the column existed.
3) Re-clone the backup's key ID to ensure the restored volume's key ID
is always unique.
Closes-Bug: #1745180
Change-Id: I6cadcbf839d146b2fd57d7019f73dce303f9e10b
The attachment_specs table's key and value columns are strict
strings, which means things like a wwpns list value for a fibrechannel
connector can't get stored there and results in a DB error during
attach with the new volume attach flow in Nova.
The attachment_specs table is arguably not the best way to store
this data, which is just a dict like the connection_info.
A better way to store this is as a serialized json blob on the
volume_attachment record itself.
This patch adds the database migration to add the column and
an online data migration routine to migrate existing attachment_specs
entries when a volume attachment object is loaded from the database.
The volume manager attachment_update flow is changed to store
new connector attachments in the volume_attachment table directly.
An online data migration hook for the CLI will be added in a follow
up change.
Change-Id: Ica1f0e06adf0afcf740aad8cdc8d133ada1760c8
Closes-Bug: #1737724
This adds a bool column to volumes to notify consumers if
the backend hosting the volume utilizes shared_targets
or not.
We use the volume-drivers capabilities report to determine
this and default to True if a driver doesn't report anything.
The purpose of the column is to notify Nova that it needs to
do some sort of locking around connect/disconnect to be sure
other volumes on the same node aren't sharing the iscsi connection.
Using a default of "True" is safe because although locking and doing
the extra checks might be somewhat inefficient it works fine because
it will just appear that there's never any other volumes in use.
So this change adds the column to the DB as well as an online migration
to go through and update any existing volumes. With this and the
service_uuid column consumers will have everything the need to:
1. determine if they need to lock
2. use the service_uuid as a unique lock name
That last remaining change in this set will be to add the fields to
the view-builder and bump the API version.
Change-Id: If600c28c86511cfb83f38d92cf6418954fb4975e
This patch implements the spec of creating volume from backup.
Change-Id: Icdc6c7606c43243a9e12d7a42df293b729f589e5
Partial-Implements: blueprint support-create-volume-from-backup
This patch adds a service_uuid FK to the volumes table.
Up until now we've just done some host name parsing and
to match up service node with where the volume is being
serviced from.
With this, we now have a unique identifier that's user
visible to indicate what node the volume-service for a
particular volume is being serviced by.
We'll use this for things like share-target locks going
forward.
Change-Id: Ia5d1e988256246e3552e3a770146503ea7f7bf73
Pass the request to scheduler rather than volume service in
order to check the backend's capacity.
Change-Id: I970c10f9b50092b659fa2d88bd6a02f6c69899f2
Partial-Implements: blueprint inspection-mechanism-for-capacity-limited-host
Fix the UUID entry for the newly added service attribute and
udpate the unit tests appropriately.
This makes the UUID entry in the service object not nullable
and fixes up the unit tests to work properly. Also introduces
a unit test specifically for the online migration api in the db.
Closes-Bug: #1727091
Change-Id: I17d3a873cfc8f056c2d31f6c8710489785998d3c
In the 112 db migration I was being lazy and generating and
updating the newly added UUID column. I also didn't update
the Service object (left that for the follow on patch).
Turns out we're going to want/need the online_data_migration
pieces for some follow up work and we might as well do this at
least a little bit more efficiently/correctly now.
This patch modifies the db migration to NOT try and populate
the newly added UUID fields in the Service table, and it
updates the Service object properly including adding the gen UUID
components to service.create()
This also fixes up what we had in place for the online_data_migration
code in cmd/manage.py; note that we had a framework there/started but
it wasn't being used up to this point.
Change-Id: I6696a15cd2c8fbf851a59b8d6d60ae1981bb1b89
Closes-Bug: #1721837
Now only volume and snapshot has metadata property.
We should support it for backup as well.
This patch added/updated the related db and ovo model,
updated the related backup CRUD APIs.
Change-Id: I6c4c175ec3be9423cdc821ccb52578dcfe442cbe
Implements: blueprint metadata-for-backup
This patch implements the spec of reverting volume to
latest snapshot.
Related tempest and client patches:
[1] https://review.openstack.org/#/c/463906/
[2] https://review.openstack.org/#/c/464903/
APIImpact
DocImpact
Partial-Implements: blueprint revert-volume-to-snapshot
Change-Id: Ib20d749c2118c350b5fa0361ed1811296d518a17
This patch adds support for replication group.
It is built upon the generic volume groups.
It supports enable replication, disable replication,
failover replication, and list replication targets.
Client side patch is here:
https://review.openstack.org/#/c/352229/
To test this server side patch using the client side patch:
export OS_VOLUME_API_VERSION=3.38
Make sure the group type has group_replication_enabled or
consistent_group_replication_enabled set in group specs,
and the volume types have replication_enabled set in extra specs
(to be compatible with Cheesecake).
cinder group-type-show my_group_type
+-------------+---------------------------------------+
| Property | Value |
+-------------+---------------------------------------+
| description | None |
| group_specs | group_replication_enabled : <is> True |
| id | 66462b5c-38e5-4a1a-88d6-7a7889ffec55 |
| is_public | True |
| name | my_group_type |
+-------------+---------------------------------------+
cinder type-show my_volume_type
+---------------------------------+--------------------------------------+
| Property | Value |
+---------------------------------+--------------------------------------+
| description | None |
| extra_specs | replication_enabled : <is> True |
| id | 09c1ce01-87d5-489e-82c6-9f084107dc5c |
| is_public | True |
| name | my_volume_type |
| os-volume-type-access:is_public | True |
| qos_specs_id | None |
+---------------------------------+--------------------------------------+
Create a group:
cinder group-create --name my_group my_group_type my_volume_type
cinder group-show my_group
Enable replication group on the primary storage:
cinder group-enable-replication my_group
Expected results: replication_status becomes “enabled”.
Failover replication group to the secondary storage.
If secondary-backend-id is not specified, it will go to the
secondary-backend-id configured in cinder.conf:
cinder group-failover-replication my_group
If secondary-backend-id is specified (not “default”), it will go to
the specified backend id:
cinder group-failover-replication my_group
--secondary-backend-id <backend_id>
Expected results: replication_status becomes “failed-over”.
Run failover replication group again to fail the group back to
the primary storage:
cinder group-failover-replication my_group
--secondary-backend-id default
Expected results: replication_status becomes “enabled”.
Disable replication group:
cinder group-disable-replication my_group
Expected results: replication_status becomes “disabled”.
APIImpact
DocImpact
Implements: blueprint replication-cg
Change-Id: I4d488252bd670b3ebabbcc9f5e29e0e4e913765a
On commit f0d34b7d9b we changed the way we
did backports for OVOs during upgrades removing the need to have or
maintain `child_versions` fields in our OVO list classes.
This patch removes these fields that were introduced afterwards
including some that had the field incorrectly defined as
`child_version`.
TrivialFix
Change-Id: I4308d5ad20175002d0924bd814a72e96b4119381
We have a test that checks that we are bumping the version of OVOs
whenever we change the fields by checking the hashes of the OVOs and
fails with a helpful reminder to bump the version if we haven't done so.
This patch adds a reminder to that message so we don't forget to add the
backporting code, as well as mentioning that if we think that
backporting code is not necessary then it means that the version bump is
most likely not necessary as well.
TrivialFix
Change-Id: I326bb5ae1e635cd76dafec1ed671f1c47254fcbd
This patch adds 2 new APIs for microversion 3.32, one to dynamically
change the log level of cinder services, and the other that allows
querying their current log levels.
DocImpact
APIImpact
Implements: blueprint dynamic-log-levels
Change-Id: Ia5ef81135044733f1dd3970a116f97457b0371de
There are some issues around new attach/detach API/CLI,
fix them step by step. This patch added attribute
'connection_info' to attachment object, also add related
testcases in API unit testcases.
Depends-On: 87982a5677
Closes-Bug: #1681297
Change-Id: Idbc1049e8adf1d5b955bda01d58bb6b89fc6c5c7
when users try to create a backup from a snapshot, the volume
related to the snapshot is set to backing-up during the action
and can't be used for other actions.
When create a backup from a large snapshot, such as larger
than 1 TB, it will cost few hours generally. It's really a
problem that the volume is not available for such a long time.
If the snapshot is provided, we change the status of the snapshot;
otherwise, we change the status of the volume as usual.
This patch added "backing-up" status for snapshot as well
DocImpact
Change-Id: I86d34c470fabbf4132b5e004d9f368e751c893a5
Closes-bug: #1670541
Currently, the results from ManageableVolumes & ManageableSnapshots list
are being returned as Dict. This needs to be modeled as OVO as this
is being returned via rpc from the driver to the api layer.
We also change all occurences of List Manageable Volumes & snapshots to ovo
Change-Id: Id63e4c35deec6dccc0ae6a82b004618cd214d96e
Replaces the original patch:
https://review.openstack.org/#/c/387712/
Use the python-cinderclient WIP:
https://review.openstack.org/#/c/387716/
Here's what you can do currently:
`cinder attachment-create <volume-uuid>`
Currently only tested/implemented the reserve piece
Will create an attachment object, set volume to a
status of 'reserved'
`cinder attachment-list`
Simple list output of attachments
`cinder attachment-show <attachment-uuid>`
Detailed list of specified attachment
`cinder attachment-delete <attachment-uuid>`
Removes an attachment
Change-Id: Ie15233c99d91de67279b56d27a5508c5ea98d769
This patch adds new methods to our failover mechanism to allow failover
to work when a backend is clustered.
Adds REST API microversion 3.26 that adds a new `failover` method
equivalent to `failover_host` but accepting `cluster` field as well as
the `host` field.
Thaw and Freeze are updated to update cluster and all services within
the cluster.
Now cluster listings accepts new filtering fields `replication_status`,
`frozen`, and `active_backend_id`.
Summary listings return `replication_status` field and detailed listings
also return `frozen` and `active_backend_id`.
Specs: https://review.openstack.org/401392
APIImpact: New service failover action and new fields in cluster listings.
Implements: blueprint cinder-volume-active-active-support
Change-Id: Id3291b28242d5814c259283fa629b48f22e70260
CG APIs work as follows:
* Create CG - Create only in groups table
* Modify CG - Modify in CG table if CG in CG table, otherwise modify
in groups table.
* Delete CG - Delete from CG or groups table depending on where it is
* List CG - Check both CG and groups tables
* List CG snapshots - Check both CG and groups tables
* Show CG - Check both tables
* Show CG snapshot - Check both tables
* Create CG snapshot - Create either in CG or groups table depending on
the CG.
* Create CG from source - Create in either CG or groups table
depending on the source.
* Create volume - Add volume either to CG or group
Additional notes:
* default_cgsnapshot_type is reserved for migrating CGs.
* Group APIs will only write/read in/from the groups table.
* Group APIs won't work on groups with default_cgsnapshot_type.
* Groups with default_cgsnapshot_type can only be operated by CG APIs.
* After CG tables are removed, we'll allow default_cgsnapshot_type
to be used by group APIs.
Partial-Implements: blueprint generic-volume-group
Change-Id: Idd88a5c9587023a56231de42ce59d672e9600770
'unmanaging' is different from 'deleting', and must
be distinguishable to know how to recover from a failed
delete/unmanage operation. The current state of things
makes it so that an 'unmanaging' volume may end up being
accidentally deleted.
Add unmanage notifications, as well.
Partial-Bug: #1478959
Change-Id: I06f60a584f219043673095346282c429773704f8
One of the useful things that was missing from the
volume_attachments code was get_all methods.
This patch adds a get_all and a get_all_by_project, it
also goes ahead and adds some filtering capability to
the existing get_by_xxxx calls since we added the framework
for it in the get_all additions.
I also looked at refactoring our db methods for attach to just:
* attach_create
* attach_update
* attach_destroy
* attach_get
* attach_get_all
This would probably be good as an independent effort to
clean things up and bring these calls more in line with
others, but there's a lot of work to update the objects
and existing code, might be better to wait until after
implementing the new attach API.
Co-Authored-By: Michał Dulko <michal.dulko@intel.com>
Change-Id: I40614fe702f726c74ff05f93faaf6ee79253447f
This change adds a new enum and field, VolumeAttachStatus
and VolumeAttachStatusField, that will hold the constants for the
'attach_status' field of the VolumeAttachStatus object. This enum
and field are based on the base oslo.versionedobjects enum and field.
This also changes over the volume object to use the new field. Finally,
all uses of strings for comparison and assignment to this field are
changed over to use the constants defined within the enum.
Partial-Implements: bp cinder-object-fields
Change-Id: Ie727348daf425bd988425767f9dfb82da4c3baa8
This commit introduces BackupDevice object to formalize data earlier
sent over RPC as an undefined dict. This is required to be able to make
non-backward compatible changes to data sent as this parameter while
maintaining compatibility with previous release - so to support rolling
upgrades.
Change-Id: Ie57d84e32ec1c5fcfac27a7bb6d4bbb189108a5b
Partial-Implements: blueprint cinder-objects
To be able to support multiple hosts working with the same resources we
have added the workers table to keep track of which host is working with
each specific resource.
This patch makes c-vol service work with this new table by adding
entries on cleanable operations and removing them once these operations
have completed.
Service cleanup on initialization has also been changed to use this new
table so hosts will cleanup only resources from operations they left on
the air and leave any operations that are being processed by other
hosts.
Specs: https://review.openstack.org/236977
Implements: blueprint cinder-volume-active-active-support
Change-Id: I4e5440b8450558add372214fd1a0373ab4ad2434
Attachments_get_by_host|instance should be just that, associating
them with a specified volume-id doesn't really solve the problem.
If that relationship is needed a simple get_by_volume will work with
some inspection.
This patch removes the volume_id arg from those get methods. A follow
up patch will add get_all with filters for more complex relationships.
Change-Id: Ic5ffdced96fdf780cce2a1227c5f2a599860f0ca
Closes-Bug: #1632433
This patch adds CinderCleanableObject class that is a Versioned Object
base class and CleanupRequest Versioned Object that will be used to pass
cleanup requests to c-vol and c-bak nodes but will not have a DB
representation.
This will be used for non Active-Active configurations as well.
Specs: https://review.openstack.org/236977
Implements: blueprint cinder-volume-active-active-support
Change-Id: Ia84b2f55a782c5e881bab03a8469b884f265910c
VolumeType OVO class does not allow lazy loading of attributes, which is
usually not a problem if you are loading the volume type using
get_by_id, but is a problem if you load for example a list of volumes
and then go to their volume_type.extra_specs, as this will raise an
error because extra_specs cannot be retrieved.
This patch changes VolumeType OVO class to allow lazy loading all
optional fields.
Change-Id: Ief143a6c981cec4bdb21888776d610aa9d5dc9d8
This is the third patch that implements the generic-volume-group
bluerpint. It adds database and object changes in order to support
group snapshots and create group from source. The API changes will
be added in the next patch.
This patch depends on the second patch which adds create/delete/update
groups support which was already merged:
https://review.openstack.org/#/c/322459/
The next patch to add volume manager changes is here:
https://review.openstack.org/#/c/361376/
Partial-Implements: blueprint generic-volume-group
Change-Id: I2d11efe38af80d2eb025afbbab1ce8e6a269f83f
This is the second patch that implements the generic-volume-group
bluerpint. It adds the groups table and introduces create/delete/
update/list/show APIs for groups.
It depends on the first patch which adds group types and group specs:
https://review.openstack.org/#/c/320165/
Client side patch is here:
https://review.openstack.org/#/c/322627/
Current microversion is 3.13. The following CLI's are supported:
cinder --os-volume-api-version 3.13 group-create --name my_group
<group type uuid> <volume type uuid>
cinder --os-volume-api-version 3.13 group-list
cinder --os-volume-api-version 3.13 create --group-id <group uuid>
--volume-type <volume type uuid> <size>
cinder --os-volume-api-version 3.13 group-update <group uuid>
--name new_name description new_description
--add-volumes <uuid of volume to add>
--remove-volumes <uuid of volume to remove>
cinder --os-volume-api-version 3.13 group-show <group uuid>
cinder --os-volume-api-version 3.13 group-delete
--delete-volumes <group uuid>
APIImpact
DocImpact
Change-Id: I35157439071786872bc9976741c4ef75698f7cb7
Partial-Implements: blueprint generic-volume-group
This patch adds support for group types and group specs.
This is the first patch to implement the blueprint
generic-volume-group.
The client side patch is here:
https://review.openstack.org/#/c/320157/
Current microversion is 3.11. The following CLI's are supported.
cinder --os-volume-api-version 3.11 group-type-create my_test_group
cinder --os-volume-api-version 3.11 group-type-list
cinder --os-volume-api-version 3.11 group-type-show my_test_group
cinder --os-volume-api-version 3.11 group-type-key my_test_group
set test_key=test_val
cinder --os-volume-api-version 3.11 group-specs-list
cinder --os-volume-api-version 3.11 group-type-key my_test_group
unset test_key
cinder --os-volume-api-version 3.11 group-type-update <group type uuid>
--name "new_group" --description "my group type"
cinder --os-volume-api-version 3.11 group-type-delete new_group
APIImpact
DocImpact
Change-Id: I38b938782e0c3b2df624f975bd07e0b81684c888
Partial-Implements: blueprint generic-volume-group
To be able to make changes in request_spec passed in create_volume calls
(in c-sch and c-vol RPC API) while supporting rolling upgrades we need
to convert it from arbitrary dict blob to a well-defined versioned
object. This commit does so while maintaining backwards compatibility
with Mitaka. When version is pinned to Mitaka we're serializing the
dict before sending and when receiving a dict we're converting it to the
object.
We actually have a lot of duplicated/unused data sent using request_spec
and filter_properties. We should start to clean that up.
The future patches should:
* Convert filter_properties to ovo.
* Make services use only one occurrence of the data.
* Remove the unused duplicates while maintaining backwards compatibility
through object translations.
Change-Id: I7dae83667a1aca92c552fbfaa1e90c6558e293bf
Partial-Implements: blueprint cinder-objects
This patch adds Cluster Versioned Object as counterpart for the Cluster
ORM class, and updates ConsistencyGroup, Volume, and Service Versioned
Objects.
Specs: https://review.openstack.org/327283
Implements: blueprint cinder-volume-active-active-support
Change-Id: Ie6857cf7db52cf284d89bad704e2c679e5549966
Change QualityOfServiceSpecs from a Python dict to using a
VersionedObject as part of moving Cinder to support rolling
upgrades.
These changes also include changes to volume/qos_specs.py:
- Removing the unused method get_qos_specs_by_name
There will be follow-up patches to transition all instances of
qos['key'] to use qos.key and updating
volume_types.get_volume_type_extra_specs to use the new VO.
Fixing the invalid UUID issues and issues raised by new test cases.
Co-Authored-By: Xinli Guan <xinli@us.ibm.com>
Change-Id: If15ea8b628a6f88211a5d5cc7aadff44f7840138
Partial-Implements: blueprint cinder-objects
Now cinder backends hava capabilities of None value.
Cinder-api also allow user to set volume type's extra_specs to None
value. However we cann't use these capabilities in cinder scheduler
filter.
This change supports None value of extra_specs in cinder-scheduler
filter.
Further more, Our VolumeType object don't allow none value of
extra_specs and this causs failure when trying to create a
volume from such volume type. This commit also changes that
field type to DictOfNullableStringsField which allows it.
Also object backporting procedure is added to make sure that before
sending data to older services we're switching None to empty string.
(PS: if older service use functions like volume.refersh to read
None value of extra_spec from db itself, it will hit bug #1588798
anyway.)
Co-Authored-By: Michal Dulko <michal.dulko@intel.com>
Change-Id: I301adc5980dbacec5e84de7c66e6d12d132643cd
Closes-bug: #1588798
In Cinder we don't have an indirection service to forward object methods
like Nova does (Conductor) so we shouldn't be defining all methods as
remotable since they are not.
If we ever decide to have a Conductor we can easily add them back where
appropriate, but for now they are only a hindrance:
- It's confusing, because most people don't understand what it actually
means in Cinder and they just add it to the methods because they see
it in other OVO methods.
- Every time we change a remote method's signature or add a new remote
method to an object the hash of that object changes and we need to
update our test_versions test, which is not necessary.
- This makes the linking of VOs with their list counterparts
problematic, because everyone will need to know that there is no need
to bump the version of a list object whenever they add or modify a
remotable method.
- We are executing unnecessary code with decorators, even if it's not
much, it's still wasted CPU time .
This patch removes all appearances of remotable and remotable_class
decorators thus making our OVOs consistent with our implementation and
removing the need of updating our tests on unrelated OVO changes.
Change-Id: I18a0f619b4b108c553d9567b5f21f978466f1670
To make objects that have other objects as fields compatible to an
earlier version, oslo versioned objects uses either a manifest passed to
obj_to_primitive or the object's obj_relationships mapping.
Which means that if we don't have any of those mechanisms in place our
rolling upgrades mechanism will fail whenever we try to backport a
Versioned Object that has set an ObjectField field because Oslo
Versioned Object will not know how to backport that related object.
This patch introduces the usage of manifests on backports when we are
doing rolling upgrades.
For the manifest, we use the data in our Objects History. Which means
that as long as we keep history in OBJ_VERSIONS right we will not have
to create and worry about keeping lists' child_versions field or our
versioned object's obj_relationships for fields with types
ListOfObjectsField and ObjectField.
We also don't have to worry about cascade version bumping, as in
changing the List OVO version whenever the OVO it contains gets bumped,
or bumping our OVO whenever one of the related OVO fields is bumped.
Closes-Bug: #1571566
Change-Id: Ibc1a1257830c925c10696c0b5aedd5f471c538d0
It seems that we've added IN_USE as a valid CG status, which actually
isn't valid. It is used only in tests. This commit removes it from the
enum and replaces occurrences in tests with equivalent valid statuses.
As CG should not ever get that status in runtime, no new version of the
object or compatibility routine is added.
TrivialFix
Change-Id: Id8290e73a38a941a2a84391ded6ddbcb87be411e
There is a potential window of opportunity where races can happen in the
API on consistency group related actions, this patch removes those
windows of opportunity using compare-and-swap for DB updates.
Races have been removed in following actions:
- create
- delete
- update
- create_cgsnapshot
- delete_cgsnapshot
Specs: https://review.openstack.org/232599/
Implements: blueprint cinder-volume-active-active-support
Change-Id: I67aec4cd8bcf2f7e09473a8d296aa383fe85ad23
Gorka Eguileor