Split off the finalization part of the volume manager's
extend_volume method and make it externally callable as the new
os-extend_volume_completion admin volume action.
This is the first part of a feature that will allow volume drivers
to rely on feedback from Nova when extending attached volumes,
allowing e.g. NFS-based drivers to support online extend.
See the linked blueprint for details.
Implements: bp extend-volume-completion-action
Change-Id: I4aaa5da1ad67a948102c498483de318bd245d86b
Add the "maximum" marker for Zed and 2023.1 at mv 3.70, which was
introduced by change I459f06504e90.
Change-Id: I7a7e45a99e0c7ec59f4e2ef7e98c9710c4cd37a1
This patch addresses a comment raised in the review of the feature,
I459f06504e90025c9c0b539981d3d56a2a9394c7.
Change-Id: I713ac84e4ed1617b29c0157e14931baefd626ce5
A new microversion 3.70 adds the ability to transfer a volume's
encryption key when transferring a volume to another project.
When the volume transfer is initiated, the volume's encryption
secret is essentially transferred to the cinder service.
- The cinder service creates a new encryption_key_id that contains
a copy of the volume's encryption secret.
- The volume (and its snapshots) is updated with the new
encryption_key_id (the one owned by the cinder service).
- The volume's original encryption_key_id (owned by the volume's
owner) is deleted.
When the transfer is accepted, the secret is transferred to the
user accepting the transfer.
- A new encryption_key_id is generated on behalf of the new user
that contains a copy of the volume's encryption secret.
- The volume (and its snapshots) is updated with the new
encryption_key_id (the one owned by the user).
- The intermediate encryption_key_id owned by the cinder service
is deleted.
When a transfer is cancelled (deleted), the same process is used
to transfer ownship back to the user that cancelled the transfer.
Implements: blueprint transfer-encrypted-volume
Change-Id: I459f06504e90025c9c0b539981d3d56a2a9394c7
NVMe-oF drivers that share the subsystem have the same race condition
issue that iSCSI volumes that share targets do.
The race condition is caused by AER messages that trigger automatic
rescans on the connector host side in both cases.
For iSCSI we added a feature on the Open-iSCSI project that allowed
disabling these scans, and added support for it in os-brick.
Since manual scans is a new feature that may be missing in a host's
iSCSI client, cinder has a flag in volumes to indicate when they use
shared targets. Using that flag os-brick consumers can use the
"guard_connection" context manager to ensure race conditions don't
happen.
The race condition is prevented by os-brick using manual scans if they
are available in the iSCSI client, or a file lock if not.
The problem we face now is that we also want to use the lock for NVMe-oF
volumes that share a subsystem for multiple namespaces (there is no way
to disable automatic scans), but cinder doesn't (and shouldn't) expose
the actual storage protocol on the volume resource, so we need to
leverage the "shared_targets" parameter.
So with a single boolean value we need to encode 3 possible options:
- Don't use locks because targets/subystems are not shared
- Use locks if iSCSI client doesn't support automatic connections
- Always use locks (for example for NVMe-oF)
The only option we have is using the "None" value as well. That way we
can encode 3 different cases.
But we have an additional restriction, "True" is already taken for the
iSCSI case, because there will exist volumes in the database that
already have that value stored.
And making guard_connection always lock when shared_targets is set to
True will introduce the bottleneck from bug (#1800515).
That leaves us with the "None" value to force the use of locks.
So we end up with the following tristate for "shared_targets":
- True to use lock if iSCSI initiator doesn't support manual scans
- False means that os-brick should never lock.
- None means that os-brick should always lock.
The alternative to this encoding would be to have an online data
migration for volumes to change "True" to "None", and accept that there
could be race conditions during the rolling upgrade (because os-brick on
computes will interpret "None" as "False").
Since "in theory" Cinder was only returning True or False for the
"shared_target", we add a new microversion with number 3.69 that returns
null when the value is internally set to None.
The patch also updates the database with a migration, though it looks
like it's not necessary since the DB already allows null values, but it
seems more correct to make sure that's always the case.
This patch doesn't close but #1961102 because the os-brick patch is
needed for that.
Related-Bug: #1961102
Change-Id: I8cda6d9830f39e27ac700b1d8796fe0489fd7c0a
This patch adds volume re-image API to enable the ability to
re-image a specific volume.
Implements: blueprint add-volume-re-image-api
Co-Authored-by: Rajat Dhasmana <rajatdhasmana@gmail.com>
Change-Id: I031aae50ee82198648f46c503bba04c6e231bbe5
Inclusion of a project_id in API URLs is now optional, and no longer
required. Removing the project_id requirement facilitates supporting
Secure RBAC's notion of system scope, in which an API method is not
associated with a specific project.
The API v3 routing is enhanced to provide duplicate routes for API
methods that traditionally required a project_id in the URL:
- The existing route for which a project_id is in the URL
- A new route for when the URL does not include a project_id
To test both routes and ensure there are no regresssions, the "API
samples" functional tests include a project_id in the URLs, and the
rest of the functional tests do not include the project_id. This is
handled by changing the 'noauth' WSGI middleware to no longer add the
project_id, and adding a new 'noauth_include_project_id' middleware
filter that implements the legacy behavior.
A new microversion V3.67 is introduced, but it only serves to inform
clients whether the project_id is optional or required. When an API
node supports mv 3.67, the project_id is optional in all API requests,
even when the request specifies a earlier microversion. See the spec
Ia44f199243be8f862520d7923007e7182b32f67d for more details on this
behavior.
Note: Much of the groundwork for this is based on manila's patch
I5127e150e8a71e621890f30dba6720b3932cf583.
DocImpact
APIImpact
Implements: blueprint project-id-optional-in-urls
Change-Id: I3729cbe1902ab4dc335451d13ed921ec236fb8fd
Introduces API microversion 3.66, which allows
snapshot creation on in-use volumes without the
force flag being passed.
Co-authored-by: Eric Harney <eharney@redhat.com>
Co-authored-by: Brian Rosmaita <rosmaita.fossdev@gmail.com>
Implements: blueprint fix-snapshot-create-force
Change-Id: I6d45aeab065197a85ce62740fc95306bce9dfc45
Change I655a47fc75dd introduced microversion 3.65. Update the
microversion history doc to reflect this.
Change-Id: I8cbe51f5e7207c241d2d490af1a6ba6e8ef74b16
Add microversion 3.64 for including the encryption_key_id in the
volume and backup details when the associated volume is encrypted.
This facilitates associating encryption keys (typically stored in
Barbican) with the volume or backup that uses it.
The encryption_key_id is included in the details only when the
associated volume is encrypted, and it isn't using the all-zeros
key ID used by the legacy fixed-key ConfKeyMgr.
APIImpact
DocImpact
Implements: blueprint include-encryption-key-id-in-details
Change-Id: I16f54e6722cdbcbad4af1eb0d30264b0039412fd
Introduces microversion (MV) 3.63 to add volume type ID
in the volume details.
This change comes to fix a problem we found in Gnocchi concerning
volumes monitored data. When a volume event is created, Cinder uses
the volume type as the volume type ID. On the other hand, Ceilometer
pollsters are always updating the volume_type value in Gnocchi with the
volume type from volume details API (which uses the volume type name
value).
That situation creates a massive number of resource revisions in
Gnocchi. This MV along with the Dynamic pollster system in Ceilometer
enables operators to overcome the issue.
Closes-Bug: https://bugs.launchpad.net/cinder/+bug/1911660
Change-Id: Icb88faeb00040250a25a5630aeb312a8434ed3c8
Signed-off-by: Rafael Weingärtner <rafael@apache.org>
* Also update max microversion for victoria (3.62) in the REST API
version history document.
* Also update vxflexos -> powerflex rebrand release note to reflect
that support for 'vxflexos' prefixed config options was yanked by
change Iaba55bc2cea1b69466d7131fd12842d3b0398513
Change-Id: Iee882657d79a568072e2687b2afe8e4d34c9f798
This patch adds a feature by which we allow setting default volume types
for projects.
The following changes are made to achieve the feature:
1) Add 4 set of APIs, set, get, get_all, unset volume type
2) All policies (except get_all) default to system/domain/project admin
3) Preference order: project default, conf default
4) Logic to not allow deletion of default type
We validate set, get and unset APIs with keystone to verify a valid
project id is passed in the request and user has proper authorization
rights to show the project.
The policies are system/domain/project admin by default except get_all
policy which defaults to system admin.
Implements: Blueprint multiple-default-volume-types
Change-Id: Idcc949ed6adbaea0c2337fac83014998b81ff1f8
cluster_name attribute is useful for admin users in Active/Active
deployments.
Change-Id: Ifab99457b8d49207e8fc7869bd82d3cdfec7a911
Closes-Bug: #1874414i
Support users can query resources by specifying the time
that resources are created at or updated at with time
comparison operators: "gt/gte/eq/neq/lt/lte",
and cinder will return all which match the condition.
The date and time stamp format is ISO 8601: CCYY-MM-DDThh:mm:ss±hh:mm.
The ±hh:mm value, if included, returns the time zone as an offset from UTC.
Add two filters updated_at and created_at in querying API.
"volumes/detail?created_at=gt:2016-01-01T01:00:00,lt:2016-12-31T01:00:00"
Change-Id: I1f43c37c2266e43146637beadc027ccf6dec017e
Partial-Implements: blueprint support-to-query-cinder-resources-filter-by-time-comparison-operators
Co-Authored-By: wangxiyuan <wangxiyuan@huawei.com>
This is to fix the following typos in docs:
modfied
certficate
transacton
specfies
Pasive
prefferably
depoying
resered
preferrable
readiblity
Change-Id: I1a5c62654d2c6da3704113299a13b5a593a40d71
This adds a note to the REST API history document to show the highest
microversion in the Train release.
Change-Id: I421933847236ae1d327c4701fb9ed11236eb6f87
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
The api documentation is now published on docs.openstack.org instead
of developer.openstack.org. Update all links that are changed to the
new location.
Note that redirects will be set up as well but let's point now to the
new location.
For details, see:
http://lists.openstack.org/pipermail/openstack-discuss/2019-July/007828.html
Change-Id: Id935a52232bc1ed22b457fc1227dcc4e8ec2babf
Add ``project_id`` to response body of list group snapshots
with detail and show group snapshot detail APIs.
Merging the group and group snapshot response in same MV
was discussed in weekly meeting[1].
[1] http://eavesdrop.openstack.org/meetings/cinder/2019/cinder.2019-02-20-16.00.log.html#l-122
Change-Id: Ided66450b5d7de32551edbce249e94f6174da2eb
Implements: blueprint add-project-id-to-group-groupsnapshot-response
This cleans up the cases where we had D001 violations so we can stop
skipping that check in doc8 runs.
Change-Id: Ie52f6ecac1a645fcbcc643b9ca63e033b622d830
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
This patch adds support for transfer pagination.
Co-Authored-By: Yikun Jiang <yikunkero@gmail.com>
Closes-bug: #1814195
Change-Id: Idb04f783b2287b2b45d626131648b0005a232fbe
Add ``project_id`` to response body of list groups with detail and show
group detail APIs.
Depends-On: I190cc67a001ffce2a533480e6e39e26cd3f981df
Change-Id: I3bc34a30b02f8a1ec2c5fecbcde6bd369c8e4a76
Implements: blueprint add-project-id-to-group-response
Add a new microversion API to add source_project_id,
destination_project_id, accepted fields to the response of follow
APIs:
- Create a volume transfer
- Show volume transfer detail
- List volume transfer and detail
And the source_project_id will be recorded when a transfer is
created, the destination_project_id and accepted will be recorded
when a transfer is accepted.
Part of blueprint: improve-volume-transfer-records
Change-Id: I3d79f9a67a9aed7272871969e7c0942a1f396ea5
Add ``user_id`` to response body of list backup with detail and show
backup detail APIs. In this way, you can get the ``user_id`` in backup
list or show response, make the admin/user easy to manage backup file.
Blueprint: add-user-id-attribute-to-backup-response
Change-Id: I3ffb544ef3ee65276cee8b1e870d524fd0e57085
We are now past feature freeze and should not have any more patches that
increment microversions this cycle. Following the convention from past
cycles, this marks the maximum API microversion for the Rocky release.
Change-Id: Idb3de6a687264ed88d697e68127f8cc3a63ec567
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
This feature changes Cinder to transfer
snapshots with volumes at the same time by default.
If user doesn't want to transfer snapshots, they could use
a new optional argument '--no-snapshots' after microversion 3.55.
And we also introduce the new V3 api 'v3/volume_transfers'
to move this API out of contrib into Cinder V3 API.
The cinderclient patch: https://review.openstack.org/#/c/577611/
Change-Id: If848d131e5edcdb77d0b3c2ca45a99c4d5e14d1e
Implements: blueprint transfer-snps-with-vols
This patch is the next step in simplifying how we create and
manage Read Only Volume Attachments.
Currently, we have multiple ways of doing this including
volume_admin_metadata and the connector that's supplied during
attachment processes. The problem is that this method requires
you to set and match both settings; and there are a lot of places
things can go wrong.
If you use volume_admin_metadata, the volume must ALWAYS receive
a connector that has read only specified in it, otherwise if the
values don't match, the attach calls fail.
If you don't use the metadata setting (ie you don't want the volume
to be exclusively Read Only) you can just specify the mode in the
connector. Rather than have these two different settings and juggle
the various attachment parameters that have been introduced
(attach_mode, access_mode, volume_admin_metadata:read_only...) it
would be much more straight forward to just allow the ability to
specify an attach mode during attachment-create and use that setting.
This change adds that option to attachment-create, but it also
keeps backward compatibility with the volume_admin_metadata for those
that are using it. Yes, we could just do a version bump and eliminate
admin_metadata; but maintaining the two independently is more disruptive
than just keeping compatibility.
One thing that this change does that might be unexpected, is it
eliminates the requirement for volume_admin_metadata and the connectors
mode parameters to match. It assumes that if an admin went through the
trouble to set a volumes admin_metadata to read only, then it should in
fact be read only regardless of the connector. So we always revert to
the admin_metadata setting, and modify the connector if they don't
match.
The next step in getting completely away from admin_metadata is to
introduce the ability to create a Read Only volume via types; that will
by the global setting for a volume in terms of its attachment setting.
None of this enforces anything on the consumer side (the old methods
didn't either) but it does set flags on the volume and attachment record
indicating what was requested and what we expect the consumer to be
doing with the volume.
Change-Id: I091f4eb8f1b464bb2dddb0a29245690a0430142e
This patch adds jsonschema validation for below Volumes API's
* POST /v3/{project_id}/volumes
* PUT /v3/{project_id}/volumes/{volume_id}
Adding new Microversion 3.53, this will have impact on both Create
and Update Volume API
* Create volume API will not accept additional properties like
(user_id, project_id, status and attach_status).
* Update volume API will require minimum one parameter in request
body otherwise it will raise 400 error.
APIImpact :
Before 3.53 version, create volume allows additional properties,
but now it will raise 400 error.
Also before 3.53 version, update volume allows empty body {},
but now it will raise 400 error.
DocImpact
Closes-Bug: #1774155
Partial-Implements: bp json-schema-validation
Change-Id: I1829b6796cd6d3fa1040dd0fb9280bec5bdbc92e
Now availability zone is highly integrated into
volume type's extra spec, it will be recognized
when creating and retyping, also we can filter
volume type by extra spec now.
Change-Id: I4e6aa7af707bd063e7edf2b0bf28e3071ad5c67a
Partial-Implements: bp support-az-in-volumetype
Adding header to denote which microversion was the maximum for
the Queens release and fix some formatting in the last addition
that caused the text to be shown as an indented note.
Change-Id: Ie786144b6417538953a9f77c69555b83be673b92
We currently limit backups to the same AZ of the volume while allowing
volume restoration to any AZ.
When having multiple AZs it is ideal to store your backups in a
different AZ than the one where the source volumes are stored.
This patch adds microversion 3.51 where we allow requesting the AZ where
we want to store backups which will allow us to have as many backup AZs
as volume AZs and do cross AZ backups for increased security, this
feature also supports having an additional AZ just for backups or the
less desirable solution where you store all your backups in a single AZ
shared with some of the volumes.
Change-Id: I595932276088d25abd464025c99dce33a2cc502b
The 3.50 microversion was added just to have a programmatic
way of determining if the service supports multiattach. The
text for 3.50 was very brief, so it wasn't really clear
what this meant. Expanding on the wording a little to try
to make it a little more clear.
Change-Id: I9b731f8cb68e9339fd5a2831b76cb57a496b6908
Currently, Cinder couldn't report backend state to service,
operators only know that cinder-volume process is up,
but isn't aware of if the backend storage device is ok.
Users still can create volume and go to fail over and over again.
This bp propose to get the backend state from driver directly,
operators or management system could query this via service list.
This function will improve the maintenance ability for Cinder.
Change-Id: I561dca3ef7c1901401621bc112389dbd178a907e
Implements: blueprint report-backend-state-in-service-list
This adds the last bits to the shared_targets and service_uuid
additions to volumes. The object and db models are already
updated and this change simply introduces API MV 3.48 to
expose the new fields to the Volume view.
These fields are intended to be used for identifying if the
back end device serving a volume is using shared targets, and
provides the service_uuid to be used as a suggested/unique
handle for locking.
Change-Id: Ibd18c5f6c772f7265175a63ed2b598fcc3c87bc1
This patch implements the spec of creating volume from backup.
Change-Id: Icdc6c7606c43243a9e12d7a42df293b729f589e5
Partial-Implements: blueprint support-create-volume-from-backup
The instance boot from volume, snapshot the instance will create
a image in glance. The image size is 0, it just a link to the
volume snapshot. When the user create volume from the snapshot
image, in the current version, it will create a blank volume. so
the user is troubled. This patch fix it.
The scheme: when the user create volume from snapshot image, the
api create interface will parse out the snapshot image corresponds
to the volume snapshot id. Then create a volume from the volume
snapshot. So, the user can use this way to restore the instance's
bootable volume.
Change-Id: Ic44dfae8bb187a5aa9c2c1f50f7b5dc0ef0f28f8
Closes-Bug: #1560867
This patch adds support for display count info
in volume, backup and snapshot's list&detail APIs
since microversion 3.45, for instance:
1. /v3/{project_id}/volumes?with_count=True
2. /v3/{project_id}/volumes/detail?with_count=True
Depends-On: 1c8fe0ade4
Change-Id: I2e92b27c36357120fcf0ec5917c6484441c946a8
Implements: bp add-amount-info-in-list-api
The REST API version history should ideally have some detailed
explanation of the changes introduced in that microversion. Barring
that, at least link to the API reference (if it exists) so client
side users have a breadcrumb trail to figure out what that microversion
introduces. That's what I'm doing here.
Change-Id: If15023b4ff9254fa6a7efa37dd132d883570faa0
Closes-Bug: #1716705
During the interop microversion discussion at the Queens PTG
a question came up about when Cinder started supporting microversions
(Mitaka) and what the max was in that first release (3.0). That wasn't
clear from the API version history doc, so it's added here.
Change-Id: I9834c5c3572a5c45326f55d5876be4d5e3812729
On Devstack, we need to cap the maximum microversion for each stable
branch after releasing like Ief5e0a1d8842fd1aed2e4493a3ef311b80322c63
This patch writes the maximum version of Pike for Cinder to help
this process.
Change-Id: I96deb2574bb5f7a71c0cebf3d2f94a3a92fbdc63
The new attachment_update method in Cinder's API creates an attachment
object and populates it with the provided connector info. In addition,
we set the volumes status to in-use and update the attachment object
status to "attached".
This isn't really accurate though, because we don't know if the volume
is actually attached (connected) by the consumer or not. Also a big
side effect here is that currently all of our tests and automation
use volume-status to determine if a volume is fully connected/ready
for use and that everything went well. It's used as an ack in most
cases.
This change goes back to using multiple states to signify where a
an attachment is in it's life-cycle:
1. reserved
We've created an empty attachment record but haven't done anything
with it yet.
2. attaching
We provided a connector and set up the TGT so that everything is
ready for a consumer to connect/use it.
3. in-use
An ACK back from the consumer letting us know that they connected
it successfully and are doing their thing.
Some consumers don't need or care about this last step, and we're going
to provide a means to set it straight to attached/in-use, but for
this bug we don't need to introduce that particular *feature*.
Sadly, this requires a micro-version bump and a new API call to
toggle the state from the API, pushing us to 3.44.
closes-bug #1710295
Change-Id: I57631d3deddb2d7cd244584e82206ee17fe2dd78
Now only volume and snapshot has metadata property.
We should support it for backup as well.
This patch added/updated the related db and ovo model,
updated the related backup CRUD APIs.
Change-Id: I6c4c175ec3be9423cdc821ccb52578dcfe442cbe
Implements: blueprint metadata-for-backup
Konrad Gube