Split off the finalization part of the volume manager's
extend_volume method and make it externally callable as the new
os-extend_volume_completion admin volume action.
This is the first part of a feature that will allow volume drivers
to rely on feedback from Nova when extending attached volumes,
allowing e.g. NFS-based drivers to support online extend.
See the linked blueprint for details.
Implements: bp extend-volume-completion-action
Change-Id: I4aaa5da1ad67a948102c498483de318bd245d86b
The various index APIs (e.g. 'GET /volumes') support two forms of sort
parameter: the legacy 'sort_key'/'sort_dir' parameters and the combined
'sort' parameter. We were documenting different parameters in different
APIs despite the fact all APIs support all parameters.
In a similar vein, the various index APIs support the same three
pagination parameters: 'offset', 'limit', and 'marker'. Once again, we
were documenting these inconsistently.
Correct both issues.
Change-Id: Ia3300a8852825f7da830f7b1ed37b27625e267e9
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
A bullet list shouldn't be indented in rST and doing so causes it to be
rendered inside a blockquote, which is not what we want.
In addition, indicating versions as vXXX is more prevalent than VXXX so
correct this.
Change-Id: I3ea8e0a8ae58d5b7da9aae85b41162b2458020d5
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
This patch adds the following tests:
1) List backup with details
2) Show backup
3) Update backup
Partially-Implements: blueprint generating-sample-files
Change-Id: I9bbfab93fb138932243d880e440c9524b5f08217
This patch adds the following tests:
1) Volume transfer for microversions 3.55 and above
Partially-Implements: blueprint generating-sample-files
Signed-off-by: Desire Barine <desirebarine16@gmail.com>
Change-Id: Ifdd99433563ab72d8d43a374af0377213834d46e
This patch adds the following tests:
1) List Backup
Partially-Implements: blueprint generating-sample-files
Change-Id: I639e2786ce0fc8d0e9422704df9cdd50d3ea84ec
Extensions don't have a namespace attribute in v3.
Change-Id: I78ff472e55d822390029e1dd3817786d9b69a742
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
This patch adds the following tests:
1) Backup create
Partially-Implements: blueprint generating-sample-files
Change-Id: Ibe820e8906e6dfdf71a9fc77f3091ccd747f033c
Due to how the Linux SCSI kernel driver works there are some storage
systems, such as iSCSI with shared targets, where a normal user can
access other projects' volume data connected to the same compute host
using the attachments REST API.
This affects both single and multi-pathed connections.
To prevent users from doing this, unintentionally or maliciously,
cinder-api will now reject some delete attachment requests that are
deemed unsafe.
Cinder will process the delete attachment request normally in the
following cases:
- The request comes from an OpenStack service that is sending the
service token that has one of the roles in `service_token_roles`.
- Attachment doesn't have an instance_uuid value
- The instance for the attachment doesn't exist in Nova
- According to Nova the volume is not connected to the instance
- Nova is not using this attachment record
There are 3 operations in the actions REST API endpoint that can be used
for an attack:
- `os-terminate_connection`: Terminate volume attachment
- `os-detach`: Detach a volume
- `os-force_detach`: Force detach a volume
In this endpoint we just won't allow most requests not coming from a
service. The rules we apply are the same as for attachment delete
explained earlier, but in this case we may not have the attachment id
and be more restrictive. This should not be a problem for normal
operations because:
- Cinder backup doesn't use the REST API but RPC calls via RabbitMQ
- Glance doesn't use this interface anymore
Checking whether it's a service or not is done at the cinder-api level
by checking that the service user that made the call has at least one of
the roles in the `service_token_roles` configuration. These roles are
retrieved from keystone by the keystone middleware using the value of
the "X-Service-Token" header.
If Cinder is configured with `service_token_roles_required = true` and
an attacker provides non-service valid credentials the service will
return a 401 error, otherwise it'll return 409 as if a normal user had
made the call without the service token.
Closes-Bug: #2004555
Change-Id: I612905a1bf4a1706cce913c0d8a6df7a240d599a
The initial cinder design[1][2][3] allowed users to create mutliattach
volumes by spcifying the ``multiattach`` parameter in the request
body of volume create operation (``--allow-multiattach`` option in
cinderclient).
This functionality changed in Queens with the introduction of
microversion 3.50[4] where we used volume types to store
the multiattach capabilities. Any volume created with a multiattach
volume type will be a multiattach volume[5].
While implementing the new functionality, we had to keep backward
compatibility with the *old way* of creating multiattach volumes.
We deprecated the ``multiattach`` (``--allow-multiattach`` on cinderclient
side) parameter in the queens release[6][7].
We also removed the support of the ``--allow-multiattach`` optional
parameter from cinderclient in the train release[8] but the API
side never removed the compatibility code to disallow functionality
of creating multiattach volumes by using the ``multiattach``
parameter (instead of a multiattach volume type).
This patch removes the support of providing the ``multiattach``
parameter in the request body of a volume create operation and will
fail with a BadRequest exception stating the reason of failure
and how it can be fixed.
[1] https://blueprints.launchpad.net/cinder/+spec/multi-attach-volume
[2] https://review.opendev.org/c/openstack/cinder/+/85847/
[3] https://review.opendev.org/c/openstack/python-cinderclient/+/85856
[4] f1bfd9790d
[5] https://docs.openstack.org/cinder/latest/admin/volume-multiattach.html#how-to-create-a-multiattach-volume
[6] 94dbf5cce2
[7] adb141a262
[8] 3c1b417959
Depends-On: https://review.opendev.org/c/openstack/tempest/+/875372
Closes-Bug: 2008259
Change-Id: I0ece6e279048abcc04b3674108290a80eca6bd62
This API is currently undocumented, as I noticed while working on OSC
changes. There isn't anything too unusual going on here. The API was
first introduced in microversion 3.7 and was then extended with
replication-related information in 3.36. Both are accounted for here.
Change-Id: I86dc774c8d203839f4dadb13e93a416d112fc7b5
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Co-authored-by: Lucas de Oliveira <lucasmoliveira059@gmail.com>
Closes-Bug: 1885544
A new microversion 3.70 adds the ability to transfer a volume's
encryption key when transferring a volume to another project.
When the volume transfer is initiated, the volume's encryption
secret is essentially transferred to the cinder service.
- The cinder service creates a new encryption_key_id that contains
a copy of the volume's encryption secret.
- The volume (and its snapshots) is updated with the new
encryption_key_id (the one owned by the cinder service).
- The volume's original encryption_key_id (owned by the volume's
owner) is deleted.
When the transfer is accepted, the secret is transferred to the
user accepting the transfer.
- A new encryption_key_id is generated on behalf of the new user
that contains a copy of the volume's encryption secret.
- The volume (and its snapshots) is updated with the new
encryption_key_id (the one owned by the user).
- The intermediate encryption_key_id owned by the cinder service
is deleted.
When a transfer is cancelled (deleted), the same process is used
to transfer ownship back to the user that cancelled the transfer.
Implements: blueprint transfer-encrypted-volume
Change-Id: I459f06504e90025c9c0b539981d3d56a2a9394c7
This patch update volume delete section in api-ref
to understand preconditions more correctly
Partial-Bug: #1918932
Change-Id: Icb5b4da3423555cd2ab0f5ee6d1dc881fc782484
NVMe-oF drivers that share the subsystem have the same race condition
issue that iSCSI volumes that share targets do.
The race condition is caused by AER messages that trigger automatic
rescans on the connector host side in both cases.
For iSCSI we added a feature on the Open-iSCSI project that allowed
disabling these scans, and added support for it in os-brick.
Since manual scans is a new feature that may be missing in a host's
iSCSI client, cinder has a flag in volumes to indicate when they use
shared targets. Using that flag os-brick consumers can use the
"guard_connection" context manager to ensure race conditions don't
happen.
The race condition is prevented by os-brick using manual scans if they
are available in the iSCSI client, or a file lock if not.
The problem we face now is that we also want to use the lock for NVMe-oF
volumes that share a subsystem for multiple namespaces (there is no way
to disable automatic scans), but cinder doesn't (and shouldn't) expose
the actual storage protocol on the volume resource, so we need to
leverage the "shared_targets" parameter.
So with a single boolean value we need to encode 3 possible options:
- Don't use locks because targets/subystems are not shared
- Use locks if iSCSI client doesn't support automatic connections
- Always use locks (for example for NVMe-oF)
The only option we have is using the "None" value as well. That way we
can encode 3 different cases.
But we have an additional restriction, "True" is already taken for the
iSCSI case, because there will exist volumes in the database that
already have that value stored.
And making guard_connection always lock when shared_targets is set to
True will introduce the bottleneck from bug (#1800515).
That leaves us with the "None" value to force the use of locks.
So we end up with the following tristate for "shared_targets":
- True to use lock if iSCSI initiator doesn't support manual scans
- False means that os-brick should never lock.
- None means that os-brick should always lock.
The alternative to this encoding would be to have an online data
migration for volumes to change "True" to "None", and accept that there
could be race conditions during the rolling upgrade (because os-brick on
computes will interpret "None" as "False").
Since "in theory" Cinder was only returning True or False for the
"shared_target", we add a new microversion with number 3.69 that returns
null when the value is internally set to None.
The patch also updates the database with a migration, though it looks
like it's not necessary since the DB already allows null values, but it
seems more correct to make sure that's always the case.
This patch doesn't close but #1961102 because the os-brick patch is
needed for that.
Related-Bug: #1961102
Change-Id: I8cda6d9830f39e27ac700b1d8796fe0489fd7c0a
According to the request documentation above this sample,
the key "cluster" should be "cluster_name".
Closes-Bug: #1970135
Signed-off-by: James Palmer <jpalmer7698@gmail.com>
Change-Id: Ifd64ba39a46cdbf7d75bc17baa2862790a7a39c1
Managing a volume to an encrypted volume type should not be allowed.
One reason is that there is no way for an operator to specify an
encryption key ID for the volume. Another is that we already don't
allow a volume of an encrypted type to be un-managed, so this change
will be symmetric.
Also update and correct the api-ref for this call.
Co-authored-by: Yadiel Xuan(轩艳东) <xuanyandong@inspur.com>
Co-authored-by: Brian Rosmaita <rosmaita.fossdev@gmail.com>
Change-Id: Ic2da41f3962c1108f974aca952bce3da6d6ac277
Closes-bug: #1944577
This patch adds volume re-image API to enable the ability to
re-image a specific volume.
Implements: blueprint add-volume-re-image-api
Co-Authored-by: Rajat Dhasmana <rajatdhasmana@gmail.com>
Change-Id: I031aae50ee82198648f46c503bba04c6e231bbe5
Inclusion of a project_id in API URLs is now optional, and no longer
required. Removing the project_id requirement facilitates supporting
Secure RBAC's notion of system scope, in which an API method is not
associated with a specific project.
The API v3 routing is enhanced to provide duplicate routes for API
methods that traditionally required a project_id in the URL:
- The existing route for which a project_id is in the URL
- A new route for when the URL does not include a project_id
To test both routes and ensure there are no regresssions, the "API
samples" functional tests include a project_id in the URLs, and the
rest of the functional tests do not include the project_id. This is
handled by changing the 'noauth' WSGI middleware to no longer add the
project_id, and adding a new 'noauth_include_project_id' middleware
filter that implements the legacy behavior.
A new microversion V3.67 is introduced, but it only serves to inform
clients whether the project_id is optional or required. When an API
node supports mv 3.67, the project_id is optional in all API requests,
even when the request specifies a earlier microversion. See the spec
Ia44f199243be8f862520d7923007e7182b32f67d for more details on this
behavior.
Note: Much of the groundwork for this is based on manila's patch
I5127e150e8a71e621890f30dba6720b3932cf583.
DocImpact
APIImpact
Implements: blueprint project-id-optional-in-urls
Change-Id: I3729cbe1902ab4dc335451d13ed921ec236fb8fd
This patch adds backups_links optional on parameters.yaml and
include as response parameter on v3 backups list(/backups)
and details(/backups/detail).
Closes-Bug: 1944463
Change-Id: I29d2fb4b33f508638d703659a7717edf2891527d
This patch adds information about default volume types i.e. project
default and deployment default (set via default_volume_type in cinder.conf)
and also regarding __DEFAULT__ type.
Change-Id: I47151b621003aea04082d38bd1d5cc911afab0f4
This patch adds information regarding the default volume types
(including __DEFAULT__) in the api-ref.
The sections modified are volume create, volume type list, show
default volume type and delete volume type.
Change-Id: I896c809bd4ff09e2f26398a1cdaa9a91dc0f8638
Introduces API microversion 3.66, which allows
snapshot creation on in-use volumes without the
force flag being passed.
Co-authored-by: Eric Harney <eharney@redhat.com>
Co-authored-by: Brian Rosmaita <rosmaita.fossdev@gmail.com>
Implements: blueprint fix-snapshot-create-force
Change-Id: I6d45aeab065197a85ce62740fc95306bce9dfc45
This patch adds a new REST API microversion to expose the new use_quota
DB field present in volumes and snapshots.
The field will be reported when listing and showing resources and can be
used when filtering as well.
The field is exposed in the REST API as `consumes_quota` to prevent
confusion for users and admins, since exposing it as `use_quota` may
give them the wrong impression that they can set it up for their own
purposes.
For users we say what is happening with the quota for that resource -it
consumes or doesn't consume quota-, whereas internally we express
instruction to the core code -whether to use quota or not-, hence the
difference in the naming.
APIImpact
DocImpact
Implements: blueprint temp-resources
Change-Id: I655a47fc75ddc11caf1defe984d9a66a9ad5a2e7
Our docs and tests are not up to date with the changes introduced to the
snapshot details introduced by the different microversion.
This patch updates the docs, samples, and tests.
Change-Id: I46fa6e1f7a5d6ad0147f17c2bafbf4fb8fa89eb8
In the documentation for importing backups the backup_id is removed
because the backup_id is generated after the upload and the user
does not need to provide it. Therefore the request parameter
backup_id has also been removed.
Closes-Bug: #1930526
Change-Id: I49e0adea8fbd97d49f4386b8240320ea2baf3661
Cinder only supports uploading volumes of encrypted volume types as
images with disk format 'raw' and container format 'bare'. Screen
for this at the REST API layer when the request is made.
Change-Id: Ibb77b8b1be6c35c5db3b07fdc4056afd51d48782
Closes-bug: #1935688
Currently the schema validation for attachment create assumes that
instance UUID will always be present in the request but that is
not the case when glance calls cinder for attachment.
Also there isn't any schema validation for MV3.54 which accepts
attachment mode in the request hence failing requests passing mode.
This patch removes the instance_uuid from required parameters and
adds schema validation for MV3.54.
Change-Id: I5108fd51effa4d72581654ed450d191a13e0e964
In this patch:
- adjusted VersionsController to return only v3
- removed cinder.api.v2.router
- adjustments to cinder.tests.unit.api.contrib to use /v3 only
- moved cinder.api.v2.snapshot_metadata (and tests) to cinder.api.v3
- moved cinder.api.v2.types (and view, tests) to cinder.api.v3
- updated versions response in api-ref
- removed unnecessary config option
- updated various sample config files
- removed experimental tempest-cinder-v2-api job
- updated some docs
- updated non-voting rally job config
Some cinder.api.v2 modules are left because the v3 classes depend on
them, but with the v2 router removed, these are unreachable via the
/v2 path.
Depends-on: https://review.opendev.org/c/openstack/rally-openstack/+/794891
(changes rally to use Block Storage API v3)
Depends-on: https://review.opendev.org/c/openstack/requirements/+/794894
(corrects regression in upper-constraint on Sphinx)
Change-Id: I2093d77db9beec7543c7524d2cd273e79dd5fd5d
We have many microversions but we but we haven't kept the samples, docs,
and tests up to date with them.
Of the many microversions this patch will focus on the ones that have
updated the fields returned for volumes updating the docs, adding the
appropriate json samples, updating the tests, and updating the
contributor docs to highlight the need to keep these up to date.
Some convenience methods have been added to facilitate the testing:
- use_versions: class decorator that allows us to run the same tests
with different microversions, each using their respective json and
templates.
- override_mv: method decorator to change the microversion in a single
test.
- common_api_sample: context manager to use the base sample instead of a
microversion specific one.
Change-Id: Ide303685a4cd1477c319b36e67a3b05c8d129126
Konrad Gube