Added new column named 'backup_type' in 'share_backups' table
and changes the share common api libs to support the dhss_true
configuration for share backup creation
Partially-implements: bp/share-backup
Change-Id: Ifb88ec096674ea8bc010c1c3f6dea1b51be3beaa
Export locations are usually too difficult to memo
rize.Currently, there is no way to determine the
export location before the share is created, so
users wait until the share creation request gets
completed, and then they check the export
locations to mount the share. The generated
export locations are often not human readable
and it is hard to memorize and control them.
Implements: bp/human-readable-export-locations
Change-Id: I72ac7e24ddd4330d76cafd5e7f78bac2b0174883
The share-server-migration-get-progress command was returning error
when executed immediataly after starting migration.
This patch fix all the errors related to this change.
Closes-bug: #2025075
Change-Id: Iaaf15906aa3a29543860d878cde305cfddc7f70e
This better aligns with 'export_location_metadata_*' APIs. The plural
'export_locations_get*' APIs are renamed to 'export_location_get_all*'.
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: I14f2b834e7ac2d8be86f9d7e381706cadbb79eb8
Rename a number of APIs to use singular, rather than plural, like every
other API uses:
- share_instances_status_update ->
share_instance_status_update
- share_instances_get_all ->
share_instance_get_all
- share_instances_get_all_by_host ->
share_instance_get_all_by_host
- share_instances_get_all_by_share_network ->
share_instance_get_all_by_share_network
- share_instances_get_all_by_share_server ->
share_instance_get_all_by_share_server
- share_instances_get_all_by_share ->
share_instance_get_all_by_share
- share_instances_get_all_by_share_group_id ->
share_instance_get_all_by_share_group_id
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: Ic48fe0d1631a6e1a8ee9a50741cc1b31c9187c37
RBAC enforcement in manila happens in stages:
1) Does user have access to the API
2) Does user have access to the resource
3) Is user permitted to perform the API action on the resource
If (1) fails, user gets a HTTP 403, if (2) fails,
they get a HTTP 404; if (3) fails, they get a HTTP 403.
More often than not, (2) prevents "existence" detection
of resources that don't belong to the user; except in
case of "public" resources (e.g.: shares can be "public").
In some share API methods, policy checks for (1) are
happening after a bunch of processing. This leads to
some inconsistency.
Fix these occurrences to ensure a consistent user
experience.
Change-Id: I5b1f1ce517efed000f17b1e0901e183a1913ba9f
Related-Bug: #2004230
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
Add CRUD APIs for resource locks with support
for preventing deletion of shares (applies to
soft-deletions and unmanage operations as well).
Change-Id: I146bc09e4e8a39797e22458ff6860346e11e592e
Implements: bp/allow-locking-shares-against-deletion
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
Add share backup feature in Data Copy Service and
Share Service. It will allow the user to create, restore
and delete backups as well as listing backups and showing
the details of a specific backup.
APIImpact
DOCImpact
Change-Id: I7d10cf47864cd21932315375d84dc728ff738f23
Implement: blueprint share-backup
Allows set max share extend size that can be created in extra_specs
for each share_type. At API level as part of share extend for admin,
max share extend size will be checked whereas for non-admin users,
max share size will be checked.
New extra_specs key is added to set max extend size of share
i.e.'provisioning:max_share_extend_size'
Closes-Bug: #1987253
Change-Id: I61e590ed1851e8fa15996dc61e8e17e9413a9d91
1. Change context as first argument to function.
2. Fix spelling mistake in version history
3. Add new host_admin RBAC policy which is applied in onlyHostFilter
since non-admin user as well needs to create share on specific host.
Change-Id: Id2c09ebab874ec983da7f26370932d46a0447801
user can create a transfer for a share. will return transfer id
and auth_key. another user can use transfer_id and auth_key to
accept this share. salt of transfer and auth_key compute crypt_hash
by sha1. then compare with crypt_hash in db.
APIImpact
DocImpact
Partially-Implements: blueprint transfer-share-between-project
Change-Id: I8facf9112a6b09e6b7aed9956c0a87fb5f1fc31f
Currently netapp_snapmirror_quiesce_timeout is an option of replica
promote that can only be set by the operator. There are scenarios where
high timeout value set by operator does not fit well e.g. disaster
stroke and user want to do an unplanned failover fast. Added new
option 'quiesce_wait_time' to share replica promote API which allows
to use specified wait_time(in seconds) instead of config option.
Closes-bug: #2000171
Change-Id: Ib02063ee8b82f7374cd89f90e7f24a845c6c7cd7
When a user is prevented from listing a non-public
share, the API service would return a 403 Forbidden.
This isn't consistent with the API SIG's guidance
on resources restricted by virtue of RBAC policy since
users with malicious intent may use the signal to
mean that the resource exists.
Depends-On: I27fdd7dfffeb15965b66dbb3f6b1568c11ff9ad4
Change-Id: I7e05dcb343c932cc7fec8d395919053d0a1801ce
Closes-Bug: #1901210
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
Added missed 'service' and 'reason' keywords for Manila exceptions unit tests.
Closes-bug: #1966473
Change-Id: I70533951f46cddd171b1979ed57a59a5448a4557
Fixed the issue in the share migration where the replica quotas
were not being accounted. Manila now tries to allocate share
replica quotas for the migrated share if the destination share type
has a `replication_type`.
If the migration fails or get cancelled, the quotas will be
reverted.
Change-Id: Ie0a08a61c79dc8ea2a7a240363e94b26a80e64a4
Closes-Bug: 1910752
Manila can now configure a share network with multiple subnets
in an availability zone. Also, it can add a new subnet for an
availability that has share servers, which will triger an update
share server allocations.
Changes:
- API:
- Bump version to 2.70.
- setup share network with multiple subents per az.
- Block manage server with multiple subnets.
- Allow add subnet for in-use share servers.
- `share_network_subnet_id` is dropped from ShareServer view
- `share_network_subnet_ids` is added in ShareServer view
- `network_allocation_update_support` is added to ShareServer and
ShareNetwork views.
- Add a check operation for share network subnet create.
- DB:
- Remove `share_network_subnet_id` from share_servers.
- Create mapping table `share_server_share_network_subnet_mappings`.
- Fix queries with new db design.
- Add migration downgrade and upgrade alembic.
- Add `share_network_subnet_id` to the NetworkAllocations.
- Scheduler:
- Change `AvailabilityZoneFilter` to take in account if the
host supports the allocation required by the setup request.
- Manager:
- Bump RPC API version.
- `_setup_server` allocating multiple subnets.
- Modify signature of driver `_setup_server` interface, passing a
list of `network_info` for each subnet.
- Share server DB creation to inform a list of subnets and
create with `network_allocation_update_support`.
- Implement `check_update_share_server_network_allocations` and
`update_share_server_network_allocations`.
- Drivers:
- For legacy compatibility, all drivers implementing `_setup_server`
consume the first element of the `network_ino`.
- Dummy Driver:
- Implement `_setup_server` with new signature as multiple subnet.
- Modify the `backend_details` to save allocations for all subnets.
- Report update allocation and share server multiple subnet support.
- Implement `check_update_share_server_network_allocations` and
`update_share_server_network_allocations` interfaces.
Signed-off-by: Felipe Rodrigues <felipefuty01@gmail.com>
Co-Authored-By: Andre Beltrami <debeltrami@gmail.com>
Co-Authored-By: Fábio Oliveira <fabioaurelio1269@gmail.com>
Co-Authored-By: Nahim Alves de Souza <nahimsouza@outlook.com>
Co-Authored-By: Caique Mello <caique_mellosbo@hotmail.com>
DocImpact
APIImpact
Partially-Implements: blueprint multiple-share-network-subnets
Change-Id: I7de9de4ae509182e9494bba604979cce03acceec
Add support share Recycle Bin, the end user can soft delete
share to Recycle Bin, and can restore the share within 7 days,
otherwise the share will be deleted automatically.
DocImpact
APIImpact
Partially-Implements: blueprint manila-share-support-recycle-bin
Change-Id: Ic838eec5fea890be6513514053329b1d2d86b3ba
e.g. manila create NFS 1 --name Share1 --share-network net1 \
--scheduler_hint="only_host=host1@generic1#GENERIC1"
Since there is no way to create share server in manila, we can use a
workaround of creating first share on specific host
(e.g. host@backend#pool). This will then create the share server
automatically on that host and admin can use idle host when other
hosts are overloaded.
New microversion 2.67 introduced.
DocImpact
Closes-Bug: #1946462
Change-Id: I603434cac246e2c0946672d3f0fe469ed5423fa4
Replaced a non-existing snapshot delete method
with an appropriate one and added tests for
the same
Closes-Bug: #1475351
Change-Id: Idb4b6de5108dd1862df7de97d0ffdc15282b1d74
A series of enhancements can be performed in the share server
migration operation, and share backends might support
nondisruptive share server migrations, which was not covered
in the previous approach.
- Skip the destination share server network allocation if the
driver is capable of reusing the source share server allocation.
Manila will switch the allocations in the migration complete phase.
- Allow share backends to reuse the share servers in case they
are able to do so in a share server migration scenario.
- Nondisruptive migration is now feasible depending on whether the
share driver supports it and if the share network parameters have
not changed.
- Share servers will be always deleted whe the share server
migration complete operation is finished.
Depends-On: I43bd3fdafea02eb8e853114a57bfb863a441f3ed
Co-Authored-By: Fabio Oliveira <fabioaurelio1269@gmail.com>
Change-Id: I48bafd92fe7a4d4ae0bafd5bf1961dace56b6005
This patch implements hard affinity and anti-affinity filter for
manila scheduler. Users can specify affinity/anti-affinity share
ids to the field "share.scheduler_hints.same_host" or
"share.scheduler_hints.different_host" in the request payload
when creating a manila share. The scheduler_hints are stored as
share metadata. The filter properties are populated from this
metadata during share migration and so filters will be applied
for share migration as well.
Both fields can be a single share UUID or multiple uuids
separated by comma. For example,
`{
"share": {
"scheduler_hints": {
"same_host": "share_uuid_1,share_uuid_2",
"different_host": "share_uuid_3"
}
}
}`
Implements: bp/affinity-antiaffinity-filter
Change-Id: Ic42d8a0c1d22e77ae64e0ca014607b28fd336467
Co-authored-by: Maurice Escher <maurice.escher@sap.com>
Add an force parameter to the API layer that lets the user choose
whether to go through the scheduler or not, which is boolean.and
default is False,set True means extend share directly, set False
means extend share will go through scheduler.
Add an new min_version 2.64 to extend share api. force parameter
only support min_version >= 2.64.
Closes-Bug:#1855391
Change-Id: I6da36a687a37c78a7fb7d3f252318d03d4a05133
Currently we register all parameters about the quota feature to
[DEFAULT] section. However it is difficult for operators to identity
that some of these parameters like reservation_expire are used in the
quota feature, based of names and descriptions of the parameters.
This change migrates all quota options to the new [quota] section,
so that operators can easily understand which parameters are used
in the quota feature.
Closes-Bug: #1934025
Change-Id: I83b5750da7083718d39efb56262a49dae0a05f48
share snap list, Fuzzy query by name or description is supported, but
the current implementation is first get all the shares, then In the API
layer, the for loop is used to achieve fuzzy matching, if the num of
shares is big,
It will seriously affect the speed of fuzzy matching.Therefore, we
should let the database do the matching to speed up the query.
Moving the pagination params (limit, offset, sorting) to the database
layer for snapshot list, to optimize query speed.
Closes-Bug:#1881865
Partial-Bug:#1831094
Change-Id: I283e78c9e7c2dd626d94cf6c1b01d4e2f9ae8097
This bugfix [1] modified the totalcount returned by pagination
query when the argument 'limit' was specified. It caused
manila to do not return precise count of shares in a query that
satisfied the conditions.
This bug has been fixed and now manila is returning the precise
values of shares matched in a given query. Also, manila is now
performing filtering actions in the database to have more
performatic results.
[1] https://review.opendev.org/#/c/688542/
Closes-Bug: #1860061
Co-Authored-By: Carlos Eduardo <ces.eduardo98@gmail.com>
Change-Id: I6ddd919bbd5180593cc52bf986912f65a2dab3a7
This is a follow up patch for [1] which removes some unused model
properties and add more unit tests to increase coverage.
[1] https://review.opendev.org/c/openstack/manila/+/774728
Change-Id: I10af132203cb07cea62839014925c7e4c99499e4
Signed-off-by: Douglas Viroel <viroel@gmail.com>
This patch implements the update of security service's association
with in-use share networks. The following changes were added:
- New share network APIs: `share_network_security_service_update`
and `share_network_reset_state`.
- A new `status` attribute was added to share network model to
identify when it's in a modification state, called 'network_change'.
Other supported status that were added: 'active' and 'error'.
- New 'security_service_update_support' property was added to both
share server and share network models, to identify when this resources
are able to process security service update for in-use share networks.
- New driver interface was added to support update of security service's
configuration of a given share server.
DocImpact
APIImpact
Partially Implements: bp add-security-service-in-use-share-networks
Co-Authored-By: Carlos Eduardo <ces.eduardo98@gmail.com>
Co-Authored-By: Douglas Viroel <viroel@gmail.com>
Co-Authored-By: Andre Beltrami <debeltrami@gmail.com>
Change-Id: I129a794dfd2d179fa2b9a2fed050459d6f00b0de
This feature allows admin to set share size limit for a project.
The defaults will either come from the default values
set in the quota configuration option or via manila.conf
if the user has configured default values for quotas there.
The quota_per_share_gigabytes defaults to -1["No Limit"] always
unless changed in manila.conf by admin.
Closes-Bug: #1811943
Change-Id: Ida126c8c419b8bf4d2a194f061a0809d52b47ab8
Allows set min/max share size that can be created in
extra_specs for each share_type.the share size will
be checked at API level as part of share create,
extend, shrink, migration_start. when manage share,
check it after get true size of share at manager layer.
new extra_specs keys are supported for set min/max
size of share.
'provisioning:max_share_size'
'provisioning:min_share_size'
Implements: blueprint share-size-limited-by-share-type
Change-Id: I5ce0fabf59bfca5ebaf0be5ffe9986e2b0480295
Non privileged users of unrelated projects
must not be able to retrieve details of an
access rule. We can add a further check to
/share-access-rules APIs to validate that
the caller has access to the share that these
rules pertain to.
Change-Id: I0009a3d682ee5d9a946821c3f82dfd90faa886aa
Closes-Bug: #1917417
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
Moves the manila shares filtering to the database in order to have
the queries performance improved.
Change-Id: I031a3b9775c50e78b6b86752ff8d1a4871a91c0c
Co-Authored-By: MaAoyu <maaoyu@inspur.com>
The share "manage" API checks whether
an existing/known share is being imported by
matching the export path provided to existing
shares.
This lookup does not consider the
fact that shares may have multiple export
locations, because it relies on an old/deprecated
"export_location" property on shares which
was added to provide backwards compatibility
to the API that presented only one export
location per share.
Further, it's possible to get a
"ERROR: Invalid share: Share already exists"
exception even when no such share exists in the
database.
Fix the lookup by using the "export_location_path"
based lookup which is faster, since it performs
a meaningful join on the export locations table;
and remove the parameters "protocol"
and "share_type_id" - these things make no
difference when there's a duplicated export
location. We'll consider "host" as a lookup
parameter since we can't be sure that export
locations are unique in a deployment - but they
ought to be unique for a given host.
Closes-Bug: #1848608
Closes-Bug: #1893718
Change-Id: I1d1aef0c2b48764789b43b91b258def6464b389f
Co-Authored-By: Goutham Pacha Ravi <gouthampravi@gmail.com>
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
Share manager may raise InvalidShare because the migration phase
has already been completed. In such cases total_progress 100 can
be reported instead of raising ShareMigrationError.
Change-Id: I6e1b1abf6a2fd8c1268e446b7ee8e364bdc496bc
Closes-Bug: #1889549
This patch adds support for migration of share servers. This
migration is performed using a two-phase approach. Administrators
are now able to request the migration of a share server within and
across backends, with the possibility of chooosing a different share
network for the destination share server.
- A new field called `task_state` was added to the share server
model in order to help the administrator to track the share server
migration steps. A new field called `source_share_server_id` was
added to link destination and source share servers.
- A new periodic task was added to track migration of share servers
and its resources.
- Two new states were added: `server_migrating` and
`server_migrating_to` to represent that share migration is in
progress.
- When performing the server migration, manila will not go to the
scheduler, instead it will provide a request spec to drivers
during migration check driver call. It'll be up to the driver
validate if there is free space to handle the share server.
- A new API called `share-server-migration-check' was added to
check the feasibility of a migration, before actually triggering
the start operation.
APIImpact
DocImpact
Partially Implements: bp share-server-migration
Co-Authored-By: Andre Beltrami <debeltrami@gmail.com>
Co-Authored-By: Carlos Eduardo <ces.eduardo98@gmail.com>
Co-Authored-By: Felipe Rodrigues <felipefuty01@gmail.com>
Change-Id: Ic0751027d2c3f1ef7ab0f7836baff3070a230cfd
Signed-off-by: Douglas Viroel <viroel@gmail.com>
This patch enables the creation of a share from snapshot
specifying another pool or backend. In the scheduler, a
new filter and weigher were implemented in order to consider
this operation if the backend supports it. Also, a new
field called 'progress' was added in the share and share
instance. The 'progress' field indicates the status
of the operation create share from snapshot (in percentage).
Finally, a new periodic task was added in order to constantly
check the share status.
Partially-implements: bp create-share-from-snapshot-in-another-pool-or-backend
DOCImpact
Change-Id: Iab13a0961eb4a387a502246e5d4b79bc9046e04b
Co-authored-by: carloss <ces.eduardo98@gmail.com>
Co-authored-by: dviroel <viroel@gmail.com>
This patch adds new quotas for share replicas and replica sizes.
This quotas can be related to either tenants and users or tenants
and share types. Now, when creating a share replica, manila will
check if there are resources available for that specific request.
Partially-Implements: bp limit-share-replicas-per-share
Change-Id: I8ba7bc6f167c28d6c169b2187d0e1bda7cad3f69
Fix:
E241 multiple spaces after ':' (or ',')
Fix also a few other problems found by hacking in files changed.
Change-Id: I4c00551f29ad65df2631c8ae55fe2f022e872544
Kiran Pawar