If we are manually creating these things, we don't want any
automatically created. This is a latent issue that was being exposed by
SQLAlchemy 2.0's tweaked session management.
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: I9d760ebbfcca176a39b44f20fb92e16eaf6284c5
Now that we have migrated all of our DB APIs to enginefacade, we can
clean up the outstanding TODOs.
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: Iee198a16f030f1205d38b7c232d4e3a42642c756
We migrate both of these at the same time since a share replica is
effectively a share instance with only the 'replica_state' field set. As
such, they share many common DB APIs.
These are mostly trivial to migrate. Just a lot of refactoring to create
private methods, avoiding nested transactions.
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: I267e2e0fac0b4130f8836de276082788fbd8d454
This better aligns with 'export_location_metadata_*' APIs. The plural
'export_locations_get*' APIs are renamed to 'export_location_get_all*'.
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: I14f2b834e7ac2d8be86f9d7e381706cadbb79eb8
Rename a number of APIs to use singular, rather than plural, like every
other API uses:
- share_instances_status_update ->
share_instance_status_update
- share_instances_get_all ->
share_instance_get_all
- share_instances_get_all_by_host ->
share_instance_get_all_by_host
- share_instances_get_all_by_share_network ->
share_instance_get_all_by_share_network
- share_instances_get_all_by_share_server ->
share_instance_get_all_by_share_server
- share_instances_get_all_by_share ->
share_instance_get_all_by_share
- share_instances_get_all_by_share_group_id ->
share_instance_get_all_by_share_group_id
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: Ic48fe0d1631a6e1a8ee9a50741cc1b31c9187c37
This reverts commit e885852648.
Reason for revert: This change is breaking drivers that are able to thin provisioning shares. The shares join in the sqlalchemy/api is not working properly, as the share doesn't have an attribute called shares. The join should be performed differently.
Change-Id: Ia5b4e0d57294c3972f55f4fa9a2750ab0251f7bf
Access rules rules allow API will now take three additional
parameters:
- lock_visibility: when True, only services, administrators and
the same user will be able to see the content of ``access_to`` and
access_key.
- lock_deletion: when True, the access rule will be locked for
deletion. Only services, administrators or the user that placed
the lock will be able to drop the access rule.
- lock_reason: a reason for the lock. This parameter should only
be provided in the presence of at least one of the former
parameters.
In order to delete an access rule that is currently locked, the
requester will need to specify ``unrestrict=True`` in the request.
In case a service placed the restrictions, only the own service or
the system administrator will be able to release it.
This change also implements filters to the access list API. It is
now possible to filter access rules based on `access_to`,
`access_type`, `access_level` and `access_key`.
DocImpact
Change-Id: Iea422c9d6bc99a81cd88c5f4b7055d6a1cf97fdc
Add CRUD APIs for resource locks with support
for preventing deletion of shares (applies to
soft-deletions and unmanage operations as well).
Change-Id: I146bc09e4e8a39797e22458ff6860346e11e592e
Implements: bp/allow-locking-shares-against-deletion
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
The scheduler need to estimate the allocated capacity for thin
provisioning hosts. To do this more efficiently, use a sum fucntion in
query, instead of loading all share instances for a specific host.
Closes-Bug: #2020187
Change-Id: I1f6883c50c74c0822e787de28ba5118a8292e348
Add share backup feature in Data Copy Service and
Share Service. It will allow the user to create, restore
and delete backups as well as listing backups and showing
the details of a specific backup.
APIImpact
DOCImpact
Change-Id: I7d10cf47864cd21932315375d84dc728ff738f23
Implement: blueprint share-backup
Added support for display count info in share snapshot
list&detail APIs:
1. /v2/snapshots?with_count=True
2. /v2/snapshots/detail?with_count=True
New microversion added 2.79
Closes-bug: #2024556
Change-Id: I37d8ca9022e2ea2c107c6695e20e951d7950043a
Another beefy one, owing to how intertwined these all are. The only
unusual thing is that we have a number of methods that are only used by
the quota sync functionality. One was exposed in 'manila.db.api' and
more were not prefixed with an underscore indicating they were private.
Since these aren't used outside this module, we can mark them all as
private and remove the sole unused public API from 'manila.db.api'.
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: I9771dbae4663461c79e5f9bcef730af2b928212d
This one is beefier than usual, since we're migrating not only share
group and share group type-related APIs, but also things like share
group type members, share group type snapshots, etc.
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: I8ff25174626385b8d1d0b11d6846e345e2d9b125
This change adds metadata controller for Share Network Subnets
resource. Bumps microversion to 2.78.
The subnet metadata is passed down to the driver together
with the network_info object.
APIImpact
Partially-implements: bp/metadata-for-share-resources
Change-Id: I8d5a03eb127941a84eea5e6e9bdf76b3489f17a8
user can create a transfer for a share. will return transfer id
and auth_key. another user can use transfer_id and auth_key to
accept this share. salt of transfer and auth_key compute crypt_hash
by sha1. then compare with crypt_hash in db.
APIImpact
DocImpact
Partially-Implements: blueprint transfer-share-between-project
Change-Id: I8facf9112a6b09e6b7aed9956c0a87fb5f1fc31f
Share replica create API does not allow to specify share network and
forces to use parent share's share network. This is problem for some
use-cases, e.g. migration from one share network to another share
network via replication is not possible. Fixed by allowing to pass
'share-network' option for share replica create API and make sure both
parent share-network and user provided share-network will have same
security service association.
Partial-Bug: #1925486
Change-Id: I9049dcd418fbb16d663ab8ed27b90c765fafc5d3
The change [1] modified the function
`share_server_get_all_by_host_and_share_subnet_valid` adding the
status option. However, it removed the actual filter from the query
wrongly. As result, the method is returning all share servers without
taking the status in account.
This patch reverts that change and also creating a new one for
collecting all share servers no matter their states.
[1] https://review.opendev.org/c/openstack/manila/+/825110
Closes-Bug: #1978962
Change-Id: I8d9437eafde67407ba5e337dd495fdb74eefec70
Adds a check when associating a security service to a share network, so
that both resources must have the same project_id. If not,
a HTTP Bad Request is raised. Affiliated tests were altered or created.
Closes-Bug: #1918323
Change-Id: Idb2a8838d492ac3c616fb21ab1272f7dc74ee589
Manila can now configure a share network with multiple subnets
in an availability zone. Also, it can add a new subnet for an
availability that has share servers, which will triger an update
share server allocations.
Changes:
- API:
- Bump version to 2.70.
- setup share network with multiple subents per az.
- Block manage server with multiple subnets.
- Allow add subnet for in-use share servers.
- `share_network_subnet_id` is dropped from ShareServer view
- `share_network_subnet_ids` is added in ShareServer view
- `network_allocation_update_support` is added to ShareServer and
ShareNetwork views.
- Add a check operation for share network subnet create.
- DB:
- Remove `share_network_subnet_id` from share_servers.
- Create mapping table `share_server_share_network_subnet_mappings`.
- Fix queries with new db design.
- Add migration downgrade and upgrade alembic.
- Add `share_network_subnet_id` to the NetworkAllocations.
- Scheduler:
- Change `AvailabilityZoneFilter` to take in account if the
host supports the allocation required by the setup request.
- Manager:
- Bump RPC API version.
- `_setup_server` allocating multiple subnets.
- Modify signature of driver `_setup_server` interface, passing a
list of `network_info` for each subnet.
- Share server DB creation to inform a list of subnets and
create with `network_allocation_update_support`.
- Implement `check_update_share_server_network_allocations` and
`update_share_server_network_allocations`.
- Drivers:
- For legacy compatibility, all drivers implementing `_setup_server`
consume the first element of the `network_ino`.
- Dummy Driver:
- Implement `_setup_server` with new signature as multiple subnet.
- Modify the `backend_details` to save allocations for all subnets.
- Report update allocation and share server multiple subnet support.
- Implement `check_update_share_server_network_allocations` and
`update_share_server_network_allocations` interfaces.
Signed-off-by: Felipe Rodrigues <felipefuty01@gmail.com>
Co-Authored-By: Andre Beltrami <debeltrami@gmail.com>
Co-Authored-By: Fábio Oliveira <fabioaurelio1269@gmail.com>
Co-Authored-By: Nahim Alves de Souza <nahimsouza@outlook.com>
Co-Authored-By: Caique Mello <caique_mellosbo@hotmail.com>
DocImpact
APIImpact
Partially-Implements: blueprint multiple-share-network-subnets
Change-Id: I7de9de4ae509182e9494bba604979cce03acceec
Add support share Recycle Bin, the end user can soft delete
share to Recycle Bin, and can restore the share within 7 days,
otherwise the share will be deleted automatically.
DocImpact
APIImpact
Partially-Implements: blueprint manila-share-support-recycle-bin
Change-Id: Ic838eec5fea890be6513514053329b1d2d86b3ba
1: As admin user, query share network list with specified
"security_service_id" and "project_id" search opts, will
get wrong result.
2: put "created_since", "created_before" search opts into database to
Increase query speed
Closes-Bug: #1923008
Change-Id: I49e412cb6c98fcda67531ff915b3b4c3edc64476
A series of enhancements can be performed in the share server
migration operation, and share backends might support
nondisruptive share server migrations, which was not covered
in the previous approach.
- Skip the destination share server network allocation if the
driver is capable of reusing the source share server allocation.
Manila will switch the allocations in the migration complete phase.
- Allow share backends to reuse the share servers in case they
are able to do so in a share server migration scenario.
- Nondisruptive migration is now feasible depending on whether the
share driver supports it and if the share network parameters have
not changed.
- Share servers will be always deleted whe the share server
migration complete operation is finished.
Depends-On: I43bd3fdafea02eb8e853114a57bfb863a441f3ed
Co-Authored-By: Fabio Oliveira <fabioaurelio1269@gmail.com>
Change-Id: I48bafd92fe7a4d4ae0bafd5bf1961dace56b6005
This patch implements hard affinity and anti-affinity filter for
manila scheduler. Users can specify affinity/anti-affinity share
ids to the field "share.scheduler_hints.same_host" or
"share.scheduler_hints.different_host" in the request payload
when creating a manila share. The scheduler_hints are stored as
share metadata. The filter properties are populated from this
metadata during share migration and so filters will be applied
for share migration as well.
Both fields can be a single share UUID or multiple uuids
separated by comma. For example,
`{
"share": {
"scheduler_hints": {
"same_host": "share_uuid_1,share_uuid_2",
"different_host": "share_uuid_3"
}
}
}`
Implements: bp/affinity-antiaffinity-filter
Change-Id: Ic42d8a0c1d22e77ae64e0ca014607b28fd336467
Co-authored-by: Maurice Escher <maurice.escher@sap.com>
share snap list, Fuzzy query by name or description is supported, but
the current implementation is first get all the shares, then In the API
layer, the for loop is used to achieve fuzzy matching, if the num of
shares is big,
It will seriously affect the speed of fuzzy matching.Therefore, we
should let the database do the matching to speed up the query.
Moving the pagination params (limit, offset, sorting) to the database
layer for snapshot list, to optimize query speed.
Closes-Bug:#1881865
Partial-Bug:#1831094
Change-Id: I283e78c9e7c2dd626d94cf6c1b01d4e2f9ae8097
This bugfix [1] modified the totalcount returned by pagination
query when the argument 'limit' was specified. It caused
manila to do not return precise count of shares in a query that
satisfied the conditions.
This bug has been fixed and now manila is returning the precise
values of shares matched in a given query. Also, manila is now
performing filtering actions in the database to have more
performatic results.
[1] https://review.opendev.org/#/c/688542/
Closes-Bug: #1860061
Co-Authored-By: Carlos Eduardo <ces.eduardo98@gmail.com>
Change-Id: I6ddd919bbd5180593cc52bf986912f65a2dab3a7
This patch implements the update of security service's association
with in-use share networks. The following changes were added:
- New share network APIs: `share_network_security_service_update`
and `share_network_reset_state`.
- A new `status` attribute was added to share network model to
identify when it's in a modification state, called 'network_change'.
Other supported status that were added: 'active' and 'error'.
- New 'security_service_update_support' property was added to both
share server and share network models, to identify when this resources
are able to process security service update for in-use share networks.
- New driver interface was added to support update of security service's
configuration of a given share server.
DocImpact
APIImpact
Partially Implements: bp add-security-service-in-use-share-networks
Co-Authored-By: Carlos Eduardo <ces.eduardo98@gmail.com>
Co-Authored-By: Douglas Viroel <viroel@gmail.com>
Co-Authored-By: Andre Beltrami <debeltrami@gmail.com>
Change-Id: I129a794dfd2d179fa2b9a2fed050459d6f00b0de
Fixes the OverQuota issue while managing shares. Now, when a share
manage request is received and there is no available quota to this
resource, manila will allow the quotas to be exceeded and the
administrator will need to adjust it.
Change-Id: If6edfa79965f1a0e6959b436c53a714217d23f7d
Closes-Bug: #1863298
We can also condense this code a bit to allow
adding more resources who have a "host" attribute.
Closes-Bug: #1895323
Change-Id: I5b750217e019b7bc5281006f63feba8fa06db534
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
This patch adds support for migration of share servers. This
migration is performed using a two-phase approach. Administrators
are now able to request the migration of a share server within and
across backends, with the possibility of chooosing a different share
network for the destination share server.
- A new field called `task_state` was added to the share server
model in order to help the administrator to track the share server
migration steps. A new field called `source_share_server_id` was
added to link destination and source share servers.
- A new periodic task was added to track migration of share servers
and its resources.
- Two new states were added: `server_migrating` and
`server_migrating_to` to represent that share migration is in
progress.
- When performing the server migration, manila will not go to the
scheduler, instead it will provide a request spec to drivers
during migration check driver call. It'll be up to the driver
validate if there is free space to handle the share server.
- A new API called `share-server-migration-check' was added to
check the feasibility of a migration, before actually triggering
the start operation.
APIImpact
DocImpact
Partially Implements: bp share-server-migration
Co-Authored-By: Andre Beltrami <debeltrami@gmail.com>
Co-Authored-By: Carlos Eduardo <ces.eduardo98@gmail.com>
Co-Authored-By: Felipe Rodrigues <felipefuty01@gmail.com>
Change-Id: Ic0751027d2c3f1ef7ab0f7836baff3070a230cfd
Signed-off-by: Douglas Viroel <viroel@gmail.com>
This patch enables the creation of a share from snapshot
specifying another pool or backend. In the scheduler, a
new filter and weigher were implemented in order to consider
this operation if the backend supports it. Also, a new
field called 'progress' was added in the share and share
instance. The 'progress' field indicates the status
of the operation create share from snapshot (in percentage).
Finally, a new periodic task was added in order to constantly
check the share status.
Partially-implements: bp create-share-from-snapshot-in-another-pool-or-backend
DOCImpact
Change-Id: Iab13a0961eb4a387a502246e5d4b79bc9046e04b
Co-authored-by: carloss <ces.eduardo98@gmail.com>
Co-authored-by: dviroel <viroel@gmail.com>
This patch adds new quotas for share replicas and replica sizes.
This quotas can be related to either tenants and users or tenants
and share types. Now, when creating a share replica, manila will
check if there are resources available for that specific request.
Partially-Implements: bp limit-share-replicas-per-share
Change-Id: I8ba7bc6f167c28d6c169b2187d0e1bda7cad3f69
Add support for querying user messages by specifying a timestamp, which
will be compared to the created_at field, and manila will return all the
messages matching to the time condition.
Partially-Implements: blueprint query-user-message-by-timestamp
Change-Id: I3d94bac4304c236f275abe7ce31432b0e2384247
When searching for compatible share servers, the share manager now
looks for share servers that matches the destination host and the
related share network subnet id.
Change-Id: I62b80e24df443789359bb8d7ceaaf3d704366bc9
This patch adds the possibility to create share networks with
multiple subnets in Manila. It also updates the share server api
to receive "share_network_subnet_id" instead of "share_network_id".
Each share network subnet must be associated with only one
availability zone. Each share network must have a single default
share network subnet.
DocImpact
APIImpact
Depends-On: I13bb48e7c03e16c26946ccf9d48e80592391a3d1
Partially-implements: bp share-network-multiple-subnets
Change-Id: Id8814a8b26c9b9dcb1fe71d0d7e9b79e8b8a9210
Closes-Bug: #1588144
Co-Authored-By: lseki <luciomitsuru.seki@fit-tecnologia.org.br>
Co-Authored-By: dviroel <viroel@gmail.com>
Currently, only the name and description and public access of share-type
is set when the share-type is created, and not allowed to be edited
after the share-type is created. We can only set extra spec for share-type.
But not name or description or public access for share-type.
Co-Authored-By: Brin Zhang <zhangbailin@inspur.com>
APIImpact
Implements: blueprint update-share-type-name-or-description
Change-Id: I4c7bdd601d48b40c01639b5089d4bff259a7b3af
Stephen Finucane