If we are manually creating these things, we don't want any
automatically created. This is a latent issue that was being exposed by
SQLAlchemy 2.0's tweaked session management.
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: I9d760ebbfcca176a39b44f20fb92e16eaf6284c5
Thankfully the APIs being migrated here were _mostly_ sharing sessions
already, so we can simply migrate from public (decorated) methods to
private methods with minimal fuss.
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: Id1b555e48106662d15e8c50567a5f3acecf6a8f1
We migrate both of these at the same time since a share replica is
effectively a share instance with only the 'replica_state' field set. As
such, they share many common DB APIs.
These are mostly trivial to migrate. Just a lot of refactoring to create
private methods, avoiding nested transactions.
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: I267e2e0fac0b4130f8836de276082788fbd8d454
Share network subnet is deleted during share network deletion which was
missing earlier causing lots of stale db entries.
Change-Id: Idb58b4ca389cfb1aaf8f9f1f8aa2fd15979c4872
Closes-bug: #2037422
We are currently running this outside of the main session of many APIs.
This is tough to do when the session is created as a decorator. Make
life easier by using a decorator, which allows us to use a separate
context and separate session.
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: I95187d5b9bc21120c74b6784489c1b0dbbf337da
This better aligns with 'export_location_metadata_*' APIs. The plural
'export_locations_get*' APIs are renamed to 'export_location_get_all*'.
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: I14f2b834e7ac2d8be86f9d7e381706cadbb79eb8
Rename a number of APIs to use singular, rather than plural, like every
other API uses:
- share_instances_status_update ->
share_instance_status_update
- share_instances_get_all ->
share_instance_get_all
- share_instances_get_all_by_host ->
share_instance_get_all_by_host
- share_instances_get_all_by_share_network ->
share_instance_get_all_by_share_network
- share_instances_get_all_by_share_server ->
share_instance_get_all_by_share_server
- share_instances_get_all_by_share ->
share_instance_get_all_by_share
- share_instances_get_all_by_share_group_id ->
share_instance_get_all_by_share_group_id
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: Ic48fe0d1631a6e1a8ee9a50741cc1b31c9187c37
Previsously, when trying to manage a share server with a given
identifier, the db API checked if the identifier is a substring
of any existing identifier and if it is True, it raised an
InvalidInput Exception.
Now, the InvalidInput expception is raised only if the
identifier is a suffix of any existing identifier and it is
ignored when it is just a preffix. Some queries from the
sqlalchemy API share_server_search_by_identifier method
were edited to implement this change.
Depends-On: I14df0c9c7326ac49274699dc0ef1adb99570784b
Closes-bug: #1982429
Change-Id: I4b340ef116625f28e431b069f399771fd29a8de9
Share/Snapshot list API returns count of shares/snapshotsi along-with
list of resource objects. The list returned by API contains correct
entries of shares/snapshots but it returns the wrong count as it
ShareInstances/SnapshotInstances. So filter queries by unique
share_id/snapshot_id.
Closes-bug: #2033604
Change-Id: I13c415767b0d126563f9553df415de564b3901d6
Access rules rules allow API will now take three additional
parameters:
- lock_visibility: when True, only services, administrators and
the same user will be able to see the content of ``access_to`` and
access_key.
- lock_deletion: when True, the access rule will be locked for
deletion. Only services, administrators or the user that placed
the lock will be able to drop the access rule.
- lock_reason: a reason for the lock. This parameter should only
be provided in the presence of at least one of the former
parameters.
In order to delete an access rule that is currently locked, the
requester will need to specify ``unrestrict=True`` in the request.
In case a service placed the restrictions, only the own service or
the system administrator will be able to release it.
This change also implements filters to the access list API. It is
now possible to filter access rules based on `access_to`,
`access_type`, `access_level` and `access_key`.
DocImpact
Change-Id: Iea422c9d6bc99a81cd88c5f4b7055d6a1cf97fdc
Add CRUD APIs for resource locks with support
for preventing deletion of shares (applies to
soft-deletions and unmanage operations as well).
Change-Id: I146bc09e4e8a39797e22458ff6860346e11e592e
Implements: bp/allow-locking-shares-against-deletion
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
Add share backup feature in Data Copy Service and
Share Service. It will allow the user to create, restore
and delete backups as well as listing backups and showing
the details of a specific backup.
APIImpact
DOCImpact
Change-Id: I7d10cf47864cd21932315375d84dc728ff738f23
Implement: blueprint share-backup
Added support for display count info in share snapshot
list&detail APIs:
1. /v2/snapshots?with_count=True
2. /v2/snapshots/detail?with_count=True
New microversion added 2.79
Closes-bug: #2024556
Change-Id: I37d8ca9022e2ea2c107c6695e20e951d7950043a
share_server_backend_details_set() add entries in db table without
checking existing entries with given combinaton of share_server_id
and key. This causes duplicate records. Fix it by validating presence
of share server id and key.
Closes-bug: #2024658
Change-Id: I58dcd9716cf95d0d696c13a4c831df787726bcda
Another beefy one, owing to how intertwined these all are. The only
unusual thing is that we have a number of methods that are only used by
the quota sync functionality. One was exposed in 'manila.db.api' and
more were not prefixed with an underscore indicating they were private.
Since these aren't used outside this module, we can mark them all as
private and remove the sole unused public API from 'manila.db.api'.
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: I9771dbae4663461c79e5f9bcef730af2b928212d
read_deleted option
'yes' indicates deleted records are visible,
'only' indicates that *only* deleted records are visible.
Change-Id: I9be0f8127fac6c2294891d209195d9d0711a907b
Closes-Bug: #2015094
This change adds metadata controller for Share Network Subnets
resource. Bumps microversion to 2.78.
The subnet metadata is passed down to the driver together
with the network_info object.
APIImpact
Partially-implements: bp/metadata-for-share-resources
Change-Id: I8d5a03eb127941a84eea5e6e9bdf76b3489f17a8
user can create a transfer for a share. will return transfer id
and auth_key. another user can use transfer_id and auth_key to
accept this share. salt of transfer and auth_key compute crypt_hash
by sha1. then compare with crypt_hash in db.
APIImpact
DocImpact
Partially-Implements: blueprint transfer-share-between-project
Change-Id: I8facf9112a6b09e6b7aed9956c0a87fb5f1fc31f
Allows to configure optional field 'default_ad_site' from version 2.76.
Restrict to make sure either server or 'default_at_site' provided, but
not both.
APIImpact
Relates-bug: #1988146
Change-Id: I8e21e9170eace134a51efed84de1ccc58eb7eaaa
Migrate service-related APIs from the legacy enginefacade to the modern
context-based enginefacade. This one is a little tricky as it highlights
two issues.
* The models in 'manila.db.sqlalchemy.models' do not match those created
by the migrations. This causes us issues since we apply the former
rather than running migrations when creating our database via the
DatabaseFixture, which means we're not actually testing against what
our customers see. One gap is closed here in order to get newly added
test working as expected but there's a larger issue that needs to be
addressed at a later date.
* We are using a pattern of "try to create and retrieve on failure"
rather than "try to get and create on failure" when determining
whether to create availability zone records. This works where each
operation results in a new transaction but does not when the
transaction is shared. We switch this.
With these two issues addressed, the rest of the migration is relatively
straightforward.
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: Icfe4b225e216dfda46366af1eb3820b193befa0f
Share replica create API does not allow to specify share network and
forces to use parent share's share network. This is problem for some
use-cases, e.g. migration from one share network to another share
network via replication is not possible. Fixed by allowing to pass
'share-network' option for share replica create API and make sure both
parent share-network and user provided share-network will have same
security service association.
Partial-Bug: #1925486
Change-Id: I9049dcd418fbb16d663ab8ed27b90c765fafc5d3
The change [1] modified the function
`share_server_get_all_by_host_and_share_subnet_valid` adding the
status option. However, it removed the actual filter from the query
wrongly. As result, the method is returning all share servers without
taking the status in account.
This patch reverts that change and also creating a new one for
collecting all share servers no matter their states.
[1] https://review.opendev.org/c/openstack/manila/+/825110
Closes-Bug: #1978962
Change-Id: I8d9437eafde67407ba5e337dd495fdb74eefec70
Manila can now configure a share network with multiple subnets
in an availability zone. Also, it can add a new subnet for an
availability that has share servers, which will triger an update
share server allocations.
Changes:
- API:
- Bump version to 2.70.
- setup share network with multiple subents per az.
- Block manage server with multiple subnets.
- Allow add subnet for in-use share servers.
- `share_network_subnet_id` is dropped from ShareServer view
- `share_network_subnet_ids` is added in ShareServer view
- `network_allocation_update_support` is added to ShareServer and
ShareNetwork views.
- Add a check operation for share network subnet create.
- DB:
- Remove `share_network_subnet_id` from share_servers.
- Create mapping table `share_server_share_network_subnet_mappings`.
- Fix queries with new db design.
- Add migration downgrade and upgrade alembic.
- Add `share_network_subnet_id` to the NetworkAllocations.
- Scheduler:
- Change `AvailabilityZoneFilter` to take in account if the
host supports the allocation required by the setup request.
- Manager:
- Bump RPC API version.
- `_setup_server` allocating multiple subnets.
- Modify signature of driver `_setup_server` interface, passing a
list of `network_info` for each subnet.
- Share server DB creation to inform a list of subnets and
create with `network_allocation_update_support`.
- Implement `check_update_share_server_network_allocations` and
`update_share_server_network_allocations`.
- Drivers:
- For legacy compatibility, all drivers implementing `_setup_server`
consume the first element of the `network_ino`.
- Dummy Driver:
- Implement `_setup_server` with new signature as multiple subnet.
- Modify the `backend_details` to save allocations for all subnets.
- Report update allocation and share server multiple subnet support.
- Implement `check_update_share_server_network_allocations` and
`update_share_server_network_allocations` interfaces.
Signed-off-by: Felipe Rodrigues <felipefuty01@gmail.com>
Co-Authored-By: Andre Beltrami <debeltrami@gmail.com>
Co-Authored-By: Fábio Oliveira <fabioaurelio1269@gmail.com>
Co-Authored-By: Nahim Alves de Souza <nahimsouza@outlook.com>
Co-Authored-By: Caique Mello <caique_mellosbo@hotmail.com>
DocImpact
APIImpact
Partially-Implements: blueprint multiple-share-network-subnets
Change-Id: I7de9de4ae509182e9494bba604979cce03acceec
Add support share Recycle Bin, the end user can soft delete
share to Recycle Bin, and can restore the share within 7 days,
otherwise the share will be deleted automatically.
DocImpact
APIImpact
Partially-Implements: blueprint manila-share-support-recycle-bin
Change-Id: Ic838eec5fea890be6513514053329b1d2d86b3ba
1: As admin user, query share network list with specified
"security_service_id" and "project_id" search opts, will
get wrong result.
2: put "created_since", "created_before" search opts into database to
Increase query speed
Closes-Bug: #1923008
Change-Id: I49e412cb6c98fcda67531ff915b3b4c3edc64476
share snap list, Fuzzy query by name or description is supported, but
the current implementation is first get all the shares, then In the API
layer, the for loop is used to achieve fuzzy matching, if the num of
shares is big,
It will seriously affect the speed of fuzzy matching.Therefore, we
should let the database do the matching to speed up the query.
Moving the pagination params (limit, offset, sorting) to the database
layer for snapshot list, to optimize query speed.
Closes-Bug:#1881865
Partial-Bug:#1831094
Change-Id: I283e78c9e7c2dd626d94cf6c1b01d4e2f9ae8097
Stephen Finucane