Commit Graph

453 Commits

Author SHA1 Message Date
Stephen Finucane 16b02bc706 tests: Don't (always) auto-create snapshot instances
If we are manually creating these things, we don't want any
automatically created. This is a latent issue that was being exposed by
SQLAlchemy 2.0's tweaked session management.

Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: I9d760ebbfcca176a39b44f20fb92e16eaf6284c5
2024-04-06 09:51:36 +01:00
Stephen Finucane 825ee37432 db: rename_table is not a batch operation
This is reported upstream [1] but I suspect the root cause is that we
were relying on a bug in Alembic: renaming tables doesn't really make
sense as a batch operation, which by definition works by recreating
tables with an updated schema (to support SQLite and its lack of full
'ALTER' support).

[1] https://github.com/sqlalchemy/alembic/issues/1453

Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: I1da6d117778bbbad64b2df2dfd2f8aeef8a7084c
2024-04-05 13:00:28 +01:00
Stephen Finucane 9c94a9adfc db: Don't use strings to indicate relationship names
Resolve the following RemovedIn20Warning warning:

  Using strings to indicate relationship names in Query.join() is
  deprecated and will be removed in SQLAlchemy 2.0.  Please use the
  class-bound attribute directly.

Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: I155b4ce4b605720c8335d465124fd32cc973a737
2024-04-04 13:30:27 +01:00
Mike Bayer 505719a50a db: turn off autocommit
after testing using an assert for DML without a transaction,
we can now turn autocommit off.   enginefacade should be used
completely now

Change-Id: I2ee07c7e41ea43c2ab24a4a095550dd0b5fe47dd
2024-04-03 19:33:24 -04:00
Mike Bayer 0ce2857d0f db: update migrations to use SQLA 2.0 patterns
All tests in test_migration.py pass with either SQLAlchemy 1.4
or SQLAlchemy 2.0 installed.

Change-Id: Ie8831c04b7b4515deea27b1ceb472d07cda91ca0
2024-04-03 19:26:23 -04:00
Stephen Finucane cae5ac7be1 db: Remove final use of legacy facade
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: I1251a5c0b2f073604ae61f378992fea7df51c07d
2024-04-03 16:23:12 +01:00
Stephen Finucane 98ef4c7755 db: Post-migration cleanup
Now that we have migrated all of our DB APIs to enginefacade, we can
clean up the outstanding TODOs.

Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: Iee198a16f030f1205d38b7c232d4e3a42642c756
2024-04-03 16:23:12 +01:00
Stephen Finucane 1330a5e34a db: Remove 'get_session'
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: I2a1a8b8eb933fc750ff8b3fc9d7ff1c7686625e6
2024-04-03 16:22:19 +01:00
Stephen Finucane d581714351 db: Remove session arguments from AZ methods
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: I38ae324d5edb3705b559c3706d6888f08473a633
2024-04-03 11:29:58 +01:00
Stephen Finucane d74c804121 db: Migrate "share" APIs to enginefacade
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: Ie5035546c3d640e3558446ad082510c249d0ce65
2024-04-02 15:40:18 +01:00
Stephen Finucane f3805e2a13 db: Migrate "share snapshot metadata" APIs to enginefacade
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: I3fa8e4e1e526d75051cb323702e3532d99008313
2024-04-02 15:40:18 +01:00
Stephen Finucane a55cafae8e db: Migrate "share snapshot export location" APIs to enginefacade
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: I3d29b5426c5a633b5590367b56d00847215d8201
2024-04-02 15:40:07 +01:00
Stephen Finucane b8f5d6199d db: Migrate "share * access" APIs to enginefacade
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: I590f4c7f78851b20a339294b820d0c223e298f3d
2024-04-02 14:12:34 +01:00
Stephen Finucane 685acf6013 db: Migrate "share snapshot", "share snapshot instance" APIs to enginefacade
Thankfully the APIs being migrated here were _mostly_ sharing sessions
already, so we can simply migrate from public (decorated) methods to
private methods with minimal fuss.

Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: Id1b555e48106662d15e8c50567a5f3acecf6a8f1
2024-04-02 14:11:07 +01:00
Stephen Finucane 3be71e3e07 db: Unwind unnecessary independent transactions
We need to maintain these on quotas (for now) but not on other
operations.

Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: I991d6127c14b5f06ca9ca8e6caccbc392e639b42
2024-03-25 11:32:49 +00:00
Stephen Finucane 36549c8b97 db: Migrate "share instance", "share replica" APIs to enginefacade
We migrate both of these at the same time since a share replica is
effectively a share instance with only the 'replica_state' field set. As
such, they share many common DB APIs.

These are mostly trivial to migrate. Just a lot of refactoring to create
private methods, avoiding nested transactions.

Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: I267e2e0fac0b4130f8836de276082788fbd8d454
2023-10-31 11:41:49 +00:00
Zuul b6010bb746 Merge "Add support for share/snapshot deferred deletion" 2024-03-12 21:51:10 +00:00
Kiran Pawar 558288e00a Add support for share/snapshot deferred deletion
Implements: bp/deferred-deletion
Change-Id: I9e55e1706fc0c3d9f65f73e13ba2a20f355c74f4
2024-03-12 14:22:25 +00:00
agireesh 6909a7c213 Share backups enhancement
Added new column named 'backup_type' in 'share_backups' table
and changes the share common api libs to support the dhss_true
configuration for share backup creation

Partially-implements: bp/share-backup
Change-Id: Ifb88ec096674ea8bc010c1c3f6dea1b51be3beaa
2024-03-12 15:56:38 +05:30
jayaanand.borra@netapp.com ea1ac5f448 Human readable export location core implementation
Export locations are usually too difficult to memo
rize.Currently, there is no way to determine the
export location before the share is created, so
users wait until the share creation request gets
completed, and then they check the export
locations to mount the share. The generated
export locations are often not human readable
 and it is hard to memorize and control them.

Implements: bp/human-readable-export-locations
Change-Id: I72ac7e24ddd4330d76cafd5e7f78bac2b0174883
2024-03-07 17:20:30 -05:00
Zuul 618576ff31 Merge "Add disabled reason field to service." 2024-02-28 21:27:48 +00:00
haixin 12ef157c3b Add disabled reason field to service.
update micversion to 2.83
user can set disabled reason for service.

Closes-Bug: #2037700

Change-Id: I3d7c46945366ac9e1d305c2f6de2233859259bf7
2024-02-27 14:27:37 +08:00
Takashi Kajinami c8ed5b05e9 db: Fix charset updates in migrations
We are failing py311 jobs with our migration checks;
and the reason is that we have some 'implied' sql defaults
that might be inconsistent with the server versions - one
of them is the charset. We recently [1] fixed a bunch of
un-annotated tables with explicit charset declarations.
This is a continuation of that attempt.

Unfortunately, we can't change history. On existing
deployments, changes to these migrations will be ignored,
and must be done manually if there are any inconsistencies
to be fixed.

While we're at it, this commit also cleans up separate
steps to set the charset and engine to pre-created
tables.

[1] I7cd6fa0cc8e054af112493746e753fef2024000f
Closes-Bug: #2053211
Change-Id: I16345f63a279549e81cbde7b60fed29521d48a02
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
2024-02-15 21:36:04 -08:00
Stephen Finucane 6a1ad1c2f6 db: Remove unnecessary helper
'_share_get_query' isn't really necessary and makes the code a little
trickier to read. Remove it.

Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: I4089c28a53846afdd8b6baa911de0120ce004ae3
2023-10-27 17:21:31 +01:00
Kiran Pawar cd767dd468 Delete share network subnet during network deletion
Share network subnet is deleted during share network deletion which was
missing earlier causing lots of stale db entries.

Change-Id: Idb58b4ca389cfb1aaf8f9f1f8aa2fd15979c4872
Closes-bug: #2037422
2023-10-11 11:28:05 +00:00
Zuul b89650f2d1 Merge "db: Remove unnecessary wrapper class" 2023-10-10 17:18:10 +00:00
Zuul 1c2e8b1756 Merge "db: Migrate remaining "transfer" APIs to enginefacade" 2023-10-10 14:42:44 +00:00
Zuul 365a028596 Merge "db: Migrate "transfer" APIs to enginefacade" 2023-10-10 14:36:04 +00:00
Zuul a6be0da020 Merge "db: Add '_share_update', '_share_instance_update'" 2023-10-10 14:36:02 +00:00
Zuul 8655b8e9fe Merge "db: Convert '_ensure_availability_zone_exists' to decorator" 2023-10-10 14:31:01 +00:00
Zuul 6250c50e19 Merge "db: Migrate "export locations", "export location metadata" APIs to enginefacade" 2023-10-07 00:03:01 +00:00
Zuul 7f654c00ff Merge "db: Add '_share_instance_get'" 2023-10-05 20:32:09 +00:00
Zuul 59be97ef8c Merge "db: Add '_share_get'" 2023-10-05 19:14:10 +00:00
Zuul 0d08682a8f Merge "db: Rename 'share_export_location_*' to 'export_location_*'" 2023-10-05 19:14:08 +00:00
Zuul d0182bf9e4 Merge "db: Rename 'share_instances_*' to 'share_instance_*'" 2023-10-04 17:17:03 +00:00
Zuul e1aaae49c2 Merge "db: Rename some methods" 2023-10-03 19:36:04 +00:00
Stephen Finucane f8ca22fca0 db: Remove unnecessary wrapper class
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: I0f07b9902514176d98c3abbea875909baf709047
2023-09-28 15:52:52 +01:00
Zuul c247ed50fc Merge "Fix share network delete procedure" 2023-09-28 14:41:22 +00:00
Stephen Finucane 8246cce49f db: Migrate remaining "transfer" APIs to enginefacade
These were kept separate since they were a little more involved.

Change-Id: Iebd498611601433a47c5de8531c8f9b75d1b6015
2023-09-28 10:21:35 +01:00
Stephen Finucane 3b779bf6e9 db: Migrate "transfer" APIs to enginefacade
Well, most of them. Two are left out since they call into 'share_*' APIs
and that's a whole other bag of string. We'll tackle those later.

Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: I170a406d4eb236d9921113270b58657a6ddaad0a
2023-09-28 10:21:35 +01:00
Stephen Finucane 8eaebc741c db: Add '_share_update', '_share_instance_update'
Another set of APIs that are commonly used from other APIs. Add private
versions of these that we can call without worrying about them creating
new sessions on us.

Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: I37f5a058dd1dec8e1a6ba129f121de12c0f2f3e1
2023-09-28 10:21:35 +01:00
Stephen Finucane db9f5dfe02 db: Convert '_ensure_availability_zone_exists' to decorator
We are currently running this outside of the main session of many APIs.
This is tough to do when the session is created as a decorator. Make
life easier by using a decorator, which allows us to use a separate
context and separate session.

Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: I95187d5b9bc21120c74b6784489c1b0dbbf337da
2023-09-28 10:21:35 +01:00
Stephen Finucane 3d8bc29615 db: Migrate "export locations", "export location metadata" APIs to enginefacade
We migrate these two at the same time since they're so intertwined.

Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: I23255a4a361602b602bca8349b5948efd147178d
2023-09-28 10:21:20 +01:00
Zuul 0629aed508 Merge "Fix py311 unit test issues" 2023-09-27 17:16:40 +00:00
Stephen Finucane 3362c5444f db: Add '_share_instance_get'
Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: Ia8c74fa493f0b0ccb36257099cc946ea18611614
2023-09-27 16:32:15 +01:00
Stephen Finucane 80d1b94bc9 db: Add '_share_get'
This is an important helper in the migration to enginefacade since
'share_get' is currently used in so many other functions. We must move
a 'get_session' call out of '_share_get_query' so that we won't
inadvertently ignore the session stored in the context.

Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: Iffe9e46ca707ef8cca3fe878faa29b62b924ab42
2023-09-27 16:32:15 +01:00
Stephen Finucane 8b148ebd41 db: Rename 'share_export_location_*' to 'export_location_*'
This better aligns with 'export_location_metadata_*' APIs. The plural
'export_locations_get*' APIs are renamed to 'export_location_get_all*'.

Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: I14f2b834e7ac2d8be86f9d7e381706cadbb79eb8
2023-09-27 16:31:07 +01:00
Stephen Finucane 204215722e db: Rename 'share_instances_*' to 'share_instance_*'
Rename a number of APIs to use singular, rather than plural, like every
other API uses:

- share_instances_status_update ->
    share_instance_status_update
- share_instances_get_all ->
    share_instance_get_all
- share_instances_get_all_by_host ->
    share_instance_get_all_by_host
- share_instances_get_all_by_share_network ->
    share_instance_get_all_by_share_network
- share_instances_get_all_by_share_server ->
    share_instance_get_all_by_share_server
- share_instances_get_all_by_share ->
    share_instance_get_all_by_share
- share_instances_get_all_by_share_group_id ->
    share_instance_get_all_by_share_group_id

Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: Ic48fe0d1631a6e1a8ee9a50741cc1b31c9187c37
2023-09-27 16:20:17 +01:00
Stephen Finucane 2a3bbe9be8 db: Rename some methods
Rename a number of APIs for consistency purposes:

- get_all_expired_transfers ->
    transfer_get_all_expired
- get_all_expired_shares ->
    share_get_all_expired
- get_shares_in_recycle_bin_by_share_server ->
    share_get_all_soft_deleted
- get_shares_in_recycle_bin_by_network ->
    share_get_all_soft_deleted_by_network
- share_transfer_get ->
    transfer_get

Signed-off-by: Stephen Finucane <stephenfin@redhat.com>
Change-Id: I805ed06feefd189768cf4b0fec9b8ff011694216
2023-09-27 16:14:29 +01:00
BubaVV 42e66d9f38 Fix share network delete procedure
Currently, delete of share network leaves security service associations
behind it. DB purge job can't process them because they are constrained
to soft-deleted networks. Proposed patch fixes both formation of
orphaned associations and seamless purging of them.

Closes-Bug: #2029366
Change-Id: I9593dd6e89b2d290d3919c92d209132e1cec84f0
2023-09-24 01:21:28 +03:00