Commit Graph

32 Commits

Author SHA1 Message Date
agireesh 6909a7c213 Share backups enhancement
Added new column named 'backup_type' in 'share_backups' table
and changes the share common api libs to support the dhss_true
configuration for share backup creation

Partially-implements: bp/share-backup
Change-Id: Ifb88ec096674ea8bc010c1c3f6dea1b51be3beaa
2024-03-12 15:56:38 +05:30
jayaanand.borra@netapp.com ea1ac5f448 Human readable export location core implementation
Export locations are usually too difficult to memo
rize.Currently, there is no way to determine the
export location before the share is created, so
users wait until the share creation request gets
completed, and then they check the export
locations to mount the share. The generated
export locations are often not human readable
 and it is hard to memorize and control them.

Implements: bp/human-readable-export-locations
Change-Id: I72ac7e24ddd4330d76cafd5e7f78bac2b0174883
2024-03-07 17:20:30 -05:00
zhongjun 0b99fdaa9a Implement share backup
Add share backup feature in Data Copy Service and
Share Service. It will allow the user to create, restore
and delete backups as well as listing backups and showing
the details of a specific backup.

APIImpact
DOCImpact

Change-Id: I7d10cf47864cd21932315375d84dc728ff738f23
Implement: blueprint share-backup
2023-08-10 11:11:42 +00:00
Kiran Pawar 5f0f14a7f8 Add quiesce_wait_time option to share replica promote API.
Currently netapp_snapmirror_quiesce_timeout is an option of replica
promote that can only be set by the operator. There are scenarios where
high timeout value set by operator does not fit well e.g. disaster
stroke and user want to do an unplanned failover fast. Added new
option 'quiesce_wait_time' to share replica promote API which allows
to use specified wait_time(in seconds) instead of config option.

Closes-bug: #2000171
Change-Id: Ib02063ee8b82f7374cd89f90e7f24a845c6c7cd7
2023-02-08 16:05:28 +00:00
Manish Honap 6431b86f19 Add config option reserved_share_extend_percentage
'reserved_share_extend_percentage' backend config option allows Manila
to consider different reservation percentage for share extend
operation. With this option, under existing limit of
'reserved_share_percentage', we do not want user to create new share if
limit is hit, but allow user to extend existing share.

DocImpact

Closes-Bug: #1961087
Change-Id: I000a7f530569ff80495b1df62a91981dc5865023
2022-06-14 23:08:31 +05:30
Fernando Ferraz 785016eb74 Fix Dummy driver fails to get subnet_allocations
With the addition of support to multiple subnets per AZ,
backend_details structure was changed in Dummy driver to store
multiple subnet primary and secondary public ips in a
subnet_allocations list dumped to a json string, breaking upgrading
environments that already contains pre-existent share servers in
old format.

This change fixes the issue by checking if subnet_allocations does
exist, otherwise ensure backward compatibility by reading public
ips information in the old format.

Change-Id: I547de2f4cd82744ecc1cc8180f8cefdf5ec485b4
2022-03-17 18:04:56 +00:00
Felipe Rodrigues 2b57d15c64 Add multiple subnets per AZ support
Manila can now configure a share network with multiple subnets
in an availability zone. Also, it can add a new subnet for an
availability that has share servers, which will triger an update
share server allocations.

Changes:
- API:
  - Bump version to 2.70.
  - setup share network with multiple subents per az.
  - Block manage server with multiple subnets.
  - Allow add subnet for in-use share servers.
  - `share_network_subnet_id` is dropped from ShareServer view
  - `share_network_subnet_ids` is added in ShareServer view
  - `network_allocation_update_support` is added to ShareServer and
    ShareNetwork views.
  - Add a check operation for share network subnet create.

- DB:
  - Remove `share_network_subnet_id` from share_servers.
  - Create mapping table `share_server_share_network_subnet_mappings`.
  - Fix queries with new db design.
  - Add migration downgrade and upgrade alembic.
  - Add `share_network_subnet_id` to the NetworkAllocations.

- Scheduler:
  - Change `AvailabilityZoneFilter` to take in account if the
    host supports the allocation required by the setup request.

- Manager:
  - Bump RPC API version.
  - `_setup_server` allocating multiple subnets.
  - Modify signature of driver `_setup_server` interface, passing a
    list of `network_info` for each subnet.
  - Share server DB creation to inform a list of subnets and
    create with `network_allocation_update_support`.
  - Implement `check_update_share_server_network_allocations` and
    `update_share_server_network_allocations`.

- Drivers:
  - For legacy compatibility, all drivers implementing `_setup_server`
    consume the first element of the `network_ino`.
  - Dummy Driver:
    - Implement `_setup_server` with new signature as multiple subnet.
    - Modify the `backend_details` to save allocations for all subnets.
    - Report update allocation and share server multiple subnet support.
    - Implement `check_update_share_server_network_allocations` and
      `update_share_server_network_allocations` interfaces.

Signed-off-by: Felipe Rodrigues <felipefuty01@gmail.com>
Co-Authored-By: Andre Beltrami <debeltrami@gmail.com>
Co-Authored-By: Fábio Oliveira <fabioaurelio1269@gmail.com>
Co-Authored-By: Nahim Alves de Souza <nahimsouza@outlook.com>
Co-Authored-By: Caique Mello <caique_mellosbo@hotmail.com>

DocImpact
APIImpact
Partially-Implements: blueprint multiple-share-network-subnets

Change-Id: I7de9de4ae509182e9494bba604979cce03acceec
2022-03-03 02:31:11 +00:00
silvacarloss 04a3db2f9a Share server migration enhancements
A series of enhancements can be performed in the share server
migration operation, and share backends might support
nondisruptive share server migrations, which was not covered
in the previous approach.

-  Skip the destination share server network allocation if the
driver is capable of reusing the source share server allocation.
Manila will switch the allocations in the migration complete phase.

- Allow share backends to reuse the share servers in case they
are able to do so in a share server migration scenario.

- Nondisruptive migration is now feasible depending on whether the
share driver supports it and if the share network parameters have
not changed.

- Share servers will be always deleted whe the share server
migration complete operation is finished.

Depends-On: I43bd3fdafea02eb8e853114a57bfb863a441f3ed
Co-Authored-By: Fabio Oliveira <fabioaurelio1269@gmail.com>
Change-Id: I48bafd92fe7a4d4ae0bafd5bf1961dace56b6005
2021-09-07 09:21:32 -03:00
kpdev 6ca10003a9 Add config option reserved_share_from_snapshot_percentage.
This config option allows different value for reservation percentage,
mostly useful on the platforms, where shares can only be created from
the snapshot on the host where snapshot was taken. The lower value of
this config option against existing (reserved_share_percentage) allows
to create shares from the snapshot on the same host up to a higher
threshold even though non-snapshot/regular share create fails.
In case this config option is not set, the shares created from snapshot
will use reservation percentage value set in 'reserved_share_percentage'.
This will be useful for users who want to keep same reservation percentage
for both non-snapshot/regular and snapshot shares.

DocImpact

Closes-Bug: #1938060
Change-Id: I390da933fe92875e3c7ee40709eacacc030278dc
2021-08-27 19:10:36 +02:00
debeltrami 2bc27c5678 Add security service update for in-use share networks
This patch implements the update of security service's association
with in-use share networks. The following changes were added:

 - New share network APIs: `share_network_security_service_update`
 and `share_network_reset_state`.

 - A new `status` attribute was added to share network model to
 identify when it's in a modification state, called 'network_change'.
 Other supported status that were added: 'active' and 'error'.

 - New 'security_service_update_support' property was added to both
 share server and share network models, to identify when this resources
 are able to process security service update for in-use share networks.

 - New driver interface was added to support update of security service's
 configuration of a given share server.

DocImpact
APIImpact
Partially Implements: bp add-security-service-in-use-share-networks

Co-Authored-By: Carlos Eduardo <ces.eduardo98@gmail.com>
Co-Authored-By: Douglas Viroel <viroel@gmail.com>
Co-Authored-By: Andre Beltrami <debeltrami@gmail.com>

Change-Id: I129a794dfd2d179fa2b9a2fed050459d6f00b0de
2021-03-12 18:37:45 -03:00
silvacarloss 2fb871cb89 Add share server limits
Add two new capabilities to manila. It is possible to set limits
to share server size and share instances in a share server by
setting `max_share_server_size` and `max_shares_per_share_server`
in a backend stanza.

Change-Id: I3170478d3aa2d09cb2adc32233dc57bc59029a56
Partially-Implements: bp new-share-server-limits
DocImpact
2021-03-08 21:35:06 -03:00
Douglas Viroel eb3c897c59 Add share server migration for Dummy driver
Partially Implements: bp share-server-migration

Change-Id: I46a0cee0a4625d950f02fa7a5bf612de926451b5
Signed-off-by: Douglas Viroel <viroel@gmail.com>
2020-09-10 19:24:30 -03:00
Douglas Viroel 6c47b193b0 Create share from snapshot in another pool or backend
This patch enables the creation of a share from snapshot
specifying another pool or backend. In the scheduler, a
new filter and weigher were implemented in order to consider
this operation if the backend supports it. Also, a new
field called 'progress' was added in the share and share
instance. The 'progress' field indicates the status
of the operation create share from snapshot (in percentage).
Finally, a new periodic task was added in order to constantly
check the share status.

Partially-implements: bp create-share-from-snapshot-in-another-pool-or-backend

DOCImpact
Change-Id: Iab13a0961eb4a387a502246e5d4b79bc9046e04b
Co-authored-by: carloss <ces.eduardo98@gmail.com>
Co-authored-by: dviroel <viroel@gmail.com>
2020-04-09 11:15:22 -03:00
Andreas Jaeger 28bb650128 Hacking: Fix E241
Fix:
E241 multiple spaces after ':' (or ',')

Fix also a few other problems found by hacking in files changed.

Change-Id: I4c00551f29ad65df2631c8ae55fe2f022e872544
2020-04-01 09:21:01 +02:00
Goutham Pacha Ravi 68a18f4947 Dummy driver: Don't fail unmanage on malformed share servers
When a share server is managed with invalid details, since
we cannot find backend network allocations in the dummy driver,
we raise an error on manage, setting the status to "manage_error".
Such a share server can be "unmanaged" or deleted, and we shouldn't
fail this request, either case.

Change-Id: I4144a532d5827b929a1dce0d4d642e6d1cee47b1
Related-Bug: #1820118
2019-03-21 14:16:30 +00:00
Rodrigo Barbieri d877b61c5e Add manage/unmanage of shares in DHSS=True
This patch adds Manage/Unmanage of share servers in
Manila. It also updates the Manage Share API to accept
a "share_server_id" parameter, and updates Unmanage
of Share and Snapshots API to allow unmanaging of
shares and snapshots in DHSS=True.

Managed share servers are not deleted automatically
by manila, and if a single share is unmanaged in
DHSS=True, the respective share server will not be
deleted automatically as well.

Managing share servers require that the driver
implements 2 functions:
- get_share_server_network_info: obtain IPs from
  share server.
- manage_server: perform required operations to
  manage and return dict of backend_details.

Unmanaging share servers require that the driver
overrides unmanage_server function.

The IPs obtained from the backend are validated
by the Network plugin, so ports with the exact
IPs must exist in the subnet and admin subnet
associated with the share network specified
when managing the share server (unless
StandaloneNetworkPlugin is used).

It is recommended to rename the backend resource
if possible when managing the share server, to
prevent issues with re-managing a share server
that has already been managed.

This patch bumps the API microversion to 2.49.

APIImpact
DocImpact

Depends-On: I17c74b2aa242918188eeff368232c762a4b31093
Partially-implements: bp manage-unmanage-with-share-servers
Change-Id: I108961e7436ba13550ef2b8f02079c6e599a6166
2019-03-05 16:03:29 -03:00
Goutham Pacha Ravi fd23a6387c Use class name in invocation of super
Fix pylint E103 warning raised by usage of
self.__class__ to refer to the derived class
in super() methods.

self.__class__ is a reasonable first argument
to super() in any method of a class, as long
as the method is not going to be invoked in
derived classes.

Python3 removes this ambiguity by not requiring
arguments for the super() method.

[1] https://docs.pylint.org/en/1.6.0/features.html#id33

Change-Id: I6071b6cfd8cff2be3853d739f71b94da990cda97
2018-06-06 17:47:18 -07:00
junboli fb17422c86 Enable some off-by-default checks
Some of the available checks are disabled by default, like:
[H106] Don't put vim configuration in source files
[H203] Use assertIs(Not)None to check for None
[H904] Use ',' instead of '%', String interpolation should be
       delayed to be handled by the logging code, rather than
       being done at the point of the logging call.

Change-Id: Ie985fcf78997a86d41e40eacbb4a5ace8592a348
2017-08-03 16:13:32 +08:00
Ben Swartzlander 5cb87d5dc3 Fix multiple issues with revert to snapshot in LVM
Reverts with the LVM driver were happening asynchronously
which resulted in unreliable behavior. This change makes
the reverts synchronous.

Also, the driver now unexports and unmounts both the share and
the snapshot during the revert (required to make the operation
synchronous) which requires changing the driver interface to
include the snapshot access rules in addition to the share access
rules.

Closes-bug: #1707943

Change-Id: Ifea1799e1d2e3963fec7e90ce3f9cb47b9f02f4f
2017-08-02 16:48:40 -04:00
zhongjun f68c1a4080 Add share usage size tracking
We need to have a way to gather information about actual share storage
usages, so cloud operators could use this information for billing,
health checks and/or other purposes.

Change-Id: Iaca1bb541a34af862b938e17e4a56d53de7a9cc4
Implement-Blueprint: share-usage-size
2017-07-26 04:09:48 +00:00
Valeriy Ponomaryov 21699451f1 [Share groups] Add scheduler filter ConsistentSnapshotFilter
That will be used for scheduling share groups based on their possibility
to create consistent snapshots.

Also apply following tempest plugin changes:
- Add new 'capability_sg_consistent_snapshot_support' tempest config
option, that will be used for creation of new share group types and used
to prove that scheduling works as expected.
- Fix some share group test attributes from 'only API involved' to
  'API and Backend are involved', because it is so indeed.

Change-Id: I05553c308ae40c4ddc2c6469ff1c1a3da36a87da
Partially-Implements BP manila-share-groups
2017-06-02 17:48:05 +03:00
Jenkins 3726c89f20 Merge "Pass access rules to driver on snapshot revert" 2017-02-07 18:31:42 +00:00
Ben Swartzlander a7d8363d32 Pass access rules to driver on snapshot revert
In order to revert to a snapshot in the LVM driver (and
very likely other drivers) the list of access rules is
needed, so this change modifies the driver interface to
provide this extra information.

This change requires preventing a revert to snapshot
operation while access rules on the affected share are
out of sync.

Closes bug: 1658133

Change-Id: Ia6678bb0e484f9c8f8b05d90e514801ae9baa94b
2017-02-07 11:15:27 -05:00
Valeriy Ponomaryov 64649b97e7 Fix default approach for share group snapshot creation
Default approach uses common driver interface of single snapshot
creation. So, emulate it correctly providing all required keys.
Also add 'provider_location' attribute to share group snapshot
members DB model, because it is required for some drivers and
was absent.

Change-Id: If120d85ef3dd3ba90e2dc12a5b81b69feecb31ea
Closes-Bug: #1660321
2017-02-07 15:04:36 +00:00
yogesh 301abe6257 Improve test coverage for share migration
- Added tests to validate the migration functionality
for a replicated share.
- Added tests to validate the extend and shrink
functionality on migrated share.
- Added tests to cover the preserve snapshot
functionality on migrated share.
- Current waiter logic in migration_progress
method of dummy driver causes every migration
test to take over 4 minutes to complete.
Therefore, updated the method to cut the
overall time in half and avoid the possible timeout.

Partially-Implements: bp ocata-migration-improvements
Change-Id: Ia7c7eb7b48b464aa670073e8c0795369bd972678
2017-02-03 23:27:34 -05:00
tpsilva 8d71932c69 Add mountable snapshots support
This new feature gives the user the ability to allow and
deny access to the snapshots, so that they could be mounted in
read-only mode to retrieve files.

APIImpact
DocImpact

Co-Authored-By: Rodrigo Barbieri <rodrigo.barbieri@fit-tecnologia.org.br>
Co-Authored-By: Alyson Rosa <alyson.rosa@fit-tecnologia.org.br>
Co-Authored-By: Miriam Yumi <miriam.peixoto@fit-tecnologia.org.br>

Partially-implements: blueprint manila-mountable-snapshots
Change-Id: I65f398a05f82eef31ec317d70dfa101483b44b30
2017-01-24 17:26:45 -02:00
Rodrigo Barbieri 53539c0e1d Share Migration Ocata Improvements
Implemented several improvements to share migration
according to spec [1].

Summary of changes:
- Snapshot restriction in API has been changed to return error only
when parameter force-host-assisted-migration is True
- Added preserve_snapshot to API and migration_check_compatibility
driver interface
- Changed all driver-assisted API parameters to be mandatory
- Added validation to prevent 'force_host_assisted_migration' to be
used alongside driver-assisted parameters
- Changed "same host" validation to reject only if the combination
of "host", "new_share_network" and "new_share_type" is the same as
the source
- Updated migration driver interfaces to support snapshots
- Updated zfsonlinux driver, defaulting preserve_snapshots to False
- Updated dummy driver to support preserve_snapshots

Spec update with latest changes since [1] merged
can be found in [2].

APIImpact
DocImpact

[1] I5717e902373d79ed0d55372afdedfaa98134c24e
[2] If02180ec3b5ae05c9ff18c9f5a054c33f13edcdf

Change-Id: I764b389816319ed0ac5178cadbf809cb632035b4
Partially-implements: blueprint ocata-migration-improvements
2017-01-20 09:45:14 -02:00
Clinton Knight d4a379d083 Implement share revert to snapshot
This commit adds the ability for Manila to revert a
share to the latest available snapshot.

The feature is implemented in the LVM driver, for
testing purposes.

APIImpact
DocImpact
Co-Authored-By: Ben Swartzlander <ben@swartzlander.org>
Co-Authored-By: Andrew Kerr <andrew.kerr@netapp.com>
Implements: blueprint manila-share-revert-to-snapshot
Change-Id: Id497e13070e0003db2db951526a52de6c2182cca
2017-01-17 15:06:01 -05:00
Clinton Knight 0d6db3588c Add create_share_from_snapshot_support extra spec
The snapshot_support extra spec has always meant two
things: a driver can take snapshots and create shares
from snapshots. As we add alternate snapshot semantics,
it is likely that some drivers will want to support
snapshots and some of the new semantics while being
unable to create new shares from snapshots.

This work adds a new extra spec,
create_share_from_snapshot_support, that removes the
overloading on snapshot_support. It also makes the
existing snapshot_support extra spec optional,
allowing admins to create types without setting
snapshot_support; shares created with such types
will not support snapshots.

APIImpact
DocImpact

Co-Authored-By: Goutham Pacha Ravi <gouthamr@netapp.com>
Implements: blueprint add-create-share-from-snapshot-extra-spec
Change-Id: Ib0ad5fbfdf6297665c208149b08c8d21b3c232be
2016-12-21 09:57:08 -05:00
Valeriy Ponomaryov 1daade76b2 [Dummy driver] Add possibility to set delays for driver methods
Dummy driver was designed to test manila main functionality that is
storage-agnostic. So, add possibility to set delays for each dummy share driver
method, to be able to simulate storage more correctly by having time delays
similar to real storage-drivers.

Two new config options were added:
- dummy_driver_default_driver_method_delay (float)
- dummy_driver_driver_methods_delays (dict of string-keys and int/float values)

Configuration example:

[DEFAULT]
enabled_share_backends = fake_backend_name

[fake_backend_name]
...
dummy_driver_default_driver_method_delay = 2.0
dummy_driver_driver_methods_delays = create_share:5.15,delete_share:1
...

Change-Id: I36598f74762f3631c6c8502148ea9989ad544b5a
2016-11-23 16:03:07 +02:00
Valeriy Ponomaryov 754750a11d [Dummy driver] Add share migration support
Add 'fake' share migration support to 'dummy' driver
for ease of interfaces testing.

Change-Id: Ifd444e38a9678e98c156b7ce658aa1b50c13b24c
2016-09-01 05:45:27 +00:00
Valeriy Ponomaryov 7145891ef8 Add dummy driver
Added dummy driver for ease of testing and development of REST APIs
and internal manila interfaces.
It can be enabled as backend in the same way as all other share drivers.
But, it does not provide any resources.

Also, update tempest to be able to work with share-networks when
networks are defined via config and not API.

CI hooks are configured to run tempest twice using both available
driver modes. Where DHSS=True mode uses StandAloneNetworkPlugin as
network backend.

Change-Id: I9053dddfc643cb5a6602f15235b91bbaea727dbd
Implements BluePrint dummy-driver
2016-07-22 17:57:59 +03:00