The protocol access mapping options should accept dictionaries whose
values are lists, not strings.
Closes-Bug: 2038607
Change-Id: Id75082507cee27bef03318dc3af5ae5c1b233505
Add share backup feature in Data Copy Service and
Share Service. It will allow the user to create, restore
and delete backups as well as listing backups and showing
the details of a specific backup.
APIImpact
DOCImpact
Change-Id: I7d10cf47864cd21932315375d84dc728ff738f23
Implement: blueprint share-backup
When manila services are stopped or restarted via stop(), the DB
entries are not deleted, they are destroyed only in kill() method. In
cluster deployments, where multiple instances of manila services are
deployed via PODs, unique hostname is derived from node name. However
if pods are deployed again and launched on new hosts/nodes, the old
entries of manila service remains as it is.
Fix it by adding 'state' column in 'services' table and introducing
per service cleanup function. On service stop, state is changed to
'stopped' and cleanup function will delete 'stopped' services unless
they are 'up' again before cleanup periodic interval.
Closes-bug: #1990839
Change-Id: I8b71c4c27ff8fcb25616a95a5ed8362a7f4ffc61
We are replacing all usages of the 'retrying' package with
'tenacity' as the author of retrying is not actively maintaining
the project. Tenacity is a fork of retrying, but has improved the
interface and extensibility (see [1] for more details). Our end
goal here is removing the retrying package from our requirements.
Tenacity provides the same functionality as retrying, but has the
following major differences to account for:
- Tenacity uses seconds rather than ms as retrying did
(the retry interface in manila exposed time in seconds as well)
- Tenacity has different kwargs for the decorator and
Retrying class itself.
- Tenacity has a different approach for retrying args by
using classes for its stop/wait/retry kwargs.
- By default tenacity raises a RetryError if a retried callable
times out; retrying raises the last exception from the callable.
Tenacity provides backwards compatibility here by offering
the 'reraise' kwarg - we are going to set this in the retry interface
by default.
- For retries that check a result, tenacity will raise if the
retried function raises, whereas retrying retried on all
exceptions - we haven't exposed this in the retry interface.
This patch updates all usages of retrying with tenacity.
Unit tests are added where applicable.
[1] https://github.com/jd/tenacity
Co-Authored-By: boden <bodenvmw@gmail.com>
Co-Authored-By: Goutham Pacha Ravi <gouthampravi@gmail.com>
Closes-Bug: #1635393
Change-Id: Ia0c3fa5cd82356a33becbf57444f3db5ffbb0dd0
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
Remove manila configuration options
and auth classes that were deprecated
before the Ussuri release.
Change-Id: I148225926cd249a0dd8d1f8c02b22ed06487f405
remove usage of six library from the following directory:
1:common
2:data
3:db
4:message
5:network
6:scheduler
Change-Id: I9db0abf2b0847157074ca6ba84b5451bfe3f20d0
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
It was deprecated in the Queens release, in
favor of ``data_node_access_ips``.
Change-Id: I01f24c2e0d66e1da4c30c579c02afc8f3930c8f8
Related-Bug: #1745436
Previously, the Data Service allowed only 1 IP to be specified
for the 'data_node_access_ip' config option, which prevented
Host-assisted Share Migration to be successful on a combination
of backends that include IPv6 and IPv4 export locations.
To fix this, we are adding a new config option and deprecating
the old one. The new one is named 'data_node_access_ips' and
accepts a list of IPs. All IPs included in this list should
correspond to interfaces on the node running the Data Service
and will be allowed access during Host-assisted Share Migration.
Additionally, we are improving the responsiveness of the
Host-assisted Share Migration by using exponential waiters
instead of polynomial. Those waiters are used when managing
resources created by Share Migration.
Closes-bug: #1745436
Partial-bug: #1708491
Change-Id: I1c0b7621ae8192f75f691987b185a4fc6a7228ce
- Pull up policy check to beginning of the APIs.
- Avoid making access rules changes when one or
more instances of the share are in an invalid state.
- Add back the per rule share instance access status.
This restoration provides better visibility for which
rules were applied successfully.
- Remove 'updating' and 'updating_multiple' as valid
states for the share instance access rules status.
- Deprecate the access rule state 'new' in favor of
'queued_to_apply' and the share instance access rules
status 'out_of_sync' in favor of 'syncing'.
In a new API micro-version:
- Allow access rule changes irrespective of the share's
access_rules_status.
- Expose new access rule states and share's
access_rules_status values.
Access rules for each share instance now transition
from 'queued_to_apply' to 'applying' to 'active' or 'error';
and from 'active', 'queued_to_apply', 'applying' or 'error'
to 'queued_to_deny' to 'denying' to 'deleted'.
APIImpact
DocImpact
Partially-implements: bp fix-and-improve-access-rules
Co-Authored-By: Mike Rooney <rooneym@netapp.com>
Change-Id: Ic25e63215b5ba723cbc8cab7c51789c698e76f28
At Austin 2016 summit there were several improvements to
Share migration feature discussed. This patch implements
these changes.
Changes are:
- Added 'Writable' API parameter: user chooses whether share must
remain writable during migration.
- Added 'Preserve Metadata' API parameter: user chooses whether
share must preserve all file metadata on migration.
- Added 'Non-disruptive' API parameter: user chooses whether
migration of share must be performed non-disruptively.
- Removed existing 'Notify', thus removing 1-phase migration
possibility.
- Renamed existing 'Force Host Copy' parameter to 'Force
Host-assisted Migration'.
- Renamed all 'migration_info' and 'migration_get_info' entries to
'connection_info' and 'connection_get_info'.
- Updated driver interfaces with the new API parameters, drivers
must respect them.
- Changed share/api => scheduler RPCAPI back to asynchronous.
- Added optional SHA-256 validation to perform additional check if
bytes were corrupted during copying.
- Added mount options configuration to Data Service so CIFS shares
can be mounted.
- Driver may override _get_access_mapping if supports a different
access_type/protocol combination than what is defined by default.
- Added CIFS share protocol support and 'user' access type
support to Data Service.
- Reset Task State API now allows task_state to be unset using
'None' value.
- Added possibility to change share-network when migrating a share.
- Bumped microversion to 2.22.
- Removed support of all previous versions of Share Migration APIs.
APIImpact
DocImpact
Implements: blueprint newton-migration-improvements
Change-Id: Ief49a46c86ed3c22d3b31021aff86a9ce0ecbe3b
Tempest tests were not appropriate for driver-assisted migration,
so this was fixed.
Also, improved docstrings and fixed workflow for drivers when
implementing 2-phase migration to be accurate with tempest and
handle AZs, which were previously locked to the source share's
AZ.
Driver-assisted migration now creates an additional
share instance to better handle and support driver methods.
Updated allow_access and deny_access APIs to allow users to mount
migrating shares before issuing 'migration-complete'.
APIImpact
Closes-bug: #1594922
Change-Id: If4bfaf7e9d963b83c13a6fea241c2eda14f7f409
Fallback share migration fails with empty files. This patch fixes it
by performing additional checks to address this specific scenario.
Change-Id: I36d59740b4e52005e6025e5df5989bf55d6bade4
Closes-bug: #1613713
By default oslo.cfg sets the default value as None. There is no
need to explicitly do this.
TrivialFix
Change-Id: I0e17a86084feb2767110b3cc38ce0c5ef47019a2
In Share migration it is always trying to remove the Copy object
from dictionary, even if it has not been added yet. So, dict.pop()
should include second argument to not throw exception.
Improved unit test to cover this variation.
Closes-bug: #1555630
Change-Id: If81c14cbeb03e41501d825cab000ceb1aa2cae7c
Removed functionality of Share Migration relying on Manila Share
Service node, moved code to Data Service node for copy phase.
Added parameter 'notify' and share/api methods for future
implementation (see dependent patches).
Added new copy operation statuses, in order to implement future
API calls to obtain progress and cancel migration.
Added possibility of 2-phase migration for driver migration and
generic (fallback) migration.
Added admin export location support and removed approach of
replacing IP with config parameter.
Added Admin-only API entry points to:
- Migration Cancel (only during copying)
- Reset Task State field
- Migration Get Progress (only during copying)
- Migration Complete (2nd phase migration)
- Notify parameter on Migrate Share
APIImpact
DocImpact
Implements: blueprint data-service-migration
Change-Id: I1d65aac2f36942cd70eb214be561d59a15a4ba26
Added a new service defined as data, whose purpose is to
receive requests, process data operations such as copying,
migration, backup, and send back the response after operation
has been completed.
In this patch the service has no methods, it is empty, the
operation it should perform will be added in subsequent patches.
Implements: blueprint data-copy-service
Change-Id: I31365c8ac4197541af175c82f8f18a3d11740a99
Takashi Kajinami