This patch adds support for migration of share servers. This
migration is performed using a two-phase approach. Administrators
are now able to request the migration of a share server within and
across backends, with the possibility of chooosing a different share
network for the destination share server.
- A new field called `task_state` was added to the share server
model in order to help the administrator to track the share server
migration steps. A new field called `source_share_server_id` was
added to link destination and source share servers.
- A new periodic task was added to track migration of share servers
and its resources.
- Two new states were added: `server_migrating` and
`server_migrating_to` to represent that share migration is in
progress.
- When performing the server migration, manila will not go to the
scheduler, instead it will provide a request spec to drivers
during migration check driver call. It'll be up to the driver
validate if there is free space to handle the share server.
- A new API called `share-server-migration-check' was added to
check the feasibility of a migration, before actually triggering
the start operation.
APIImpact
DocImpact
Partially Implements: bp share-server-migration
Co-Authored-By: Andre Beltrami <debeltrami@gmail.com>
Co-Authored-By: Carlos Eduardo <ces.eduardo98@gmail.com>
Co-Authored-By: Felipe Rodrigues <felipefuty01@gmail.com>
Change-Id: Ic0751027d2c3f1ef7ab0f7836baff3070a230cfd
Signed-off-by: Douglas Viroel <viroel@gmail.com>
Fix:
E241 multiple spaces after ':' (or ',')
Fix also a few other problems found by hacking in files changed.
Change-Id: I4c00551f29ad65df2631c8ae55fe2f022e872544
Previously, the Data Service allowed only 1 IP to be specified
for the 'data_node_access_ip' config option, which prevented
Host-assisted Share Migration to be successful on a combination
of backends that include IPv6 and IPv4 export locations.
To fix this, we are adding a new config option and deprecating
the old one. The new one is named 'data_node_access_ips' and
accepts a list of IPs. All IPs included in this list should
correspond to interfaces on the node running the Data Service
and will be allowed access during Host-assisted Share Migration.
Additionally, we are improving the responsiveness of the
Host-assisted Share Migration by using exponential waiters
instead of polynomial. Those waiters are used when managing
resources created by Share Migration.
Closes-bug: #1745436
Partial-bug: #1708491
Change-Id: I1c0b7621ae8192f75f691987b185a4fc6a7228ce
- Add Database migration to introduce the column on the
share instances model.
- Set the field to True if creating read-only secondary
replicas, unset while promoting them.
- Set the field to True if drivers don't support writable access
to migrating shares, or if using host assisted migration.
Unset if migration fails, or is canceled.
- Expose the field via share-instances and share-replicas
APIs to administrators.
Supporting read only-access rules is part of the minimum
driver requirements in manila.
APIImpact
DocImpact
Implements: bp fix-and-improve-access-rules
Co-Authored-By: Rodrigo Barbieri <rodrigo.barbieri@fit-tecnologia.org.br>
Change-Id: Ie8425f36f02cbcede0aaa9f3fe1f5f3cf23df8b8
- Pull up policy check to beginning of the APIs.
- Avoid making access rules changes when one or
more instances of the share are in an invalid state.
- Add back the per rule share instance access status.
This restoration provides better visibility for which
rules were applied successfully.
- Remove 'updating' and 'updating_multiple' as valid
states for the share instance access rules status.
- Deprecate the access rule state 'new' in favor of
'queued_to_apply' and the share instance access rules
status 'out_of_sync' in favor of 'syncing'.
In a new API micro-version:
- Allow access rule changes irrespective of the share's
access_rules_status.
- Expose new access rule states and share's
access_rules_status values.
Access rules for each share instance now transition
from 'queued_to_apply' to 'applying' to 'active' or 'error';
and from 'active', 'queued_to_apply', 'applying' or 'error'
to 'queued_to_deny' to 'denying' to 'deleted'.
APIImpact
DocImpact
Partially-implements: bp fix-and-improve-access-rules
Co-Authored-By: Mike Rooney <rooneym@netapp.com>
Change-Id: Ic25e63215b5ba723cbc8cab7c51789c698e76f28
For drivers that implement update_access always through recovery
mode, access rules previously set to read-only were being reset
to read-write when the Data Service was adding/removing its access
rule.
Fixed it by integrating the logic that casts DB rules to read-only
into access helper class.
Change-Id: Ife35dcdb99dffa2f266b5c2f6a68fe163da7edd3
Closes-bug: #1626523
Share migration code [1] merged in newton intending to allow admins
to mount the share between phase1 and phase2, but API code for
allow_access and deny_access was set incorrectly, blocking it.
After discussing this feature's purpose further, we decided we do
not want this feature at this moment, so we are just fixing the
allow_access and deny_access error messages.
Also, addressed a small case of concurrency that was happening
once in a while in CI. Update_access was being invoked while
other rules were being applied, thus setting the access_rule_state
to "UPDATING_MULTIPLE", ignoring the migration access rule change
RPC request completely, failing migration. By refreshing the
model we are able to assign the proper access_rule_state at the
time the function is invoked, setting the access_rule_state
correctly.
[1] If4bfaf7e9d963b83c13a6fea241c2eda14f7f409
APIImpact
Closes-bug: #1623051
Closes-bug: #1623052
Change-Id: I76a7d8c3bdd597b951e700350f8f3f82bfb21e03
This patch adds a 'new_share_type_id' parameter to Share Migration,
where the destination share can be provisioned under a different
share type of choice.
Host-assisted migration handles it by creating a totally new share,
as before.
Driver-assisted migration handles by creating the destination
instance model with the new share type, the driver is responsible
for making the necessary changes to satisfy the provided model.
In order to accomplish this, a database change was required,
transferring the 'share_type_id' field from the 'shares' table
to the 'share_instances' table.
APIImpact
Partially implements: blueprint newton-migration-improvements
Change-Id: I3200eaaa5b66d9b8ce1cbd16c1658db8516c70fb
At Austin 2016 summit there were several improvements to
Share migration feature discussed. This patch implements
these changes.
Changes are:
- Added 'Writable' API parameter: user chooses whether share must
remain writable during migration.
- Added 'Preserve Metadata' API parameter: user chooses whether
share must preserve all file metadata on migration.
- Added 'Non-disruptive' API parameter: user chooses whether
migration of share must be performed non-disruptively.
- Removed existing 'Notify', thus removing 1-phase migration
possibility.
- Renamed existing 'Force Host Copy' parameter to 'Force
Host-assisted Migration'.
- Renamed all 'migration_info' and 'migration_get_info' entries to
'connection_info' and 'connection_get_info'.
- Updated driver interfaces with the new API parameters, drivers
must respect them.
- Changed share/api => scheduler RPCAPI back to asynchronous.
- Added optional SHA-256 validation to perform additional check if
bytes were corrupted during copying.
- Added mount options configuration to Data Service so CIFS shares
can be mounted.
- Driver may override _get_access_mapping if supports a different
access_type/protocol combination than what is defined by default.
- Added CIFS share protocol support and 'user' access type
support to Data Service.
- Reset Task State API now allows task_state to be unset using
'None' value.
- Added possibility to change share-network when migrating a share.
- Bumped microversion to 2.22.
- Removed support of all previous versions of Share Migration APIs.
APIImpact
DocImpact
Implements: blueprint newton-migration-improvements
Change-Id: Ief49a46c86ed3c22d3b31021aff86a9ce0ecbe3b
Tempest tests were not appropriate for driver-assisted migration,
so this was fixed.
Also, improved docstrings and fixed workflow for drivers when
implementing 2-phase migration to be accurate with tempest and
handle AZs, which were previously locked to the source share's
AZ.
Driver-assisted migration now creates an additional
share instance to better handle and support driver methods.
Updated allow_access and deny_access APIs to allow users to mount
migrating shares before issuing 'migration-complete'.
APIImpact
Closes-bug: #1594922
Change-Id: If4bfaf7e9d963b83c13a6fea241c2eda14f7f409
Share Migration must transfer access rules from source instance
to destination instance, but creating new ones instead of creating
new mapping between existing and destination instance. This
patch addresses this by copying the mapping.
Closes-bug: #1555677
Change-Id: Ie5a50e4117c14f822c22cace91bb8b0b95d0799d
Removed functionality of Share Migration relying on Manila Share
Service node, moved code to Data Service node for copy phase.
Added parameter 'notify' and share/api methods for future
implementation (see dependent patches).
Added new copy operation statuses, in order to implement future
API calls to obtain progress and cancel migration.
Added possibility of 2-phase migration for driver migration and
generic (fallback) migration.
Added admin export location support and removed approach of
replacing IP with config parameter.
Added Admin-only API entry points to:
- Migration Cancel (only during copying)
- Reset Task State field
- Migration Get Progress (only during copying)
- Migration Complete (2nd phase migration)
- Notify parameter on Migrate Share
APIImpact
DocImpact
Implements: blueprint data-service-migration
Change-Id: I1d65aac2f36942cd70eb214be561d59a15a4ba26
Since merge of update_access(), share migration tests in Manila gate
have been facing concurrency issues on RPCAPI requests, contributing
to gate instability, while scenario test has been broken.
This patch provides an immediate fix to share migration tests so
Manila gate can be more stable.
Change-Id: I151d2a4352892096f6f3336f88e470223f792090
Closes-bug: #1546946
- Add update_access() method to driver interface
- Move all code related to access operations to ShareInstanceAccess
class
- Statuses from individual access rules are now mapped to
share_instance's access_rules_status
- Add 'access_rules_status' field to share instance, which indicates
current status of applying access rules
APIImpact
Co-Authored-By: Rodrigo Barbieri <rodrigo.barbieri@fit-tecnologia.org.br>
Co-Authored-By: Tiago Pasqualini da Silva <tiago.pasqualini@gmail.com>
Implements: bp new-share-access-driver-interface
Change-Id: Iff1ec2e3176a46e9f6bd383b38ffc5d838aa8bb8
We bump hacking>=0.10.0, and hacking removed some rules, for
the full list of rules please see [1]. So don't need them any more.
Hacking related commits:
Remove H904 in commit b1fe19ebebe47a36b905d709467f5e82521bbd96
Add H105 using author tags
Add H238 hacking no old style class
Add H501 hacking no locals
[1]https://github.com/openstack-dev/hacking/blob/master/setup.cfg#L30
Change-Id: I6a1f3c3ef22be0962ffb81a5da5660837552e5d7
Share Migration allows a share to be migrated from
one host#pool to another host#pool through the
"manila migrate <share> <host#pool>" command. It first
calls the driver to perform it in an optimized way if
possible. If the driver returns that it did not migrate,
it performs a generic migration.
A new field has been added to "shares" table: task_state,
which tracks migration status.
For driver migration, the method migrate_share in driver
base class should be overridden.
For generic migration, drivers may use new config options
to achieve the necessary configuration:
- migration_mounting_backend_ip: If backend has additional
exports IP for admin network, specify it here.
- migration_data_copy_node_ip: IP of entity performing
migration between backends, such as manila node or
data copy service node. This may not apply for
DHSS = true drivers.
- migration_protocol_mount_command: specify mount command
with protocol and additional parameters. Advisable to restrict
protocols per backend. Defaults to "mount -t <share_proto>".
If additional customization is needed, drivers may override
certain methods:
- _mount_share: return the mount command.
- _umount_share: return the umount command.
- _get_access_rule_for_data_copy: return an access rule with
the IP address which will allow the manila node or data copy node
to mount the share after added permission through
allow-access API command.
Change-Id: I8dde892cb7c0180b2b56d8c7d680dfe2320c2ec7
Implements: blueprint share-migration
Douglas Viroel