NetApp driver changes to accommodate human readable
share location. Export path is updated with
human frendly value if present else use share-id.
partially-implements: bp human-readable-export-locations
Depends-On: I72ac7e24ddd4330d76cafd5e7f78bac2b0174883
Change-Id: I2f5bfdbc9d0458c7b9198a3eb94b3b095d5b5e04
Implement the share backup feature for NetApp driver.
NetApp SnapVault technology is used to create and restore
the backup for NetApp ONTAP share. backup delete workflow
just delete the transferred snapshot from destination
backup volume.
Depends-On: Ifb88ec096674ea8bc010c1c3f6dea1b51be3beaa
Change-Id: I5a4edbf547e7886fb4fa9c1bed90110a33f9bf3b
If share created from snapshot is deleted immediately after creation
and if clone split operation is in progress, then delete call fails.
Fix this issue by first stopping the clone split job and then continue
with deletion.
Closes-bug: #1960239
Change-Id: If9844b3da70cec427c6260ee239c8c6131ed77ed
When creating a share using CIFS protocol users can face errors related
with incorrect credentials or insufficient privileges which have not been properly described by the exception error message.
This patch add a more clearer error message to address this kind of
error to the user.
Closes-Bug: #1900755
Change-Id: I589c218f2c1072e17e76a6a8d8d81541d5072ad1
Implement method share_server_migration_get_progress to get the
share server migration percent based in the total size of shares
(GB) tranfered from source to destination.
Depends-On: I9eae95ff3f66a3497b00ca582491afec58ae6dc3
Closes-bug: #2030969
Change-Id: If4bf3378388cc0d9ea03f58b0ab5abd9a268bfdd
The certificate is automatically created on NetApp with 1 year i.e. 365
days of expiration time, and admin needs to manually extend it. It would
be nice Manila can take care to create certs with admin configuable
expiration time. Manila should first create the new cert with given
expiration time and if successful, delete the old cert.
Closes-bug: #2011693
Change-Id: I37e52b94dc492e91fe9e673b3619e6716737d39a
Migrate non-disruptive cifs share from different pools change the
export location. When the non-disruptive migration complete
process is started a new share and export location is created. As
result, Manila finds a conflict between the old export location and
the new one.
This patch add a condition to skip export location creation when
a CIFS migration is in progress, also change the way that the export
location is created. Instead of create the export path with share
name, the new one is taken from the backend. The fix is only for
ZAPI API calls.
Change-Id: I1bb888a0b644f0b071816d275d464c4dd27125a7
Co-authored-by: Lucas Oliveira <lucasmoliveira059@gmail.com>
Closes-bug: #1920937
This patch contains the functions related to DHSS True only
functions, that is, the operations related to share servers and
network configurations.
Many functions that are used on DHSS True operations were already
covered in previous patches because they are also used
by DHSS False operations.
This patch covers DHSS True specific implementation for:
> DHSS True driver initialization and periodic tasks
> Setup share server
> Delete share server
> Security services configurations (AD, LDAP and Kerberos)
> Create share from snapshot
> Share server migration
partially-implements: bp netapp-ontap-rest-api-client
Change-Id: Ia8992e8125c4fc7f30fb94f19449cd62b95a4831
Co-authored-by: Caique Mello <caiquemellosbo@gmail.com>
Co-authored-by: Felipe Rodrigues <felipefuty01@gmail.com>
Co-authored-by: Helena Dantas <helenamylena@gmail.com>
Co-authored-by: Matheus Andrade <matheus.andrade@netapp.com>
Co-authored-by: Lucas Oliveira <lucasmoliveira059@gmail.com>
Co-authored-by: Luisa Amaral <luisarfamaral@gmail.com>
Co-authored-by: Luiz Santos <luizfelipe.looze@gmail.com>
Co-authored-by: Raffaela de Castro Cunha <raffaelacunha@gmail.com>
Co-authored-by: Renan Vitor <renanv@netapp.com>
From version 2.76, 'default_ad_site' which if provided set domain
controller discovery mode to 'site'.
Related-bug: #1988146
Depends-On: I8e21e9170eace134a51efed84de1ccc58eb7eaaa
Change-Id: I251e4f94ef04e9ad2fe24844fb8ce1947f42d752
When share server is in error state in Manila and backend vserver has
not been created, there exist entries of share server and shares in
database. Deletion of share in such case cause share in error_deleting
state. In order to delete share correctly, we need to fix _get_vserver
which should check vserver existence before building the vserver
client.
Closes-Bug: #1964592
Change-Id: I1f6b94a7e54403a559a5d77b4d93b939a40e76bf
If deletion with username + password is not working.
Fixes error: 'NetApp API failed. Reason - 19149:Username cannot be empty.'
The parameter username is marked as optional, but it has to be given
either the force option or username + password.
Closes-Bug: #1995733
Change-Id: I778e47dabbd518dd4f7797b99f09e52fa7f56a3d
If the user is configuring "Servers" as AD Server in the Security
Service then the domain controller discovery mode should be changed
to "none" and only these servers should be used.
Closes-bug: #1987537
Change-Id: I5cc181ac32529174e5e73dfde4604a470bd32442
The NetApp driver is not reporting the home state of the aggregate
pools. This information is useful during maintenance tasks, since
not home aggregate cannot create shares.
This patch adds to the report netapp capabilities the boolean
`netapp_is_home`.
Closes-Bug: #1927823
Change-Id: I8e98541d8e457e9e4609410853b50d6156465f61
This patch adds support to multiple subnets in a same network
segment (and AZ). All subnets assigned to a share must have the
same VLAN id, otherwise the ONTAP driver will fail on share server
creation.
Depends-On: I7de9de4ae509182e9494bba604979cce03acceec
Implements: blueprint ontap-multiple-subnets-same-segment-id
Change-Id: If5db09627a2a9a98951a972e15b8af0857598d1e
Python2 is no longer supported, so in this patch
set we remove the usage of the six (py2 and py3
compatibility library) in favor of py3 syntax.
Change-Id: I3ddfad568a1b578bee23a6d1a96de9551e336bb4
The issue happens when the parameter 'encrypt-destination' is sent to
OnTap devices with version lower than 9.0 or OnTapi lower then 1.110.
This patch adds a condition to check these versions and ignore the flag
addition if needed on '_send_volume_move_request'.
Closes-Bug: 1915237
Change-Id: Id9faa4f73996cb7467b7361d0406c980f06905c2
The NetApp ONTAP driver was creating thin provisioned volumes on
AFF systems even when the thin provisioned extra spec was saying
the contrary.
ONTAP AFF systems create thin provisioned volumes by default, which
differs from FAS systems, that creates thick provisioned volumes by
default.
The NetApp ONTAP driver was modified to fix this misbehavior.
Change-Id: I9218bf0d7cb607dd75e892ce86af958d43a7fcdb
Closes-Bug: #1929421
The NetApp driver has been working with FlexVol ONTAP volumes.
The driver does not support scaling FlexVol volumes higher than
100 TiB, which was a theoretical limit for the large namespace that
these containers were meant to handle. ONTAP's Flexgroup volumes
eliminate such limitations. So, added the support for provisioning
share as FlexGroup in the NetApp driver.
The FlexGroup provision is enabled by new option
`netapp_enable_flexgroup`, which will make the driver report a single
pool represeting all aggregates. The selection on which aggregates the
FlexGroup share will reside is up to ONTAP. If the administrator desires
to control that selection through Manila scheduler, it must inform the set
of aggregates that formss FlexGroup pool in the new option
`netapp_flexgroup_pool`.
Each NetApp pool will report now the capability: `netapp_flexgroup`
informing which type the pool is.
The following operations are allowed with FlexGroup shares (DHSS
True/False and NFS/CIFS):
- Create/Delete share;
- Shrink/Extend share;
- Create/Delete snapshot;
- Revert to snapshot;
- Manage/Unmanage snapshots;
- Create from snapshot;
- Replication[1]
- Manage/Unmanage shares;
The backend with one FlexGroup pool configured will drop the consistent
snapshot support for all pools.
The driver FlexGroup support requires ONTAP version 9.8 or greater.
[1] FlexGroup is limited to one single replica for ONTAP version
lower than 9.9.1.
DocImpact
Depends-On: If525e97a5d456d6ddebb4bf9bc8ff6190c95a555
Depends-On: I646f782c3e2be5ac799254f08a248a22cb9e0358
Implements: bp netapp-flexgroup-support
Change-Id: I4f68a9bb33be85f9a22e0be4ccf673647e713459
Signed-off-by: Felipe Rodrigues <felipefuty01@gmail.com>
Implements share server migration using a proper mechanism
provided by ONTAP. In case the driver identifies that the ONTAP
version matches the version where this mechanism is available,
ONTAP will automatically chose to use this instead of SVM DR.
- Implemented new methods for migrating a share server using a
new mechanism provided by ONTAP, when both source and destination
clusters have versions >= 9.10. This new migration mechanism
supports nondisruptive migrations in case there aren't network
changes in the migration.
- The NetApp now does not need to create an actual share server in
the backend prior to the migration, in case SVM Migrate is being
used.
- The NetApp ONTAP driver can now reuse network allocations from
the source share server in case a share network change wasn't
identified.
Change-Id: Idf1581d933d11280287f6801fd4aa886a627f66f
Depends-On: I48bafd92fe7a4d4ae0bafd5bf1961dace56b6005
Implement the `readable` replication type for the NetApp driver.
The driver will keep having support for the `dr` type as well, being
the driver replication type a list containing them.
The replicas for readable style are mounted, created the export and
applied the QoS. When promoting, the original active replica does
not need to be unmounted. The user just loses the write access.
The update access interface is now applying rules for non active
replicas that are readable.
Implements: bp netapp-readable-replica
Change-Id: Icc74eaecc75c3064715f91bebb994e93c0053663
Signed-off-by: Felipe Rodrigues <felipefuty01@gmail.com>
We are replacing all usages of the 'retrying' package with
'tenacity' as the author of retrying is not actively maintaining
the project. Tenacity is a fork of retrying, but has improved the
interface and extensibility (see [1] for more details). Our end
goal here is removing the retrying package from our requirements.
Tenacity provides the same functionality as retrying, but has the
following major differences to account for:
- Tenacity uses seconds rather than ms as retrying did
(the retry interface in manila exposed time in seconds as well)
- Tenacity has different kwargs for the decorator and
Retrying class itself.
- Tenacity has a different approach for retrying args by
using classes for its stop/wait/retry kwargs.
- By default tenacity raises a RetryError if a retried callable
times out; retrying raises the last exception from the callable.
Tenacity provides backwards compatibility here by offering
the 'reraise' kwarg - we are going to set this in the retry interface
by default.
- For retries that check a result, tenacity will raise if the
retried function raises, whereas retrying retried on all
exceptions - we haven't exposed this in the retry interface.
This patch updates all usages of retrying with tenacity.
Unit tests are added where applicable.
[1] https://github.com/jd/tenacity
Co-Authored-By: boden <bodenvmw@gmail.com>
Co-Authored-By: Goutham Pacha Ravi <gouthampravi@gmail.com>
Closes-Bug: #1635393
Change-Id: Ia0c3fa5cd82356a33becbf57444f3db5ffbb0dd0
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
In the ONTAP 9.8 and older, there is a bug in the zapi call
`snapmirror-release-iter` when using the scoped account
configuration. The operation is returning success, but the release
is not occurring.
To avoid this issue, the Netapp driver was changed to release with
the `snapmirror-release` zapi call, instead of the one that has a
bug. This new zapi call requires the field `relationship-id` to be
specified and it is being retrieved from ONTAP before calling it.
A small fix, not related to the bug, was made in the zapi call to
`snapmirror-release-iter` because the `relationship-info-only`
field was in the wrong place according to the documentation.
Closes-Bug: #1934889
Change-Id: I21633d4ffe14983365b0b6239216ed5d0bbfaace
This patch is a follow up of [1] to address some comments
added in the review process. It also adds more unit tests to
validate the new code added.
[1] https://review.opendev.org/c/openstack/manila/+/775032
Change-Id: If7b8628fa05200363a129eb19d9dc80fb7e3bc36
Signed-off-by: Douglas Viroel <viroel@gmail.com>
This patch implements support for security service updates
for in use share networks. It works with all three security
service types. For 'active_directory' and 'kerberos', the 'domain'
attribute update isn't supported, since it can might affect
user's access to all related shares.
Change-Id: I8556e4e2e05deb9b116eacbd5afe2f7c5d77b44b
Depends-On: I129a794dfd2d179fa2b9a2fed050459d6f00b0de
Depends-On: I5fef50a17bc72ba66a3a9d6f786742bcb5745d7b
Implements: bp netapp-security-service-update
Co-Authored-By: Carlos Eduardo <ces.eduardo98@gmail.com>
Signed-off-by: Douglas Viroel <viroel@gmail.com>
This patch adds support for automated creation of FPolicy policies
and association to a share. The FPolicy configuration can be added using
the extra-specs 'netapp:fpolicy_extensions_to_include',
'netapp:fpolicy_extensions_to_exclude' and 'netapp:fpolicy_file_operations'.
Change-Id: I661de95bfb6f8e68b3a8c58663bb6055e9b809f6
Implements: bp netapp-fpolicy-support
Signed-off-by: Douglas Viroel <viroel@gmail.com>
This patch fixes some issues with LDAP client configuration on
ONTAP SVMs. With ldap security service, users should be able to
configure a LDAP client that can be used for authentication and
name mapping. The name service switch order remains: ldap,files.
Issues fixed:
- The driver now identifies when user provide a Active Directory
domain or a Linux/Unix LDAP server IP and sets the correct schema.
- LDAP configuration parameter `servers` was replaced by `ldap-servers`
in ONTAP 9.2, and now accepts host names too.
- Fix DNS configuration for LDAP security service
- User can now specify base search DN for LDAP queries, which can be
mandatory for Unix/Linux servers, using the security service `ou`
parameter.
Closes-Bug: #1916534
Change-Id: Ieaa53abbe50e7b708e508c132dfc4bb36b71a4f5
Signed-off-by: Douglas Viroel <viroel@gmail.com>
- Fixes the zapi calls for setting up a kerberos, which have
changed since ONTAP 8.3.
- Fixes kerberos configuration cleanup when deleting a
share server.
- Fixes access rules authentication methods for NFS when a
share server is configured for Kerberos.
Change-Id: I60b4f92979045b1fdb90ad8df4f65c1dfe463ae8
Closes-Bug: #1901189
Closes-Bug: #1904746
Closes-Bug: #1907669
Co-Authored-By: Felipe Rodrigues <felipefuty01@gmail.com>
Signed-off-by: Douglas Viroel <viroel@gmail.com>
Set following 4 attributes when creating nfs shares:
showmount: enabled
v3-ms-dos-client: support
v3-connection-drop: disable
ejukebox-errors: disable
Change-Id: I86c3de32d36940e85545fcb337723e4b3edce216
Closes-bug: 1901937
Co-authored-by: Maurice Escher <maurice.escher@sap.com>
This patch is a follow up from the main change[1] that adds support
for Adaptive QoS policies that have been pre-created in the storage.
Improvements added in this patch:
- Fail earlier when using this configuration with DHSS=True mode
and for shares that support replication.
- Fail earlier if no cluster credentials where provided to configure
volumes with QoS.
- Add support for migration and manage share operations.
Closes-Bug: #1895361
[1] https://review.opendev.org/#/c/740532/
Change-Id: I210994b84548ed6857e338c8e1f41667fa844614
Signed-off-by: Douglas Viroel <viroel@gmail.com>
This patch is a follow up patch of the main change[1] that adds
support for share server migration in NetApp driver.
It fixes two issues:
- Data motion 'get_backend_configuration' now avoids sending vserver
configuration, available only in DHSS=False mode, when driver is
configured with DHSS=True.
- After migrating a share server, all volumes appears with different
autosize configuration and need to be manually updated to the
original values. Provisioning options are also being updated in the
same method.
[1] https://review.opendev.org/#/c/747048/
Change-Id: I28a47417ec5dda0ed1f6c64fae37f5af6ca057e6
Signed-off-by: Douglas Viroel <viroel@gmail.com>
Added support for Adaptive QoS policies that have been pre-created on
the storage system, with clustered ONTAP version 9.4 or higher. To use
this feature, configure a Manila share type with the extra-spec
"netapp:adaptive_qos_policy_group" and value set to the qos policy
group on the ONTAP storage system, for example:
netapp:adaptive_qos_policy_group=platform3
Note that a cluster scoped account must be used in the driver
configuration in order to use QoS in clustered ONTAP. Other notes:
-This only works for backends without share server management.
-This does not work for share replicas or share migration.
Partially-Implements: bp netapp-adaptive-qos-support
Change-Id: I3cc1d2fa2a8380ca925538cab5a3414ac2141d70
This patch adds support for share server migration between NetApp
ONTAP drivers. This operation is now supported for migrating a share
server and all its resources between two different clusters.
Share server migration relies on ONTAP features available only in
versions equal and greater than ``9.4``. Also, in order to have share
server migration working across ONTAP clusters, they must be peered in
advance.
At this moment, share server migration doesn't support migrate a share
server without disrupting the access to shares, since the export locations
are updated at the migration complete phase.
The driver doesn't support changing security services while changing the
destination share network. This functionality can be added in the future.
Co-Authored-By: Andre Beltrami <debeltrami@gmail.com>
Implements: bp netapp-share-server-migration
Depends-On: Ic0751027d2c3f1ef7ab0f7836baff3070a230cfd
Change-Id: Idfac890c034cf8cbb65abf685ab6cab5ef13a4b1
Signed-off-by: Douglas Viroel <viroel@gmail.com>
From this this change, shares and share groups can be created
upon share servers configured with specific NFS max transfer
limits. An administrator is now able to set the share type
extra-specs `netapp:udp_max_xfer_size` and
`netapp:tcp_max_xfer_size`. While creating a share server or
providing a share server to a share or a share group, the NetApp
driver will consider these extra-specs to decide whether to create
or reuse a share server.
Share server now contains the details:nfs_config field, which
stores the server NFS configuration dictionary. In case the server
does not have a NFS configuration requirement, it saves the
default NFS values, retrieved at the driver startup. A server
without details:nfs_config is considered as using the default one.
The share server manage operation was modified to also retrieve
its NFS max transfer configurations.
The share manage operation was modified to check whether the NFS
max transfer extra-specs are matching the share server configured
values.
It relies on ONTAP features available only in versions equal and
greater than ``9.4``.
Implements: bp netapp-share-server-nfs-modify
Change-Id: Iaddb771ae28ec59dd125af0bf638f591f5662bfc
Depends-On: I8daf919a764075998be95c5845807bec37104c78
ONTAP 9.5 adds vserver cifs domain preferred-dc configuration
validation. The parameter 'skip-config-validation' is mandatory
for 'cifs-domain-preferred-dc-add' now and is set to false,
i.e. an invalid config will raise early.
Change-Id: I5eb97ba1b29ad24b296e12064f81eb93533c6b99
Closes-Bug: 1887643
This patch improves the operation of creating share from snapshot
to accept new destinations that can be different pools or
back ends.
Change-Id: Id3b3d5860d6325f368cbebfe7f97c98d64554d72
The NetApp ONTAP driver is now fixed to allow extension
and shrinking of share replicas after they get promoted.
Change-Id: Iea92feaf5894c10674d2ec4c1a4d7b0191e6d5b4
Closes-Bug: #1700871
This patch adds support for replication with DHSS=True on
NetApp driver. It now handles peering operations between
share servers and shares.
Change-Id: I93888bcc6a0ca672671cf2aa254ceb23c4cbf692
This patch implements the functionality of managing and
unmanaging of share servers to the NetApp driver,
allowing for shares and snapshots to be managed
and unmanaged in DHSS=True driver mode.
Implements: bp netapp-driver-manage-unmanage-with-share-servers
Change-Id: I6051cf038dcf9f175e0610fff0adf360230dc23c
Depends-On: I452c2a99b186f53d737cb7fbd7eabfcfd9b249d6
The security service dns_ip parameter also takes a list of
comma separated DNS IPs for vserver dns configuration.
Allows HA setup, where DNS can be down for maintenance.
Change-Id: I1da99d0b44778d3ccaaff5871dfcb175b8cdbc02
jayaanand.borra@netapp.com