Implement the share backup feature for NetApp driver.
NetApp SnapVault technology is used to create and restore
the backup for NetApp ONTAP share. backup delete workflow
just delete the transferred snapshot from destination
backup volume.
Depends-On: Ifb88ec096674ea8bc010c1c3f6dea1b51be3beaa
Change-Id: I5a4edbf547e7886fb4fa9c1bed90110a33f9bf3b
Implement method share_server_migration_get_progress to get the
share server migration percent based in the total size of shares
(GB) tranfered from source to destination.
Depends-On: I9eae95ff3f66a3497b00ca582491afec58ae6dc3
Closes-bug: #2030969
Change-Id: If4bf3378388cc0d9ea03f58b0ab5abd9a268bfdd
The certificate is automatically created on NetApp with 1 year i.e. 365
days of expiration time, and admin needs to manually extend it. It would
be nice Manila can take care to create certs with admin configuable
expiration time. Manila should first create the new cert with given
expiration time and if successful, delete the old cert.
Closes-bug: #2011693
Change-Id: I37e52b94dc492e91fe9e673b3619e6716737d39a
Asynchronous SnapMirror schedules are set using netapp config option
'netapp_snapmirror_schedule'. The delta for determining replica is
in-sync or out-of-sync updated to twice the schedule time seconds.
Also, not only new snapmirrors, but also old ones should
have a schedule according to the current
'netapp_snapmirror_schedule' config.
Closes-bug: #1996859
Depends-On: I0390f82dfdc130d49e3af6928996dd730e3cf69f
Change-Id: Ifbe0575f6c359929344763666e4d93d8c6084e83
This patch contains the functions related to DHSS True only
functions, that is, the operations related to share servers and
network configurations.
Many functions that are used on DHSS True operations were already
covered in previous patches because they are also used
by DHSS False operations.
This patch covers DHSS True specific implementation for:
> DHSS True driver initialization and periodic tasks
> Setup share server
> Delete share server
> Security services configurations (AD, LDAP and Kerberos)
> Create share from snapshot
> Share server migration
partially-implements: bp netapp-ontap-rest-api-client
Change-Id: Ia8992e8125c4fc7f30fb94f19449cd62b95a4831
Co-authored-by: Caique Mello <caiquemellosbo@gmail.com>
Co-authored-by: Felipe Rodrigues <felipefuty01@gmail.com>
Co-authored-by: Helena Dantas <helenamylena@gmail.com>
Co-authored-by: Matheus Andrade <matheus.andrade@netapp.com>
Co-authored-by: Lucas Oliveira <lucasmoliveira059@gmail.com>
Co-authored-by: Luisa Amaral <luisarfamaral@gmail.com>
Co-authored-by: Luiz Santos <luizfelipe.looze@gmail.com>
Co-authored-by: Raffaela de Castro Cunha <raffaelacunha@gmail.com>
Co-authored-by: Renan Vitor <renanv@netapp.com>
From version 2.76, 'default_ad_site' which if provided set domain
controller discovery mode to 'site'.
Related-bug: #1988146
Depends-On: I8e21e9170eace134a51efed84de1ccc58eb7eaaa
Change-Id: I251e4f94ef04e9ad2fe24844fb8ce1947f42d752
As the result of adoption of a REST API by NetApp ONTAP, this patch
is adding the basic structure to migrate from ZAPI Client to REST
incrementally.
To avoid adding bugs to the pre-existent ZAPI client and easily backport
the new client to older releases, the new REST implementation is a
completely new code on the client layer: a new client REST and API REST
classes. The driver layer should be agnostic from this transition, the
interfaces that the driver calls should keep the same as before.
The REST client contains a fallback mechanism, which will call the
ZAPI client when the operation is not implemented yet. This first
patch is only implementing the get ONTAP version operation using
REST, all other operations keep working with ZAPI.
The REST client can only work with ONTAP 9.11.1 and upper.
A new configuration `netapp_use_legacy_client` is added to enable
the operator to switch between new REST client implementation and the
old ZAPI implementation. The default is `True`, given that the REST
client is still experimental.
partially-implements: bp netapp-ontap-rest-api-client
Change-Id: I0bbc7609df66b72e9f8e26f4abb8092ae68cbe63
The NetApp driver has been working with FlexVol ONTAP volumes.
The driver does not support scaling FlexVol volumes higher than
100 TiB, which was a theoretical limit for the large namespace that
these containers were meant to handle. ONTAP's Flexgroup volumes
eliminate such limitations. So, added the support for provisioning
share as FlexGroup in the NetApp driver.
The FlexGroup provision is enabled by new option
`netapp_enable_flexgroup`, which will make the driver report a single
pool represeting all aggregates. The selection on which aggregates the
FlexGroup share will reside is up to ONTAP. If the administrator desires
to control that selection through Manila scheduler, it must inform the set
of aggregates that formss FlexGroup pool in the new option
`netapp_flexgroup_pool`.
Each NetApp pool will report now the capability: `netapp_flexgroup`
informing which type the pool is.
The following operations are allowed with FlexGroup shares (DHSS
True/False and NFS/CIFS):
- Create/Delete share;
- Shrink/Extend share;
- Create/Delete snapshot;
- Revert to snapshot;
- Manage/Unmanage snapshots;
- Create from snapshot;
- Replication[1]
- Manage/Unmanage shares;
The backend with one FlexGroup pool configured will drop the consistent
snapshot support for all pools.
The driver FlexGroup support requires ONTAP version 9.8 or greater.
[1] FlexGroup is limited to one single replica for ONTAP version
lower than 9.9.1.
DocImpact
Depends-On: If525e97a5d456d6ddebb4bf9bc8ff6190c95a555
Depends-On: I646f782c3e2be5ac799254f08a248a22cb9e0358
Implements: bp netapp-flexgroup-support
Change-Id: I4f68a9bb33be85f9a22e0be4ccf673647e713459
Signed-off-by: Felipe Rodrigues <felipefuty01@gmail.com>
Implements share server migration using a proper mechanism
provided by ONTAP. In case the driver identifies that the ONTAP
version matches the version where this mechanism is available,
ONTAP will automatically chose to use this instead of SVM DR.
- Implemented new methods for migrating a share server using a
new mechanism provided by ONTAP, when both source and destination
clusters have versions >= 9.10. This new migration mechanism
supports nondisruptive migrations in case there aren't network
changes in the migration.
- The NetApp now does not need to create an actual share server in
the backend prior to the migration, in case SVM Migrate is being
used.
- The NetApp ONTAP driver can now reuse network allocations from
the source share server in case a share network change wasn't
identified.
Change-Id: Idf1581d933d11280287f6801fd4aa886a627f66f
Depends-On: I48bafd92fe7a4d4ae0bafd5bf1961dace56b6005
This patch implements support for security service updates
for in use share networks. It works with all three security
service types. For 'active_directory' and 'kerberos', the 'domain'
attribute update isn't supported, since it can might affect
user's access to all related shares.
Change-Id: I8556e4e2e05deb9b116eacbd5afe2f7c5d77b44b
Depends-On: I129a794dfd2d179fa2b9a2fed050459d6f00b0de
Depends-On: I5fef50a17bc72ba66a3a9d6f786742bcb5745d7b
Implements: bp netapp-security-service-update
Co-Authored-By: Carlos Eduardo <ces.eduardo98@gmail.com>
Signed-off-by: Douglas Viroel <viroel@gmail.com>
This patch adds support for automated creation of FPolicy policies
and association to a share. The FPolicy configuration can be added using
the extra-specs 'netapp:fpolicy_extensions_to_include',
'netapp:fpolicy_extensions_to_exclude' and 'netapp:fpolicy_file_operations'.
Change-Id: I661de95bfb6f8e68b3a8c58663bb6055e9b809f6
Implements: bp netapp-fpolicy-support
Signed-off-by: Douglas Viroel <viroel@gmail.com>
This patch fixes some issues with LDAP client configuration on
ONTAP SVMs. With ldap security service, users should be able to
configure a LDAP client that can be used for authentication and
name mapping. The name service switch order remains: ldap,files.
Issues fixed:
- The driver now identifies when user provide a Active Directory
domain or a Linux/Unix LDAP server IP and sets the correct schema.
- LDAP configuration parameter `servers` was replaced by `ldap-servers`
in ONTAP 9.2, and now accepts host names too.
- Fix DNS configuration for LDAP security service
- User can now specify base search DN for LDAP queries, which can be
mandatory for Unix/Linux servers, using the security service `ou`
parameter.
Closes-Bug: #1916534
Change-Id: Ieaa53abbe50e7b708e508c132dfc4bb36b71a4f5
Signed-off-by: Douglas Viroel <viroel@gmail.com>
- Fixes the zapi calls for setting up a kerberos, which have
changed since ONTAP 8.3.
- Fixes kerberos configuration cleanup when deleting a
share server.
- Fixes access rules authentication methods for NFS when a
share server is configured for Kerberos.
Change-Id: I60b4f92979045b1fdb90ad8df4f65c1dfe463ae8
Closes-Bug: #1901189
Closes-Bug: #1904746
Closes-Bug: #1907669
Co-Authored-By: Felipe Rodrigues <felipefuty01@gmail.com>
Signed-off-by: Douglas Viroel <viroel@gmail.com>
NetApp driver is hard-coding the location of CA certificates for SSL
verification during HTTPS requests. This location may change depending
on the environment or/and backend.
This patch adds the `netapp_ssl_cert_path` configuration, enabling
each backend to choose the directory with certificates of trusted CA
or the CA bundle. If set to a directory, it must have been processed
using the c_rehash utility supplied with OpenSSL. If not informed,
it will use the Mozilla's carefully curated collection of Root
Certificates for validating the trustworthiness of SSL certificates.
Closes-Bug: #1900191
Change-Id: Idbed4745104de26af99bb16e07c6890637dfcfd1
This patch is a follow up from the main change[1] that adds support
for Adaptive QoS policies that have been pre-created in the storage.
Improvements added in this patch:
- Fail earlier when using this configuration with DHSS=True mode
and for shares that support replication.
- Fail earlier if no cluster credentials where provided to configure
volumes with QoS.
- Add support for migration and manage share operations.
Closes-Bug: #1895361
[1] https://review.opendev.org/#/c/740532/
Change-Id: I210994b84548ed6857e338c8e1f41667fa844614
Signed-off-by: Douglas Viroel <viroel@gmail.com>
This patch is a follow up patch of the main change[1] that adds
support for share server migration in NetApp driver.
It fixes two issues:
- Data motion 'get_backend_configuration' now avoids sending vserver
configuration, available only in DHSS=False mode, when driver is
configured with DHSS=True.
- After migrating a share server, all volumes appears with different
autosize configuration and need to be manually updated to the
original values. Provisioning options are also being updated in the
same method.
[1] https://review.opendev.org/#/c/747048/
Change-Id: I28a47417ec5dda0ed1f6c64fae37f5af6ca057e6
Signed-off-by: Douglas Viroel <viroel@gmail.com>
This patch adds support for share server migration between NetApp
ONTAP drivers. This operation is now supported for migrating a share
server and all its resources between two different clusters.
Share server migration relies on ONTAP features available only in
versions equal and greater than ``9.4``. Also, in order to have share
server migration working across ONTAP clusters, they must be peered in
advance.
At this moment, share server migration doesn't support migrate a share
server without disrupting the access to shares, since the export locations
are updated at the migration complete phase.
The driver doesn't support changing security services while changing the
destination share network. This functionality can be added in the future.
Co-Authored-By: Andre Beltrami <debeltrami@gmail.com>
Implements: bp netapp-share-server-migration
Depends-On: Ic0751027d2c3f1ef7ab0f7836baff3070a230cfd
Change-Id: Idfac890c034cf8cbb65abf685ab6cab5ef13a4b1
Signed-off-by: Douglas Viroel <viroel@gmail.com>
From this this change, shares and share groups can be created
upon share servers configured with specific NFS max transfer
limits. An administrator is now able to set the share type
extra-specs `netapp:udp_max_xfer_size` and
`netapp:tcp_max_xfer_size`. While creating a share server or
providing a share server to a share or a share group, the NetApp
driver will consider these extra-specs to decide whether to create
or reuse a share server.
Share server now contains the details:nfs_config field, which
stores the server NFS configuration dictionary. In case the server
does not have a NFS configuration requirement, it saves the
default NFS values, retrieved at the driver startup. A server
without details:nfs_config is considered as using the default one.
The share server manage operation was modified to also retrieve
its NFS max transfer configurations.
The share manage operation was modified to check whether the NFS
max transfer extra-specs are matching the share server configured
values.
It relies on ONTAP features available only in versions equal and
greater than ``9.4``.
Implements: bp netapp-share-server-nfs-modify
Change-Id: Iaddb771ae28ec59dd125af0bf638f591f5662bfc
Depends-On: I8daf919a764075998be95c5845807bec37104c78
With python3.7, the eventlet is breaking the ssl.py, so the https
is not working. This patch fixes it by changing the request library
(urllib by requests), the new library can be built over the
pyopenssl.py instead of ssl.py.
Closes-Bug: #1878993
Change-Id: I9c0b1f332ead25634f3dc3aebfdc8b51dfbc4178
This patch improves the operation of creating share from snapshot
to accept new destinations that can be different pools or
back ends.
Change-Id: Id3b3d5860d6325f368cbebfe7f97c98d64554d72
This patch adds support for replication with DHSS=True on
NetApp driver. It now handles peering operations between
share servers and shares.
Change-Id: I93888bcc6a0ca672671cf2aa254ceb23c4cbf692
NetApp ONTAP Multi-SVM driver was raising an error while trying to
create shares on multiple subnets that belong to the same neutron
network, as it was trying to map multiple ipspaces to the same VLAN
port.
This fix allows the driver to use the same ipspace and VLAN port across
all subnets belonging to the same neutron network.
Change-Id: If9cbb34a890ee44806c404085e40cc924a1296a7
Closes-Bug: #1774159
NVE is a software-based technology for encrypting
data at rest one volume at a time. An encryption
key, accessible only to the storage system,
ensures that volume data cannot be read if the
underlying device is repurposed, returned,
misplaced or stolen.
Signed-off-by: Erlon R. Cruz <erlon@netapp.com>
Signed-off-by: Tiago Pasqualini <tiagod@netapp.com>
Change-Id: Ib622c3d64cbec5a7254f6074f07d6f56f6492ca4
Implements: blueprint netapp-encrypted-shares
API Tracing is valuable when diagnosing problems or
unexpected behaviors with the ONTAP Manila driver.
However, turning it on may spam logs and make it rather
harder to trace through specific API calls.
Change-Id: I3c91638138ec1a2652efdadbc86176afa295abd8
ONTAP supports assigning QoS policy groups to storage
objects and workloads. [1]
Expose this functionality through the ONTAP manila
drivers (DHSS=True/False, NFS, CIFS).
The drivers will set the capability "qos" to True if the
configured credentials have access to create qos policy
groups on the configured ONTAP backend. When 'qos'
extra-spec is set in share types, scoped extra-specs can
be used to specify QoS ceiling values in iops or bps.
The drivers support the following QoS specs:
'netapp:maxiops', 'netapp:maxiopspergib', 'netapp:maxbps',
'netapp:maxbpspergib'. Policies are created on-demand
and manipulated as and when shares are manipulated
through manila.
[1] http://docs.netapp.com/ontap-9/index.jsp?topic=%2Fcom.netapp.doc.pow-perf-mon%2FGUID-38357C43-FB36-419D-B31F-6FD75B47254D.html
Implements: blueprint netapp-cdot-qos
Change-Id: I6f82c012ea60cfb1e9f82a696e2346ee95c60df3
Add tenant routes/gateway to Vservers created by the driver.
Change-Id: Id33c0e13d265d50f74f86ab8fb2c533eefa4b783
Closes-Bug: #1698258
Closes-Bug: #1612655
The DHSS=True cDOT driver can be used with non-segmented
networks, and the driver should not assume that network
interfaces are always assigned to VLAN ports.
Change-Id: Ifffae0f5e42d16ea40d693116b720c7f7d57c971
Closes-Bug: #1698250
This commit adds support for the revert-to-snapshot feature to
the NetApp cDOT drivers for both normal and replicated shares.
Implements: blueprint netapp-cdot-share-revert-to-snapshot
Change-Id: Ia939eba03b3db9cbba0cc6c16184578e8c8893d1
Add migration support in the cDOT driver.
Change-Id: I72e045b1c978b752f38cc3018cb2a7084e3f3e36
Implements: blueprint netapp-cdot-driver-optimized-migration
The NetApp cDOT drivers now include the cluster node utilization
metrics for each pool reported to the manila scheduler. These
values are designed to be included in the filter & goodness functions
used by the scheduler, so the cDOT drivers now also report those
functions to the scheduler for each pool.
Implements: blueprint netapp-cdot-goodness-functions
Change-Id: I3fca5c0ece1819eb4b3b98ed3fd9471cc5045977
The NetApp cDOT driver now explicitly filters root aggregates
from the pools reported to the manila scheduler if the driver
is operating with cluster credentials.
Change-Id: I659edada559e50d2332790025c65fae265a27c3d
Closes-Bug: #1624526
A customer has requested the ability to selectively enable NFS versions
3, 4, and 4.1 in the cDOT multi-SVM drivers. This can be accomplished
via a config option.
Change-Id: Ia16a3bde81568096c118d61e808614d630616d0d
Implements: blueprint netapp-cdot-configure-nfs-versions
NetApp cDOT controllers can mix SSDs and spinning disks
in the same aggregate, where the SSDs are used for a
cache. This commit reports the hybrid aggregate attribute
as well as the aggregate name to the scheduler for each
pool for use in extra specs matching.
Implements: blueprint netapp-report-cdot-hybrid-aggrs-manila
Change-Id: Iaa0bcd79789449f977b48f1de2adf997c936db61
With snapshot manage / unmanage in Manila core, we can
support that in the NetApp single-SVM driver.
Implements: blueprint netapp-cdot-snapshot-manage-unmanage
Change-Id: I7c6c005fb3fd8613da9e9ac04b9dd832781e35ca
Currently when DHSS=true with a Netapp cmode backend
during vserver creation, lifs and VLANs get created.
But on cleanup everything gets removed except the VLANs.
This patch removes the vlans at the end of vserver deletion.
Netapp prevents deleting VLANs with existing lifs on it.
Change-Id: Id0b56bbce8e5e9b8707f22d401d99c7867372a50
Closes-Bug: #1580163
Managing a share with a share type that has replication_type
extra_spec must be allowed. Drivers are expected to fail
this operation if the share was part of a replication relationship
that Manila does not know about.
Unmanaging a share with replicas must not be permitted
until all replicas are removed.
Managing and unmanaging of snapshots must not
be permitted for a share that has replicas.
Modify the NetApp driver for manage_existing to check
for existing replicas.
Also fix issue with manage retry where the share
data was being altered inappropriately by a DB API.
Closes-Bug: #1561641
Closes-Bug: #1565903
Co-Authored-By: Goutham Pacha Ravi <gouthamr@netapp.com>
Change-Id: I82f1fef1e30114e017efd00fa7da70aceecab94c
When promoting or updating a replica that is on a backend that is unreachable,
the status and replica_state fields for the replica should be ERROR.
Closes-Bug: #1554170
Change-Id: I7e4dd53a70aea2beeed1404c54e354f773a94427
agireesh