NetApp driver changes to accommodate human readable
share location. Export path is updated with
human frendly value if present else use share-id.
partially-implements: bp human-readable-export-locations
Depends-On: I72ac7e24ddd4330d76cafd5e7f78bac2b0174883
Change-Id: I2f5bfdbc9d0458c7b9198a3eb94b3b095d5b5e04
Implement the share backup feature for NetApp driver.
NetApp SnapVault technology is used to create and restore
the backup for NetApp ONTAP share. backup delete workflow
just delete the transferred snapshot from destination
backup volume.
Depends-On: Ifb88ec096674ea8bc010c1c3f6dea1b51be3beaa
Change-Id: I5a4edbf547e7886fb4fa9c1bed90110a33f9bf3b
If share created from snapshot is deleted immediately after creation
and if clone split operation is in progress, then delete call fails.
Fix this issue by first stopping the clone split job and then continue
with deletion.
Closes-bug: #1960239
Change-Id: If9844b3da70cec427c6260ee239c8c6131ed77ed
Add the new scheduler weigher NetAppAIQWeigher that relies on
the NetApp Acitve IQ software to weigh the hosts. It only
works with NetApp only hosts.
It is also adding a new NetApp specific pool information
called ``netapp_cluster_name`` that contains the name
of the cluster where the pool is located.
Implements: netapp-active-iq-scheduler-weigher
Signed-off-by Felipe Rodrigues <felipefuty01@gmail.com>
Change-Id: I36b08066545afdaa37e053eee319bc9cd489efdc
When a volume is created in NetApp ONTAP, it has a few autosize
attributes that are set by default. The values of the attributes
are defined according to the volume type (DP/RW).
During the replica promotion, the types are swaped between source
and destination volumes, but the autosize values were not being
updated. This patch fixes this behavior, calling an autosize reset
after promoting the replica.
Closes-Bug: #1957075
Change-Id: I9a4e5763927b7585a8fbd6b0004d6a123dcd7fae
Changes were done in create_share_from_snapshot
method so that a scoped account does not need to get the
source and the destination cluster names and does not have the
permissions.
Closes-Bug: #1922512
Change-Id: Ib36c81c213a374a918378854ce0a89ce70acf1d0
Asynchronous SnapMirror schedules are set using netapp config option
'netapp_snapmirror_schedule'. The delta for determining replica is
in-sync or out-of-sync updated to twice the schedule time seconds.
Also, not only new snapmirrors, but also old ones should
have a schedule according to the current
'netapp_snapmirror_schedule' config.
Closes-bug: #1996859
Depends-On: I0390f82dfdc130d49e3af6928996dd730e3cf69f
Change-Id: Ifbe0575f6c359929344763666e4d93d8c6084e83
Replica promote is retaining unneeded snapshots from previous
SnapMirror relationships and increasing the amount of space
consumed from snapshots in the storage system.
This patch fixes the issue by calling the snapmirror release
operation after resync completes its transferring, which allows
the SnapMirror software to properly cleanup unneeded resources.
Closes-Bug: #1982808
Change-Id: I516fb3575e30d18d971d6a1b7f3b9ad7120c3bbd
Currently netapp_snapmirror_quiesce_timeout is an option of replica
promote that can only be set by the operator. There are scenarios where
high timeout value set by operator does not fit well e.g. disaster
stroke and user want to do an unplanned failover fast. Added new
option 'quiesce_wait_time' to share replica promote API which allows
to use specified wait_time(in seconds) instead of config option.
Closes-bug: #2000171
Change-Id: Ib02063ee8b82f7374cd89f90e7f24a845c6c7cd7
Managed snapshot has its name changed. When the volume is reverted
to the snapshot, the name is reverted to the old too. As result,
the Manila loses the snapshot reference, causing some issues.
This patch is removing the code that renames the snapshot during
management, so the snapshot name is kept as informed, saving it
to the provider location field.
Closes-bug: #1936648
Change-Id: Ib1a928d453de80d6841524cdb86c4708a5b0dbdf
In order to determine replica state from snapmirror, in addition to
existing check of last-transfer-end-timestamp', also add new checks
of `last-transfer-size` and `last-transfer-error`. New config option
`netapp_snapmirror_last_transfer_size_limit` added with default value
of 1MB. The last-transfer-size above this value or presence of any
last-transfer-error is considered as replica is out_of_sync.
Closes-bug: #1989175
Change-Id: I6d038244493583cc943063b50d731b8c1ef5ed28
Added missed 'service' and 'reason' keywords for Manila exceptions unit tests.
Closes-bug: #1966473
Change-Id: I70533951f46cddd171b1979ed57a59a5448a4557
The NetApp driver is not reporting the home state of the aggregate
pools. This information is useful during maintenance tasks, since
not home aggregate cannot create shares.
This patch adds to the report netapp capabilities the boolean
`netapp_is_home`.
Closes-Bug: #1927823
Change-Id: I8e98541d8e457e9e4609410853b50d6156465f61
The NetApp driver has been working with FlexVol ONTAP volumes.
The driver does not support scaling FlexVol volumes higher than
100 TiB, which was a theoretical limit for the large namespace that
these containers were meant to handle. ONTAP's Flexgroup volumes
eliminate such limitations. So, added the support for provisioning
share as FlexGroup in the NetApp driver.
The FlexGroup provision is enabled by new option
`netapp_enable_flexgroup`, which will make the driver report a single
pool represeting all aggregates. The selection on which aggregates the
FlexGroup share will reside is up to ONTAP. If the administrator desires
to control that selection through Manila scheduler, it must inform the set
of aggregates that formss FlexGroup pool in the new option
`netapp_flexgroup_pool`.
Each NetApp pool will report now the capability: `netapp_flexgroup`
informing which type the pool is.
The following operations are allowed with FlexGroup shares (DHSS
True/False and NFS/CIFS):
- Create/Delete share;
- Shrink/Extend share;
- Create/Delete snapshot;
- Revert to snapshot;
- Manage/Unmanage snapshots;
- Create from snapshot;
- Replication[1]
- Manage/Unmanage shares;
The backend with one FlexGroup pool configured will drop the consistent
snapshot support for all pools.
The driver FlexGroup support requires ONTAP version 9.8 or greater.
[1] FlexGroup is limited to one single replica for ONTAP version
lower than 9.9.1.
DocImpact
Depends-On: If525e97a5d456d6ddebb4bf9bc8ff6190c95a555
Depends-On: I646f782c3e2be5ac799254f08a248a22cb9e0358
Implements: bp netapp-flexgroup-support
Change-Id: I4f68a9bb33be85f9a22e0be4ccf673647e713459
Signed-off-by: Felipe Rodrigues <felipefuty01@gmail.com>
Implements share server migration using a proper mechanism
provided by ONTAP. In case the driver identifies that the ONTAP
version matches the version where this mechanism is available,
ONTAP will automatically chose to use this instead of SVM DR.
- Implemented new methods for migrating a share server using a
new mechanism provided by ONTAP, when both source and destination
clusters have versions >= 9.10. This new migration mechanism
supports nondisruptive migrations in case there aren't network
changes in the migration.
- The NetApp now does not need to create an actual share server in
the backend prior to the migration, in case SVM Migrate is being
used.
- The NetApp ONTAP driver can now reuse network allocations from
the source share server in case a share network change wasn't
identified.
Change-Id: Idf1581d933d11280287f6801fd4aa886a627f66f
Depends-On: I48bafd92fe7a4d4ae0bafd5bf1961dace56b6005
Implement the `readable` replication type for the NetApp driver.
The driver will keep having support for the `dr` type as well, being
the driver replication type a list containing them.
The replicas for readable style are mounted, created the export and
applied the QoS. When promoting, the original active replica does
not need to be unmounted. The user just loses the write access.
The update access interface is now applying rules for non active
replicas that are readable.
Implements: bp netapp-readable-replica
Change-Id: Icc74eaecc75c3064715f91bebb994e93c0053663
Signed-off-by: Felipe Rodrigues <felipefuty01@gmail.com>
In order to optimize the NetApp ONTAP driver, this patch is caching
the status of driver pools and reusing for the each share server,
given that the pool is not separated by share server.
The option `netapp_cached_aggregates_status_lifetime` is added
for controlling the time that the cached values is considered
valid.
Closes-Bug: #1900469
Change-Id: I14a059615fc29c7c173c035bb51d39e0bbb8b70a
This patch adds support for automated creation of FPolicy policies
and association to a share. The FPolicy configuration can be added using
the extra-specs 'netapp:fpolicy_extensions_to_include',
'netapp:fpolicy_extensions_to_exclude' and 'netapp:fpolicy_file_operations'.
Change-Id: I661de95bfb6f8e68b3a8c58663bb6055e9b809f6
Implements: bp netapp-fpolicy-support
Signed-off-by: Douglas Viroel <viroel@gmail.com>
This patch is a follow up from the main change[1] that adds support
for Adaptive QoS policies that have been pre-created in the storage.
Improvements added in this patch:
- Fail earlier when using this configuration with DHSS=True mode
and for shares that support replication.
- Fail earlier if no cluster credentials where provided to configure
volumes with QoS.
- Add support for migration and manage share operations.
Closes-Bug: #1895361
[1] https://review.opendev.org/#/c/740532/
Change-Id: I210994b84548ed6857e338c8e1f41667fa844614
Signed-off-by: Douglas Viroel <viroel@gmail.com>
This change fixes the NetApp promote back issue when using CIFS
protocol. When promoting a replica, the NetApp ONTAP driver
attempts to create a new CIFS share entity (an access point as
defined in [1]) for the new active replica. This behavior
causes a failure since the storage identifies that a current
backend CIFS share with the same name exists, considering
that the reffered replica was once the active one.
This issue is addressed by removing the related CIFS share
entity when the replica gets promoted.
[1] https://library.netapp.com/ecmdocs/ECMP1401220/html/GUID-1898D717-A510-4B3D-B2E3-CCDDD5BD0089.html
Closes-Bug: #1879368
Change-Id: Id9bdd5df0ff05ea08881dd2c83397f0a367d9945
Added support for Adaptive QoS policies that have been pre-created on
the storage system, with clustered ONTAP version 9.4 or higher. To use
this feature, configure a Manila share type with the extra-spec
"netapp:adaptive_qos_policy_group" and value set to the qos policy
group on the ONTAP storage system, for example:
netapp:adaptive_qos_policy_group=platform3
Note that a cluster scoped account must be used in the driver
configuration in order to use QoS in clustered ONTAP. Other notes:
-This only works for backends without share server management.
-This does not work for share replicas or share migration.
Partially-Implements: bp netapp-adaptive-qos-support
Change-Id: I3cc1d2fa2a8380ca925538cab5a3414ac2141d70
This patch adds support for share server migration between NetApp
ONTAP drivers. This operation is now supported for migrating a share
server and all its resources between two different clusters.
Share server migration relies on ONTAP features available only in
versions equal and greater than ``9.4``. Also, in order to have share
server migration working across ONTAP clusters, they must be peered in
advance.
At this moment, share server migration doesn't support migrate a share
server without disrupting the access to shares, since the export locations
are updated at the migration complete phase.
The driver doesn't support changing security services while changing the
destination share network. This functionality can be added in the future.
Co-Authored-By: Andre Beltrami <debeltrami@gmail.com>
Implements: bp netapp-share-server-migration
Depends-On: Ic0751027d2c3f1ef7ab0f7836baff3070a230cfd
Change-Id: Idfac890c034cf8cbb65abf685ab6cab5ef13a4b1
Signed-off-by: Douglas Viroel <viroel@gmail.com>
From this this change, shares and share groups can be created
upon share servers configured with specific NFS max transfer
limits. An administrator is now able to set the share type
extra-specs `netapp:udp_max_xfer_size` and
`netapp:tcp_max_xfer_size`. While creating a share server or
providing a share server to a share or a share group, the NetApp
driver will consider these extra-specs to decide whether to create
or reuse a share server.
Share server now contains the details:nfs_config field, which
stores the server NFS configuration dictionary. In case the server
does not have a NFS configuration requirement, it saves the
default NFS values, retrieved at the driver startup. A server
without details:nfs_config is considered as using the default one.
The share server manage operation was modified to also retrieve
its NFS max transfer configurations.
The share manage operation was modified to check whether the NFS
max transfer extra-specs are matching the share server configured
values.
It relies on ONTAP features available only in versions equal and
greater than ``9.4``.
Implements: bp netapp-share-server-nfs-modify
Change-Id: Iaddb771ae28ec59dd125af0bf638f591f5662bfc
Depends-On: I8daf919a764075998be95c5845807bec37104c78
When the NetApp backend starts, it needs to know whether the
`revert_to_snapshot` support exist. So, it was retrieving the
licenses and checking if the `SnapRestore` is included. Using a
scoped account, the backend cannot retrieve that information,
though. So, this patch solves it by sending a fake operation
`revert_to_snapshot` that must fail. Analyzing the given error, it
sets the backend support field.
Closes-Bug: #1882590
Change-Id: Ib71a6cec939288498e48736f129fbfdacaabe9da
NetApp ONTAP share delete operation can fail sometimes when is triggered
immediately after migration cancelation on a overloaded NetApp backend.
Migration cancelation invokes "abort_volume_move" which is an asynchronous
API. If share delete operation is requested immediately after call the
former API, it fails because the "abort_volume_move" is still in progress.
Now NetApp cDOT driver checks, for a period of time, if the
``volume-move-abort`` operation has ended before report migration
cancelation success.
Change-Id: I76e11fef27c9723f019cfdfdc6ea86878db78776
Closes-Bug: #1688620
This patch fixes the error raised by NetApp driver when the user try
to shrink a share to a size smaller than the current used space. Now
the driver raises "shrinking_possible_data_loss" instead of
"shrinking_error".
Change-Id: I01a549a0e299ae0d0d9b603a9d0797f16f4b68b3
Closes-Bug: #1858328
This is to fix the following assert statements:
* asser_called_once_with, which should be assert_called_once_with
* asser_called_once, which should be assert_called_once
Change-Id: I0025b06d6b10249b118a46e67d8ca6ba56d4befd
This patch improves the operation of creating share from snapshot
to accept new destinations that can be different pools or
back ends.
Change-Id: Id3b3d5860d6325f368cbebfe7f97c98d64554d72
Fix:
E241 multiple spaces after ':' (or ',')
Fix also a few other problems found by hacking in files changed.
Change-Id: I4c00551f29ad65df2631c8ae55fe2f022e872544
NetApp cDOT driver is now fixed to remove the QoS Policy on the
backend volume when a share is migrated from an extra-spec which had
QoS defined to another extra-spec which has no QoS defined in it.
Change-Id: I57c2e15fc22eb2bf69ebee25a92ba10746505e73
Closes-Bug: #1706137
The NetApp cDOT driver is now fixed to unmount the original
active share volume after one of its replica gets promoted.
Change-Id: Iecb734b431f0b7bc9a905b0a672013869ff60fd8
Closes-Bug: #1634278
Added check to validate SnapRestore license for
NetApp ONTAP pools before setting revert_to_snapshot_support
to true/false for the respective pool.
This was always set to True before.
Change-Id: I4750d28c8acfe6d68a91e129e767c6e8607e6a75
Closes-Bug: #1678524
This patch implements the functionality of managing and
unmanaging of share servers to the NetApp driver,
allowing for shares and snapshots to be managed
and unmanaged in DHSS=True driver mode.
Implements: bp netapp-driver-manage-unmanage-with-share-servers
Change-Id: I6051cf038dcf9f175e0610fff0adf360230dc23c
Depends-On: I452c2a99b186f53d737cb7fbd7eabfcfd9b249d6
By default, every share created allows access of its .snapshot
where files of each taken snapshot can be accessed. As per
some use-cases, it is desirable to not allow access to
the .snapshot folder.
This can now be done by using the extra_spec netapp:hide_snapdir.
When set to True, it will hide the .snapshot directory for every
newly created share.
Also, for existing shares, a config option named
netapp_reset_snapdir_visibility can be used to reset
all existing shares' setting to either hide or display
the .snapshot visibility on driver restarts.
Implements blueprint: netapp-snapdir-visibility
Change-Id: I30619bb13de528538b9887b00f39482f91a8db49
NVE is a software-based technology for encrypting
data at rest one volume at a time. An encryption
key, accessible only to the storage system,
ensures that volume data cannot be read if the
underlying device is repurposed, returned,
misplaced or stolen.
Signed-off-by: Erlon R. Cruz <erlon@netapp.com>
Signed-off-by: Tiago Pasqualini <tiagod@netapp.com>
Change-Id: Ib622c3d64cbec5a7254f6074f07d6f56f6492ca4
Implements: blueprint netapp-encrypted-shares
The ONTAP driver was ignoring the size requested when
creating a share from an existing snapshot.
Closes-Bug: #1717263
Change-Id: I335f72a8dea49d5855201fc63a7ac22240b60bbf
In course of normal operation, the backend snapshot
name is associated with the the ID of the snapshot instance
in manila.
However, when a share is replicated or is being migrated,
the backend snapshot is not renamed. Instead, we simply
store the original name in the "provider_location" field of
the snapshot model.
Fix revert-to-snapshot methods to identify snapshots by
"provider_location"s rather than the instance IDs.
Co-Authored-By: Goutham Pacha Ravi <gouthampravi@gmail.com>
Change-Id: I4900c7aecc3da6640ea9c0d4d08012ff5b68cc58
Closes-Bug: #1694768
jayaanand.borra@netapp.com