This patch enables Active/Active support in NetApp NFS driver.
To support replication in A/A mode, the method failover_host was
splitted in two phases (failover and failover_completed) as
required by the spec [1].
[1] https://specs.openstack.org/openstack/cinder-specs/specs/ocata/ha-aa-replication.html
Change-Id: I401ca89440d44e04f460741981cf24a42e5264a0
The NetApp NFS driver is spawning a Python native thread to
schedule image cache cleanups, which can lead to unpredictable
behavior such as thread starvation due to its hunger to preempt
other Cinder's fellow threads.
This patch fixes it by reimplementing the NFS image cache cleanup
task as a green thread with a customized periodic interval.
The patch adds the configuration
option ´netapp_nfs_image_cache_cleanup_interval´ to allow
operators to set an interval in seconds between calls to the image
cleanup task.
Closes-Bug: #2008017
Change-Id: I609a4f056d19f67aa7b8113bb38b9e9083f24124
The copy offload tool is no longer available available, so this patch adds
an alternative approach using file copy operations to the scenarios where
the tool was required. During the clone image operation, if the source
volume is not available in the same pool or cache, the copy file operation
is called to perform the action.
The copy offload tool will keeping working and it will be removed in the
next release (Antelope). There patch [1] deprecates the tool.
[1] https://review.opendev.org/c/openstack/cinder/+/847733
Implements: blueprint netapp-nfs-copy-offload-image
Change-Id: I4e2163661b913800a5a1179a5899ec1384f8c167
This patch contains the implementation of the core functions of the
NetApp drivers (NFS, iSCSI and FCP). The functions were migrated
from ZAPI to REST API, but ZAPI client was not removed since it is
being used as a fallback mechamism when a function does not have an
equivalent in the REST API.
In summary, the features implemented in this patch are related to:
> Periodic tasks - methods used during driver initialization and
executed periodically to get information about the volumes,
performance metrics and stats
> Basic volume operations - methods used to create, delete,
attach, detach and extend volumes
Co-authored-by: Fábio Oliveira <fabioaurelio1269@gmail.com>
Co-authored-by: Fernando Ferraz <sfernand@netapp.com>
Co-authored-by: Luisa Amaral <luisaa@netapp.com>
Co-authored-by: Matheus Andrade <matheus.handrade15@gmail.com>
Co-authored-by: Vinícius Angiolucci Reis <angiolucci@gmail.com>
Change-Id: I67eb7f6264cf5eea94c0a364082b530b9cdc1ae3
partially-implements: blueprint netapp-ontap-rest-api-client
This patch adds option ´netapp_driver_reports_provisioned_capacity´,
for driver to calculate the storage provisioned capacity by queryng
volume sizes directly from the storage system and reporting as a
pool capability ´provisioned_capacity_gb´, instead of relying
on the default behavior of using scheduler and volume service
internal states as done by ´allocated_capacity_gb´.
Implements: blueprint ontap-report-provisioned-capacity
Change-Id: I97625de865a63b0cf725f61d9d2ea3c44740d9e7
This patch adds support to revert to snapshot for NFS, FC and
iSCSI drivers with FlexVol pool.
Adds a method on client_cmode to rename a NFS volume file using
the file-rename-file zapi call.
The revert steps are:
1. Create a clone volume file/LUN from snapshot.
2. Change the original volume file/LUN path to a temporary path.
3. Change clone file/LUN path to the original path.
4. Delete the volume file/LUN on temporary path.
If any step fails, the original volume file/LUN is preserved and
the clone file/LUN is deleted.
For NFS, ONTAP clone file is not supported on FlexGroup pool, in
this case the generic implementation will perform the revert to
snapshot.
Implements: blueprint ontap-revert-to-snapshot
Co-Authored-By: Fabio Oliveira <fabioaurelio1269@gmail.com>
Change-Id: I347f0ee63d8d13ff181dd41a542a006b7c10b488
This patch adds support for storage assisted migration
(intra-cluster) to NetApp ONTAP drivers (iSCSI/FC/NFS), for the
following use cases:
1) Between pools in a same vserver and backend/stanza. This
operation is non-disruptive on iSCSI and FC drivers.
2) Between pools in a same vserver but in a different
backend/stanza. This operation is disruptive in all cases
and requires the volume to be in `available` status.
3) Between pools in a different vserver in a different
backend/stanza. This operation is disruptive in all cases
and requires the volume to be in `available` status.
Storage assisted migration is only supported within the same
ONTAP cluster. If a migration between two different clusters is
requested, driver will automatically fallback to host assisted
migration.
Implements: blueprint ontap-storage-assisted-migration
Change-Id: Iaad87c80ae37b6c0fc5f788dc56f1f72c0ca07fa
It is a follow up to the patch [1], providing the fixes:
1. Check the ONTAP storage version for the FlexGroup feature.
The feature is only accepted whether the ONTAP is 9.8 or greater.
2. Remove multiattach support for flexgroup pools.
3. Start bumping the driver version with its documentation.
4. Update the flexgroup release notes.
[1] https://review.opendev.org/c/openstack/cinder/+/776713
Change-Id: I0636b6530a79f8ca86db52f87273ec822c3cb406
Adds the support for FlexGroup pool using the NFS storage mode.
The FlexGroup pool has a different view of aggregate capabilites,
changing them by a list of elements, instead of an element. They
are: `netapp_aggregate`, `netapp_raid_type`, `netapp_disk_type`
and `netapp_hybrid_aggregate`. The `netapp_aggregate_used_percent`
capability is an average of used percent of all FlexGroup's
aggregates.
The `utilization` capability is not calculated to FlexGroup pools,
it is always set to default value.
The driver cannot support consistency group with volumens that are
over FlexGroup pools.
ONTAP does not support FlexClone for file inside a FlexGroup pool,
so the operations of clone volume, create snapshot and create volume
from an image are implemented as the NFS generic driver.
The driver with FlexGroup pools has the snapshot support disabled, requiring
that setting the `nfs_snapshot_supprot` to true on the backend definition.
This config is the same as the NFS generic driver.
The driver image cache relies on FlexClone for file, so it is not applied
for volumes over FlexGroup pools. It can use the core cache image, though.
The QoS minimum is only enabled for FlexGroup pool if all nodes of the
FlexGroup support it.
Implements: blueprint netapp-flexgroup-support
Change-Id: I507083c3e34e5a5cf1db9a3d1f6bef47bd51a9f8
NetApp ONTAP 9.4 or newer supports Adaptive QoS, which scales the
throughput according to the volume size.
In Victoria release, ONTAP Cinder driver added support for assigning
pre-existing Adaptive QoS policy groups to volumes.
This patch allows the dynamic creation of Adpative QoS, by reading new
back-end QoS specs.
Implements: blueprint netapp-ontap-dynamic-adaptive-qos
Change-Id: Ie35373c7b205ffa12c4bb710255f1baf7a836d9f
Currently, the ONTAP Cinder driver only supports the max (ceiling)
throughput QoS specs.
This patch adds support for min (floor) throughput QoS policy specs
``minIOPS`` and ``minIOPSperGiB``, which can be set individually or
along with the max throughtput specs.
Added a new driver specific capability called `netapp_qos_min_support`.
It is used to filter the pools that has support to the Qos minimum
(floor) specs during the scheduler phase.
The feature is supported by ONTAP AFF with version equal or greater
than 9.2 for iSCSI/FCP and 9.3 for NFS, ONTAP Select Premium with
SSD and ONTAP C190 with version equal or greater than 9.6.
Implements: blueprint netapp-ontap-min-throughput-qos
Implements: blueprint netapp-ontap-min-throughput-qos-capability
Co-Authored-By: Felipe Rodrigues <felipen@netapp.com>
Change-Id: Ic6579d459670fec4e5295e51c12fd807d980bb81
md5 is not an approved algorithm in FIPS mode, and trying to
instantiate a hashlib.md5() will fail when the system is running in
FIPS mode.
md5 is allowed when in a non-security context. There is a plan to
add a keyword parameter (usedforsecurity) to hashlib.md5() to annotate
whether or not the instance is being used in a security context.
In the case where it is not, the instantiation of md5 will be allowed.
See https://bugs.python.org/issue9216 for more details.
Some downstream python versions already support this parameter. To
support these versions, a new encapsulation of md5() has been added to
oslo_utils. See https://review.opendev.org/#/c/750031/
This patch is to replace the instances of hashlib.md5() with this new
encapsulation, adding an annotation indicating whether the usage is
a security context or not.
Reviewers need to pay particular attention as to whether the keyword
parameter (usedforsecurity) is set correctly. Almost all instances
of md5 usage appear to be to refer to etags, to do checksums, or to
generate uuids for paths.
I had hoped to update the bandit config to enable scanning for instances
of md5 and bad algorithms, so that instances would not creep in in future,
but I couldn't find the bandit config.
With this patch (and the corresponding os-brick and oslo-versioned_object
dependent changes) all the functional tests and alnmost all the unit tests
pass on a FIPS enabled system.
Issues I found were as follows:
- Cinder appears to be using md5 in a security context in
cinder/volume/drivers/synology/synology_common.py. If this is really
the case, then we'll need to consider how to replace md5 in this usage.
This case did not appear to exercised in the unit or functional tests I ran.
- Cinder appears to use md5 in a security context in
cinder/volume/drivers/stx/client.py, which resulted in the failed unit test
cinder.tests.unit.volume.drivers.test_seagate.TestSeagateClient.test_login
This was the only unit test that failed.
Change-Id: I57ec3e7e99c78535fa8051d011d970adb7fb89ab
Depends-On: https://review.opendev.org/#/c/756151
Added support for Adaptive QoS policies that have been pre-created on
the storage system, with the NetApp driver and clustered ONTAP version
9.4 or higher. To use this feature, configure a Cinder volume type with
the following extra-specs::
netapp:qos_policy_group=<name_of_precreated_aqos_policy>
netapp:qos_policy_group_is_adaptive="<is> True"
Note that a cluster scoped account must be used in the driver
configuration in order to use QoS in clustered ONTAP.
Partially-implements: bp netapp-adaptive-qos-support
Co-Authored-By: Lucio Seki <lucioseki@gmail.com>
Change-Id: Idcdbd042bb10f7381048fe4cb9fb870c3eebb6ce
test.py was previously not in the tests
directory. This means that downstream packagers
of Cinder have to specifically exclude it from
the main Cinder package (which does not typically
include unit tests).
Move it under the cinder/tests/unit/ dir where it
should be, to clean this up.
Change-Id: I65c50722f5990f540d84fa361b997302bbc935c5
This adds usage of the flake8-import-order extension to our flake8
checks to enforce consistency on our import ordering to follow the
overall OpenStack code guidelines.
Since we have now dropped Python 2, this also cleans up a few cases for
things that were third party libs but became part of the standard
library such as mock, which is now a standard part of unittest.
Some questions, in order of importance:
Q: Are you insane?
A: Potentially.
Q: Why should we touch all of these files?
A: This adds consistency to our imports. The extension makes sure that
all imports follow our published guidelines of having imports ordered
by standard lib, third party, and local. This will be a one time
churn, then we can ensure consistency over time.
Q: Why bother. this doesn't really matter?
A: I agree - but...
We have the issue that we have less people actively involved and less
time to perform thorough code reviews. This will make it objective and
automated to catch these kinds of issues.
But part of this, even though it maybe seems a little annoying, is for
making it easier for contributors. Right now, we may or may not notice
if something is following the guidelines or not. And we may or may not
comment in a review to ask for a contributor to make adjustments to
follow the guidelines.
But then further along into the review process, someone decides to be
thorough, and after the contributor feels like they've had to deal with
other change requests and things are in really good shape, they get a -1
on something mostly meaningless as far as the functionality of their
code. It can be a frustrating and disheartening thing.
I believe this actually helps avoid that by making it an objective thing
that they find out right away up front - either the code is following
the guidelines and everything is happy, or it's not and running local
jobs or the pep8 CI job will let them know right away and they can fix
it. No guessing on whether or not someone is going to take a stand on
following the guidelines or not.
This will also make it easier on the code reviewers. The more we can
automate, the more time we can spend in code reviews making sure the
logic of the change is correct and less time looking at trivial coding
and style things.
Q: Should we use our hacking extensions for this?
A: Hacking has had to keep back linter requirements for a long time now.
Current versions of the linters actually don't work with the way
we've been hooking into them for our hacking checks. We will likely
need to do away with those at some point so we can move on to the
current linter releases. This will help ensure we have something in
place when that time comes to make sure some checks are automated.
Q: Didn't you spend more time on this than the benefit we'll get from
it?
A: Yeah, probably.
Change-Id: Ic13ba238a4a45c6219f4de131cfe0366219d722f
Signed-off-by: Sean McGinnis <sean.mcginnis@gmail.com>
Much of our code renames this at import already --
just name it "volume_utils" for consistency, and
to make code that imports other modules named "utils"
less confusing.
Change-Id: I3cdf445ac9ab89b3b4c221ed2723835e09d48a53
Move code into more specific locations where it is
applicable rather than the Cinder-wide utils.py.
Change-Id: I87f80f041cec255d51145e39bc0c0781a81e6db8
This patch moves the netapp exception to the netapp/utils.py
for common use among all the netapp drivers.
Change-Id: I4bed2917bdb7d9929c0fbfa2dcfe4067cb3d963f
Move touch to privsep and remove it from volume.filters.
Change-Id: I169b45f1176537448871cf23b6362c7ffbb16fff
Signed-off-by: Chuck Short <chucks@redhat.com>
Remove the resolve_hostname function since
the resolve_hostname in cinder.utils does the exact same thing
so we can reuse it.
Change-Id: I3fcdaf8c9d91845bada9d2bff8967b04ab700992
Signed-off-by: Chuck Short <chucks@redhat.com>
Signed-off-by: zhu.boxiang <zhu.boxiang@99cloud.net>
NetApp ONTAP driver currently have some issues with IPv6:
- The URL is not properly formatted when using in management path
- NFS driver breaks when handling IPv6 addresses
- iSCSI driver creates a URL that is not properly formatted when
using IPv6 addresses
This patch fixes all issues related to IPv6 on NetApp ONTAP drivers.
Closes-bug: 1788419
Closes-bug: 1788460
Change-Id: I6eeca47997c7134d6604874bea48eab7cab6c1a2
Cinder scheduler now checks backend capability online_extend_support
before performing an online volume extend operation. This patch makes
NetApp ONTAP NFS driver report to the scheduler that it doesn't support
this feature, thus avoiding leaving a volume in error_extending state
after an online extending attempt.
Change-Id: Iafe3e68d6e9c20064a8825848b7b2d33ce7f4cff
Depends-On: I2c31b5c171574074a8fc7ba86f94f983fc9658f7
Related-Bug: #1765182
API Tracing is valuable when diagnosing problems or
unexpected behaviors with the ONTAP Cinder drivers.
However, turning it on may spam logs and make it rather
harder to trace through specific API calls.
Added an API trace pattern filter in order to filter out
undesired API calls from the DEBUG log.
Change-Id: Ic0563848205a941cf8e779eee42e24ecdaf847dd
On ONTAP NFS driver, the export path was being used as the volume
name. If the export path is different than the volume name, the
API call to delete files would fail and the driver would invoke
a fallback method that deletes the files manually. This patch fixes
that by finding the correct volume name.
Change-Id: Ice78889573a36ff5e8873a0d316ddcf180d0263f
Closes-bug: #1690954
After the addition of the auto-max-over-subscription on Cinder, drivers that
somehow used the CONF.max_over_subscription_ratio in the driver where marked as
not supporting the feature. As Netapp Ontap drivers used the option to do some
calculations, they were marked as so.
In this patch we remove the function that did this calculation as the function
(_share_has_space_for_clone() is not actually necessary as currectly all volume
clones are checked by the scheduler.)
Change-Id: I25713b402ad93e0b4aa1861b4ccd9e05ee496200
Depends-on: If30bb6276f58532c0f78ac268544a8804008770e
Improves NetApp cDOT block and file drivers suport for SVM scoped user
accounts. Features not supported for SVM scoped users include QoS,
aggregate usage reporting, and dedupe usage reporting.
Change-Id: I2b42622dbbb0f9f9f3eb9081cf67fb27e6c1a218
Closes-Bug: #1694579
When 'NetApp NFS Copy Offload' tool is configured to download Glance
images, these images are downloaded twice because the tool is doing
the job after Cinder has already done it.
This patch fixes the bug by executing the copy offload tool inside the
clone_image function instead of using the copy_image_to_volume.
Closes-bug: #1632333
Change-Id: I5c6ad150543213acfd0c78dbbdb1dc1584d22d26
The Unified Driver for NetApp Storage in Cinder
supports two families of ONTAP, 7mode and Clustered
Data ONTAP.
ONTAP 7 is now officially nearing the end-of-life
of product support. The deprecation notice [1]
for these drivers in Cinder was issued in the Newton
Release, so it is time to remove them from tree.
[1] http://lists.openstack.org/pipermail/openstack-operators/2016-November/011957.html
Implements: bp remove-netapp-7mode-drivers
Change-Id: I129ca060a89275ffd56481b8f64367b0d803cff5
The ONTAP drivers in Cinder ("7mode" and "cmode") cannot
reliably and efficiently track provisioned_capacity_gb as expected
by the Cinder scheduler.
The driver authors originally assumed that provisioned_capacity_gb
is consumed space on the backend. This results in miscalculation of
over subscription in the Cinder scheduler.
The fix adopted here is to remove this wrong reporting and rely on
calculation of the provisioned_capacity_gb in the scheduler.
Change-Id: Ic106dbcae8ceaac265b710756ab1874e445ca826
Closes-Bug: #1714209
Currently two repeating tasks are initialized, during driver
initialization, to handle deletion of unneeded QoS policies on the
NetApp backend. This change removes one of the tasks.
Change-Id: Ibb9c28937f9e1912cf3293fcf4ca83c5d76d7f75
Closes-Bug: #1713774
Adding support for generic volume groups to NetApp cDOT's NFS driver.
CG methods are moved to the 7-mode NFS driver. Announcement was made
in a previous release for the removal of the NetApp 7-mode drivers.
Generic groups will only be implemented for the NetApp cDOT drivers.
Change-Id: I5edf62ccfbbb7a5f3e64e0baebe9eb4a955cbdf5
Implements: blueprint netapp-add-generic-group-support-cdot
failover_host is the interface for Cheesecake.
Currently it passes volumes to the failover_host
interface in the driver. If a backend supports both
Cheesecase and Tiramisu, it makes sense for the driver
to failover a group instead of individual volumes if a
volume is in a replication group. So this patch passes
groups to the failover_host interface in the driver in
addition to volumes so driver can decide whether to
failover a replication group.
Change-Id: I9842eec1a50ffe65a9490e2ac0c00b468f18b30a
Partially-Implements: blueprint replication-cg
After image cloning the NFS client cache needs to be refreshed.
This can be accomplished by touching the directory hosting the
cached image file.
See also: https://bugs.launchpad.net/nova/+bug/1617299
Co-Authored-By: Sebastian Schee <sebastian.schee@sap.com>
Co-Authored-By: Goutham Pacha Ravi <gouthampravi@gmail.com>
Closes-bug: #1679716
Change-Id: If392f41f65978721668b53cfab94393f074d24e9
This patch disables setting multiattach flag to True in driver capabilities.
Since we reworked the driver attach/detach API, we need to ensure that the
issue of detaching a shared volume works properly on the Nova side. A shared
flag might need to be required to return in initialize_connection to support
multiattach, so for now we will disable the capability.
Change-Id: I4f8229a4465d009e06de86da255f442020151113
The copyoffload tool is optional and not always in place. This fixes a
misleading error message during image clone creation when the tool is
not in place.
Change-Id: I66f28faaac629fbfa7ec57fa5238451eb7c6820a
Closes-bug: #1660927
mock_object method in TestCase class now accepts keyword arguments that
will be properly handled when no new object is provided, so we can now
simplify calls in our tests.
This patch changes calls like
self.mock_object(os.path, 'exists', mock.Mock(return_value=True))
into
self.mock_object(os.path, 'exists', return_value=True)
Change-Id: I904ab4956a081c0292e2c982320e5005b055d3d5
The NetApp DOT drivers log some info about OpenStack deployments
already, and more info is needed about the specific storage
resources managed by Cinder.
Implements: blueprint netapp-dot-enhanced-support-logging
Change-Id: Id8937a181ee7ab1b985b6aa4143f68026b02656d
The maximum amount of shared (deduplicated, cloned) data on
a Data ONTAP FlexVol (i.e. a Cinder pool) is 640TB. The only thing
more surprising about that number is that we have customers hitting
it. The symptom is that operations such as cloning Cinder volumes
fail because no more blocks may be shared. The fix is to report the
level of consumption to the scheduler for optional incorporation
into the filter & goodness functions, so that pools nearing the shared
block limit may be shielded from further provisioning requests.
Implements: blueprint netapp-cdot-report-shared-blocks-exhaustion
Change-Id: I01b7322f7ddb05ee5e28bcb1121a90a6ea307720
Host level replication capability was added to the NetApp
cDOT Block and File drivers through
I87b92e76d0d5022e9be610b9e237b89417309c05.
However, the replication capability was not being reported
at the pool level.
Add the field at pool level allowing for compatibility with
volume types that may include the `replication_enabled`
extra-spec.
Change-Id: Ic735a527c609166884668c84c589da521769500b
Closes-Bug: #1615451
This fixes the issue of deleting temporary snapshots created during the
consistency group creation process. These temporary snapshots may not be
deleted if the system is under load and the temporary snapshot remains
in a "busy" state after the consistency group creation process is
otherwise complete.
This change also reduces lines of code by implementing a manager for the
creation of FixedIntervalLoopingCall instances in the ONTAP drivers.
This looping call manager also provides the ability to start all
registered looping calls after the driver has been properly
initialized. The looping call manager also makes it easy to ensure that
FixedIntervalLoopingCall instances are not instantiated in Unit Tests.
Closes-Bug: #1596679
Change-Id: I13096a8c94a32e68814f81900032dbcc6a4a9806
Nahim Alves de Souza