Conditionally Import Parakimo
Separate SSH functions into ssh_utils.py for safe conditional import.
Change-Id: Ia1a3ee69bef76b52e4e6df1e73488c018ac0f3c9
'reserved_share_extend_percentage' backend config option allows Manila
to consider different reservation percentage for share extend
operation. With this option, under existing limit of
'reserved_share_percentage', we do not want user to create new share if
limit is hit, but allow user to extend existing share.
DocImpact
Closes-Bug: #1961087
Change-Id: I000a7f530569ff80495b1df62a91981dc5865023
Python2 is no longer supported, so in this patch
set we remove the usage of the six (py2 and py3
compatibility library) in favor of py3 syntax.
Change-Id: I3ddfad568a1b578bee23a6d1a96de9551e336bb4
This config option allows different value for reservation percentage,
mostly useful on the platforms, where shares can only be created from
the snapshot on the host where snapshot was taken. The lower value of
this config option against existing (reserved_share_percentage) allows
to create shares from the snapshot on the same host up to a higher
threshold even though non-snapshot/regular share create fails.
In case this config option is not set, the shares created from snapshot
will use reservation percentage value set in 'reserved_share_percentage'.
This will be useful for users who want to keep same reservation percentage
for both non-snapshot/regular and snapshot shares.
DocImpact
Closes-Bug: #1938060
Change-Id: I390da933fe92875e3c7ee40709eacacc030278dc
Remove manila configuration options
and auth classes that were deprecated
before the Ussuri release.
Change-Id: I148225926cd249a0dd8d1f8c02b22ed06487f405
This patch enables the creation of a share from snapshot
specifying another pool or backend. In the scheduler, a
new filter and weigher were implemented in order to consider
this operation if the backend supports it. Also, a new
field called 'progress' was added in the share and share
instance. The 'progress' field indicates the status
of the operation create share from snapshot (in percentage).
Finally, a new periodic task was added in order to constantly
check the share status.
Partially-implements: bp create-share-from-snapshot-in-another-pool-or-backend
DOCImpact
Change-Id: Iab13a0961eb4a387a502246e5d4b79bc9046e04b
Co-authored-by: carloss <ces.eduardo98@gmail.com>
Co-authored-by: dviroel <viroel@gmail.com>
Fix:
W605 invalid escape sequence
This is the final change I plan for hacking, the remaining problems
need further investigation by manila team and decision whether and how
to solve them.
Change-Id: I73d73e044eaaf412bf7ace358a3f07c8d269d6cf
Pylint does not play very well with dynamic object
manipulation in python This creates a lot of
false-positives in the code-base which affects
contributors looking for genuine failures.
So, this change
- adds pylint ignore statements where appropriate
to disable testing these lines of code and failing.
- replaces all the pylint error codes (they are
hard to remember/relate to) with error names
which are easier to understand when reading the
code.
- initializes sqlalchemy model objects as dictionaries
which is a valid representation over None.
- removes ignore directives on six.moves which
is globally ignored in our pylintrc.
- adds alembic.op to the ignored
modules list since they are not supported by
pylint and have known issues.
This patch is the beginning of a series of
commits to use pylint in a sane way on manila code.
Change-Id: I44616821c5311d6f14986697efbbe5624de364a5
Some of the available checks are disabled by default, like:
[H106] Don't put vim configuration in source files
[H203] Use assertIs(Not)None to check for None
[H904] Use ',' instead of '%', String interpolation should be
delayed to be handled by the logging code, rather than
being done at the point of the logging call.
Change-Id: Ie985fcf78997a86d41e40eacbb4a5ace8592a348
Spectrum Scale provide NFS service with Ganesha server
by default (protocol node). So altering the default value
of NFS server for easiness.
Implements: blueprint gpfs-nfs-server-type-default-value-change
Change-Id: I2e3adeca58505c2da4a5675f7342bb2a174078cc
Due to lack of GPFS path ('/usr/lpp/mmfs/bin/') export, driver
is not able to find the GPFS commands causing the initialization
failure.
With this patch, GPFS will be able to run all GPFS commands
irrespective of GPFS path export.
Change-Id: I1aa8baf884dfaa42c37e575f0b6e1317f18543a1
Closes-bug: #1690785
Some configuration options were accepting both IP addresses
and hostnames. Since there was no specific OSLO opt type to
support this, we were using ``StrOpt``. The change [1] that
added support for ``HostAddressOpt`` type was merged in Ocata
and became available for use with oslo version 3.22.
This patch changes the opt type of configuration options to use
this more relevant opt type - HostAddressOpt.
[1] I77bdb64b7e6e56ce761d76696bc4448a9bd325eb
TrivialFix
Change-Id: I44ba478ff14a6184434dd030efd9b7fa92458c7a
Add update_access() implementation to replace the deprecated
allow_access() and deny_access().
Change-Id: I206f1284bddc02452087e24061619ca3c04395a4
Implements: blueprint gpfs-update-access
Added support for manage/unmanage in GPFS NFS driver.
This patch added functions that allow share on Spectrum Scale
node to be managed by OpenStack if existing fileset is an
independent fileset and doesn't have any NFS export
over the fileset path. Also, share can be unmanaged from
OpenStack but still left in Spectrum Scale cluster.
Implements: blueprint gpfs-manage-support
Change-Id: I9134408b59c30ac4bc593f287294741f6e996136
Use looser error code checking and add a verify
after deny access so that the call will succeed
whenever it can be confirmed that the access no
longer exists.
This fixes a variety of situations where previously
a return code of 1 while attempting to remove access
would cause the manila access rule to get stuck while
the actual client access export was removed or never
existed.
Change-Id: Ie058a6185e3f5d91fb1cf232301eb0ac6ddcea7e
Closes-Bug: #1651587
Fix GPFS KNFS allow/deny access so that it does not reuse
the ssh prefix when looping through NFS server commands for
allow and deny access.
This fixes GPFS KNFS allow/deny access incorrect behavior
when multiple NFS servers are configured (gpfs_nfs_server_list)
and any of the servers are remote (except the last one).
Change-Id: I8c182eed386e8325b087c3192d363502bb848633
Closes-Bug: #1651578
Several bugs are caused by an error code and
exception raised when a path has no NFS exports.
Use the machine-readable mmnfs export list command
(the -Y option) which does not cause an exception
when checking for exports.
Co-Authored-By: digvijay2016 <digvijay.ukirde@in.ibm.com>
Change-Id: I770756e0a36c5b61386878164b651fadf9730b7f
Closes-Bug: #1650043
Closes-Bug: #1650044
Closes-Bug: #1650045
Spectrum Scale supports NFS ganesha through CES protocol services. This
change replaces the current NFS ganesha support with the Spectrum Scale
CES infrastructure.
Implements: blueprint gpfs-ces-support
Change-Id: I8eb307c19761788ea71c15c889794f8b5f110253
Logging format arguments are not supposed to be tuples, but
are rather to be supplied separately - i.e., use
LOG.debug('%s is %s', arg1, arg2)
instead of
LOG.debug('%s is %s', (arg1, arg2))
Rid the manila codebase of this logging format error and
add a hacking check for this. The hacking check will not
catch all corner cases but should help prevent this error
in most cases.
Change-Id: Ibea39f1f90c0444eae5cfcd6090b2a00d84ab923
Closes-Bug: #1585394
Adding GPFS driver back as the third party CI system
for GPFS driver is up and running.
This is a revert of commit 720ec90445
Change-Id: I86f9b7da21716346f96bae3ce30446589e4c4c37
The Manila community set a deadline of 3 Sep 2015 for all drivers
to have reporting CI systems, and there is no CI system for the
IBM GPFS driver. As agreed, the driver will be removed until such
time that a CI system exists.
Change-Id: I76f238a1a6eb039e4ddc1cb189115ab3e686c56b
This change helps extend the size of the share by
changing the quota on GPFS fileset created for
that share.
Change-Id: I8076200b1350feadbb782ccd330e477c6edd52aa
Closes-Bug: #1481180
- Remove passing DB reference to drivers in __init__() method
- Remove db reference from Generic driver and service_instance
- Remove db reference from Netapp share driver
- Remove db reference from Glusterfs share driver
- Remove db reference from Glusterfs_Native share driver
- Remove db reference from Quobyte share driver
- Remove db reference from IBM GPFS driver
- Remove db reference from HDS_SOP driver
- Remove db reference from HDFSNative driver
- Remove db reference from fake driver
- Remove db reference from unit tests.
Change-Id: I74a636a8897caa6fc4af833c1568471fe1cb0987
- Use '%' for variable substitution in the message used for logging a
call and throwing an exception.
- Use log marker functions(_LI(), _LW(), _LE(), and _LC()) to send
message directly to the log.
- Use ',' for variable subsitution in the message in a log marker
function.
Change-Id: Ib925014e79d2c380954d952f5dbc835971a0b320
Closes-Bug: #1439762
Improving the error handling and error messages as follows:
- Check for exit status for the commands executed over SSH
- Following Log Translation guidelines
Change-Id: Ide6a37d9c00062ce4b025063da97f2e236c4f986
Closes-Bug: #1420739
Module 'log' from oslo-incubator was removed after release of oslo_log library.
So, start using oslo_log, but keep oslo-incubator code yet other common modules
within Manila codebase use it.
Implements bp use-oslo-log-lib
Change-Id: I88224f7c2bd99adb78140dfc3fa73cea437f29cd
We have explicit definition for supported share protocols.
Using "startswith" to search for substring is not a right way.
Change-Id: I8f482aea74249efca3bd490d971cf3e944919bb8
The oslo team is recommending everyone to switch to the
non-namespaced versions of libraries. Updating the hacking
rule to include a check to prevent oslo.* import from
creeping back in.
oslo.messaging is the only exception because this package doesn't
currently support non-namespaced imports.
Change-Id: I3987e651bc880c8ffa7c0105df0298679dcd3a43
Due to unclear meanings of existing names for share drivers modes it was
decided to replace string driver modes with boolean value, since we have only
two possible values that will clearly say what it used for by name of opt.
This replacement includes following changes:
- String opt 'share_driver_mode' is replaced with
bool opt 'driver_handles_share_servers'. New bool opt does not have
default value and should be defined explicitly.
- Class ShareDriver (parent class for share drivers) now expects additional
argument 'driver_handles_share_servers' which is mandatory and should be
provided by child classes. Expected values are boolean or tuple/list/set of
booleans that says what modes are supported. Above new config opt will be
compared to these.
- Update interfaces 'setup_server' and 'teardown_server' of class ShareDriver.
These interfaces now consider driver modes and call new additional private
methods only when driver is enabled to mode with share servers handling.
These new private methods are '_setup_server' and '_teardown_server', they
should be redefined by child classes when share servers should be handled by
Manila.
- To get know current driver mode within child classes just call property
'driver_handles_share_servers'. It can not be changed by child classes and
returns value that is set by config opt with same name.
- Remove methods 'setup_server' and 'teardown_server' from all share drivers,
that do not support handling of share servers.
- Rename methods 'setup_server' and 'teardown_server' to appropriate
private methods for drivers that do support handling of share servers.
- Update unit tests related to all changed places.
- Make Devstack set new mandatory bool opt.
Implements bp rename-driver-modes
Change-Id: I33402959bc4bbc34cfd4d7308ad0a7bcff3669b5
Driver mode functionality was implemented to be able to specify how
driver should work and filter backends scheduling share creation based on this.
Add to all drivers update of attr 'mode' based on its current behavior.
Set 'share_driver_mode' extra spec to volume/share type with one of available
values. Scheduler will use it for host filtering.
Implements blueprint driver-modes-for-scheduler
Change-Id: Ida644f630ee07c51c02aea5d6280980b5d704c2f
Method 'get_share_stats' of share drivers uses private method called
'_update_share_status'. But it should use 'stats' instead of 'status'.
Some drivers have redefined it. So, rename it for base driver class and
remove duplicated redefinitions of 'get_share_stats' in share drivers.
Change-Id: Ib9a1184256380268376b1da426526f8394c2308a
Introduce the ganesha share driver helper module
which provides the GaneshaNASHelper class from which
share drivers can derive NFS-Ganesha backed protocol
helpers.
Some utility functions are also added to ease
integration.
Partially implements blueprint gateway-mediated-with-ganesha
Change-Id: I8683ea5eb43d7a8eaf0dfa6af3791782d32b944a
Several things are implemented:
- Allocation/deallocation now handled by drivers instead of share manager.
It provides flexibility for drivers.
- Network plugin interface was updated to support new approach for
configuration options setting.
Config opts for network plugin can be defined via three sources:
a) using separate config group
b) using config group of back end
c) using DEFAULT config group
Variants (a) and (b) are mutually exclusive, there are switched by
opt 'network_config_group' that belongs to share driver interface.
Implements bp network-helper
Change-Id: I3b05369f01777675c1b834af5ee076d8b7219a0f
Supporing GPFS exports over kernel NFS and Ganesha NFS
Supporting following Manila functionality:
1. Create, List, Delete Shares
2. Create, List, Delete Share Snapshots
3. Create Share from a Share Snapshot
4. Allow, Deny access to a share based on IP
Supports, local and remote GPFS nodes to the Manila service node,
in a GPFS cluster
Limitation:
1. While using remote GPFS node, with Ganesha NFS, 'gpfs_private_key'
for remote login to the GPFS node must be specified and there must be
a passwordless authentication already setup between the Manila and the
remote GPFS node.
DocImpact
Implements: blueprint gpfs-driver
Change-Id: I6664054ba52d03814cea846cb0d79cd853632814
ashrod98