This method of installing manila-tempest-plugin
was deprecated several releases ago, but compatibility
was maintained for third party CI jobs. Since sufficient
time has passed allowing third party CI to adopt,
lets drop this from the devstack plugin
Change-Id: I59ffe6ebe4b2e80be1e6ae97872cf4db6527da62
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
MANILA_MULTI_BACKEND has been deprecated for five years now, we should remove
it from our code base.
This variable was removed from the settings scripts along with:
MANILA_BACKEND1_CONFIG_GROUP_NAME and MANILA_SHARE_BACKEND1_NAME;
MANILA_BACKEND2_CONFIG_GROUP_NAME and MANILA_SHARE_BACKEND2_NAME.
Because they work in the same context.
Instead of them, the already implemented and in use,
MANILA_ENABLED_BACKENDS variable was placed to garantee the successful
back-end setup. The same replacement was made in the contribution
samples scripts.
Apart from this, we avoid configuring generic1 and generic2 if
another backend/s are selected.
Closes-Bug: #1898791
Closes-Bug: #1878477
Change-Id: I67036a65da9255694a00a9c8d56cfdefbdf23c05
This patch adds support for migration of share servers. This
migration is performed using a two-phase approach. Administrators
are now able to request the migration of a share server within and
across backends, with the possibility of chooosing a different share
network for the destination share server.
- A new field called `task_state` was added to the share server
model in order to help the administrator to track the share server
migration steps. A new field called `source_share_server_id` was
added to link destination and source share servers.
- A new periodic task was added to track migration of share servers
and its resources.
- Two new states were added: `server_migrating` and
`server_migrating_to` to represent that share migration is in
progress.
- When performing the server migration, manila will not go to the
scheduler, instead it will provide a request spec to drivers
during migration check driver call. It'll be up to the driver
validate if there is free space to handle the share server.
- A new API called `share-server-migration-check' was added to
check the feasibility of a migration, before actually triggering
the start operation.
APIImpact
DocImpact
Partially Implements: bp share-server-migration
Co-Authored-By: Andre Beltrami <debeltrami@gmail.com>
Co-Authored-By: Carlos Eduardo <ces.eduardo98@gmail.com>
Co-Authored-By: Felipe Rodrigues <felipefuty01@gmail.com>
Change-Id: Ic0751027d2c3f1ef7ab0f7836baff3070a230cfd
Signed-off-by: Douglas Viroel <viroel@gmail.com>
Replace "legacy" style jobs with native zuulv3
ones built within the manila-tempest-plugin
repository. This refactoring is driven by a
community wide goal to stop relying on devstack-gate,
a deprecated component of the OpenDev CI system [1]
Remove in-tree devstack gate hook script portions
that are no longer necessary; and mark the rest as
deprecated, since third party CI could still be
relying on these hooks.
Replace "manila-tempest-dsvm-mysql-generic",
"manila-tempest-dsvm-postgres-generic-singlebackend",
"manila-tempest-dsvm-generic-no-share-servers" and
"manila-tempest-dsvm-scenario" jobs with
"manila-tempest-plugin-generic" which preserves
coverage for the generic driver, while removing
unnecessary duplication/separation. See commit
message in [2] for more details.
Remove older CentOS and HDFS jobs since they
were not being executed.
Remove "custom" image jobs targeting custom
manila-image-element based builds, the jobs
now originate in the manila-image-elements
repository. [3]
The dummy driver job has been split into two parts
for testing DHSS=True and DHSS=False in separate
jobs. Read more about the motivation in the commit
message for the change that created that job [4]
While some name tags such as "postgres" and
"ipv6-only" have been removed from the job
names, the functionality of the job hasn't
been changed.
This change will not be backported. Instead, new
changes akin to this may directly target older
branches of manila.
[1] https://governance.openstack.org/tc/goals/selected/victoria/native-zuulv3-jobs.html
[2] https://review.opendev.org/726291/
[3] d9d2f16360/zuul.d/manila-image-elements-jobs.yaml (L34-L67)
[4] https://review.opendev.org/740534/
Change-Id: I2be6fc12c787c05c7df68297990fcdca96f57c5d
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
Switch to using the LVM driver in the grenade
job that allows us to add a minor data
path test to prove that upgrading manila
has no impact on data path connectivity
to resources created by manila.
Change-Id: I8588e8f988d85dc64e19e7a44a25c3dd0b776892
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
To set up some first party backends such as
ZFSOnLinux, CephFS via NFS gateway, Container
(where the NAS server is containerized) and LVM,
manila's devstack plugin creates a NAS server
on the devstack host.
On test machines, access to this NAS server is
firewalled from networks outside of the host's
internal network namespace (including from private
project networks that are in different network
namespaces, on the same devstack host).
We currently use a legacy devstack-gate script
to disable firewall on NFS ports; however,
anyone that installs devstack with LVM, Container,
ZFSOnLinux, CephFS-NFS drivers will need these
firewall ports to be opened to be able to mount
shares exported off their devstack host machines.
Move these firewall commands to the devstack plugin.
These commands can be invoked by setting the localrc
variable MANILA_ALLOW_NAS_SERVER_PORTS_ON_HOST to True.
The value of this variable is False by default,
to preserve existing behavior.
Change-Id: Ic9cad47662f1edf2e5c710dbe64d580bc5f01d44
This patch improves the operation of create share from snapshot
to support different backends/hosts for the zfsonlinux driver
Partially-implements: bp create-share-from-snapshot-in-another-pool-or-backend
Depends-On: Iab13a0961eb4a387a502246e5d4b79bc9046e04b
Change-Id: I124803734c81d3630c5147f5f3bb75724489c929
This patch enables the replication test to run on Dummy driver
when configured with DHSS True. This change depends on subnets
implementation to properly work.
Change-Id: Ib6a44d8f0d8c56acd149b8d80549079f771ffbe0
Depends-On: Icb93fc346b71edc7bb3ac989e9751899bb80fbe5
manila-tempest-plugin can be installed with its
devstack plugin; Installing it via manila's plugin
is unnecessary. So, deprecate its installation
in the DevStack plugin.
Change-Id: I21c08069ff82b3bfb52ef7ac960183ddc866c2ee
Restoring the default IPv6 route is not needed in CI and
fails when there are multiple defaults, but it is useful
in local devstacks where multiple default routes are not
typical.
Add a variable in settings and use it to make this behavior
conditional and set it to False for the lvm job.
Closes-bug: #1836788
Change-Id: Id73de8100509ec5935641f5f35f93f482d108bcd
The pre test hook for gate tests has a faulty test
for whether the PYTHON3_VERSION variable is set and
as a result puts a blank value for this variable in
local.conf when it is not set instead of leaving it
out of local.conf altogether. This causes jobs
running with USE_PYTHON3=True and no explict version
setting to actually run manila under python 2.7 because
that is what devstack itself defaults for pip installs
if the version is not set (or auto-detected in devstack
when local.conf does not contain the PYTHON3_VERSION
variable at all).
So correct the condition in our pre-test hook so that
it only inserts a PYTHON3_VERSION in local.conf if
that version has been supplied.
Closes-bug: #1822178
Change-Id: Ie37595bde17d18770d82fb7fd6475dcfab74a543
Switch manila to use devstack common functions
for deploying a wsgi app under uwsgi and apache.
This change aims to fulfill the community requirement
for all projects on OpenStack to switch devstack jobs
to deploy control-plane API services under uwsgi with
Apache acting as a front end proxy.
More details in https://governance.openstack.org/tc/goals/pike/deploy-api-in-wsgi.html
We used to deploy with mod_wsgi, but mod_wsgi has many
issues when being used in devstack for development or testing.
Now we leave mod_wsgi as optional and we default to uwsgi.
Implements: bp wsgi-web-servers-support
Depends-On: https://review.openstack.org/#/c/643188/
Change-Id: I12d4c864f2ceb0f3555e32d2dc893e00dbef0d96
And use a 'file://' url for the custom manila image.
See also https://review.openstack.org/#/c/623330. These
are interdependent patches but no harm is done by merging
this one first and having 623330 depend on it since retrieval
of the image name is not working correctly anyways.
Partial-Bug: #1807969
Change-Id: Ib3a74d170f159dfbc38b5302f00800f3530f4921
Make default share type and group type
creation optional in
manila-tempest-minimal-dsvm-dummy job.
Depends-On: https://review.openstack.org/#/c/540070/
Change-Id: I585c15d9eb4eefb843df62f7aad08838c83611e2
'standalone_network_plugin_ip_version' option has been
deprecated. This option has been replaced by
'network_plugin_ipv4_enabled' and
'network_plugin_ipv6_enabled' options.
Change-Id: I0251fecddac74578698bcda5d46e44ca16bd8a76
TEMPEST_SERVICES global variable is not supported by devstack since long back.
- I380dd20e5ed716a0bdf92aa02c3730359b8136e4
- I9c24705e494689f09a885eb0a640efd50db33fcf
Service availability of tempest known services will be set by devstack with
local check.
- I02be777bf93143d946ccbb8e9eff637bfd1928d4
This commit removes the unused TEMPEST_SERVICES setting
Change-Id: I2c6f3cd67094002c415ce9d1018c0f08d9a2a22f
Related-Bug: #1743688
This change complements the work of preceding IPv6 scenario
test changes [1,2] by enabling the settings that allow the tests
to be run in the Upstream CI.
[1] I4ca727f92618998242af18908bbbda6bb5f86303
[2] Id8b005cdd429d53a75624885fe7ca795746c3ede
Change-Id: I705b7d9cbc14d4bbc4d89b4c343205908758a5cb
Depends-on: Id8b005cdd429d53a75624885fe7ca795746c3ede
Depends-on: I4ca727f92618998242af18908bbbda6bb5f86303
This patch makes necessary changes to devstack manila
plugin in order to be able to run IPv6 scenario tests.
Part of the changes included the dependency of
neutron-dynamic-routing plugin and the installation
of quagga, so routes in devstack host can be created
automatically for each router and private network
created by tempest during the tests.
Also, added a new config option "override_ip_for_nfs_access"
for manila tempest plugin that overrides the access rules used
for NFS scenario tests. This option can be set by exporting the
environment variable "OVERRIDE_IP_FOR_NFS_ACCESS" with the
intended value before running devstack.
This change is enabled by a following change and
tested alongside a change on manila-tempest-plugin
project.
Please note that we are temporarily disabling IPv6
functionality in Host-assisted Share Migration, as
the Data Service is not able to handle IPv4 + IPv6
scenarios.
Change-Id: I4ca727f92618998242af18908bbbda6bb5f86303
Needed-By: Id8b005cdd429d53a75624885fe7ca795746c3ede
One of the goals for Pike [1] is to make each API service be able to
run under web servers that support WSGI applications,
such as Apache (+mod-wsgi) and Nginx (+uWSGI).
Do following to address governance requirements:
- Split existing manila/wsgi.py module into 3 modules:
First (manila/wsgi/eventlet_server.py) is used by
eventlet-based WSGI application approach.
Second (manila/wsgi/wsgi.py) is used for WSGI web servers.
And third (manila/wsgi/common.py) is common code for both.
All three are made in cinder-like way to have alike-approach.
- Reuse common code from "oslo_service/wsgi.py" module that
allows us to remove code duplication.
- Delete config opts that are defined by newly reused common code.
- Register new entry point that will be manila wsgi app: "manila-wsgi".
- Fix "manila/api/openstack/wsgi.py" module to be compatible
with str/bytes handling approach used by Apache mod-wsgi plugin using
different python versions (2/3).
- Add web server config template "devstack/apache-manila.template"
- Add devstack support where usage of this feature can be
enabled or disabled using "MANILA_USE_MOD_WSGI" env var.
It is set to "True" by default, because it is requirement for Pike
release - to have it running in all CI jobs.
Disable it only for one CI job that uses dummy driver and tests
various manila core features that are not covered by other CI jobs.
[1] https://governance.openstack.org/tc/goals/pike/deploy-api-in-wsgi.html
Partially-Implements BluePrint wsgi-web-servers-support
DocImpact
Change-Id: Ibdef3c6810b65a5d6f3611e2d0079c635ee523ab
Manila project uses special images for some of its share drivers.
So, add possibility to configure manila and run tempest tests using
custom image that was built with proposed changes to
'manila-image-elements' project.
Supported share drivers are 'generic' and 'container'.
Change-Id: I005d1b204ea25ced8913351d19eac282ca465fb6
Commit [1] fixed approach for setting devstack config according to
changes [2] in devstack. But it started updating only 1 of 2 possible
devstack directories that are 'old' and 'new'. Where 'old' is the one
that is used only in 'grenade' job as base installation for performing
project migration. 'new' is used in all other single-devstack
installations. So, with [1] everything started working except
'grenade' job.
Fix grenade job by copying devstack config to 'old' dir if it exists.
[1] Idfa8e0cb3f446361daa1449e0854d95cfb379f46
[2] http://lists.openstack.org/pipermail/openstack-dev/2017-February/112872.html
Change-Id: Ic00a1fe296fa68a65672fa680ebd8a9336ed7cef
Improve CI execution time by decreasing the interval
to wait for next 'migration_continue' driver call.
TrivialFix
Change-Id: I6134d3b0cc05ab17837639f1bc209d5f3ed99658
This new feature gives the user the ability to allow and
deny access to the snapshots, so that they could be mounted in
read-only mode to retrieve files.
APIImpact
DocImpact
Co-Authored-By: Rodrigo Barbieri <rodrigo.barbieri@fit-tecnologia.org.br>
Co-Authored-By: Alyson Rosa <alyson.rosa@fit-tecnologia.org.br>
Co-Authored-By: Miriam Yumi <miriam.peixoto@fit-tecnologia.org.br>
Partially-implements: blueprint manila-mountable-snapshots
Change-Id: I65f398a05f82eef31ec317d70dfa101483b44b30
This commit adds the ability for Manila to revert a
share to the latest available snapshot.
The feature is implemented in the LVM driver, for
testing purposes.
APIImpact
DocImpact
Co-Authored-By: Ben Swartzlander <ben@swartzlander.org>
Co-Authored-By: Andrew Kerr <andrew.kerr@netapp.com>
Implements: blueprint manila-share-revert-to-snapshot
Change-Id: Id497e13070e0003db2db951526a52de6c2182cca
The snapshot_support extra spec has always meant two
things: a driver can take snapshots and create shares
from snapshots. As we add alternate snapshot semantics,
it is likely that some drivers will want to support
snapshots and some of the new semantics while being
unable to create new shares from snapshots.
This work adds a new extra spec,
create_share_from_snapshot_support, that removes the
overloading on snapshot_support. It also makes the
existing snapshot_support extra spec optional,
allowing admins to create types without setting
snapshot_support; shares created with such types
will not support snapshots.
APIImpact
DocImpact
Co-Authored-By: Goutham Pacha Ravi <gouthamr@netapp.com>
Implements: blueprint add-create-share-from-snapshot-extra-spec
Change-Id: Ib0ad5fbfdf6297665c208149b08c8d21b3c232be
In addition to functional tests.
Also, make some corrections to scenario tests code to make it work
with shares located on localhost.
Co-Authored-By: Alexey Ovchinnikov <aovchinnikov@mirantis.com>
Co-Authored-By: Valeriy Ponomaryov <vponomaryov@mirantis.com>
Change-Id: Ice61a7593f5e4e443cffbe78ecd71d683a9c274e
3rd party CIs cannot set this config by exporting an env var
or local.conf setting, as it is always being overriden in code.
Change-Id: Ia2a4790d3a0d03faa231b400c32f6c1928bdca1f
Closes-bug: #1649249
Make devstack grenade code use 'newton' as base branch for migration to
master (ocata).
After switch [1] to dummy driver we need to override defaults.
So, update pre_test_hook approach for saving configuration data properly.
[1] I2b698f69a29ef983b13534b2f4d393570be4dac8
Change-Id: Ie5fa0f78b0e3caa06d34a0822a8ed60b69406f9d
Closes-Bug: #1631412
Improve coverage by adding tests that validate the share-type
change while also changing the driver mode.
Closes-bug: #1620800
Change-Id: I924c34aa69591754b437d75f43db91d77e73fb07
Current implementation of share migration works only if we use
local shell executor. So, refactor approach for it to make work
with both shell executors - local and remote (SSH).
Change-Id: Ibe31bf52006040ea26badfa7236ed67559f7aded
Closes-Bug: #1620502
Add support of share migration feature to ZFsonLinux driver.
Implements BP zfsonlinux-share-migration
Change-Id: If9e1fec9a6d4f96d2bb5c176c6249fafc7e3e891
At Austin 2016 summit there were several improvements to
Share migration feature discussed. This patch implements
these changes.
Changes are:
- Added 'Writable' API parameter: user chooses whether share must
remain writable during migration.
- Added 'Preserve Metadata' API parameter: user chooses whether
share must preserve all file metadata on migration.
- Added 'Non-disruptive' API parameter: user chooses whether
migration of share must be performed non-disruptively.
- Removed existing 'Notify', thus removing 1-phase migration
possibility.
- Renamed existing 'Force Host Copy' parameter to 'Force
Host-assisted Migration'.
- Renamed all 'migration_info' and 'migration_get_info' entries to
'connection_info' and 'connection_get_info'.
- Updated driver interfaces with the new API parameters, drivers
must respect them.
- Changed share/api => scheduler RPCAPI back to asynchronous.
- Added optional SHA-256 validation to perform additional check if
bytes were corrupted during copying.
- Added mount options configuration to Data Service so CIFS shares
can be mounted.
- Driver may override _get_access_mapping if supports a different
access_type/protocol combination than what is defined by default.
- Added CIFS share protocol support and 'user' access type
support to Data Service.
- Reset Task State API now allows task_state to be unset using
'None' value.
- Added possibility to change share-network when migrating a share.
- Bumped microversion to 2.22.
- Removed support of all previous versions of Share Migration APIs.
APIImpact
DocImpact
Implements: blueprint newton-migration-improvements
Change-Id: Ief49a46c86ed3c22d3b31021aff86a9ce0ecbe3b
In Scenario job, tests consist of VMs mounting shares
provided by share servers.
Admin network is used by share migration tests and is
by default used in other tests as well. Previously it
used the same network address range as tenant networks
(used by VMs) and there was no conflict.
Recently some unknown change broke connectivity between
VMs and share server in scenario job. It seems as there
is now a conflict between those network addresses ranges.
This patch changes the admin network address range,
removing the existing conflict.
Closes-bug: #1616115
Change-Id: I5bdb70cae4eab2930f3f559a621643ad732909bb
This driver exploits a Docker container as a share server instead
of a virtual machine. The container acts as a CIFS server
and shares logical volume attached to it. The volume gets created
simultaneously to the container on a host in a volume group
specified by the user.
DocImpact
Change-Id: I711813a2022d765ab7a70ed670e34f68f9cf1399
Implements: blueprint manila-container-driver
Added dummy driver for ease of testing and development of REST APIs
and internal manila interfaces.
It can be enabled as backend in the same way as all other share drivers.
But, it does not provide any resources.
Also, update tempest to be able to work with share-networks when
networks are defined via config and not API.
CI hooks are configured to run tempest twice using both available
driver modes. Where DHSS=True mode uses StandAloneNetworkPlugin as
network backend.
Change-Id: I9053dddfc643cb5a6602f15235b91bbaea727dbd
Implements BluePrint dummy-driver
Fix small incompatibility that left to be done and bump Tempest version
to unblock our CI jobs.
Also, remove temporary fix [1] that was done for unblocking CI.
[1] I44655c83cc5588c5b94d0fad81162dd0c238b5b4
Change-Id: I12e2652638ba26833f9365f06fa84a98ca24765c
Currently, Manila CI Tempest jobs use deprecated options of Tempest.
Recently, latest version of Tempest dropped them [1].
Manila CI jobs work ok, because they check out specific version of
Tempest. But it will work until first Devstack update for some one
more Tempest feature and then we will be forced to use latest Tempest
again. Switch to supported approach of setting up users in Tempest,
to avoid such sudden blockers.
[1] I8c24cd17f643083dde71ab2bd2a38417c54aeccb
Change-Id: I5b3ebab52ea1401f6f7a116d1260268eb10ebe0c
Closes-Bug: #1586129
Recently merged commit to "neutron-fwaas" project [1] started using
one of Tempest's modules that appeared after version our plugin uses.
So, implement hotfix-workaround until we make all compatibility changes
to latest Tempest in our plugin.
For the moment, we cannot switch to latest Tempest as we already
have incompatibility for it, see bug [2].
[1] https://review.openstack.org/#/c/326150/
[2] https://bugs.launchpad.net/manila/+bug/1586129
Change-Id: I44655c83cc5588c5b94d0fad81162dd0c238b5b4
Closes-Bug: #1589964
Generic driver fails from time to time on Cinder not being able
to schedule shares because of exceeded size limit.
Since we do not use size in API tests at all increase
Cinder oversubscription ratio from 20.0 to 100.0
to make it enough for sure.
Such error appears from time to time and
more often with addon of new Tempest tests that create shares.
TrivialFix
Change-Id: I8a14f806c0235f5a12930eee362a838404cd5c06
This fix basically ports nova fix https://review.openstack.org/#/c/136931/
plus additional bug fixing to Manila. It creates a common class to get
used for all clients (neutron, nova, cinder).
Idea is to create an auth session and pass this to the client object
instead of let the client do that. The auth session will be created by
a config group which means the configuration for the clients needed to be
heavily reworked. Patch is also backward compatible with old options but
flag them as deprecated.
DocImpact
Change-Id: Ic211a11308a3295409467efd88bff413482ee58d
Closes-bug: #1555093
SSH is required for replication to work in the ZfsOnLinux
driver, but the gate defaults to disabling it. Make it
configurable and make sure to preinstall host key.
Change-Id: Iece2ad2e966fafe3ab4d63c82d062830ac9f3170
Closes-Bug: 1552244
Download of Generic/Windows image for Nova should be disabled for
all other share drivers in CI.
Change-Id: Ib2c952e85dd0a6eb36fe945e0a33950c38dfdfae
Closes-Bug: #1553343
This driver exploits a container as a share server instead
of a virtual machine. The container acts as a NFS/CIFS server
and shares logical volume attached to it. The volume gets created
simultaneously to the container on a host in a volume group
specified by the user. Currently NFS is limited to v3 since the
only userspace NFS server that works in a container out of the box
seems to be unfs3. Later it will be replaced with nfs-ganesha.
Co-Authored-By: Alexey Ovchinnikov <aovchinnikov@mirantis.com>
Implements: blueprint lxc-driver
Depends-On: Ib1896997f2e7a505b5bf8ec0c9e5ee35942f79a6
Change-Id: Iea73bd34b94919d77ebe17cf054dde1f580384be
Goutham Pacha Ravi