... because these options are configured by devstack. This allows us to
honor some options which determine whether these accounts should be
used(eg. TEMPEST_HAS_ADMIN)
Also, this fixes the inconsistent default value of admin password (it
defaults to 'secret' in devstack).
Change-Id: I6221168ad400346bf1542ed7715c481295e42e04
We need an admin to setup share and group
types and per recent changes [1], this needs to
be a user with the "admin" role, operating
with "project" scope and not "system" scope.
Also remove a TODO in the code that was completed
with a prior SRBAC patch.
[1] https://review.opendev.org/c/openstack/manila/+/856394
Change-Id: I7bd2bca2bac6b892e7b8a07654d33a56d915e3bb
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
brctl is being deprecated in some Linux distros, so
change container driver to start using iproute2 commands.
Closes-bug: #1877410
Change-Id: If679e79fa3242ee1cd8610b5525deca35b41c87e
Manila plugin is setting the availability zone of the
backends without checking if it is set. As result, a CI
configuring that field will not be respected.
Fixed by only setting the field in case its value was
not set before. Otherwise, keep as it is.
Change-Id: I546e5f057f3b748417dcbcfe9c3f3ae4a5d51556
... because the policy rules are implemented in code. Also now Manila
uses policy.yaml instead of policy.json.
Change-Id: I4b03ec81a5a72491734ca1c76b50162399d5c370
RSA key is no longer enabled by default in recent openssl library.
Because of this change, RSA key does not work in recent Fedora and
CentOS Stream 9 unless it is enabled somehow.
This introduces a new MANILA_KEY_FORMAT parameter to devstack plugin
so that a different ssh key format (eg. ecdsa) can be used.
Change-Id: Ibe60da0b06cade641002d5a1650c38aa57c2d131
manila's tempest plugin is branchless and can at
any time add tests for API microversions incompatible
with stable branches.
We used to configure test API versions via the post test
hook when relying on devstack-gate. However, API version
config makes sense whenever we use devstack and have
tempest enabled. This will help us simplify default
job configuration in the manila-tempest-plugin repository.
Since the API version won't change on a stable branch,
hard-coding the versions (as other micro-versioned services
do in their devstack) doesn't pose any risk of failing
reality.
Change-Id: Ia671fa74c0ee338199bd92a1613882328d24c9e2
Related-Bug: #1928879
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
We used a workaround to solve the lack of legacy credentials
in our devstack plugin script by sourcing credentials early [1].
This workaround breaks when using the devstack-system-admin
persona from the /etc/openstack/clouds.yaml that devstack
sets up, because we'd be sourcing project credentials that aren't
popped off in time [2]. Cloud profiles work with OSC, we could
switch to that in the devstack plugin rather than rely on legacy
credential helpers from devstack. The only remaining piece is the
group type setup [3] which is expected to be complete soon.
[1] https://review.opendev.org/c/openstack/manila/+/818617
[2] https://review.opendev.org/c/openstack/devstack/+/817074
[3] https://review.opendev.org/c/openstack/python-manilaclient/+/805064
Change-Id: Icc15f427cc02054021fa23626ea14e6cd37b18fb
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
A change that stops loading credentials from userrc_early file [1]
was merged. Creating resources using the python-manilaclient
requires some parameters that were set while loading such file.
This change starts loading the credentials during the manila setup,
so the client can normally create the resources.
Change-Id: Id872f6e62b6464544a4edfbc2d3e7c954042f6e2
Use install_package instead of using a specific package manager
while installing docker on fedora systems. With DNF also a
possibility, the better option is to do not take a stand in chosing
which package manager will be used to install packages.
We have a install_package function that will use the appropriate
package manager to install the programs we need to install.
Change-Id: I0524554dae2d6970191ba3afa583480c391b5c3c
This method of installing manila-tempest-plugin
was deprecated several releases ago, but compatibility
was maintained for third party CI jobs. Since sufficient
time has passed allowing third party CI to adopt,
lets drop this from the devstack plugin
Change-Id: I59ffe6ebe4b2e80be1e6ae97872cf4db6527da62
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
A few releases ago, service types were standardized
across OpenStack [1] and manila's service type was
recommended as "shared-file-system" [2]. Tools were
meant to look at the service-types-authority [3]
and os-service-types repository [4] to find a list
of standard service type names for various first
party OpenStack services.
The openstacksdk currently relies on the service
type to be "shared-file-system" [5], so let's set
one up in devstack, to aid contributors working on
openstacksdk.
It is desirable to have all clients to initially
look for the standard service type name in the
cloud's service catalog, and fall back to the
legacy name ("sharev2") in environments that
call the v2 API that.
[1] https://specs.openstack.org/openstack/service-types-authority/
[2] https://service-types.openstack.org/service-types.json
[3] https://opendev.org/openstack/service-types-authority
[4] https://opendev.org/openstack/os-service-types
[5] f4dd6fe5fd/openstack/_services_mixin.py (L71-L72)
Change-Id: I6a1ada102a40f3e83fe8e5287856d01dd137ea95
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
As of API version 2.60, a project_id is no
longer needed in the API URLs. We can stop
devstack from setting up an endpoint with
project_id in it.
Create a "sharev2_legacy" endpoint that
contains the project_id for testing the
compatibility with the older style of URLs.
Change-Id: I25aeb1b6dd1a4150c6e542e29b7d43e18d9ad94c
Implements: bp remove-project-id-from-urls
Depends-On: I5127e150e8a71e621890f30dba6720b3932cf583
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
MANILA_MULTI_BACKEND has been deprecated for five years now, we should remove
it from our code base.
This variable was removed from the settings scripts along with:
MANILA_BACKEND1_CONFIG_GROUP_NAME and MANILA_SHARE_BACKEND1_NAME;
MANILA_BACKEND2_CONFIG_GROUP_NAME and MANILA_SHARE_BACKEND2_NAME.
Because they work in the same context.
Instead of them, the already implemented and in use,
MANILA_ENABLED_BACKENDS variable was placed to garantee the successful
back-end setup. The same replacement was made in the contribution
samples scripts.
Apart from this, we avoid configuring generic1 and generic2 if
another backend/s are selected.
Closes-Bug: #1898791
Closes-Bug: #1878477
Change-Id: I67036a65da9255694a00a9c8d56cfdefbdf23c05
We were using an env var with the IPv4 gateway config
that is not always present. This was causing devstack to fail
in developer environment. Use the local variable instead.
Closes-Bug: #1910760
Change-Id: Iede8a9e59b96d0f21c117ab1464a0a9e3477c24b
When you run unstack.sh from devstack, other devstack
services are stopped and disabled to provide a clean environment
for a restack, but manila services are left running.
This doesn't matter for CI where a new VM is stood up for each
devstack but it's inconvenient for local devstack and if you
restack without restarting the services manually the results you
see may not actually match the environment you intended.
Change-Id: I6761619042e4bc36ec2f1cab4be33cb1b39d00d7
This patch adds support for migration of share servers. This
migration is performed using a two-phase approach. Administrators
are now able to request the migration of a share server within and
across backends, with the possibility of chooosing a different share
network for the destination share server.
- A new field called `task_state` was added to the share server
model in order to help the administrator to track the share server
migration steps. A new field called `source_share_server_id` was
added to link destination and source share servers.
- A new periodic task was added to track migration of share servers
and its resources.
- Two new states were added: `server_migrating` and
`server_migrating_to` to represent that share migration is in
progress.
- When performing the server migration, manila will not go to the
scheduler, instead it will provide a request spec to drivers
during migration check driver call. It'll be up to the driver
validate if there is free space to handle the share server.
- A new API called `share-server-migration-check' was added to
check the feasibility of a migration, before actually triggering
the start operation.
APIImpact
DocImpact
Partially Implements: bp share-server-migration
Co-Authored-By: Andre Beltrami <debeltrami@gmail.com>
Co-Authored-By: Carlos Eduardo <ces.eduardo98@gmail.com>
Co-Authored-By: Felipe Rodrigues <felipefuty01@gmail.com>
Change-Id: Ic0751027d2c3f1ef7ab0f7836baff3070a230cfd
Signed-off-by: Douglas Viroel <viroel@gmail.com>
When the image count is over 25, there might not get
manila-service-image, because current manila shares
creation is using novaclient to get image info, but
novaclient can only get 25 images due to pagination
of glance server, So this change is to switch to use
glanceclient instead of novaclient to get image info,
because glanceclient can iter all image info, while
novaclient is rarely maintained with stuff of image
API.
Change-Id: Id1715d0b9cb3a4aeedeb23d9b1d9924a78d18dc6
Closes-Bug: #1741425
After fixing the uwsgi installation from source that was broken[1][2],
Manila jobs started to fail since no uwsgi wasn't found in the
expected path. This fix now uses 'which' command to search for
uwsgi pathnames in the system.
[1] https://bugs.launchpad.net/devstack/+bug/1883468
[2] https://review.opendev.org/#/c/577955/
Related-Bug: #1883468
Closes-Bug: #1883715
Change-Id: I8d8b2fe07d86899c694cb73a81087d25311d30a5
If the cephfs protocol is enabled, clients
need to access the ceph daemons.
We also don't need to enable access to NFS
ports when not using NFS.
Change-Id: I077d12785f94c940716f0e479d43dbb29ddc3c3c
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
When running IPv6 tests, we disable the creation of
the public network in devstack, and handle it in the
devstack plugin. So, we'll need to configure it
in tempest in the plugin.
Change-Id: Icf5f26397e16d7fbfae77aa246ac5e35581f441e
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
Devstack no longer installs this package after [1]. The driver
needs to replace the use of brctl to completely remove this
package from our test environments.
[1] https://review.opendev.org/724443/
Partial-Bug: #1876820
Change-Id: Id6094827341bf6ef8856cd4e7af11b36e9afb560
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
We currently use a legacy devstack-gate script
to create the bgp speaker, and peer required
to communicate to and from the tenant private
IPv6 networks.
Move this logic to the devstack plugin so that
we can invoke it automatically with the existing
devstack variable MANILA_SETUP_IPV6.
Change-Id: Iea90e3f04ae05e5783c3163bbf2d2dabd128c7b5
A fix was committed [1] to the python-openstackclient
project to handle VMs with multiple ports. We can
drop the workaround of setting a floating IP to the
VM's port.
[1] 013c9a4f3a
Change-Id: I67c38b4e6695c05a8272a86448ed248b492a279d
Closes-Bug: #1747721
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
To set up some first party backends such as
ZFSOnLinux, CephFS via NFS gateway, Container
(where the NAS server is containerized) and LVM,
manila's devstack plugin creates a NAS server
on the devstack host.
On test machines, access to this NAS server is
firewalled from networks outside of the host's
internal network namespace (including from private
project networks that are in different network
namespaces, on the same devstack host).
We currently use a legacy devstack-gate script
to disable firewall on NFS ports; however,
anyone that installs devstack with LVM, Container,
ZFSOnLinux, CephFS-NFS drivers will need these
firewall ports to be opened to be able to mount
shares exported off their devstack host machines.
Move these firewall commands to the devstack plugin.
These commands can be invoked by setting the localrc
variable MANILA_ALLOW_NAS_SERVER_PORTS_ON_HOST to True.
The value of this variable is False by default,
to preserve existing behavior.
Change-Id: Ic9cad47662f1edf2e5c710dbe64d580bc5f01d44
This patch enables the use scheduler creating share from
snapshot flag in order to run properly the manila tempest tests.
Partially-implements: bp create-share-from-snapshot-in-another-pool-or-backend
Change-Id: I15311089c45be1574857d46caa073e89424e716d
Today we either expose the CephFS back end via the CephFS native
protocol, in which case kernel NFS and Samba are not needed, or
by NFS-Ganesha, in which case Samba is not needed and kernel NFS
is incompatible (we just turn it off again later).
In the future someone could develop support for exposing CephFS via
Samba so we'd add specific checks for that then and install Samba.
Change-Id: Id65ce2536a3deb32a04a70bd0c49081297741965
It was deprecated in the Queens release, in
favor of ``data_node_access_ips``.
Change-Id: I01f24c2e0d66e1da4c30c579c02afc8f3930c8f8
Related-Bug: #1745436
configure_auth_token_middleware was replaced by
configure_keystone_authtoken_middleware in
Id0dec1ba72467cce5cacfcfdb2bc0af2bd3a3610. Manila's
DevStack plugin no longer needs to create a cache
directory to use as the signing_dir for keystone
authtoken.
Change-Id: I18fa04e6c3a832e17ade3f84d8b43b15686f06fe
manila-tempest-plugin can be installed with its
devstack plugin; Installing it via manila's plugin
is unnecessary. So, deprecate its installation
in the DevStack plugin.
Change-Id: I21c08069ff82b3bfb52ef7ac960183ddc866c2ee
Restoring the default IPv6 route is not needed in CI and
fails when there are multiple defaults, but it is useful
in local devstacks where multiple default routes are not
typical.
Add a variable in settings and use it to make this behavior
conditional and set it to False for the lvm job.
Closes-bug: #1836788
Change-Id: Id73de8100509ec5935641f5f35f93f482d108bcd
Switch manila to use devstack common functions
for deploying a wsgi app under uwsgi and apache.
This change aims to fulfill the community requirement
for all projects on OpenStack to switch devstack jobs
to deploy control-plane API services under uwsgi with
Apache acting as a front end proxy.
More details in https://governance.openstack.org/tc/goals/pike/deploy-api-in-wsgi.html
We used to deploy with mod_wsgi, but mod_wsgi has many
issues when being used in devstack for development or testing.
Now we leave mod_wsgi as optional and we default to uwsgi.
Implements: bp wsgi-web-servers-support
Depends-On: https://review.openstack.org/#/c/643188/
Change-Id: I12d4c864f2ceb0f3555e32d2dc893e00dbef0d96
Enabling tls-proxy allows devstack to
set up a tls proxy server that front-ends
interactions with the manila-api and
terminates tls connections.
Also enable tls-proxy in dummy and lvm
jobs. The dummy driver job is configured
to run the in-built wsgi server, the lvm
job is configured to use mod-wsgi.
Closes-Bug: #1816836
Change-Id: I48b0ccc082604d78242ba61bee94a45efeb2467b
Every enabled backend gets assigned to its own AZ.
Test cases in manila-tempest-plugin already exercise
creating shares (and replicas) across AZs when
multiple AZs are available.
This is done for all back-end drivers that are not
the Generic driver. Configuring backend AZ for each
Generic driver based back end will require configuring
Nova and Cinder in a similar fashion.
Partially-implements: bp per-backend-availability-zones
Depends-On: https://review.openstack.org/#/c/630886/
Depends-On: https://review.openstack.org/#/c/629958/
Change-Id: I1b6ff535a22f10e70d379377767c8ffac3ef5286
There might be a rescan for volume groups needed before checking if
the volume group is already there. Otherwise, the check for the volume
group fails which means the code tries to create the volume, but that
fails then because the volume is already there.
Here is the devstack run log:
[...]
plugin.sh:configure_backing_file:538 sudo losetup -f --show /opt/stack/data/lvm-shares-backing-file
plugin.sh:configure_backing_file:538 DEV=/dev/loop2
plugin.sh:configure_backing_file:543 sudo vgs lvm-shares
Volume group "lvm-shares" not found
Cannot process volume group lvm-shares
plugin.sh:configure_backing_file:543 sudo vgcreate lvm-shares /dev/loop2
Physical volume '/dev/loop2' is already in volume group 'lvm-shares'
Unable to add physical volume '/dev/loop2' to volume group 'lvm-shares'
/dev/loop2: physical volume not initialized.
plugin.sh:configure_backing_file:1 exit_trap
[...]
Change-Id: I0189580ae40b180249fd5846aa986052e44ce2c2
Generic driver jobs are failing because of timeouts when
establishing the initial ssh connection from manila-share
to the service VM.
Bump up the default value of the connection timeout for paramiko
client and also set the banner timeout since the failure occurred
during banner exchange. Set the two timeouts to the same value
for now. This ensures that the connection timeout is at least as
long as the banner timeout and there is no current need in manila
to control these independently.
This is more of a workaround than a real fix since a real fix
would remove the delay during banner exchange. I suspect that
the real fix will need to be in neutron/ovs though.
Change-Id: Ib5e59faaf9667b9cb5e7d4072531b7d6c3d4da39
Partial-bug: #1807216