This job is non-voting against the manila repos currently.
We've observed sporadic failures in data intensive scenario
tests. There are a number of manila tests that write a few
gigabytes of data onto mounted shares. While we've tried
to optimize test concurrency and the kind of data being written,
sometimes, things seem to fall over and a test case runs into
a generic I/O error. So maybe more work is necessary to
get this job to be stable; but that work may not be in this
devstack plugin repository.
Change-Id: Id89f161ba6dd5b5fbb5b82042a3e94c92797a881
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
Add podman ceph-common and jq as part of preinstall dependency.
Add REMOTE_CEPH capabilities to CEPHADM deployment.
Removed set_min_client only if cinder is enabled, this should be set
in any case.
Get FSID from ceph.conf in /etc/ceph to avoid unnecessary override.
Part of an effort to test multinode deployments with cephadm.
Needed-By: I5162815b66d3f3e8cf8c1e246b61b0ea06c1a270
Change-Id: I84249ae268dfe00a112c67e5170b679acb318a25
Following up on the usage of inclusive jargon [1] and to ensure
consistency in code, there is a need to replace deprecated term
tempest_test_blacklist with a new term tempest_test_exclude_list.
[1] https://review.opendev.org/c/openstack/tempest/+/768583
Change-Id: I1692876c9d42b3ac857fdb61dcdc2caa61cd349d
We are about to drop Fedora support as the latest image in upstream
has been transitioned to EOL. Centos 9 Stream has evolved as replacement
platform for new features. Review which deletes nodeset from devstack:
https://review.opendev.org/c/openstack/devstack/+/885467
Change-Id: I3b1bece44b9ad2f4bf4d822f261b141ddf3031fd
On the multinode job we need to enable validation like the base job
and also disable block migration for live migration that we get from
our parent job.
Change-Id: I076527536e19f7fa2c0cd177bebb1df22db51a0a
The overrides in "devstack-plugin-ceph-cephfs-nfs"
are being applied to the parent job, so they
can be removed here.
Depends-On: I6abf7a8eb275c5b29cfe29b3a0f54de76c5bb1e8
Change-Id: I35ebd069399d3a8d8e2bb56542a01993914b1d73
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
This reverts commit 863a01b032.
Partial revert only for the pin to focal, leaves the broken other jobs
commented out.
Update paste-deploy workaround to be used always.
Add qemu-block-extra and podman deps to the debs list.
Running on the newer ceph and distro causes some quite different
performance characteristics that cause tests that used to pass to fail
more often. This includes some performance optimizations to help
reduce the memory footprint, as well as depends on changes to
tempest tests to improve the reliability of those tests by enabling
validation via SSH.
This also moves the cephadm job to be the voting/gating job as that
seems to be the clear consensus about "the future" of how we deploy
ceph for testing.
Depends-On: https://review.opendev.org/c/openstack/cinder-tempest-plugin/+/881764
Co-Authored-By: Dan Smith <dms@danplanet.com>
Change-Id: I899822fec863f43cd6c58b25cf4688c6a3ac1e9b
Package based deployment of ceph isn't a Ceph-community
supported deployment strategy. We are trying to migrate
to using cephadm orchestrated ceph in the CI one job
at a time. See [1]
[1] https://review.opendev.org/c/openstack/devstack-plugin-ceph/+/865315/11
Depends-On: I66ac85b735566f8f7dd75234eb8e0be7773fee3a
Change-Id: I21688daa7ac21eb395897daba61a5fdba2697b5f
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
The cephfs-nfs job was turned off [1] for perma-failing.
This commit adds the original non-voting job back into the
check queue and fixes some installation issues:
1) use ceph "quincy" release: Ceph Pacific's end of life
is 2023-06-01 [2]. The manila community thinks deployers
are more likely to use quincy with the 2023.2 (bobcat)
release of OpenStack.
2) run the job with centos-stream-9: There are no packages
currently available for Jammy Jellyfish on download.ceph.com [3].
The OS shouldn't really matter for this CI job that is meant to
test feature functionality provided by manila. At this time, we'd
like to stick with builds provided by the ceph community instead
of the distro since it may take a while to get bugfixes into distro
builds.
3) The install script uses "nfs-ganesha" builds for ubuntu and
centos hosted by the nfs-ganesha community [4]. We will not rely on
the ceph community to provide the latest builds for
nfs-ganesha any longer.
This commit also cleans up the unnecessary condition in the
ceph script file pertaining to configuring ceph packages for
Jammy Jellyfish. This step wasn't doing anything.
Ubuntu packages don't work at the moment and that requires some more
investigation.
[1] Id2ae61979505de5efb47ce90a2bac8aac2fc5484
[2] https://docs.ceph.com/en/latest/releases/
[3] https://www.spinics.net/lists/ceph-users/msg74312.html
[4] https://download.nfs-ganesha.org/
Change-Id: I40dfecfbbe21b2f4b3e4efd903980b5b146c4202
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
As per the community wide goal to migrate the CI/CD from
Ubuntu Focal to Ubuntu jammy, we need to merge the devstack, tox base
jobs to jammy on Nov 18. But ceph jobs are failing on Ubuntu Jammy.
To move ahead to merge the base job patches we need to pin the
ceph jobs nodeset to Focal until this is fixed for Jammy.
Since the devstack-plugin-ceph-tempest-py3 is currently permanently
failing, although this patch is a no-op, make it non-voting. Also don't
run 100% failing non-voting jobs until they are fixed.
Needed-By: https://review.opendev.org/c/openstack/devstack/+/860795
Related-bug: #1996628
Change-Id: Id2ae61979505de5efb47ce90a2bac8aac2fc5484
Add the option to deploy the Ceph cluster
with the cephadm tool.
Depends-On: I799521f008123b8e42b2021c1c11d374b834bec3
Co-Authored-By: Francesco Pantano <fpantano@redhat.com>
Change-Id: Id2a704b136b9e47b7b88ef586282cb5d0f754cf1
The parent jobs of the basic tempest jobs defined here enabled
horizon since the merge of tempest-horizon into tempest (see
Id2ced856a41548a0b49e594ee5fed6ed28785f24).
But horizon is not really needed here, as the default list of
tests does not incldue any horizon test. Disabling it helps
freeing up a bit of resources for the job and also helps
the ongoing effort of testing FIPS, which would otherwise
require a patched django at this point.
Change-Id: I05ea7f675aafb85eb54427ca95167b4c8f0eabdf
This changeset adds a config option (TEST_MASTER)
to toggle testing with the current Ceph master.
The builds are retrieved from Shaman, which is the
platform used by the Ceph community to submit their
latest builds.
The main goal of this change is to use this on our CI
and be able to catch bugs in OpenStack services that use
Ceph as a backend more quickly. Because of this, the
required non-voting job is also added.
Change-Id: Ibe26f89e7be3061cdb7acb441727efb2c396e29e
devstack-plugin-ceph is now branched and each branch
will maintain the respective branch specific code and
testing.
Change-Id: I9edf3ff59055e1ce6cd1630ea8e747ff5b498f02
The risk of breaking the gates by forgetting to run the jobs
against the old branches is too high
(it has happened with https://review.opendev.org/778960/ for example)
Related-Bug: #1921897
Change-Id: Ieab4e0ee482604802485cdf573c31e3c373a8553
This change introduces a multinode ceph job based on
tempest-multinode-full-py3 to be used to provide live migration and
other multinode coverage initially in the openstack/nova project.
As this job uses ceph for both Nova ephemeral
([libvirt]/imagebackend=rbd) and Cinder volume storage the live block
migration tests are skipped.
Change-Id: I9ffdff44a3ad42ebdf26ab72e24dfe3b12b1ef8b
Update the list of tested stable branches (excluding stein which is EM).
The nodesets for the stable branches are now pinned because
their testing environment are well defined and won't change.
Also, remove the -py3 suffix: those jobs are all -py3 based
(the suffix was mostly requires in the transitional time
between py2 and py3).
Change-Id: I48f8574960df51fc47f80fa17091fd13ec71aa99
Consume packages from download.ceph.com for ceph octopus
Our focal fossa based CI jobs are getting octopus from
ubuntu ppas instead of getting the packages from
download.ceph.com.
This happens because, even though we were defaulting to
nautilus, ubuntu got the later version available
(in this case, the packages in ubuntu ppas).
Also, remove CEPH_STABLE_RELEASE env var, unify to CEPH_RELEASE
Stable branches CI versioning is done in the jobs definition now
Change-Id: I1edd5a50079f325fa143a7e0d51b3aa3bb5ed45d
Given the issues introduced by I0eab8ad25112511b0c4dd4b7796b4cda821bca8b
it would be nice to have a simple RPM based job somewhere in the check
queue to avoid a repeat in the future.
Change-Id: I601b77f2da0574d38ccef7d05db4933f6fdae998
This job covers the popular usecase of not using rbd for ephemeral
storage for nova while it is used for Glance and Cinder.
This job additionally enables the recently implemented direct download
of rbd hosted Glance images into the file based ephemeral store of Nova.
Change-Id: Ieddd803fccb097657fffe30ef87c1c8385e7eb28
- manila-tempest-minimal-dsvm-cephfs-native and
manila-tempest-minimal-dsvm-cephfs-nfs are both
legacy jobs that use the now deprecated
devstack-gate.
- We no longer need the CentOS based experimental
jobs for manila since the manila team plans to
delete them from their tree. Newer CentOS experiments
can happen with the native zuulv3 style jobs.
- The jobs added are inheriting manila-tempest-plugin jobs,
but currently don't override anything - there's a plan
to supply overrides soon to accommodate the ceph support
matrix that the manila team wants to test via this
repository.
- Also make minor refactors to existing base
jobs to make reuse of certain things easier
A future commit will remove the pre-test-hook and
post-test-hook scripts; they're presently there to allow
testing legacy jobs against older releases of
openstack/manila.
Change-Id: I20ab47cf7846e7c0291ef4771cc93fd85d0452b0
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
devstack-plugins-ceph does not have the doc infra set
up and no doc publish job.
This is need to add the contributor doc as per the ussuri
community goal.
Change-Id: Ib62c34b52b28b753f7f3046d51492fd8d9fb6193
While the job itself is not currently voting within openstack/nova the
use of the plugin essentially is within the nova-live-migration job.
As such we should really enable voting of this job within this project
with the eventual goal of also enabling it within openstack/nova and
other projects using it.
Change-Id: Id243b92e800e2d6e8b2228fadbd1c72b194bbfb9
devstack-plugin-ceph-tempest is py2 job and not supposed
to run on master gate. devstack-plugin-ceph master gate already
run its py3 version devstack-plugin-ceph-tempest-py3.
We need to keep the devstack-plugin-ceph-tempest definition
for stable branches gate.
Change-Id: I6c512b29a0346db1f92de34b1dbfb56958fa2cad
devstack-plugin-ceph is branchless and its jobs run on
stable branches. To tests the devstack-plugin-ceph changes
for stable branches, let' add the stable version of
devstack-plugin-ceph-tempest-py3 job in experimental pipeline.
Adding in experimental pipeline as we do not really have to run
those all the time on check gate but can be run on demand.
Change-Id: I9010f0d6226304537adcbe4e3f97250c754a7de9
The manila CentOS jobs break often, and are not
maintained. Let's move them to experimental
so they don't gate every patch. The other manila
jobs currently provide necessary test coverage
to cephfs-native and cephfs-nfs protocols/drivers.
These jobs also run with python3, and we don't
need to support python3.5 any longer.
Change-Id: Ibb752a1d9aa818dd6862bcbbce4d6b6e366b4800
Devstack and tempest default to enable ssh validation in
a lot of API tests which can fail intermittently:
http://status.openstack.org/elastic-recheck/#1808010
Also see bug 1802971.
Since a ceph backend for storage shouldn't really have
any impact on whether or not ssh is going to work in a
tempest run, this change disables it in the
devstack-plugin-ceph-tempest jobs which should make the
job faster and more stable.
Change-Id: Iea948bcc43315286e5c130485728152d4710bfcb
This is a mechanically generated change to replace openstack.org
git:// URLs with https:// equivalents.
This is in aid of a planned future move of the git hosting
infrastructure to a self-hosted instance of gitea (https://gitea.io),
which does not support the git wire protocol at this stage.
This update should result in no functional change.
For more information see the thread at
http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003825.html
Change-Id: I0ebfde9478ec07f9f8649e239adfccbf6f7431c6
Below patch enabled the multiattach feature in devstack ceph job
without branch variant. Multiattach feature is only available
from stein.
- https://review.openstack.org/#/c/638499/2
This commit makes two variant of job with multiattach enable from
stein onwards only.
Change-Id: Ib0a5667a6125fa5ae430086db93baf5a53be4877
Closes-Bug: #1820098
Tempest change I522a15ba3dbfee5d8ef417e43288a12319abf6ff is
enabling volume multiattach testing in the tempest-full and
tempest-full-py3 job configurations which we inherit for
devstack-plugin-ceph-tempest and devstack-plugin-ceph-tempest-py3
respectively. Since the rbd driver in cinder does not support
multiattach yet we need to disable multiattach testing for these
jobs.
Change-Id: Ia9bd6a42f33a7d9fe3041cf5c751ab691df19cb7
Finding this file via zuul.project only works if
the project is devstack-plugin-ceph, look for it via
zuul.projects which should work for any run of this
job.
Change-Id: I4bc982fad3b7de9582ef30b115c6829980f61047
This migrates the legacy jobs to zuulv3 native ones.
While migrating the jobs I got the impressions that they have not
bin maintained recently, which is possible since they are not
voting / experimental.
The regex in the py27 job includes cinder.* tests, and the job
used the all-plugin tox env (which has site-packages enabled) so
my guess is that the job used to run cinder in tree tests.
Those tests are now available in the cinder-tempest-plugin repo
instead, which could be added to these jobs if needed.
Following a similar setup in the cinder-tempest-plugin[0] we add
a file `tempest_blacklist.txt` that can by amended to list tests
that are known to fail for a ceph based deployment.
[0] https://review.openstack.org/#/c/531206/
Co-Authored-By: Jens Harbott <j.harbott@x-ion.de>
Change-Id: I27f85991ca947809d46ae2dee57704e779e6dfb6
This is a mechanically generated patch to complete step 1 of moving
the zuul job settings out of project-config and into each project
repository.
Because there will be a separate patch on each branch, the branch
specifiers for branch-specific jobs have been removed.
Because this patch is generated by a script, there may be some
cosmetic changes to the layout of the YAML file(s) as the contents are
normalized.
See the python3-first goal document for details:
https://governance.openstack.org/tc/goals/stein/python3-first.html
Change-Id: I65217d73f55132dbd1ca48ac421cba504543d1cb
Story: #2002586
Task: #24327
Goutham Pacha Ravi