Commit Graph

317 Commits

Author SHA1 Message Date
Goutham Pacha Ravi 6e0ab534c0 Set CephFS/NFS job to non-voting
This job is non-voting against the manila repos currently.

We've observed sporadic failures in data intensive scenario
tests. There are a number of manila tests that write a few
gigabytes of data onto mounted shares. While we've tried
to optimize test concurrency and the kind of data being written,
sometimes, things seem to fall over and a test case runs into
a generic I/O error. So maybe more work is necessary to
get this job to be stable; but that work may not be in this
devstack plugin repository.

Change-Id: Id89f161ba6dd5b5fbb5b82042a3e94c92797a881
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
2024-03-05 09:38:39 -08:00
Goutham Pacha Ravi 24d9517b78 Fix cephadm binary URL to the numeric ceph version
download.ceph.com used to maintain named versions of ceph
releases, which were hardlinked folders corresponding
to the latest minor release from a stable branch; we've noticed
that these folders can be deleted. Let's instead look for
cephadm under the corresponding numeric release tag folder.

Change-Id: Ic39b48fb2dd48f47d5b3c6165e4f4c6b1c47cc7d
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
2024-03-04 19:07:16 +00:00
Zuul b513540955 Merge "Add ingress deamon" 2024-02-06 19:40:02 +00:00
ashrod98 a42ee1269b Add ingress deamon
Adds ingress deamon, for 18.0+ Ceph versions

Change-Id: Ic345784873161df2326344c8993f5691fbadb40c
Depends-On: Idf15ccf996eb6cdc05a960bdc6411f6fdb9e5fea
Depends-On: I85ad659bf1ad36cb5340a53cd57603451fc77147
2024-02-02 15:37:47 +00:00
Ashley Rodriguez c7fb07d479 Bump to Reef
Bumps ceph versions to Reef to enable ingress service deployments.

Change-Id: I85ad659bf1ad36cb5340a53cd57603451fc77147
2024-02-02 10:37:09 -05:00
Francesco Pantano 4c22c3d090
Add os_release variable to get the right cephadm version
For releases older than reef (e.g., octopus, pacific), packages are not
always available under the el9 subdirectory.
This patch introduces a switch case to make sure we're able to match the
right version of cephadm.

Change-Id: I0ee37b832f1ea47961528f074f2d42492b0ac755
2023-12-11 09:35:37 +01:00
Goutham Pacha Ravi 7b209845d5 Update default ceph image tag to remove patch version
Ceph release tags adhere to a versioning scheme x.y.z [1], where:
- x = major release number (e.g.: quincy is 17, reef is 18)
- y = 1 or 2, where 1 is an release candidate, and 2 is a stable release
- z = patch/updates

We shouldn't hardcode a patch version in the default container
image we're fetching in our jobs, unless absolutely necessary
for some bugfix/feature that we rely on.

[1] https://docs.ceph.com/en/latest/releases/general/
Related-Bug: #1989273

Change-Id: Iea541d2edefc871bcac2d965997c88462fcbe521
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
2023-10-23 12:26:01 -07:00
ashrod98 120041215c Revert bad rebase
Reverting a bad rebase to fix cephadm binary source
Also adds catatonit to rpms file to fix CI issues

Change-Id: Ie1b1dc0ef2508eae38ae7954fb0bb62653780644
2023-10-13 17:21:55 +00:00
Ashley Rodriguez b663a9fb83 Remote Ceph with cephadm
Add podman ceph-common and jq as part of preinstall dependency.
Add REMOTE_CEPH capabilities to CEPHADM deployment.
Removed set_min_client only if cinder is enabled, this should be set
in any case.
Get FSID from ceph.conf in /etc/ceph to avoid unnecessary override.

Part of an effort to test multinode deployments with cephadm.

Needed-By: I5162815b66d3f3e8cf8c1e246b61b0ea06c1a270
Change-Id: I84249ae268dfe00a112c67e5170b679acb318a25
2023-09-12 20:51:41 +00:00
Zuul f493a2b408 Merge "Replace deprecated terms" 2023-08-14 11:53:36 +00:00
Zuul 491c0b9293 Merge "[CI] Drop overrides in cephfs-nfs job" 2023-08-07 22:01:26 +00:00
Zuul 95b230b1ff Merge "Update location of cephadm script" 2023-08-01 21:27:29 +00:00
Katarina Strenkova 8ae707f234 Replace deprecated terms
Following up on	the usage of inclusive jargon [1] and to ensure
consistency in code, there is a need to replace deprecated term
tempest_test_blacklist with a new term tempest_test_exclude_list.

[1] https://review.opendev.org/c/openstack/tempest/+/768583

Change-Id: I1692876c9d42b3ac857fdb61dcdc2caa61cd349d
2023-07-27 13:50:11 +02:00
Goutham Pacha Ravi d1433e6408 Update location of cephadm script
We don't need to get cephadm from the source tree.
The ceph community publishes this binary on download.ceph.com [1]

[1] https://docs.ceph.com/en/latest/cephadm/install/#curl-based-installation

Change-Id: I9c1f46b15a03768aa92803e7c51522c3553b1595
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
2023-07-19 12:48:51 -07:00
Zuul bdce6ce508 Merge "Drop Fedora support" 2023-07-18 11:45:39 +00:00
Zuul fab466e73c Merge "Update default ceph version to "quincy"" 2023-07-17 21:02:45 +00:00
Dan Smith c4d753d37a Unskip rebuild_volume_backed_server test
This was actually due to a too-strict-for-AIO-machines default timeout,
which is bumped in the dependent patch.

Closes-Bug: #2025096
Depends-On: https://review.opendev.org/c/openstack/devstack/+/887110
Change-Id: I5b852797dc05f1de55a2b294289938b9962c38e1
2023-06-27 12:18:56 -07:00
Ghanshyam Mann 10670ac532 Skip test_rebuild_volume_backed_server test for ceph job
There is some issue in ceph job while password injection
during rebuid and due to that test is failing 100% failure

- https://b932a1446345e101b3ef-4740624f0848c8c3257f704064a4516f.ssl.cf5.rackcdn.com/883557/2/check/nova-ceph-multistore/d7db64f/testr_results.html

Skip test to unblock the gate for now and we can unskip it once
issue it fixed.

Related-Bug: #2025096
Change-Id: Ica868be8ff0e1795d8de204640045691b0261b29
2023-06-26 22:43:34 +00:00
jskunda a7f2662a81 Drop Fedora support
We are about to drop Fedora support as the latest image in upstream
has been transitioned to EOL. Centos 9 Stream has evolved as replacement
platform for new features. Review which deletes nodeset from devstack:
https://review.opendev.org/c/openstack/devstack/+/885467

Change-Id: I3b1bece44b9ad2f4bf4d822f261b141ddf3031fd
2023-06-21 23:45:03 +02:00
Goutham Pacha Ravi 6c11c78631 Update default ceph version to "quincy"
Affects only the package based install script.

Pacific is going to be EOL'ed soon [1], we do not
expect folks to deploy the next version of OpenStack
(2023.2/Bobcat) with Ceph Pacific; moreover, all our
CI jobs are using Quincy by default, and the cephadm
script uses quincy by default.

[1] https://docs.ceph.com/en/latest/releases/index.html#active-releases

Change-Id: I6abdb1241e99d76bcee35b331e1059c4fe48296b
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
2023-05-17 13:35:09 -07:00
Dan Smith bf4598d923 Enable validation and disable block-migration
On the multinode job we need to enable validation like the base job
and also disable block migration for live migration that we get from
our parent job.

Change-Id: I076527536e19f7fa2c0cd177bebb1df22db51a0a
2023-05-11 11:45:10 -07:00
Zuul d1c7a2db8b Merge "Run the native CephFS job with cephadm" 2023-05-10 17:06:16 +00:00
Goutham Pacha Ravi 09c3caa13d [CI] Drop overrides in cephfs-nfs job
The overrides in "devstack-plugin-ceph-cephfs-nfs"
are being applied to the parent job, so they
can be removed here.

Depends-On: I6abf7a8eb275c5b29cfe29b3a0f54de76c5bb1e8
Change-Id: I35ebd069399d3a8d8e2bb56542a01993914b1d73
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
2023-05-08 17:53:57 -07:00
Dr. Jens Harbott 41b6a8c227 Revert "Temporary pin the ceph jobs nodeset to Focal"
This reverts commit 863a01b032.

Partial revert only for the pin to focal, leaves the broken other jobs
commented out.

Update paste-deploy workaround to be used always.
Add qemu-block-extra and podman deps to the debs list.
Running on the newer ceph and distro causes some quite different
performance characteristics that cause tests that used to pass to fail
more often. This includes some performance optimizations to help
reduce the memory footprint, as well as depends on changes to
tempest tests to improve the reliability of those tests by enabling
validation via SSH.

This also moves the cephadm job to be the voting/gating job as that
seems to be the clear consensus about "the future" of how we deploy
ceph for testing.

Depends-On: https://review.opendev.org/c/openstack/cinder-tempest-plugin/+/881764
Co-Authored-By: Dan Smith <dms@danplanet.com>
Change-Id: I899822fec863f43cd6c58b25cf4688c6a3ac1e9b
2023-05-03 12:06:27 -07:00
Goutham Pacha Ravi ef8898fd8a Run the native CephFS job with cephadm
Package based deployment of ceph isn't a Ceph-community
supported deployment strategy. We are trying to migrate
to using cephadm orchestrated ceph in the CI one job
at a time. See [1]

[1] https://review.opendev.org/c/openstack/devstack-plugin-ceph/+/865315/11
Depends-On: I66ac85b735566f8f7dd75234eb8e0be7773fee3a
Change-Id: I21688daa7ac21eb395897daba61a5fdba2697b5f
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
2023-04-27 02:35:33 +00:00
Goutham Pacha Ravi 563cb5deeb Cleanup installation and revive cephfs-nfs job
The cephfs-nfs job was turned off [1] for perma-failing.
This commit adds the original non-voting job back into the
check queue and fixes some installation issues:

1) use ceph "quincy" release: Ceph Pacific's end of life
   is 2023-06-01 [2]. The manila community thinks deployers
   are more likely to use quincy with the 2023.2 (bobcat)
   release of OpenStack.
2) run the job with centos-stream-9: There are no packages
   currently available for Jammy Jellyfish on download.ceph.com [3].
   The OS shouldn't really matter for this CI job that is meant to
   test feature functionality provided by manila. At this time, we'd
   like to stick with builds provided by the ceph community instead
   of the distro since it may take a while to get bugfixes into distro
   builds.
3) The install script uses "nfs-ganesha" builds for ubuntu and
   centos hosted by the nfs-ganesha community [4]. We will not rely on
   the ceph community to provide the latest builds for
   nfs-ganesha any longer.

This commit also cleans up the unnecessary condition in the
ceph script file pertaining to configuring ceph packages for
Jammy Jellyfish. This step wasn't doing anything.
Ubuntu packages don't work at the moment and that requires some more
investigation.

[1] Id2ae61979505de5efb47ce90a2bac8aac2fc5484
[2] https://docs.ceph.com/en/latest/releases/
[3] https://www.spinics.net/lists/ceph-users/msg74312.html
[4] https://download.nfs-ganesha.org/

Change-Id: I40dfecfbbe21b2f4b3e4efd903980b5b146c4202
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
2023-03-14 16:58:03 -07:00
Ghanshyam Mann 863a01b032 Temporary pin the ceph jobs nodeset to Focal
As per the community wide goal to migrate the CI/CD from
Ubuntu Focal to Ubuntu jammy, we need to merge the devstack, tox base
jobs to jammy on Nov 18. But ceph jobs are failing on Ubuntu Jammy.

To move ahead to merge the base job patches we need to pin the
ceph jobs nodeset to Focal until this is fixed for Jammy.

Since the devstack-plugin-ceph-tempest-py3 is currently permanently
failing, although this patch is a no-op, make it non-voting. Also don't
run 100% failing non-voting jobs until they are fixed.

Needed-By: https://review.opendev.org/c/openstack/devstack/+/860795

Related-bug: #1996628
Change-Id: Id2ae61979505de5efb47ce90a2bac8aac2fc5484
2022-11-19 18:36:58 +01:00
Dmitriy Rabotyagov e80282791f Use default Ubuntu Jammy ceph repos
Ubuntu 22.04 does provide Ceph Quincy (17.2.*) out of the box, thus
there's no urgent need to have comunity repos and we can simply rely
on distro-provided ones. We can remove logic once community publish
Ceph packages for Ubuntu Jammy (22.04).

Change-Id: I169971ef77f11ceb01a5db87441051dcb33555f7
2022-11-16 11:08:28 +00:00
Zuul 19db7b46ab Merge "Set snapshot_support to true for cephfsnfstype" 2022-09-30 10:55:07 +00:00
Victoria Martinez de la Cruz 6b73342c9c Set snapshot_support to true for cephfsnfstype
CephFS driver supports snapshots, therefore the snapshot_support
extra spec for the cephfsnfstype we create should be True.

Change-Id: I97b58697f27824a97cfd31ed21d79916b9e270cc
2022-09-26 13:13:09 +00:00
Victoria Martinez de la Cruz 46186d062b Add cephfs_nfs_cluster_id option
As part of change I1826f2970528928a31b32a664013380e38bbd7c9
we added a configuration option to the manila cephfs nfs driver
when deployed using cephadm.

We also need this option set here to get the m-shr service
to pick the right helper when deploying with DevStack

Change-Id: If99714e07f4b75c76db29a660ad8d1e93f7055e5
2022-09-14 17:12:43 +00:00
Victoria Martinez de la Cruz 3c0ce20d00 Use a standard port for NFS
We should be using a standard port for NFS

Most frequently used ports for NFS are 111 and 2049.

Change-Id: I78503c7780feebdc51ab32973b5cb7b8cea30272
2022-08-24 16:41:28 +00:00
Zuul 536335b9f8 Merge "Bump Ceph default version to Quincy in cephadm deploy" 2022-08-17 00:18:04 +00:00
Victoria Martinez de la Cruz 0f018f081b Bump Ceph default version to Quincy in cephadm deploy
Latest Ceph stable version is Quincy. This patch set
sets Quincy as default for the cephadm deploy.
This change won't have an impact in the script being
currently used in our CI.

Change-Id: I2d87ec0e93853cd0852944b30a87f6127f491550
2022-08-10 09:02:45 +00:00
Zuul 956b123b28 Merge "Remove pg_num and pgp_num config from bootstrap_ceph" 2022-08-02 15:22:13 +00:00
Zuul b50fae99db Merge "Set default value for MANILA_CEPH_GANESHA_RADOS_STORE" 2022-08-02 10:48:19 +00:00
Victoria Martinez de la Cruz 22d94f462e Remove pg_num and pgp_num config from bootstrap_ceph
The configuration that we are using by default that
sets pg_num and pgp_num values in bootstrap_ceph
breaks devstack-plugin-ceph when using the latest Ceph version (quincy).

This patch sets removes the config for pg_num and pgp_num,
so we delegate to Ceph the pools autoscaling.

Closes-Bug: #1983107
Change-Id: Iecd949ef2258ae8a6ded596219bb993aeff20de5
2022-07-29 14:01:20 +00:00
jiaqi07 400b1011be Use TOX_CONSTRAINTS_FILE
UPPER_CONSTRAINTS_FILE is old name and deprecated
This allows to use upper-constraints file as more
readable way instead of UPPER_CONSTRAINTS_FILE=<lower-constraints file>.

Change-Id: I71bfcf0576eda9f72d963c4f5d92464c5a04734e
2022-07-26 11:42:19 +08:00
Goutham Pacha Ravi ed3c978f11 Set default value for MANILA_CEPH_GANESHA_RADOS_STORE
It's not a required configuration item; and its not required
with cephadm deployed NFS-Ganesha daemon/s.

Change-Id: I54380f1cb905dfa5ab287ba423561aa75bc1d2f4
Signed-off-by: Goutham Pacha Ravi <gouthampravi@gmail.com>
2022-07-19 01:16:52 +05:30
Zuul f1ad6ff275 Merge "Deploy with cephadm" 2022-06-30 12:14:30 +00:00
Victoria Martinez de la Cruz bba01bdd2f
Deploy with cephadm
Add the option to deploy the Ceph cluster
with the cephadm tool.

Depends-On: I799521f008123b8e42b2021c1c11d374b834bec3
Co-Authored-By: Francesco Pantano <fpantano@redhat.com>

Change-Id: Id2a704b136b9e47b7b88ef586282cb5d0f754cf1
2022-06-27 18:54:15 +02:00
Luigi Toscano e222cc9769 Ceph set minimum client version
This patch adds a new VAR for when the user wants to set the minimum
client version used in a Manila/Glance/Nova/Cinder test job in addition
to using devstack.

A new configuration option, CEPH_MIN_ CLIENT _VERSION, has been
introduced to specify a Ceph-minium client version that allows proper
handling when deleting images and snapshots with dependencies, etc. The
default value is null.

Co-Author-By: Sofia Enriquez <lsofia.enriquez@gmail.com>
Change-Id: Id8e581893ee4b373b268acc7c59b670985cedc2f
2022-06-23 21:57:03 +00:00
Luigi Toscano b0418e177f Disable horizon from basic tempest tests
The parent jobs of the basic tempest jobs defined here enabled
horizon since the merge of tempest-horizon into tempest (see
Id2ced856a41548a0b49e594ee5fed6ed28785f24).

But horizon is not really needed here, as the default list of
tests does not incldue any horizon test. Disabling it helps
freeing up a bit of resources for the job and also helps
the ongoing effort of testing FIPS, which would otherwise
require a patched django at this point.

Change-Id: I05ea7f675aafb85eb54427ca95167b4c8f0eabdf
2022-02-11 20:23:47 +00:00
Abhishek Kekane 8a27b7bdd8 Allow glance ceph osd cinder,nova pool access
Recently cinder has utilized clone v2 support of Ceph for its
RBD backend, since then if you attempt to delete an image from
glance that has a dependent volume, all future uses of that
image will fail in error state. Despite the fact that image
itself is still inside of Ceph/Glance. This issue is reproducible
if you are using ceph client version greater than 'luminous'

To resolve this issue glance RBD driver now checks whether original
image has any dependency before deleting/removing it's snapshot and
returns 409 response if it has any dependency. To check this
dependency glance osd needs 'read' access to cinder and nova
side RBD pool.

This change allows glance keyring/osd a read access on cinder and nova
side RBD pool.

Related-Bug: #1954883
Change-Id: I2e6221e6de23920998bb5f32b2323704b3c89f74
2021-12-20 04:59:38 +00:00
Sean Mooney 4d9f4b2235 remove python3-logutils
this change ensure that python3-logutils is removed so that
that it can be installed by pip later.

Before today in passing build, there were no python3-logutils pulled by neutron deps so
it was not installed or tried to uninstall so all good
- https://zuul.openstack.org/build/590c5996ca1b402486bfe1c7e1d08535/log/job-output.txt

But from today (10th Dec), python3-logutils pulled by neutron deps and failure started
- https://zuul.opendev.org/t/openstack/build/722c6caf8e454849b897a43bcf617dd2/log/job-output.txt#9419

The root cause of why this issue started happening today is not known. May be its
pecan===1.4.1 ? I8ee467bbb363f428a005f92554812bfdae95881a making it install but there
is no change for logutils as deps in pecan previous version 1.3.3 also[1]).  Or it may
be cpeh/ubuntu packaging.

But it is clear that python3-logutils  is coming from somewhere and causing gate blocker
in Nova gate, let's remove it and later we can find the root cause if anyone need this
package for their ceph job.

[1] https://github.com/pecan/pecan/blob/1.3.3/requirements.txt#L5

Closes-bug: #1954427

Change-Id: Icb63649b252fd6eb229adeae454b5ec3c6b79cad
2021-12-10 20:09:14 +00:00
Zuul bb7ea57a58 Merge "Fix Ganesha repository broken urls" 2021-10-18 18:59:31 +00:00
silvacarloss 0e27c6d650 Fix Ganesha repository broken urls
Few URLs from some ganesha repositories were not corresponding
to actual valid URLs, where packages could be searched in
the repository.

These broken URLs have now been fixed.

Change-Id: If27b488cfec29731b74e7db774c4811b0e34c14e
2021-10-15 16:33:43 +00:00
Zuul 81754183c1 Merge "Initialize RBD pool" 2021-10-08 14:23:44 +00:00
Ghanshyam Mann bf07abf7a9 Fix job definition as devstack-plugin-ceph is branched
As devstack-plugin-ceph is branched, we moved stable branch
jobs definition in their respective branch

- https://review.opendev.org/c/openstack/devstack-plugin-ceph/+/786308
- https://review.opendev.org/c/openstack/devstack-plugin-ceph/+/786309
- https://review.opendev.org/c/openstack/devstack-plugin-ceph/+/786310
- https://review.opendev.org/c/openstack/devstack-plugin-ceph/+/786311

But we did not cleanup the master version of job definition so that we
do not need to do it for future stable branches. When stable/xena is cut
from master the ./.zuul.yaml version of jobs in stable/xena has the stable
branches variant which is being picked up by zuul for stable branch gate
instead of their own version defined in their stable branch.

- https://zuul.opendev.org/t/openstack/build/bc088154e50440fe9a2cacf52426748a/log/zuul-info/inventory.yaml#241

And it started failing ceph job when ceph version was update for master and
same is being picked by stable branch testin so due to above branch matching
logic.

To avoid this in future when we will have stable/yoga or so we need to cleanup
this in master. Master repo will only keep the master version of jobs and no
stable branch jobs.

Other stable/rocky and older jobs will be moved to stable/stein so that we have
them there if anyone cut the branch.

Closes-Bug: #1945358
Change-Id: I7061f8d1491ff957452c9c777e40186a4e9c324e
2021-09-28 18:09:27 -05:00
Giulio Fidente ea0c1c768c Use Ceph Pacific for devstack-plugin-ceph-tempest-py3 jobs
Move tempest py3 jobs to latest released stable Ceph, which is
Pacific.

Change-Id: I6cd5424f023d2a895f8ed259d21944490bc1de5f
2021-09-20 11:56:17 +00:00