Commit Graph

841 Commits

Author SHA1 Message Date
Sean Mooney 2523aa0b41 update grenade-base to use 8G of swap
we have notice failures in the grenade jobs
corralated with OOM events. grenade-base
previously used the default value of 1G
for configure_swap_size: but now its updated
to configure_swap_size: 8192

most devstack jobs have already been changed
to either 4096 or 8192. all jobs that inherit
form devstack-tempest use 4096 where as more
complex jobs such as those that have ceph use
8192. as grenade is a non trivial job this patch
uses 8192 MB

Change-Id: Ibf6dfe84f15019d0360e2c2042856aeecb22209d
2024-03-13 19:55:06 +00:00
melanie witt f1980a682b Set NOVA_VNC_ENABLED: true for subnode in the grenade job
Currently, because we don't run the n-novnc (console proxy) service on
the subnode, devstack won't enable VNC for guests hosted on the
subnode.

We see occasional failures in the gate when VNC tempest tests attempt
to obtain a VNC console for a guest that happened to get scheduled to
the subnode with the following error:

  Details: {'code': 400, 'message': 'Unavailable console type novnc.'}

This sets NOVA_VNC_ENABLED = true for the subnode only, with the
thinking that the enabled n-novnc service is sufficient for the
controller node.

Jobs that derive from the "devstack" job [1] are already setting
NOVA_VNC_ENABLED: true for both the controller node and subnode.

[1] 3ffcc89d5d/.zuul.yaml (L535)

Change-Id: I1acbdb605e0e0d9e5b4604fcd21af850b06823b9
2023-11-06 23:44:17 +00:00
Dan Smith 36c8d242f8 Increase cinder verify sync timeout
This is occasionally failing with a timeout. It has been for months,
but it seems like the frequency has increased lately.

Change-Id: Ib1270e4f5bada8680a5d19133a888a8ade8f73c3
Closes-Bug: #2039707
2023-10-19 11:41:31 -07:00
Ghanshyam Mann 0f0d9dcb9e Make octavia-grenade job voting
octavia-grenade job is fixed and passing now,
so making it back to voting.

Change-Id: I33f878dd0da28fd3985347f2ed7ef6b70cd5acc9
2023-10-05 08:46:59 +00:00
Ghanshyam Mann 159054a927 Update grenade setting on current master
grenade is branched for stable/2023.2, so updating
the current master default setting for:
- Update grenade testing from stable/2023.2 to master.
- Update grenade-skip-level and grenade-skip-level-always
  jobs to upgrade from stable/2023.1 -> master

Depends-On: https://review.opendev.org/c/openstack/grenade/+/897381
Change-Id: I1dbea0caad676d6be7a4a58495a76a351e3c41e5
2023-10-05 05:44:21 +00:00
Martin Kopec 5d8b7569ee Use re2 compatible regexes in branch matchers
Python regexes are deprecated in Zuul, so update the use of regexes
in branch matchers with re2-compatible syntax.
The commit also removes matches for branches that are EOL - all
branches until stable/rocky (including).

Change-Id: I43c696ef2029cbf351f902b293df05f296dbb2a2
2023-09-20 15:38:19 +02:00
Zuul 67858b98bd Merge "Disable using GLOBAL_VENV for now" 2023-07-01 01:58:05 +00:00
Dr. Jens Harbott 3e5b00403b Disable using GLOBAL_VENV for now
We don't want to go through the hassle of changing existing deployments
from global pip installs to using a virtual environment, so disable this
feature in grenade jobs for now until we can use it on the old side,
too.

Needed-for: https://review.opendev.org/558930
Change-Id: I5e5a67cceade05a4a32ddd1b86802f9496d1bb79
2023-04-13 07:59:51 +02:00
Julia Kreger d8294440e7 [ironic] Add an instance wait setting for nova resources
In troubleshooting grenade failures in ironic,
we've observed a sporatic failure causing the
ironic-grenade job to fail upwards of 20-40%
of the time where it seems cirros, due to the
boot sequencing and interfaces, is not
online for networking until after sixty seconds
have passed. In one case 83 seconds before the
networking was fully online.

In consulting with neutron contributors, a random
job check seems to reveal that even with pure VM
workloads, it takes on average 35 seconds into the
ping check. As such, it seems reasonable to make
the setting configurable so ironic-grenade can
execute with an increased timeout more appropriate
to the job settings and test environment.

Adds a INSTANCE_WAIT variable, which defaults to
the prior setting which was static, 60 seconds.

Change-Id: Ie5acf1ad7f8dca39db07f7e61035a8916439004d
2023-04-05 10:56:41 -07:00
Ghanshyam Mann 4c9c059883 Fix tempest venv constraints for target branch tempest run
We fixed the tempest venv constraints setting for tempest test run
on base branch but forgot to do the same for target branch test run.
It cause issue when base and target branch have different Tempest
and constraints to use (for example stable/wallaby use old tmepest
and stable/wallaby constraints but stable/xena use tempest master
and master constraints), in such cases we need to set proper constraints
defined in devstack and then run tempest.

We do not see this issue as none of the job run tempest on target except
heat (https://review.opendev.org/c/openstack/heat/+/872055/5).

I reproduced the issue in grenade stable/xena by enaling the tempest run
on target:
- https://review.opendev.org/c/openstack/grenade/+/878247/1
- https://zuul.opendev.org/t/openstack/build/1b503d359717459c9c77010608068e27/log/controller/logs/grenade.sh_log.txt#17184

Closes-Bug: 2003993
Change-Id: I05fcdb5045a33997bd1a4c340a16458d88399f5f
2023-03-30 20:58:04 -05:00
Ghanshyam Mann 1ce132fdaf Update grenade setting on current master
grenade is branched for stable/2023.1, so updating
the current master default setting for:
- Update grenade testing from stable/2023.1 to master.
- Update grenade-skip-level job not to run for current non SLURP master
release

octavia-grenade job is broken due to diskimage-builder so to unblock
grenade gate, stop running it and we can run it once DIB fix is merged
and released
- https://review.opendev.org/c/openstack/diskimage-builder/+/878089

Change-Id: I400c7f386e5b900774039223c9199508229494a2
2023-03-21 11:06:26 -05:00
Zuul a8cbe6d1a9 Merge "Fix setting the tempest virtual env constraints env var" 2023-03-06 17:45:40 +00:00
Dan Smith 3877df6af1 Factor out a grenade-skip-level-always job
For projects that want to opt-in to testing N-2 upgrades on all
releases (for early warning of issues, best practices, etc), this
pulls out a grenade-skip-level-always job that can track N-2 regardless
of whether master or N-2 is a SLURP release. The regular
grenade-skip-level job can inherit from that, with the branch
restriction and most recent SLURP as the from_branch.

Note that this makes the specific job a subset of the generic for
easier application of config to both. Right now they use different
nodeset rules, but in the next release they will be the same, so it
seems clearer to make it a hierarchy.

Change-Id: I39d1cbc4a2633c9908ba0e25205fb2f6a4aa729e
2023-03-01 14:37:15 -08:00
Dan Smith 8fdda79453 Generate SSH key locally for cinder resources
Nova has deprecated the server-side keypair generation process, which
is used by cinder's resource create phase in grenade. Even though it
still works for old microversions, it generates RSA keys which will
not be accepted (by default) in newer distros. This makes us generate
the keypair locally and upgrade the public key instead of generating
it server-side.

Change-Id: I0275a44f2ec4977b085b8129fd06e80a0ed6e68d
2023-03-01 11:17:00 -08:00
yatinkarel 3f9fe2e8fc Bump ping timeout to 60 seconds
The jobs fails intermittently on ping check
with current timeout of 30 seconds.

The testvm can take time to boot and have network
ready on slow systems and using qemu hypervisor
and lead to ping timeout.

With [1] included noticed it taking more than 40 seconds
to boot testvm, so bumping timeout to 60 seconds will
help in such cases.

[1] https://review.opendev.org/c/openstack/grenade/+/874417

Related-Bug: #1463631
Related-Bug: #2007357
Change-Id: Ibdd030e126d508e6ff24cde180c611ada7f24cb3
2023-02-23 12:39:48 +05:30
yatinkarel adcb563b18 Dump Console log if ping fails
Console log will be useful in debugging
ping check fails.

Related-Bug: #2007357
Change-Id: I0f1603154eac85353aeb3a0c42da83c268174b05
2023-02-21 19:43:10 +05:30
yatinkarel 79a138f866 Don't enable the dstat service in CI jobs
It's already disabled for devstack jobs with
[1]. Let's do for grenade too as those too
seeing this issue.

[1] https://review.opendev.org/q/I32ef8038e21c818623db9389588b3c6d3f98dcad

Related-Bug: #1943184
Change-Id: I72db099726ec6f737bc8eccd94a44fd715552728
2023-02-14 21:37:14 +05:30
Ghanshyam Mann 01e143de49 Fix setting the tempest virtual env constraints env var
Grenade set the env var TOX_CONSTRAINTS_FILE/UPPER_CONSTRAINTS_FILE
which are used to use the constraints during Tempest virtual env
installation. Those env var are set to non-master constraint when
we need to use non-master constraints but when we need to use the
master constraints we do not set/reset them point to master constraints.

This create the issue when grenade script install the tempest second time
on new devstack where we need to use master constraints but env var
TOX_CONSTRAINTS_FILE/UPPER_CONSTRAINTS_FILE set previously in tempest
installation in old devstack point to the stable constraints.

One case where it started failing when we tried to pin the stable/wallaby with
Tempest 29.0.0
- https://review.opendev.org/c/openstack/devstack/+/871782

and stable/xena grenade job failed (stable/xena use master tempest and supposed
to use master constraints)
- https://zuul.opendev.org/t/openstack/build/fb7b2a8b562c42bab4c741819f5e9732/log/controller/logs/grenade.sh_log.txt#16641

We should set/reset those constraint env var to master constraints if configuration
tell to use the master constraints.

Closes-Bug: #2003993
Change-Id: I8c623008292f3b2dd2cd6d596feea552194acd89
2023-01-27 00:50:29 -06:00
Zuul c7a6dff51d Merge "Remove lib/neutron-legacy leftovers" 2023-01-04 17:29:49 +00:00
Zuul 3d9486a1ab Merge "Update the rootwrap config when upgrading Neutron" 2022-12-22 11:47:51 +00:00
Rodolfo Alonso Hernandez 7c3144ce84 Update the rootwrap config when upgrading Neutron
As reported in the related bug, the rootwrap filters are not updated
between versions. This patch calls the devstack method that setups
the Neutron rootwrap configuration and filters.

Closes-Bug: #1999235
Change-Id: Iaebf1b33ccf3bfd64191f9a898408bcfe11dd557
2022-12-21 17:10:42 +00:00
Sławek Kapłoński 94814b7425 Remove lib/neutron-legacy leftovers
In [1] we finally got rid of the unfinished lib/neutron module and kept
only lib/neutron-legacy. It's renamed to lib/neutron now and it's the
only neutron related module in Devstack.
So this patch removes leftovers related to the old lib/neutron-legacy.

[1] https://review.opendev.org/c/openstack/devstack/+/865014

Change-Id: I2a856b15eda992f0e78ee8eff65f39646e24c936
2022-12-20 11:15:47 +01:00
Slawek Kaplonski 0dd1e186b5 Use default neutron backend in the base grenade job
When devstack was switching to ML2/OVN as default backend,
in the base grenade job it was explicitly configured to use old default
ML2/OVS backend. It was done like that to avoid problems with upgrade
and change backend in the same job.
Now, as ML2/OVN is default backend in Devstack since at least
couple of cycles it's fine to switch grenade jobs to use default
backend.

Depends-On: https://review.opendev.org/c/openstack/ironic/+/866993
Depends-On: https://review.opendev.org/c/openstack/devstack/+/867065

Change-Id: Iede9fe71d81fc86a92122800d5a16f45442dd79e
2022-12-09 11:48:00 +00:00
Martin Kopec b1cd7a1866 Update the master grenade setting for zed to master upgrade
Change-Id: Ic7b6a33e674fc9e292a85de6d818b8adad784baf
2022-09-28 17:26:52 +00:00
Slawek Kaplonski 793f553bdd Define NEUTRON_ENDPOINT_SERVICE_NAME variable as empty string
This new Devstack's variable is introduced by patch [1] but as it's not
in the stable branches of Devstack yet, it can't be used in Grenade jobs
with default setting.
So lets set it to the emtpy string to make it working as it was before
[1].

[1] https://review.opendev.org/c/openstack/devstack/+/849145

Change-Id: I2de65d4764f23ae85a086d4d09ed80c81777434b
2022-07-19 12:30:53 +02:00
Zuul a3e31e3d69 Merge "docs: fix typo in README" 2022-06-16 09:08:14 +00:00
yatinkarel b4c6851ceb Skip stop of ovn services
Grenade jobs stop services, check fip connectivity
for a nova server and then upgrade to next release.

But since ovn data plane and db services are stopped along
with other services, fip connectivity fails as a result.

We shouldn't stop these services along with other
neutron services. This patch sets "SKIP_STOP_OVN" to True
to skip stop of ovn services.

This fixes ovn grenade jobs.

Depends-On: https://review.opendev.org/c/openstack/devstack/+/839362
Change-Id: I2bd3f7e5f0af9a6532db7a1cdb4bc45a963042ca
2022-06-02 10:08:06 +05:30
Ghanshyam Mann fc8b623145 Do not run grenade-skip-level job on older stable branches
grenade-skip-level is added and supposed to be run from Yoga
release. We have added this job in integrated template which
is in branchless tempest side and applicable for stable branches
also.

We can control to run this job on Yoga and SLURP release onwards but
that leave if anyone running it directly without template So controlling
the branches in the job definition will work for everyone.

Change-Id: Ie560b25f83c3f102f408077f5023f07facf6095d
2022-05-20 14:20:54 -05:00
Zuul bdc4d7388e Merge "Disable auto discovery" 2022-04-25 20:04:20 +00:00
Brian Rosmaita 53056fc212 docs: fix typo in README
Change-Id: I642ca62018d2133eb3048a91ab2fa605b999c24a
2022-04-25 13:43:33 -04:00
Martin Kopec b960348631 Update the master grenade setting for yoga to master upgrade
Change-Id: I076ee646ca0e017748a0eeba715e5604ec6b68eb
2022-03-29 04:36:03 +00:00
Ghanshyam Mann e10ac41b71 Remove sourcing of userrc_early
Devstack has stopped creating the userrc_early in yoga
and also fixed the start_swift function to stopm relying
on userrc_early.

- https://review.opendev.org/c/openstack/devstack/+/780417

It start failing when grenade started upgrade testing from
stable/yoga to master.

- https://review.opendev.org/c/openstack/grenade/+/835232

Change-Id: I6caaac9d84e4f3925378f496acd06fc38bb0d2d8
2022-03-28 22:46:11 -05:00
Jiri Podivin 3114725b50 Disable auto discovery
Tripleo-ci jobs are broken after latest release of setuptools 61.0
because of breaking changes which are not backwork compatible,
details in related bug and [1].

Users that don't set ``packages``, ``py_modules``, or `configuration`
are still likely to observe the auto-discovery behavior, which may
halt the build if the project contains multiple directories and/or
multiple Python files directly under the project root.

To disable auto discovery, one can do below in setup.py

~~~
setuptools.setup(..,packages=[],..)
~~~

or

~~~
setuptools.setup(..,py_modules=[],..)
~~~

[1] https://github.com/pypa/setuptools/issues/3197

Releated-Bug: #1966382

Signed-off-by: Jiri Podivin <jpodivin@redhat.com>
Change-Id: I8ba2f6eccb9f279406321915793709e50e8279d4
2022-03-28 12:39:49 +02:00
Ghanshyam Mann 68aa50b183 Do not run integration test job for gitreview file change
Change-Id: I3bb398a5853cfd732ad150314cc0eac4440f0efe
2022-03-25 09:29:15 -05:00
Dan Smith 9b223d3e22 Add grenade-skip-level job
This adds a grenade job that specifically skips the last release,
to prove that we can do it. Such a job would facilitate a new
goal of a tick/tock release arrangement where slower-moving operators
could stick on the "tick" release and know that they can upgrade to
the next "tick" release directly.

Currently this tests from wallaby to yoga (master) and is added as
non-voting until the TC resolution passes (if it does).

Change-Id: I5a8fcc1097956ebbbe7b5ea0bed0889eeeca1d80
2022-02-21 14:14:51 -08:00
Dan Smith bf235cd904 Enable nova workaround for upgrade bug
Nova recently introduced an issue that breaks FFU due to requiring
new compute service records to be recorded before control services are
allowed to start. However the former depends on the latter, which
creates an unsatisfiable circular dependency. Since this has been
both released and backported, Nova introduced a workaround config flag
specifically to get out of that loop, which we set here if needed.

Related-Bug: #1958883
Change-Id: If4820dd4ec7e05ade6eb4a82a328797102704570
Depends-On: https://review.opendev.org/c/openstack/nova/+/826097
2022-02-21 14:14:46 -08:00
Lee Yarwood 6aae2a9cfa zuul: Add the neutron devstack plugin and enable neutron-trunk service
Depends-On: https://review.opendev.org/c/openstack/devstack/+/811303

Closes-Bug: #1945346
Change-Id: I3326ef7ea5f7340c8dc1e8c21fa977bc1b0ddba6
2021-09-28 20:01:08 +00:00
Ghanshyam Mann e830e1d708 Update the master grenade setting for xena to master upgrade
Change-Id: I614031450d7ac179826d3876d39323a1dd8a35f1
2021-09-24 15:04:05 -05:00
Rajat Dhasmana d61a95325b Unblock Gate: create encryption type with right params
Recently we added a check in cinder for cipher algorithm[1]
Currently grenade is not passing cipher when creating an
encrypted type, hence grenade job is failing in glance and cinder gate.
This patch addresses the same by passing the cipher and other
properties.

[1] https://review.opendev.org/c/openstack/cinder/+/800268

Closes-Bug: #1938763

Change-Id: I1eb352deca11916f4db1010a50d15a83e33a61bb
2021-08-03 08:01:08 +00:00
Zuul f0215b0319 Merge "Update IRC network to OFTC" 2021-05-31 23:40:58 +00:00
Ghanshyam Mann 731940d4d4 Update IRC network to OFTC
Change-Id: I13b5772a0d04278b26afcc5ea07ae51a8333d557
2021-05-31 15:05:17 -05:00
Brian Rosmaita e3cf221066 Use Block Storage API v3
The openstackclient has defaulted to using the v3 API since 4.0.0 [0],
so there is no reason for grenade to insist on using v2, especially since
v2 is being removed in Xena [1].

[0] a96089ff6d
[1] https://review.opendev.org/c/openstack/cinder/+/792299

Change-Id: Ie558d45374f92f79942b80a9c27fd393ad4f4761
2021-05-20 09:50:55 -04:00
yangyawei e816fafffc setup.cfg: Replace dashes with underscores
Setuptools v54.1.0 introduces a warning that the use of dash-separated
options in 'setup.cfg' will not be supported in a future version [1].
Get ahead of the issue by replacing the dashes with underscores. Without
this, we see 'UserWarning' messages like the following on new enough
versions of setuptools:

  UserWarning: Usage of dash-separated 'description-file' will not be
  supported in future versions. Please use the underscore name
  'description_file' instead

[1] https://github.com/pypa/setuptools/commit/a2e9ae4cb

Change-Id: Id782291f7892b93d8452c836bfdd7f47f8e940ef
2021-05-14 15:55:45 +08:00
Anand Bhat c7f6f5d1fa Changed minversion in tox to 3.18.0
The patch bumps min version of tox to 3.18.0 python in order to
replace tox's whitelist_externals by allowlist_externals option:
https://github.com/tox-dev/tox/blob/master/docs/changelog.rst#v3180-2020-07-23

Change-Id: Icea0fbbd3e389800d7fcfdd6ecd11f3b05b2a892
2021-05-06 20:02:50 +05:30
Zuul 703926636d Merge "Remove the old release upgrade extra scripts" 2021-05-05 22:31:08 +00:00
Ghanshyam Mann f94c0ae3e2 Remove the old release upgrade extra scripts
Grenade does not support the fast forward upgrade so old release
wise script are not in use in current grenade support.

from-* script in current grenade will only be run via from-$BASE_DEVSTACK_BRANCH
- 1a1f58a69e/inc/upgrade (L92)

and BASE_DEVSTACK_BRANCH will be wallaby for current master grenade or
immediate previous release which means we do not need to keep the older
releases upgrade extra scripts.

Ditto for within-* script.

Change-Id: I66b021f03faa38f976243b7f194454360def215f
2021-05-04 11:46:46 -05:00
Martin Kopec af1b4bf7d6 Remove neutron workaround for mitaka and newton
Remove condition for checking if BASE_DEVSTACK_DIR/lib/neutron
exists which was introduced ~5 years ago.
Based on the comment it looks like it can be deleted.

Change-Id: Ice9a709ad75bb5794aebed379fb35f78f0b3422f
2021-04-27 15:07:20 +00:00
Radosław Piliszek 1a1f58a69e Do not suggest Gerrit as source of clones
Make infra-root happy.

Change-Id: Id2b87753236dcc055c8cd9e42e23cf5d3faee4c0
2021-04-27 06:58:19 +00:00
Anand Bhat 41364698b2 References to review.openstack.org are replaced with review.opendev.org
Closes-Bug: #1832367
Change-Id: If38096aedac5c528a4feb5f70992afa4e13d6f67
2021-04-26 19:58:41 +00:00
Zuul a1b98d6d7a Merge "Make heat and octavia grenade jobs as voting" 2021-04-13 04:10:10 +00:00