Commit Graph

1958 Commits

Author SHA1 Message Date
Clark Boylan 9cfd5cca0a Fix test-matrix shebang for ansible 6
Ansible 6 interprets the shebang line to know what version of python to
run a module under. Unfortunately, this is at odds with using the
shebang line normally on a unix system to execte the module as a normal
script. Devstack-gate uses the test-matrix library in both ways as an
Ansible module and as a normal script. To reconcile the differences we
drop the shebang entirely to force Ansible to use a default
interpretation, and on the script side we force users to run the script
with the python they want explicitly.

This is not ideal, but there isn't a better way to reconcile the
difference in Ansible 6 expectations and using things as a normal
script.

Change-Id: I9a331d508276d9abe72c789fd91fc77a4136c5ca
2022-10-11 08:18:44 -07:00
Dr. Jens Harbott 2c891c51d5 [CI] Move queue setting to project level
Per [1]. Stop running the legacy-tempest-neutron-full-stable job since
it is broken. Stop running the devstack job since it no longer includes
the roles from this repo.

[1] http://lists.zuul-ci.org/pipermail/zuul-discuss/2022-May/001801.html

Change-Id: Iebcd055d7e8439ff67723ab2e1b93f20c114112d
2022-10-07 08:50:56 +02:00
Elvira García 3ab1cf3fe4 Remove old Neutron job
The job called neutron-tempest-dvr-ha-multinode-full does not use
devstack-gate any longer.

See https://review.opendev.org/c/openstack/neutron/+/848461 for more
information.

Change-Id: I91b86913007e29b71ff4f4de739ac53c83e5bf58
2022-07-05 18:57:04 +02:00
Ghanshyam Mann 9086c2b017 Use stable constraints for Tempest venv for stable/ussuri|victoria
we do have legacy jobs in ussuri and might be in victoria also and
as we are pinning tempest in both stable branches they need to use the
respective stable branches constraints instead of master constraints.

- https://review.opendev.org/q/topic:ussuri-pin-tempest
- https://review.opendev.org/q/topic:victoria-pin-tempest

Hoping this is last time we are updating the tempest setting in d-g
as all jobs in victoria onwards should be migrated to zuulv3 native.

Change-Id: I07e1cb9b038e8d0c8ce83e8554188f9eb72cb339
2022-05-27 16:01:43 -05:00
yatinkarel fd322f7961 Use stable constraints for Tempest venv for stable/train testing
Without this grenade job in stable/train fails as it uses
master constraints and master is dropping py3.6 support,
like oslo-log already dropped that and is causing issues[2].

Tempest in stable/train is already pinned in devstack[1]
to 26.1.0. So let's use stable constraints for Tempest venv.

[1] https://github.com/openstack/devstack/commit/8a22f738
[2] https://review.opendev.org/c/openstack/neutron/+/841698

Change-Id: I81233ea91cdd24919db245dd8475341d7532daf3
2022-05-25 07:26:24 +00:00
yatinkarel 28382fcf1e Switch legacy-tempest-neutron-full-stable to focal
legacy-tempest-neutron-full-stable deploy's stable/wallaby
and is running on ubuntu-bionic(py3.6) and failing in tempest
setup as tempest master no longer works with py3.6.

Master is dropping py3.6 support, currently oslo.log has already
dropped the support and it's latest release is not installable
on py3.6.

Pin Markusafe, as since version 2.1.0 it dropped
module 'soft_unicode' which is required by Jinja2.

Also update ansible version to 2.9.6 as of focal
which should auto discover python as per distro.

So let's switch this job to ubuntu-focal which is py3.8.

Change-Id: I37887bf430b4bce2b9d2179b2d6dd6c050bb1768
2022-05-25 11:31:28 +05:30
Ghanshyam Mann 83493cc0f8 Update README file for d-g retirement date
In TC PTG, we decided to retire the d-g once
stable/wallaby is in EM state and update it
in README file so that anyone still using it
will know the retirement date.

- https://etherpad.opendev.org/p/tc-zed-ptg#L379

Change-Id: I3e7eed4d260051d72e3ee9d6ed2ce726e85583fe
2022-04-29 19:15:58 -05:00
Clark Boylan 8e3d669ada Pin Ara install deps
The old version of Ara used by d-g doesn't cap off dependencies which
are now no longer working together. We manually cap things off in our
Pip install step.

Note users should stop using this tool. There are alternatives like
zuul-jobs' ensure-devstack role [0] for Zuul/Ansible users. The issue is
that d-g is not very maintainable and on its last legs for the openstack
until the Xena release is no longer maintained. This is noted in the d-g
README.rst file.

[0] https://opendev.org/zuul/zuul-jobs/src/branch/master/roles/ensure-devstack

Story: 2009973
Change-Id: I781ce6e2be88d6724c6908490e17ba8acaa3b911
2022-04-07 08:46:03 -07:00
Dr. Jens Harbott 0596760d94 Cap PyYAML to stay compatible with our code
It seems PyYAML==6.0 breaks the test-matrix code, let's just cap it.

Signed-off-by: Dr. Jens Harbott <harbott@osism.tech>
Change-Id: Id7bc0433315334657f78c0ee29bd100213479939
2021-10-14 09:30:19 +02:00
Zuul a26706bb29 Merge "Retire django-openstack-auth" 2021-08-03 19:30:00 +00:00
Elod Illes 7caf665cf6 Remove neutron-lbaas from PROJECTS list
neutron-lbaas was retired [1] and old branches that still use
devstack-gate started to fail with the following error:

functions.sh:setup_workspace:L461: rsync -a '/home/zuul/src/*/openstack/neutron-lbaas/' neutron-lbaas
rsync: change_dir "/home/zuul/src/*/openstack/neutron-lbaas" failed: No such file or directory (2)

This patch removes the neutron-lbaas from the PROJECTS list to fix the
stable gate jobs that are based on devstack-gate.

[1] I04bf076a051e0bf8333f5dc7c000952048dcb523

Change-Id: Ia62245bbeaf4f1a4b50c07efe14b58b550a9817f
2021-08-03 15:53:41 +02:00
manchandavishal 9e22c31171 Retire django-openstack-auth
``django_openstack_auth`` code was merged into the main horizon
repository during the queens release. This repo is already
deprecated and now we can retire it. This patch removes all
references of ``django_openstack_auth`` as mentioned in [1].

[1] https://docs.openstack.org/project-team-guide/repository.html#retiring-a-repository

Needed-By: https://review.opendev.org/c/openstack/project-config/+/800532/
Change-Id: I556f97e1dded51a6be4f9f9be64ad7eb64d5aca2
2021-07-13 22:55:23 +05:30
Ghanshyam Mann 943c9dbeea Moving IRC network reference to OFTC
Change-Id: I8dd178094c03640db938a258c0c30f4708271af4
2021-07-08 18:36:50 -05:00
Ghanshyam Mann 225e56e8fb Keep only legacy jobs which actually test the d-g code path
Only legacy jobs test the devstack-gate code and all the zuulv3 native jobs
use the devstack only.

Many of the current job running in this gate has been converted to
zuulv3 with same name so we are running zuulv3 native job which does
not use d-g code on d-g gate.

Example: Ibe613a48116aed9cbcc7c689a50c1ef9fba60f05 migrated the neutron
greande job to zuulv3 native.

Also this remove ocata legacy job which is EOL.

To cover the testing for supporting the stable branches in d-g, we have
'legacy-tempest-neutron-full-stable' job running on last supported stable
branch (stable/wallaby - https://review.opendev.org/c/openstack/devstack-gate/+/785003).
Adding it in gate pipeline too to have 'integrated' queue testing.

Also removing the below broken jobs, they need more work to migrate to latest distro
or so but we do not have bandwidth to fix them now.
- legacy-tempest-dsvm-neutron-full-centos-7:
- legacy-tempest-dsvm-multinode-full:

We are not adding any stable branch grenade legacy job here as preparing the
one will be more work. If any issue occur for grenade legacy job side then we can
fix/test it via depends-on.

Change-Id: If7b870331cae42e3250bdf548c572f7067d835f5
2021-06-11 05:35:12 -05:00
Ghanshyam Mann 67487fbffa Add deprecation warning in README file
We are officially deprecating the devstack-gate for
master (Xena onwards) but continue supporting the
old stable branch until stable/wallaby.

Change-Id: I5063de76d531d9b017cbee70a17e6ca945848acc
2021-06-10 09:15:14 -07:00
Clark Boylan 2315ee2e1c Run hooks job in gate
This will ensure we have a test to run when only rst files update.

Change-Id: Ia325732b6ea10d01416121a4e63f6479b8795abc
2021-06-10 09:15:13 -07:00
Ghanshyam Mann 7a9cb8e3c4 Move devstack-gate jobs list to in-tree
It is easy to know all the jobs running on devstack-gate
if they are listed in-tree.

This will also help in deprecation of devstack-gate for master
(Xena onwards) while converting those jobs to run for stable branch.

Depends-On: https://review.opendev.org/c/openstack/project-config/+/795379

Change-Id: Id5262a22fb119447eecb0a77f1d2be2940786226
2021-06-08 11:49:12 -05:00
Zuul 913dd53cd9 Merge "Update grenade setting for stable/wallaby" 2021-04-13 06:27:35 +00:00
Ghanshyam Mann ccdfff263e Update grenade setting for stable/wallaby
There are still grenade legacy jobs in few projects,
so for them we need to update the grenade setting in d-g.

Hope this is last release we need to support in d-g.

Depends-On: https://review.opendev.org/c/openstack/devstack/+/784974
Change-Id: I5f98212be14ac3ebad961595e60a099c58fedada
2021-04-06 11:08:38 -05:00
Ghanshyam Mann f3d3018044 Move stable branch gate job to stable/wallaby
We have stable/wallaby branch as latest stable and
d-g test only latest stable branch to make sure the compatibility
of legacy jobs. Hope this is last stable we need to update/support
in d-g.

This commit moves the legacy stable job testing to stable/wallaby.

Depends-On: https://review.opendev.org/c/openstack/devstack/+/784974
Change-Id: I5542dca0f8050a6e21aff773f49eb4799ec26828
2021-04-06 10:47:40 -05:00
Elod Illes b5edf8508b Set up stable/wallaby
* update the feature list
* add wallaby to allowed stable branch names list

Change-Id: I92942eedaca9b5babeb0e664acd143038e974db1
2021-03-02 05:14:58 +01:00
Ghanshyam Mann d20e76343a Use stable constraints for Tempest venv for stable/stein testing
As stable/stein is in Extended Maintenance state, we use old Tempest
26.0.0 for its testing
- https://review.opendev.org/c/openstack/devstack/+/774103

For old Tempest, we need to use the stable constraints otherwise
it recreate the tempest venv and remove the initial installed tempest
plugins which causes the error while running plugins tests

Example: https://zuul.opendev.org/t/openstack/build/bae42db334ca4b69ab3f4b1c440bcb6a/log/job-output.txt#2444-2454

For Devstack side, I am making the constraints setting more
standardize in below series but for legacy job we need the fix here.
- https://review.opendev.org/q/topic:%22standardize-tempest-tox-constraints%22+(status:open%20OR%20status:merged)

Depends-On: https://review.opendev.org/c/openstack/devstack/+/776811
Depends-On: https://review.opendev.org/c/openstack/tempest/+/774060

Change-Id: I36799ee9d4db20104377e2772c86229b8083448d
2021-02-21 00:12:38 +00:00
Masayuki Igawa b821373c51 Move stable branch gate job to stable/victoria
We have stable/victoria branch as latest stable and
d-g test only latest stable branch to make sure the compatibility
of legacy jobs.

This commit moves the legacy stable job testing to stable/victoria.

Change-Id: I78a0f55c360df2804200cc801109741276248550
2020-10-02 15:47:47 +00:00
Ghanshyam Mann 22b0f318ca Update grenade setting for stable/victoria
Depends-On: https://review.opendev.org/#/c/755333/
Depends-On: https://review.opendev.org/#/c/755230/
Change-Id: I71f9f1018dde422b80693b018c08e23a8dca3d5c
2020-10-02 15:47:36 +00:00
Foundjem 2d240977d7 Set up stable/victoria
Change-Id: I5b1fc4e7ea3cae6792966e9b351ec8d358092683
Signed-off-by: Foundjem <foundjem@ieee.org>
2020-10-02 00:23:44 +00:00
Ghanshyam Mann 9f8a6d546e Run bashate job on bionic nodeset
bashate job on d-g fail with error, as d-g
is moving towards deprecated phase let's keep
bashate job running on bionic.

Change-Id: Ibe644c9ed207bf2f76895319774d7376e08391e3
2020-10-01 19:21:48 -05:00
Ghanshyam Mann 7a70f559c5 Always assume glance is uwsgi mode for grenade
Since older devstack versions do not support glance
in standalone mode, make grenade configure the base
and target for glance deployed with uwsgi.

It is failing in legacy grenade jobs[1], for devstack based
jobs it is taken care in https://review.opendev.org/#/c/741479/2

[1] http://lists.openstack.org/pipermail/openstack-discuss/2020-July/016006.html

Change-Id: Ic84b2e5a2f3677156f6ddb523b9dc91abeda992b
2020-07-20 16:53:30 +00:00
Ian Wienand ba0dea73c3 Ansible 2.9 fixes; fix match filter and sudo flags
As of Ansible 2.9, using a jinja test as a filter fails; switch it to
"is" format.

The "--sudo" flag has become "--become", switch it.

Change-Id: If75cb934d386ab25240df9f21f9c948843bdb8fb
2020-06-17 05:57:31 +10:00
Zuul 4ecb62ca55 Merge "Convert testmatrix run to venv" 2020-05-06 01:41:22 +00:00
Zuul 582cd5df4c Merge "Move stable branch gate job to stable/ussuri" 2020-04-29 00:37:20 +00:00
Ghanshyam Mann b08dd82400 Update grenade setting for stable/ussuri
Depends-On: https://review.opendev.org/#/c/723599/
Depends-On: https://review.opendev.org/#/c/723605/

Change-Id: Ib593eaa5b356b2779d3e8abfbe7536d18e52732e
2020-04-27 17:23:25 +00:00
Ghanshyam Mann 1f7fd5e0d5 Move stable branch gate job to stable/ussuri
We have stable/ussuri branch as latest stable and
d-g test only latest stable branch to make sure the compatibility
of legacy jobs.

This commit moves the legacy stable job testing to stable/ussuri.

Depends-On: https://review.opendev.org/#/c/723605/
Change-Id: Id57e5590a5b24c2d821381f52e1c445cbbd1105b
2020-04-27 12:20:14 -05:00
Andreas Jaeger c9a0437ea6 Set TOX_CONSTRAINTS_FILE
UPPER_CONSTRAINTS_FILE is deprecatedi[1], set the newer
TOX_CONSTRAINTS_FILE as well when interacting with other repos for the
transition.

[1] https://zuul-ci.org/docs/zuul-jobs/python-roles.html#rolevar-tox.tox_constraints_file

Change-Id: Ib02819e805d216ead178300df0b250f85f9f27b7
2020-04-21 21:51:52 +02:00
Elod Illes 184a23bb30 Set up stable/ussuri
* update the feature list
* add ussuri to allowed stable branch names list

Change-Id: Ia1de95b01da1007d6eef1f95cfba8fed15ffef2e
2020-04-16 23:46:00 +02:00
Ian Wienand b365a67c7e Convert testmatrix run to venv
As part of our efforts to ship bare nodes, we want to install as
little as possible in any sort of pre-run jobs.  Since all infra nodes
have python3 available, use venv to create this small environment for
the test matrix production, and avoid having to pull in any other
dependencies.

Change-Id: Ie8b09ab60841f9d1b15ae1a7afec51906cd89f95
2020-03-12 11:03:14 +11:00
Ian Wienand 7974a61f83 Fix virtualenv detection
This checks if the task failed; which it never does because of the
failed_when: false.  What we want to check is the return value of that
check.

Change-Id: Ic975b53d60f44e8e743bcc31fc864cc8416b3a92
2020-03-12 10:45:40 +11:00
Zuul 4985fd75b4 Merge "Do not gzip files under logs in job results" 2020-02-19 01:13:40 +00:00
Zuul 4d64b8529d Merge "Just use the pip from the virtualenv" 2020-02-13 13:53:15 +00:00
Elod Illes 42bc6f1e03 Do not gzip files under logs in job results
In zuul job results the 'logs' directory contains gzip'd files which are
now unbrowsable. This happened due to changes in swift behavior [1].

This patch removes unnecessary compression of files under logs
directory.

[1] http://lists.openstack.org/pipermail/openstack-discuss/2020-January/011875.html

Change-Id: I1d0e7869d418612bb9a20fdf01da2641693ed8bb
2020-02-13 10:51:16 +01:00
Monty Taylor baa4cd731e Just use the pip from the virtualenv
There is only one python, so there is only one pip.

Change-Id: I31bc5a7bad603f3f58a3d2ac1501271935830cdd
2020-02-12 16:48:15 -06:00
Ghanshyam Mann ecf9df2e93 Cap virtualenv to a version < 20
There are errors with the recently released
virtualenv 20 for example: https://github.com/pypa/virtualenv/issues/1551

It has been capped on devstack side also
I3b447557f013934719aa357efeffb093c7f6bd95

Let's use the lower version and we can move to
latest once issues are fixed.

Change-Id: I341437856c709e1e0e281357b96901ae6ef7cd00
2020-02-12 18:03:13 +00:00
Ghanshyam 51a074c302 Use stable constraints in Tempest tox till stable/rocky
Stable branches till stable/rocky is using python
version <py3.6. Tempest test those branch in venv
but Tempest tox use the master upper-constraint[1]
which block installation due to dependencies
require >=py3.6. For exmaple, oslo.concurrency 4.0.0
is not compatible for <py3.6.

As we pin Tempest for EM stable brach, we should be
able to use stable constraint for Tempest installation
as well as while running during run-tempest playbook.

tox.ini is hard coded to use master constraint[1] which force
legacy ocata job to use the master constraints and fail
- https://review.opendev.org/#/c/706161/

Devstack can set stable u-c to use via env var but this is
not enough as Tempest recreate the tox env with master u-c
when run test via devstack-gate.

All other branches have been fixed by fixing the run-tempest playbooks
in same way.
- https://review.opendev.org/#/c/705870/

Because stable/ocata and grenade job use devstack-gate, we need to fix this.

This is required to fix the grenade jobs on each stable branch to
get the other fixes in.

[1] bc9fe8eca8/tox.ini (L14)

Change-Id: Ibfc5c3d5e64e274fd674e6c42878d98bb194fa73
2020-02-07 14:51:27 +00:00
Dr. Jens Harbott a9b0d5ddb1 Drop peakmem_tracker service
When devstack reverted the pip cap in [0], it was necessary to disable
the peakmem_tracker service as there was no easy way to make it work in
that setup. So we need to delete it from the test-matrix here, too.

We could decide to add it only for older stable/branches, but as ds-gate
is to be obsoleted anyway, I'm not convinced that that is worth the
effort.

[0] https://review.opendev.org/561597

Change-Id: I9dd57d3bdb56c64377efbdd55429bd075223c746
2020-01-22 13:31:41 +00:00
Zuul 8342c061cd Merge "nova: Enable n-cauth on stable/rocky for cells v1 jobs" 2020-01-15 11:03:07 +00:00
Eyal 579487f20b Use python3 for virtualenv if exists
Depends-On: https://review.opendev.org/#/c/702162/
Change-Id: Ib07ea45bf0a6e202751f6ddf7fcb159da310f5c8
2020-01-14 12:29:08 +02:00
Lee Yarwood 988cc19138 nova: Enable n-cauth on stable/rocky for cells v1 jobs
Ia85ea5d338f07cb248f60de44b21826862d09d26 had enabled n-cauth again
correctly on branches prior to Rocky but missed that while deprecated it
is still required on stable/rocky for cells v1 jobs:

https://docs.openstack.org/nova/rocky/configuration/config.html#workarounds.enable_consoleauth

Change-Id: I74fd0ba8d2fb1ed25210576fabe957167dc6ca90
2020-01-11 16:12:05 +00:00
Lee Yarwood 1b5f905c13 nova: Renable n-net on stable/rocky|queens|pike|ocata
Following on from Ia85ea5d338f07cb248f60de44b21826862d09d26 n-net also
needs to be renabled on these older branches to allow cellsv1 testing.

Change-Id: I300a93142355517382c8ba55105ac7c1e88f29c8
2020-01-10 13:47:11 +00:00
Ghanshyam Mann cdb438afb0 Re-enable the n-cauth required service for stable/queens|pike|ocata
n-cauth serviec has been remvoed from nvoa in Ussuri and that is why
it was removed from default services[1]. But that is requried
for stable/queens|pike|ocata which still depends on text-matrix
for ENABLED_SERVICES. Since stable/rocky we have migrated the
service enable in devstack base job[2].

This commit fix the Tempest master and stable/queens|pike|ocata
gate by re-enable the n-cauth service for listed stable branches
only. Later we can discuss to remove the test-matrix dependency for
these branches also.

Related-Bug: 1858666
[1] https://review.opendev.org/#/c/700217/
[2] https://review.opendev.org/#/c/546765/

Change-Id: Ia85ea5d338f07cb248f60de44b21826862d09d26
2020-01-07 17:07:01 +00:00
Matt Riedemann 183a58eab1 Add a feature support matrix test for grenade subnode
test-features.sh wasn't testing anything for subnodes
so this adds a subnode feature support matrix test for
grenade. It also highlights that the grenade config
is applied to all nodes regardless of role so a TODO
is added to test_matrix.py in case someone needs to make
that per-role, per-config service override support at
some point (hopefully that never really happens and all
jobs are moved to zuulv3 by that point).

Change-Id: I485d76c1db5c3a4363c3e26fea204cc7782887f3
2019-12-20 15:38:13 -05:00
Matt Riedemann cab3fb81ab Disable c-bak and etcd3 in grenade jobs
Grenade only runs tempest smoke tests and there are no
cinder-backup service related smoke tests in the main
tempest repo so running the c-bak service on the primary
and subnode (in multinode jobs) is just a waste of resources
since it's not used in testing.

Similarly, etcd3 is really only interesting for cinder in
a multinode job but in grenade multinode jobs etcd3 is not
enabled on the subnode, so only the primary node cinder
services are configured to use etcd3, so again it's a waste
of resources.

There is a theory that bug 1844929 is failing primarily on
OVH nodes because they aggressively restrict iops so we're
running into swap issues [1]. Freeing up more resources on
the nodes used in grenade jobs might alleviate some of that
swap pressure.

To do this using the feature test matrix in devstack-gate
the test_matrix.py script has to be updated such that
services can be added/removed per config rather than per
feature because otherwise grenade says it wants cinder but
cannot remove cinder-specific services like c-bak.

[1] http://lists.openstack.org/pipermail/openstack-discuss/2019-November/010506.html

Change-Id: Ide146793053ae6b62f93a5af45c6130a21514e87
Related-Bug: #1844929
2019-12-20 15:27:17 -05:00