Commit Graph

394 Commits

Author SHA1 Message Date
Clark Boylan 4195dcf119 Revert "Remove rax from zuul log upload config"
This reverts commit 9d25a857da.

We revert this as our testing shows the MFA updates to our regular
accounts don't seem to affect swift uploads currently. That may change
when MFA is required across the board on the 26th but we can reapply
this sort of disablement and testing then and in the meantime continue
to have two providers of log upload locations.

Change-Id: I9f24f67253934a6a128a6cee3cceb9c1f0bcdf37
2024-03-20 11:55:18 -07:00
Clark Boylan 9d25a857da Remove rax from zuul log upload config
Rax is requiring everyone to use multi factor auth by March 26, 2024.
We're currently transitioning to MFA early to control when and how it
happens. One benefit of doing this early is we can pull rax out of the
log upload destination list to enable us to test it still works after
the MFA switch.

Do this by removing Rax from the prod uploads and removing ovh from the
test uploads so that only rax is used in base-test. After we update to
MFA we can check that uploads for base-test still work then revert this
change.

Change-Id: I8dafb5ea7ad6b10989ca6258c3f56bc8b91d0e06
2024-03-20 09:53:42 -07:00
Clark Boylan 578252a1c6 Remove centos-7 nodeset
We are cleaing up the centos-7 nodes as it is near EOL and this will
free up space on nodepool builders and our mirrors for other newer node
types. Land this when we are happy sufficient cleanup has been performed
within our Zuul tenants.

Change-Id: I8e7bf2776679647ffdfc6a3f4dedbf69f62b91fb
2024-03-13 08:22:34 -07:00
Clark Boylan 5db056ec44 Drop debian-buster nodesets
We are removing this node type and image from Nodepool and our mirrors
entirely. Buster has been replaced by bullseye and bookworm and we don't
need to keep an old largely unused debian release around.

Depends-On: https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/910008
Depends-On: https://review.opendev.org/c/openstack/project-config/+/910011
Depends-On: https://review.opendev.org/c/opendev/system-config/+/909787
Depends-On: https://review.opendev.org/c/zuul/zuul-jobs/+/909786
Change-Id: Id15101787af3284de352e7d3887218010f0bf80a
2024-02-23 08:44:21 -08:00
Clark Boylan 10243087b7 Drop the opensuse-15 nodeset
We are dropping support for opensuse leap 15 test environments because
our current leap 15 env is based on 15.2 which is quite ancient at this
point. Additionally leap as a distro appears to be going away and will
be replaced with some other dev type distro in that ecosystem.

Note the depends on below are a subset of the removals from repos but
they are the ones that I think we should get done before we land this
change.

Depends-On: https://review.opendev.org/c/zuul/zuul-jobs/+/909772
Depends-On: https://review.opendev.org/c/opendev/bindep/+/909771
Depends-On: https://review.opendev.org/c/openstack/openstack-zuul-jobs/+/909770
Depends-On: https://review.opendev.org/c/openstack/project-config/+/909775
Change-Id: Icda87007f38f4aa9f53010c45fdd070ed646165d
2024-02-21 09:33:29 -08:00
James E. Blair ddb3137d87 Revert "Temporarily disable uploads to ovh_gra"
This restores our standard log upload config now that the outage
in ovh is resolved.

This reverts commit babd74eb4f.
This reverts commit 7e033243f3.

Change-Id: Ib308489085a6ad2ace501e4ae078606634eda50a
2023-12-11 09:10:56 -08:00
James E. Blair babd74eb4f Force base-test to upload to ovh_gra
Test if the recent outage is resolved.

Change-Id: I3b83b084ed2fd8adf09f7dc12cae825fb53384f2
2023-12-11 08:45:01 -08:00
James E. Blair 7e033243f3 Temporarily disable uploads to ovh_gra
We've seen a high incidence of failures uploading to ovh_gra recently.

Change-Id: I687427d9eab5754f7db6021a2250c78ea1053b05
2023-12-11 07:29:52 -08:00
Clark Boylan 84cffab894 Reset log upload targets
This resets the log upload targets to our full list of options. This
should only be landed after the previous commit has beened used to
confirm all is well in rax iad and dfw again.

Change-Id: Icc22521b812bfb10b158c5e1e07665f3e45aaa4e
2023-11-06 13:56:11 -08:00
Clark Boylan a6da5a83e1 Force base-test to upload only to rax-iad and rax-dfw
These were disabled due to service issues. Independent testing seems to
indicate that things are happy again. Set up base-test to upload only to
these to regions to confirm. We'll reset base and base-test jobs to the
full set of upload targets if this goes well.

Change-Id: I90141b8f666101fabbcc0b820b370fe20af73632
2023-11-06 13:54:03 -08:00
Clark Boylan 1e5f2cd932 Temporarily disable log uploads to rax dfw and iad swift
We've seen errors with uploads to rax iad and rax dfw resulting in jobs
reporting POST_FAILURE with no logs uploaded. Testing container listings
against iad and dfw results in 500 errors but the same requests succeed
against ord so we leave it in place.

Based on the 500 errors received in testing above we believe this issue
is not on our end (client/sdk updates for example) but rather on the
server/service side and we need to disable the endpoints and ride it
out.

Change-Id: Ic38fb681c1de84bdb96edb1ecaa7c632a5bec706
2023-11-06 12:34:44 -08:00
Clark Boylan 10a6896e40 Remove fedora-latest nodeset
We haven't kept up with fedora releases and the release we have images
for has been sunsetted upstream (they removed package mirrors under us).
In response to this we're going to wind down fedora images entirely and
that requires removal of this nodeset.

Users can look to centos stream which is a forward looking upstream to
rhel to fill a similar role as fedora. The upside to centos stream is
that it has much longer support periods so we don't have to update
images frequently.

Depends-On: https://review.opendev.org/c/opendev/bindep/+/892378
Change-Id: I81bb306a9d5d9f12a703ede382985fa0dec0b697
2023-08-22 09:39:02 -07:00
Zuul 7be4c15207 Merge "Add standard Debian Bookworm nodesets" 2023-08-11 17:22:05 +00:00
Jeremy Stanley 00a7562d3e Pin Pillow<10 for blockdiag incompatibility
This is essentially the same stop-gap zuul/zuul-jobs put in place
with I93b9634f7d2813cb1a17aeda2d8ef2ec188f86f3 for now, since it's
generating the same diagrams.

Change-Id: I682ffe9e89ed83b09405940d5cc9bdc66643e131
2023-08-11 15:15:33 +00:00
Dr. Jens Harbott e1b31d2a0d Add standard Debian Bookworm nodesets
This adds nodesets debian-bookworm and debian-bookworm-arm64.

Change-Id: I5c98aa8c5f09858596038d6b365e1b838f2b78c0
2023-08-11 09:19:25 +02:00
Radosław Piliszek ca59b60fbe buildset-registry: Always use Docker
Since run-buildset-registry depends on the container_command,
but buildset-registry supports only Docker, we need to enforce it.

Change-Id: I8966251030dcb3342befa727b2cc6e20b7229b11
2023-05-22 21:44:40 +02:00
Clark Boylan 3fc688b08d Run ensure-quay-repo in our base container jobs
This will ensure that properly defined container images are created in
quay registries for us. You need to do this out of band of the docker
push if you want the image repos to be public. The ensure-quay-repo role
should ignore images that don't have the correct metadata making this
safe for all container base jobs.

Depends-On: https://review.opendev.org/c/zuul/zuul-jobs/+/881521
Change-Id: Ic358f5e2f44c2a1e02140f8c848fe352214ba65a
2023-04-25 15:28:39 -07:00
James E. Blair baae244431 Set default job vars for container image promote
This sets some boilerplate variables for the container image promote
jobs.  This does not change the promote method to intermediate-registry
but makes it easier to use.

Change-Id: Ia545b993d04bd3e29c474a3559804a72bedaf687
2023-04-13 12:58:01 -07:00
James E. Blair dbd7b981db Move pull-from-intermediate-registry to localhost
This should run on the executor, not the nodes.

Change-Id: I61fd52982c81d6dfe309b641cdb28278b4b438f2
2023-03-23 13:19:19 -07:00
James E. Blair 30b02a86dc Fix container-image pre playbook container_command default
This variable can be undefined, so use the documented default.

Change-Id: Ied1f79a5ec0ca769301ed2ecd38e67f1f511aa50
2023-03-23 12:55:33 -07:00
James E. Blair 326c244f8c Add ensure-* roles to container image jobs
This is in the zuul-jobs pre-playbook, but we don't actually inherit
from those jobs so we need to duplicate it.

Change-Id: I875df74936736b80dbb2f29bbb474b993f4616ea
2023-03-23 11:51:10 -07:00
James E. Blair 4119042c7e Add container-image jobs
These are the analogs of the opendev-build-docker-image jobs,
using the newer container roles.

Change-Id: Ifec8fd7db3b238536b396a9012bdf93d0d19547e
Depends-On: https://review.opendev.org/c/zuul/zuul-jobs/+/878291
2023-03-23 10:47:33 -07:00
Clark Boylan 69efb45018 Revert "Temporarily disable rax swift log uploads"
This reverts commit 5ce784d816.
This reverts commit 5497c3aa3a.

Revert the two chagnes that were used to disable rax and then force rax
under base-test for testing. The testing performed after that change
landed seems to indicate things are working.

Change-Id: Ibc3e71399205895d1508786f1eb40cb13d44817a
2023-02-09 14:18:18 -08:00
Clark Boylan 5497c3aa3a Update base-test to only upload to rax
We recently disabled rax swift log uploads due to errors. Force all
uploads under the base-test job to go to rax so that we can test this
more now that the immediate fire is contained.

Change-Id: I7cb8b312356fbaf0d8b4db02b6cc9363f3b13c6f
2023-02-09 11:22:24 -08:00
Clark Boylan 5ce784d816 Temporarily disable rax swift log uploads
We are seeing failures to these regions. Disable them until we can debug
further to avoid unnecessary job failures.

Change-Id: If47636adf08279f8c691c3e9b6351b08067f3191
2023-02-09 11:05:47 -08:00
Dr. Jens Harbott 16470c3780 Revert "Disable OVH BHS1 and GRA1 log uploads"
This reverts commit 022af868f1.

Reason for revert: OVH issue fixed

Change-Id: I7413b87e6c7e520661fd6e51f7ba417eed042225
2023-01-11 13:18:10 +00:00
Clark Boylan 022af868f1 Disable OVH BHS1 and GRA1 log uploads
These regions are returning 503s for file retrievals and some jobs are
failing to upload logs there. Disable until it stabilizes.

Change-Id: Ic5d75b95bf8e3c71025c7297644e7fb3ed2fd9b3
2023-01-10 14:47:14 -08:00
Clark Boylan 7081efc3e0 Add opendev nox docs promotion
This adds a new publish-opendev-nox-docs project-template which will run
opendev-nox-docs and opendev-promote-docs (with the appropriate var for
nox jobs set) to promote and publish our docs using nox based jobs.

Change-Id: I7ed1f439062f13a15fd5a24fbb77d4135cfb08ee
2023-01-09 15:03:07 -08:00
Clark Boylan 57c5e353e5 Fix tox.ini for tox v4
This will probably go away eventually but before that can happen the
child change adding nox docs support needs to land.

Change-Id: Ibf25e07ceb80e54263223c7dc900a444143ecbac
2023-01-09 09:22:36 -08:00
James E. Blair 675ff8b712 Add nox-docs base jobs
This adds a copy of the tox-docs related jobs but using nox instead.

Depends-On: https://review.opendev.org/868134
Change-Id: I445202f366c748191fe6a05e145c05cbad1bb8f5
2022-12-20 09:17:00 -08:00
Clark Boylan 8a7b3895d4 Revert "Disable ovh swift endpoints due to errors"
This reverts commit 85e1ff20ea.

The incident [0] has been marked as resolved by our provider. We should
be good to return to our full set of swift backends for log storage.

[0] https://public-cloud.status-ovhcloud.com/incidents/by8279p6sdjd

Change-Id: I46d5ae367412081808c22f6b2626fbb83fe2e34c
2022-11-16 11:50:42 -08:00
Clark Boylan 85e1ff20ea Disable ovh swift endpoints due to errors
There is an incident producing errors with ovh swift object storage [0].
Disable these regions until that incident is resolved.

Note we disable rax on the test job so that we can easily test things
are functional once this incident is resolved. Reverting this change
will reenable all swift endpoints in base and base-test jobs.

[0] https://public-cloud.status-ovhcloud.com/incidents/by8279p6sdjd

Change-Id: I8f0655f95308a31881680d1b0c25ed6af8f54fb7
2022-11-16 07:49:45 -08:00
Ian Wienand 543a02f059
infra-prod: use prod_bastion group
This is similar in purpose to
I137ab824b9a09ccb067b8d5f0bb2896192291883 to separate out where we are
talking to the bastion host from the executor, versus the nested
ansible CD run.

Add the host in the "prod_bastion" group, and switch the source setup
playbook to use "prod_bastion[0]".  This reduces the number of places
you have to update the bridge name when you change the host.

Change-Id: I66df4057b3990eed2230d894ff42d0a425a2381a
2022-11-04 09:23:46 +11:00
Ian Wienand f53b34c171
infra-prod: Move project-config reset into base-jobs
Currently we reset trees to master in two places; here and in
sync-project-config (Ib999731fe132b1e9f197e51d74066fa75cb6c69b).  This
is a bit confusing, and requires delegating tasks to the bridge node
which isn't great.  Also, as we think about trying to make jobs run in
parallel it's another place to get things wrong.

This merges the update into one place.

Change-Id: I6ffeb6e6562fb34db89f4e475da27b60e30f6fe7
2022-10-28 18:00:34 +11:00
Zuul e76a51f9cf Merge "Switch to bridge01.opendev.org" 2022-10-25 22:38:41 +00:00
Jeremy Stanley 132e746c8f Switch default nodeset to ubuntu-jammy
As announced[*], we are moving from ubuntu-focal to ubuntu-jammy as
our default nodeset. Any jobs which don't specify a nodeset will
inherit this change in default value.

[*] https://lists.opendev.org/pipermail/service-announce/2022-October/000047.html

Change-Id: I6b349a19b4dee397c1928c24eab9647e7c9aec30
2022-10-25 16:11:09 +00:00
Ian Wienand e9526fe69e
Switch to bridge01.opendev.org
Switch to the new bastion host

Change-Id: I8b7547af99f8858934af2593f8ac9b4172484895
2022-10-25 16:07:42 +11:00
Zuul 15c7802445 Merge "setup-keys: add bridge node to "bastion" group" 2022-10-23 23:46:45 +00:00
Dr. Jens Harbott f6d2034eb8 Drop ara related vars from the base jobs
We no longer use ara since we have the zuul web console now. This change
has effectively been tested for a long while since these variable were
never added to the base-test job.

Change-Id: I74dba3acf63fde543eee865c2d3e65528cbf0c23
2022-10-12 04:41:18 +00:00
Ian Wienand 94857d2f38
setup-keys: add bridge node to "bastion" group
This puts the dynamically added bridge node in the "bastion" group.
This way the production jobs can refer to the generic group name, and
be abstracted from the actual hostname.

Change-Id: Ie35f3f003f21472be2ca87ab962141d17fc2a7b6
2022-10-12 14:12:37 +11:00
Ian Wienand accdc49eef
Fix zuul_console_disabled typo
Similar to Ie0a0d8f4ae137dc12f4c13f901096ee39d9a088e in system-config;
fix the typo on this variable name.

Change-Id: I579af80831ec6c317aa4c03d68a1e1934c2fe16c
2022-10-11 13:52:26 +11:00
Ian Wienand 83e03c36e8
Add zuul_console_disable flag to added hosts
This stops the bridge trying to write out console streaming files that
will never be read, because we don't allow connections to the
streaming port.  c.f. Ifbb5b8acb1f231812905cf9643bfec6fbbd08324

Change-Id: I82f194631c2a6d4ed2e46e057a609e5d68ffd2dc
2022-10-11 10:05:48 +11:00
Clark Boylan d1dc777fd1 Use test-prepare-workspace-git in base-test
This will enable us to test changes to test-prepare-workspace-git and
ultimately prepare-workspace-git.

Change-Id: Ic6badd58a7021595508cad0d3ecb9c7d80780858
2022-09-22 10:49:29 -07:00
Clark Boylan 81494e12fd Remove ansible-version: 2.8
Zuul is removing support for old ansible versions like 2.8 remove our
use of it. While we are syncing base-test with base a bit add abstract:
true to base-test to match base as well.

Change-Id: I949c6c7d93db7e3862ec9169ee94fd501539863a
2022-09-21 08:45:37 -07:00
Ian Wienand c81fc34da4
configure-mirrors: enable extras-common for 9-stream
This uses the new argument provided in the dependent change to enable
the extras-common repo for 9-stream.  Since this is already running
with the default arguments, it should be low-risk to change them here
and only affect CentOS 9-stream.

Change-Id: I185657987fd1b454db683bd1329a985940014750
2022-09-20 08:47:00 +10:00
Ian Wienand ceba1e4f5a
base-test: add descriptive names
This ports I40f2592a316bb8293f91d90be3996a6c697de196 to keep this file
in sync with base/pre.yaml

Change-Id: Ie0063edc6e6ae9e1c478b538cbda010dd03177c9
2022-09-20 08:47:00 +10:00
Ian Wienand 3bfdda452c
Revert "Switch base-test to test-prepare-workspace-git"
This reverts commit 88e7b0da57.  The
testing is complete.

Change-Id: I4e5420e05bc8ef8ece56fb53746236e751869cd7
2022-09-20 08:46:55 +10:00
James E. Blair d48513b084 Fix pypi upload secret
The switch to api_token in I8fd7310a91ff023cc348466a9cf20a144a4ff691
was incomplete.  The username must be removed in order to pass
the validation check in zuul-jobs.

Change-Id: I395868f41cc20da9fbbcbd0493a7631ab332104e
2022-09-05 13:01:17 -07:00
Zuul 7935fe7b12 Merge "Add Rocky Linux 9 nodeset" 2022-09-02 01:24:51 +00:00
Zuul 3b6e6acf06 Merge "Update fedora-latest to Fedora 36" 2022-09-02 01:24:49 +00:00