The publish jobs build and copy the wheels to the AFS r/w partitions,
but they are not released to the mirrors until a final "vos release"
of their volume.
Previously this happened in a final job that would not trigger if any
of the wheel builds failed. This has meant thing like out-of-sync
centos mirrors or arm64 node failures have stopped all wheel
publishing previously. There's really no need to block other
platforms publishing if one of these jobs fails.
This converts the jobs to have a release job syncrhonized by a
semaphore. We don't want many "vos release" processes running all at
once, as it has been a source of failure with openafs before. Since
releasing requires the admin key, we keep it in a separate job from
the building.
Change-Id: I10c307c2d46c0e5b86732943208c3167da28a694
This was missed with Ia44c384072c0482cfd11c642013fd51004f85c8b when we
removed the Xenial ARM64 jobs.
Also cleanup the grafana; additionally remove CentOS 7 ARM64
references which are not present.
Change-Id: If5fb2a1fe2cce058f1d516d5f5f7180a6cab06cf
Since Debian Buster can not be used with nova 23.0.0 because of the
min required libvirt version, we should make Bullseye available for CI
to ensure that OpenStack Wallaby release will run on it smoothly.
Depends-On: https://review.opendev.org/c/openstack/diskimage-builder/+/783790
Change-Id: I9c1bb7aaa02ba60ee52e2d7b990e2e6e1212317f
We ended up not creating centos7 arm64 wheels and don't have this
volume, don't try and release it.
Change-Id: Ibe4d459ce6c9c6e26330944f30272925b930ba3d
We can not speculatively test changes to the wheel build jobs while it
is project config. These jobs are in the gate for the requirements
project, which run them when bindep.txt changes.
https://review.opendev.org/731394 moved the build-wheel-mirror-base
job/playbook/role into openstack-zuul-jobs, renamed as
build-wheel-cache-* to a) distinguish the name and b) make it a little
clearer that we're building the wheels that goes on the mirror, rather
than passing through to mirroring something existing.
This updates the job names in project-config to reference these new
jobs.
Note the publishing steps are kept here along with the AFS secret.
Depends-On: https://review.opendev.org/732085
Depends-On: https://review.opendev.org/732087
Depends-On: https://review.opendev.org/732083
Depends-On: https://review.opendev.org/732084
Depends-On: https://review.opendev.org/732086
Change-Id: Iac3a906803177e8365f4cfb611800b5ccaed4a6e
This removes trusty from the repo and thus from OpenDev.
Afterwards the AFS volume mirror.wheel.trustyx64 can be deleted.
Depends-On: https://review.opendev.org/702771
Depends-On: https://review.opendev.org/702818
Change-Id: I3fa4c26b0c8aeacf1af76f9046ea98edb2fcdbd0
This role uses the system-config:install_puppet.sh script to first
install puppet on the wheel-builder host, and then uses
puppet-kerberos/puppet-openafs to bring up the AFS environment.
This is replaced with ansible roles to do the same AFS bring-up. A
new "openafs-client" playbook is added to configure this for the infra
environment. The wheel-building base job is reworked to use this.
Roles have been tested under zuul with
https://review.openstack.org/596018
kerberos-client role:
Depends-On: https://review.openstack.org/593486
openafs-client role:
Depends-On: https://review.openstack.org/593487
add system-config to linters:
Depends-On: https://review.openstack.org/596645
Change-Id: I9a3af36048e4a4439fe86b36582667ff0af68035
The releases project has been updated to support ubuntu-bionic wheels,
start publishing them to AFS.
Change-Id: I30c1a7c702a7f2cf7f1544ddd62a7a0e719d9572
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
No need to use magic names since we now have host_vars, instead set
wheel_python explicitly.
Remove TODO comment.
Change-Id: I5051a32d843b106b013b94d57fd24b63896b741a
We only need the three wheel-X scripts from jenkins/scripts and need
them only for the playbook and its roles. Move the scripts in the
roles directory and copy them only if needed - using the scripts module.
Remove copying of complete jenkins/script for this playbook using the
role legacy-copy-project-config-scripts, it's not needed anymore.
Change-Id: I2e481fcf7ca148aac8a36ae99d8598ba26078a25
All of these invocations of the bindep role are passing what amount to
the defaults. In zuul-jobs, doing bindep_dir: "{{ zuul_work_dir }}" is
important because it allows in-repo uses of those jobs to override how
they work for things like triggering from one repo but running on
another.
The publication jobs in project-config do not need that level of
flexibility, so just remove params and let defaults take over.
As a follow up, we could also add defaults for zuul_work_dir - and
honestly for bindep_dir - to the base job. That way we can always count
on them being set to a sane default value, and they're still
overrideable in a job.
Change-Id: I5bcda84aefa4726d0a904e8bc9698c21ca07f9a5
The openstack/requirements projects needs to pull in a few more
dependencies to build wheels. Add back centos-release-openstack-pike
which was recently deleted from base images.
Change-Id: I2093905f8b48d9dfedd07bcf8f3721c658c4acf8
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Because we run multi-node jobs of our wheel-mirror jobs, we need to
hash by the version of python the node is running. Otherwise, both
nodes will upload logs to the same place.
Change-Id: I5557747db0773873a6542c5ebbac25ca4d4c779e
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
This allows us to add the build-wheel-mirror jobs to
openstack/requirements check pipelines, but not release wheels to afs.
Making it easier to properly debug these jobs, otherwise an infra-root
must manually enqueue-ref for periodic pipeline.
Change-Id: Iabedbfeb62f1773d10462e8b07bbe82f1fb76389
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Because, why can't things be the same across distros? We need
ansible_distribution_major_version for centos (7) and
ansible_distribution_version for ubuntu (16.04).
Change-Id: I06f3b587a78165674d0d09432164d79caa5ecf02
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
We generate build logs when we build wheels, collect them so we can
review if there is an issue.
Change-Id: Ibc69a6c1ffa87a698c8452fcd03951f2a7aff5a7
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Currently, our secrets are not included in the
build-wheel-mirror-base, so this means we cannot access the afs secret
to copy files into /afs/.openstack.org. Move this into pre distro jobs
where the secret lives.
Again, once we have protected final jobs, we can clean this up a bit.
Change-Id: I8e8b92af771bc84a3930a8f98c96c8216388d1cf
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Because centos-7 doesn't have an openafs-client, we have to install
the version we built ourself.
Change-Id: If8fbb9bf7c585eebd2eb7abbb220375bdf38f682
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Doesn't look like --syntax-check or linters caught this. Update to be
more correct.
Change-Id: I95f34611ef5733e930eead1cbe5ee4573ea7aadf
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
We need this be done before we call the bindep role. Oops.
Change-Id: Id25594a3ba3bfc4964d558fddfa0c5660ec11c7f
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Right now, it seems we only need parallel from EPEL. Rather then
enabling the whole repo, we can just opt to install it first, then
bindep will be happy.
Change-Id: Ifd1eb186f78c395f549d1430cfe4414ac1121f63
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
While we are using multi-node jobs, neither host needs to talk to the
other. So, have the race fast as possible to the end of their
playbooks.
Change-Id: I6fa39a5444d1c3e6188e1067ffaa45d0fe97e90d
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Our wheels will end up requiring dependencies, so add support for
bindep.
Change-Id: Ifa73ba475284ac8435ba29ffaf228a54dce59b09
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Including puppet madness!
We don't need the long-lived wheel builders - we build from scratch
every time. So all we need is AFS running on a node.
AFS, however, is tricky on CentOS. Rewriting all that right now is meh.
We can do that later.
This gets us to not needing special nodes for wheel builders. Yay!
Change-Id: I6d92c47f4f7a26ca290cd400452cb0a73872b24d
NOTE: It may be inefficient to run this on our low-bandwidth nodes.
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Pass the wheel keytab across and aklog with it.
Also, split the build role into a build and a copy so that we only have
the token when we need it, and not while we're running the wheel builds
themselves. And split release from build so that we can release the
mirror once for a consistent state across python2 and python3.
Change-Id: Id30a99f5b515dd520d013fabbd356e32c82030a0
We need to run periodic jobs to build the wheel mirrors.
This removes them from the v2 config because we really should not run
these jobs from both systems at the same time and once this lands it'll
be live.
It copies the scripts from project-config to the homedir, so we can
ditch the puppetted copies of them.
Change-Id: I87a78509a97d53eb627a84c646a6a3f0a227d9a1
Ian Wienand