The recent pyyaml 6 release, incorporated in the latest
nodepool-builder/dib images, requires a specific loader now. Use
safe_load as there's nothing crazy going on here.
Change-Id: I26e5e1cf6778cb51872d6a65b2ac683335201bb4
Some distros (gentoo) do not come with six in the base image (stage3),
it is installed later in the image build. Use the native
urlopen/URLError if the six version is not available.
Change-Id: Id1c52aa17e565c16369645508e43e4f485defa66
The dib 2.34.0 release uncapped hacking and has found some new minor
issues. Add missing readmes and fix whitespace.
Change-Id: Ia05e54c26988774bf03b0764a6df5e60e8ddaca8
This is what prepare-workspace-git expects. This change should be
merged after we have used the symlink (removed in this change) to
migrate to that role.
Change-Id: I8ae788dfc98f5102aefef6b57f2b82e7c50922ac
Depends-On: https://review.opendev.org/680696
Use new locations following OpenDev changes.
Node: This changes the on-disk repo path to /opt/git/openstack/devstack
Change-Id: I7042913fefa64dcec4044779dbeb13f86daea858
There is no jenkins, only zuul! With zuulv3 running in production,
there is no more need to add jenkins user or scripts to our images.
Move chmod of /opt/cache/files from
nodepool/elements/jenkins-slave/install.d/20-jenkins-slave to
nodepool/elements/zuul-worker/install.d/60-zuul-worker
We also leave the jenkins-slave elements for now, in case 3rd party CI
use them.
Change-Id: Ia9750877fbc1a17ec467ca4ac685afdb9c1627f8
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
This commit is part of a series to retire the Packaging Deb project.
Step 3 is to set the Gerrit ACLs for these repositories to
read-only, remove them from CI configuration and IRC bots (as in
https://docs.openstack.org/infra/manual/drivers.html#retiring-a-project).
Change-Id: I05c9e4a37ecb1eefe1909be3cb5fa56ee10710ff
git:// is not a secure protocol. It's also harder to do apache rewrite
tricks and whatnot when we eventually flatten namespaces. Change
occurances of git://git.openstack.org to https://git.openstack.org.
Change-Id: Ic54f1b84346e37a76baa87deadbb540662889bca
fix a "bug" in that the flake8 configuration in tox.ini was exclusively
selecting H231 as the only error it would report, so it was missing the
errors in the python modules (such as submit_log_processor_jobs). Due to
this being the case for a long time (since 2004) limit the more thorough
linting to the roles/ and playbooks/ directories where we'll be adding
ansible plugins/modules/etc. Also, lint in jenkins/script and nodepool.
Fix problems found.
We can lint everything with pep8 once the zuul v2 scripts are removed,
not worth patching them right now.
Change-Id: I479f010643cf3b67c183d763510f07a33400d38b
Co-Authored-By: Jesse Keating <omgjlk@us.ibm.com>
This element is no longer used. jenkins/cache/files is now a symlink
and jenkins/cache/pip no longer seems used.
Change-Id: I558b3bff7da97853d53ab5ef71b0a9854238e592
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
Add python3 support for these functions. Tested with a python3 based
dib in a Xenial environment. Add __future__ imports so we don't
regress on these bits.
Change-Id: Iddca88cb6c9fbff6383c3597eb81133948b3b420
Move non-puppet-related parts of the puppet element into other elements,
remove the puppet element, and stop running the prepare-node script.
\o/
Change-Id: Ib241ea976ca0a3d661599f36e3f5a2d4eb023c08
system-config was cloned twice in order to not interfere with
slave repo cache during image creation.
This clone is not used anymore and can be eliminated
Change-Id: I0ff0675ead5f4c192c863f6b0be84f3364f84c0a
Do not clone repos that are retired. Since they will all use now
"retired.config" as file name, we can easily find them.
For this, move project filtering from main to get_project_list so that
we have access to the full yaml file and not only to the list of repo
names.
Change-Id: Iea6781c19b4828fd9bae0122add2f83e2586f190
If you specify a CUSTOM_PROJECTS_LIST_URL that doesn't exist you get a
message but the script continues. Since you clearly meant to specify
something we should bail so the user can correct the problem.
Additionally, add some documentation to the build-image.sh script to
describe how to use this to cut down build times considerably for
initial testing purposes.
Change-Id: I02d1cd341f793471b1ad4374e98031ea53db7f49
Building a nodepool image is incredibly slow due to the git repo
precaching and it doesn't help that everything is cached twice.
Change-Id: I665bda6cde29aea9ad5285d8d6981ad9c19fc5a7
Adding the ability to selectively cache only some git repos in
nodepool slave.
This is useful in some environments where it is needless to cache
all the git repos that appear in gerrit.
For example, in a 3rd party CI environment, each nodepool slave
needs only few openstack projects in order to check commits to
a specific project. So it is needless to cache > 1500 projects
currently in gerrit
Change-Id: Ie7302594b24d2bdcc157d3cf64b1f219e7ef0205
We were missing the * ref which will cache all branches and tags when
building the source repo cache in our images. Add it in so those things
are included.
Change-Id: I1ce6dd0f737a4bb6e6a0a73bf8c010d9ab11c581
This moves prepare-node out of node-devstack so that cache-devstack is
not included unnecessarilly. We change openstack-repos and
node-devstack to depend on this element. The evenual goal is to remove
node-devstack, which will be done in a followup commit.
Change-Id: I4d9ebc3659f5f59cab4d4ab0762e157b6ff11eda
openstack-repos:install.d/95-chown-jenkins relies on the jenkins user
being around, which isn't done until the puppet stuff in
node-devstack:install.d/20-prepare-node.
Express this dependency
Change-Id: I1cf006d709a54b9da31c2375a0941fe497c3c761
We have attic and stackforge projects all of which are now inactive.
Because they are inactive we don't need to be caching these repos on
every test slave image.
Change-Id: I4df78c3d542758ce6159c195c1407f1d56f565a0
Since I6c5a962260741dcf6f89da9a33b96372a719b7b0 dib has had a
standardised method for ensuring consistency of tracing and error
detection. Bring the tracing for these elements up to that standard,
but maintain the status-quo of flags such as "-e" and "pipefail" by
adding ignore flags where appropriate (we can update these separately
to avoid breakage)
Other minor changes are alphabetical-ordering in the element-deps
files and permissions on prepare-node script
With this, "tox -edib" passes
Change-Id: Ibba1dadb9e819f94294c9d583b83ff698252f93f
Passing a source-repositories ref of "*" signals fetching all heads
and tags rather than just a specific refname. This is desirable for
the warm cache on our job workers, since jobs may run against
arbitrary branches (or possibly even tags). The point is to have a
reasonably complete mirror of each repo, rather than a fairly
minimal clone.
Change-Id: I624ed1e259e007d4246afb45c3a0560598bfbe3f
Depends-On: I4562c9689a8d235ebe09b2f7178aa5890dbc85f1
We are renaming openstack-infra/config to
openstack-infra/system-config. This patch edits paths for
nodepool.
Change-Id: I75de3128e9d179b700df465726ed057439e5401e
This repo was created from filter branching the openstack-infra/
config repo. This process brought a lot of cruft with it in the
form of directories that we no longer need. This patch removes
that cruft so we begin with a tidier repo.
Change-Id: Ibffad1b11c0c5f84eedfb0365369f60c4961a0f3
Ian Wienand