The opendev project has been moving away from puppet and this is one of
the puppet modules that is no longer used. To simplify things for us we
are taking the extra step of retiring this repo.
Change-Id: I1d3a8a3999e75be00844a314fa7f52c37e28f9d4
We need to communicate to apache vhost config that we want LE certs to
be used in some cases. Add a vhosts_flags flag to signal this then plumb
the correct data into the vhost when set.
Note this requires the vhost name and the le cert name to match as that
is our mapping between puppet and ansibe here.
Change-Id: I40131525c10152ebb0a7495588f137e3c019bbc5
ARA 1.0 will be released in the near future and isn't backwards
compatible. Pin it so it doesn't break things unexpectedly.
Change-Id: I6e239bb4b445163a3f0dd8c7557119b057da4ff0
The objgraph library is used to grab object counts for memory leak
detection. We already have this installed on zuul01, but that appears
to have been done manually. This cements the install.
Since puppet-nodepool requires the same packages (yappi and objgraph),
wrap both of the installs in a conditional check to prevent puppet
errors about being defined in multiple places.
Change-Id: I63ab5a21f7b98341bb203883f40c2df5404687ba
There are two deployment options for the new multi-ansible support:
the executor can install Ansible on startup (the default), or we
can pre-install ansible when we install Zuul, so that it's ready
for the next executor restart. This change implements the latter.
Change-Id: I0c75724b4376f68128590eaabdd8a10ed63dcda0
This updates the status_backups resource definition to allow for
specific uris to be set. This allows you to do per tenant backups
without tenant white labeling in the web server.
Change-Id: Ia94a16ef8b2a1069b9265bae2773cd323ac9a188
FreeBSD libc's malloc implementation, jemalloc, performs better than
glibc's for some tasks. In the case of our Zuul executors, we've
seen useful improvements in memory efficiency. Update the Puppet
module to install the libjemalloc1 package on Zuul executor servers
and export the necessary LD_PRELOAD envvar to use it when launching
executor daemons.
Change-Id: I8d8de3c118bd09d408b23a16482f71198a1232fc
This updates the zuul web config to manage both global web server and
whitelabeled per tenant webserver configs. A new set of config hashes is
introduced to do this. For backward compatibility we construct the
hashes from existing parameters if the new hashes are unset.
Change-Id: Ie8ba46111530e74b8b1d0ec1746df2e09754fe67
This is to deal with pip10 issue around python-yaml.
| Jul 23 16:52:29 ze11 puppet-user[14344]: (/Stage[main]/Zuul/Exec[install_zuul]/returns) Cannot uninstall 'PyYAML'. It is a distutils installed project and thus we cannot accurately determine which files belong to it which would lead to only a partial uninstall.
Change-Id: I3f7b209f4b8b5ed7f826369c9f8ad171cb530925
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
PyYAML is already a dependency of zuul, so the distro package doesn't
need to be installed. This attempts to clean pip10 issues with OS
packages.
Change-Id: Ia17aa61ab3c153c82aeae7b99747d142dd75ff61
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
These are used by the zuul.conf template to write the zuul-web
configuration. The template compares the values to the empty string
to decide if they should be written, since they are undefined, they
are always written, but with no value. This adds them to the class
with empty-string defaults which should cause them to not appear in
the config file unless needed.
Change-Id: I065d18a0743179cb2c9827a8e6b1b70c169d9199
Without this patch, puppet 4 apply tests can't find the mod defined type
and fail with:
Evaluation Error: Resource type not found: Mod at /etc/puppetlabs/code/modules/zuul/manifests/web.pp:284:15
This patch ensures the resource reference is namespaced just like all
the other resource references in the manifest already are.
Change-Id: Iac45cd1fe28fa9cb42751d2d879224ddc7f255ce
Commit 0c3b8fb96 of zuul added a dependency on fb-re2 which requires the
libre2-dev package to build. Ensure the puppet module installs it.
Change-Id: I5e66a5a2176012663add76ea5d68a22b04b93cf9
There is no such default fact as 'operatingsystemmajversion'. On puppet
3, this would not raise an error, but on puppet 4 this causes an error
when used in the versioncmp function which expects a string and not
undef. The correct name of the fact is 'operatingsystemmajrelease'.
Change-Id: Iafc4554cffd4a5406535e00e158af7bd04ca2f13
The previous layout meant we couldn't really do apache offloading of
static content and instead were proxying them to the aiohttp server.
The new update moves api calls below /api. Update the rewrite rules to
deal with this. Also, update the document root to point to where we
deploy the zuul dashboard so that it's picked up by apache.
This also updates the apache config to work for non-whitelabeled deploys
as well.
Depends-On: https://review.openstack.org/556967
Change-Id: I6e662e7bc97de96969828801bcb18c5f354965c5
In v3, we need to be able to use some variables from zuul/web.pp in the
vhost template, but the vhost is created in ::zuul because of how v2
worked.
Split the config, and split where we're defining it so that in the next
patch we can make use of variables from zuul/web.pp.
Change-Id: I3e1c72bb773be050854f5563d09f6c19af8bc6dc
Depends-On: https://review.openstack.org/557085
In the next patch we'll start using them, but we need to be able to add
them to system-config so that the acceptance tests will run.
Add the parameters but don't do anything with them yet.
Change-Id: I4a2062fc35f8861267f8b8ce3e6d0e9f1d6cffdc
Other packages, such as bup (for backups) depend on this, so removing
it also removes them. Just leave it alone.
Change-Id: I4d3d69f943cf5f625ff1d2af475e6a6ed0dfa411
With the update to the use of yarn/webpack for the zuul-web javascript
stack, there is now a pre-built tarball of the web assets. Update the
code to use it.
An alternative to this approach would be to install nodejs and yarn on
the machine and just do a pip install . in the zuul source dir which
would also work ... but since there is an option that doesn't require
the need to run the build tools, why not use it?
Because spec/acceptance/fixtures/default.pp is hard-coded to the
zuulv3 install path, we can remove the now unnecessary checks from it.
Move the status backups to /var/lib/zuul/backup, instead of www/backup
as the www directory no longer exists. Also remove it as the
documentroot.
Depends-On: https://review.openstack.org/547790
Change-Id: Ib88b48ddea26ef5d8d0d152774112844b180f77e
Zuul is about to remove .json suffixes from the REST endpoints. Update
the puppet to deal with that.
Change-Id: Ic1df0c01eec57f163ce332518ebc560e77b92bf3
Depends-On: https://review.openstack.org/537010
OpenStack white-labels the dashboard specifically at the OpenStack
level, but there are some URLs, like the github connection payload
webhook receiver, that are global and not per-tenant. It's conceivable
that we might want to add richer support for single-tenant whitelabels
as well.
Change-Id: I03ea09e86168ed4e896f74db1b715131ad61f600
We have extra whitespace in the pip installed package listing that we
don't have in the upstream pypi package search so our string comparison
of the versions breaks. Fix this by trimming extra whitespace.
Change-Id: I04b28508ff9371397d42957f0d086aa9bb63a4f7
Since the contents of the status.json is only available in-memory from
the Zuul scheduler, the contents are lost forever if the scheduler,
or the server it is hosted on, crash.
This implements two crons:
1) A curl to download a timestamped status.json, if it is available
2) File deletion so we keep no more than 180 timestamped status files
at any time.
The files are stored at a location within the scope of the zuul-web
vhost in order to make it easily possible to dump queues with something
like:
/opt/zuul/tools/zuul-changes.py http://zuulv3.openstack.org/backup/status_timestamp.json
Change-Id: Ia1923bfae0222559b8d2dcb5e0618848b3a618d0
This updates the rewrite rules to the current state of the zuulv3
dashboard, and removes the overlap with zuul v2 urls.
Change-Id: Idd84facca4a0e0170234bf7d4ee2b42811f52b67
Until we have the more generalized javascript tooling in place for zuul,
we need to add libraries directly here. The zuul web console requires
angular, so add it.
Some of the same dependencies are used in zuulv2 and zuulv3, but in a
different location. Symlink them as appropriate
Change-Id: I75d5263ab2af3666d162b66394a537dfbe88010c
Zuulv3 now suport command socket to stop both merger and schedulers,
add logic to install new init scripts for them depending on zuulv2 or
zuulv3.
Change-Id: Iecc8c03896c145d29e42e6be6194d7db893c8515
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
We're moving the default pidfile location in
https://review.openstack.org/517381
This change creates default files so that when running v3, the
new location is used. The init scripts themselves are still
compatible with v2 and v3.
Change-Id: I57c486f4506114eb0447be2014baa42fc22cc8a2
We want to ensure that our jobdir and the git repo cache are
on the same filesystem. Configure the executors to use a dir
under /var/lib/zuul for the jobdir. We will mount a new (large)
filesystem at /var/lib/zuul to accomodate both.
Change-Id: I91472fa609274aadcf4945aad44f25014c07b105
zuul v3 supports optionally collecting data via ara if it's installed.
Install it so that we can produce ara html reports for all of our runs.
The current implementation is similar to the devstack-gate
implementation - it uses a sqlite db per job. The bubblewrap invocation
sets $HOME to the work_dir, so the sqlite db for each job goes in to the
root of the work_dir with no further config.
Change-Id: Iefeda37338c57466d671ea07e7d318fa92aafc35
Depends-On: I8facdf0b95b83d43c337058d70fe6bf71e17d570
Because we need to update zuul.conf with the path to the file, we
actually need to configure this variable within init.pp.
Change-Id: I734f2c338ceab2b1da2a3245423cb912b7dd8c00
Depends-On: I7983ff9cfaea1ee6d3b099824b5f31df98ce72a5
Signed-off-by: Paul Belanger <pabelanger@redhat.com>