Commit Graph

123 Commits

Author SHA1 Message Date
Clark Boylan 278862b3f2 Retire this repo
The opendev project has been moving away from puppet and this is one of
the puppet modules that is no longer used. To simplify things for us we
are taking the extra step of retiring this repo.

Change-Id: I1d3a8a3999e75be00844a314fa7f52c37e28f9d4
2022-02-17 13:02:43 -08:00
Clark Boylan fb45aaaeec Add option for LE provisioned certs to zuul::web
We need to communicate to apache vhost config that we want LE certs to
be used in some cases. Add a vhosts_flags flag to signal this then plumb
the correct data into the vhost when set.

Note this requires the vhost name and the le cert name to match as that
is our mapping between puppet and ansibe here.

Change-Id: I40131525c10152ebb0a7495588f137e3c019bbc5
2020-01-06 12:53:25 -08:00
James E. Blair 9f5225752d Fix a typo with web_root
Apparently more @ is needed.

Change-Id: I9580873f1e4995662ee7616a7e80808e6ced9816
2019-08-09 09:12:58 -07:00
James E. Blair 67dc98c908 Add web_root option to zuul.conf
Change-Id: I8020c166b4896041abd063ddabc62780c5237915
2019-08-09 08:12:31 -07:00
James E. Blair 5e9a106e6d Update executor ansible on installation
There are two deployment options for the new multi-ansible support:
the executor can install Ansible on startup (the default), or we
can pre-install ansible when we install Zuul, so that it's ready
for the next executor restart.  This change implements the latter.

Change-Id: I0c75724b4376f68128590eaabdd8a10ed63dcda0
2019-03-18 08:35:35 -07:00
James E. Blair 96641b0edb Set fingergw user to zuul
The user field no longer has a default, so if we want fingergw
to drop root privs, we need to set it.

Change-Id: I0bd16cbd23a30d2c643d1ac1840868e63f7adb4e
2019-03-18 07:13:07 -07:00
Clark Boylan 375befe790 Don't rewrite t/ prefix on global zuul vhost
We want the t/ prefix to survive to requests to the backend when we are
hosting a global tenant UI. This rewrite appears to be causing 404s when
/t/tenant/status is rewritten to /tenant/status.

The upstream docs [0] also don't include this rewrite.

[0] https://zuul-ci.org/docs/zuul/admin/installation.html#static-offload

Change-Id: Id8f94581b9a1c8a383c7a0d40fb89ce49c50b2e3
2019-03-05 08:38:09 -08:00
Clark Boylan f85574113e Allow for both global zuul web and whitelabeled tenant
This updates the zuul web config to manage both global web server and
whitelabeled per tenant webserver configs. A new set of config hashes is
introduced to do this. For backward compatibility we construct the
hashes from existing parameters if the new hashes are unset.

Change-Id: Ie8ba46111530e74b8b1d0ec1746df2e09754fe67
2019-01-28 16:25:48 -08:00
James E. Blair 8d7b9c8ab4 Add relative_priority scheduler option
Change-Id: Ide901da7bc3068f3aafe3b35a110cc7c2de445b0
2018-11-30 08:42:07 -08:00
Tristan Cacqueray 425f80b2e2
Revert "Revert "Revert "Revert "Add rewrite rules for HTML5 url""""
This change also removes backward compat /status rewrite
The new URL scheme actually uses /status for the status page,
and it should be rewritten to the index.html below instead.

Change-Id: I7ff5a8387ec6fb65bda55d3233b670180682d08d
2018-10-09 05:58:06 +02:00
James E. Blair 002efe1bbd Revert "Revert "Revert "Add rewrite rules for HTML5 url"""
This reverts commit 0bdd3c9a15.

The js tarball publication job is broken:

http://logs.openstack.org/c7/c7148815378b9f26751ec0729a96894e7cd6b4c3/post/publish-openstack-javascript-content/05c6626/ara-report/result/72fa45d7-c12e-4af5-b520-5e1e73e4853b/

Change-Id: I76a50203968de07b5449fbf46c2dd2d227cc2cfe
2018-10-08 14:40:43 -07:00
James E. Blair 0bdd3c9a15 Revert "Revert "Add rewrite rules for HTML5 url""
This reverts commit ba6e0e77dc.

Change-Id: I8b06980c7b052e90422bc3ca0913e45eb3a5031c
2018-10-08 08:41:57 -07:00
James E. Blair ba6e0e77dc Revert "Add rewrite rules for HTML5 url"
This reverts commit ff1f91895d.

See https://review.openstack.org/606611 which reverted the react
rewrite.

Change-Id: I41f12b0733a7ba3f0dfff265f806f3562879a160
2018-09-29 11:52:30 -07:00
Tristan Cacqueray ff1f91895d Add rewrite rules for HTML5 url
This change makes the ProxyPass rewrites Last to stop as soon as they
match. Then any requests that doesn't match a file is served the
index.html.

Depends-On: https://review.openstack.org/591604
Change-Id: I363c2c4d3924074c8ff5c7fd9a410e357ed71b5d
2018-09-21 03:16:11 +00:00
James E. Blair 3d46444494 Increase apache disk cache max size
This allows us to cache larger status.json files (which is needed
at OpenStack-infra scale).

Change-Id: I462ea29c9b5e45ced8110750a96f53a666e8623e
2018-06-06 21:01:59 +00:00
Colleen Murphy ced484d8c2 Fix scope of vhost template variables
Dynamic scoping for variables in ERB templates was removed in puppet
4[1] which means that the variables defined in the zuul and zuul::web
classes cannot be found when it is referenced in the httpd::vhost
defined type and will be evaluated as nil when puppet runs. Use
scope.lookupvar instead to be explicit about the variable's source.

[1] https://puppet.com/docs/puppet/4.10/lang_updating_manifests.html#dynamic-scoping-in-erb

Change-Id: I321cc7215387723c787ebc5d359b27866da26995
2018-04-27 19:17:48 +02:00
Monty Taylor b94e292457
Support updated REST and HTML layout
The previous layout meant we couldn't really do apache offloading of
static content and instead were proxying them to the aiohttp server.

The new update moves api calls below /api. Update the rewrite rules to
deal with this. Also, update the document root to point to where we
deploy the zuul dashboard so that it's picked up by apache.

This also updates the apache config to work for non-whitelabeled deploys
as well.

Depends-On: https://review.openstack.org/556967
Change-Id: I6e662e7bc97de96969828801bcb18c5f354965c5
2018-03-27 19:04:22 -05:00
Monty Taylor fa66845b5b
Split v2 and v3 web config
In v3, we need to be able to use some variables from zuul/web.pp in the
vhost template, but the vhost is created in ::zuul because of how v2
worked.

Split the config, and split where we're defining it so that in the next
patch we can make use of variables from zuul/web.pp.

Change-Id: I3e1c72bb773be050854f5563d09f6c19af8bc6dc
Depends-On: https://review.openstack.org/557085
2018-03-27 19:04:13 -05:00
Monty Taylor 2fb12bb339
Update zuul-web install for javascript stack
With the update to the use of yarn/webpack for the zuul-web javascript
stack, there is now a pre-built tarball of the web assets. Update the
code to use it.

An alternative to this approach would be to install nodejs and yarn on
the machine and just do a pip install . in the zuul source dir which
would also work ... but since there is an option that doesn't require
the need to run the build tools, why not use it?

Because spec/acceptance/fixtures/default.pp is hard-coded to the
zuulv3 install path, we can remove the now unnecessary checks from it.

Move the status backups to /var/lib/zuul/backup, instead of www/backup
as the www directory no longer exists.  Also remove it as the
documentroot.

Depends-On: https://review.openstack.org/547790
Change-Id: Ib88b48ddea26ef5d8d0d152774112844b180f77e
2018-03-07 18:22:47 -06:00
Monty Taylor 8f68109b2e
Move unanchored general rules after specific ones
The match ^/jobs doesn't have an ending anchor, so it matches /jobs
and /jobs.html - which makes the url
https://zuul.openstack.org/jobs.html return JSON. While the JSON is
valid job data, raw JSON is less pleasant for a human to read than HTML
(go figure)

Move the less specific entries to after the more specific entries.

Change-Id: I80f7fbe54d96cc151ef8da28465445f93092aadd
2018-02-20 06:38:01 -06:00
Monty Taylor 7e99dca02d Update apache and backups for json suffix removal
Zuul is about to remove .json suffixes from the REST endpoints. Update
the puppet to deal with that.

Change-Id: Ic1df0c01eec57f163ce332518ebc560e77b92bf3
Depends-On: https://review.openstack.org/537010
2018-02-19 12:08:34 -08:00
Monty Taylor eb2a5f81b1
Add support for specifying a tenant explicitly
OpenStack white-labels the dashboard specifically at the OpenStack
level, but there are some URLs, like the github connection payload
webhook receiver, that are global and not per-tenant. It's conceivable
that we might want to add richer support for single-tenant whitelabels
as well.

Change-Id: I03ea09e86168ed4e896f74db1b715131ad61f600
2018-02-15 17:13:17 -06:00
James E. Blair 5c759a128d Zuul v3: handle github paylods with zuul-web
Zuul-web now handles this, rather than the old webapp.

Change-Id: Ie6cf1892626097d1837870d9e4d1237d689674dc
2018-02-05 12:28:51 -08:00
James E. Blair d44c025626 Revert "Zuulv3: Convert all proxy rules to zuul-web"
This reverts commit 1ccf68fc96.

Change-Id: I18770c65c6dc9c1e2d91e08e7e1336e06d13d974
2018-02-05 12:27:33 -08:00
James E. Blair 1ccf68fc96 Zuulv3: Convert all proxy rules to zuul-web
All Zuul v3 web services are handled by zuul-web now.

Change-Id: Id34949a364c80b86685aa5444750da0daabdf6f8
2018-02-03 07:53:16 -08:00
Jeremy Stanley 8683244caf Remove [webapp] section from v3 config
As of I8278d9ca81ed7b0a2a2189d42b8b69c5eea2bab5 the last remaining
use for the [webapp] config section (passing in the status_url) is
no longer required, so it is safe to remove in its entirety at this
point.

Change-Id: I9ceb695cf54397fa2ccbbd4e580843a3f37a997c
2018-02-03 15:44:09 +00:00
Jeremy Stanley 9753b1b889 Also include status_url in [web]
Starting with I8278d9ca81ed7b0a2a2189d42b8b69c5eea2bab5 the github
reporter looks for its status_url in the [web] section of the
configuration instead of [webapp], so put a copy there as well.

Change-Id: I55ec498e1e35832723d6ffeb62ffa06b9ee4c930
2018-02-03 01:27:24 +00:00
James E. Blair 970916b50e Add public key hosting to SSL site
It was only enabled on the non-ssl site.  Oh the irony.

Change-Id: Iaf5850e6a1c721fb93557d9c6f32e119493d9bfb
2018-01-25 13:04:15 -08:00
Monty Taylor 84d7483224
Add /connection to rewrite rules
We need to forward requests for /connection to zuul_status_url, which is
the scheduler port 8001.

This will need to get updated when we land the move-gh-to-zuul-web
patches.

Change-Id: I6d10e090b37218176e969128c337b0d431870d41
2018-01-20 12:22:53 -06:00
James E. Blair dcde52761b Fix typo in keys rewrite rule
Change-Id: I4bf13add2ae04df2e479258a78d822f4a61d3442
2017-12-21 09:15:30 -08:00
James E. Blair 04f80070d3 Update zuulv3 rewrite rules for keys
This is now tenant-scoped.

Change-Id: Ib85e25bc789ed31872413a2f4bc599012adb6893
2017-12-14 16:49:19 -08:00
James E. Blair a405aafdf5 Update Zuulv3 url rewrites
This updates the rewrite rules to the current state of the zuulv3
dashboard, and removes the overlap with zuul v2 urls.

Change-Id: Idd84facca4a0e0170234bf7d4ee2b42811f52b67
2017-12-08 14:48:15 -08:00
James E. Blair 22b49ea1e5 Move statsd to correct zuul.conf file
This was added to zuulv3, not zuulv2.

Change-Id: Ieb36a6122f9b4c30d547326e7fcd8377857e7d78
2017-10-15 15:17:37 -07:00
James E. Blair 42065b22ee Add statsd to zuul.conf
We're adding statsd support to zuul.conf; support it there and the
existing defaults file for compatability.

Change-Id: I08feb2dbb18146a1a962aa8b96e3d33e5f794459
2017-10-10 13:20:00 -07:00
James E. Blair 0ec00b15bc Add support for zookeeper timeout
Change-Id: I743bfa28d2faaa93bba6fc975d575696bd8a5a70
2017-09-28 10:40:14 -07:00
James E. Blair 1f43ca60a9 Add gerrit_strip_branch_ref option
Change-Id: Id388274a7b628ad40a0e943087ec82596791c982
See: I012485065e0fd8692e4f2ef787dd0a62be303314
2017-09-19 07:51:34 -07:00
James E. Blair a7ef39c2c0 Zuulv3: move the job dir under /var/lib/zuul
We want to ensure that our jobdir and the git repo cache are
on the same filesystem.  Configure the executors to use a dir
under /var/lib/zuul for the jobdir.  We will mount a new (large)
filesystem at /var/lib/zuul to accomodate both.

Change-Id: I91472fa609274aadcf4945aad44f25014c07b105
2017-08-16 10:10:28 -07:00
Paul Belanger 0cc4c05cbd Move zuul site variables file into init.pp
Because we need to update zuul.conf with the path to the file, we
actually need to configure this variable within init.pp.

Change-Id: I734f2c338ceab2b1da2a3245423cb912b7dd8c00
Depends-On: I7983ff9cfaea1ee6d3b099824b5f31df98ce72a5
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-08-03 21:02:54 -04:00
Paul Belanger 90df637aaa
Use zuul_status_url for /keys endpoint
Change-Id: I908e219cb16c3d6a9554916da805b8722552d940
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-08-01 13:30:23 -04:00
Paul Belanger 81e1edc0f1
Add /keys to apache vhost
We need to expose our public keys for secrets.

Change-Id: I3d80aaa008b3b0d18f6e8a6bc479352eb9c076d7
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-08-01 12:35:48 -04:00
Monty Taylor 392bc6a954
Add paths bind mount option names to match zuul
We just made this change in zuul v3 itself, update the puppet to expose
it.

Change-Id: I448f6710b54d0e1d0955b7c657a84a0fcc8b405f
2017-07-28 16:39:15 -05:00
James E. Blair a98e898002 Add support for disk_limit_per_job
Change-Id: Ic267a156ee7eef4eb9e5f431cc22b236c017d0e8
Depends-On: I415e5930cc3ebe2c7e1a84316e78578d6b9ecf30
2017-07-28 13:14:54 -07:00
Jenkins 267330257e Merge "Add support for a parallel set of secrets for connections" 2017-07-26 20:56:12 +00:00
Monty Taylor afbbf2eca6
Add support for a parallel set of secrets for connections
github and mysql connections both contain secret data that is not
appropriate for public hiera. Create a second list of data that can go
into private hiera. If it's found, it will be added to the connection
config.

Change-Id: I265b445cfbf4b6d4555621e3663e71e9a0f85813
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-07-26 15:30:47 -04:00
Monty Taylor bdaeb08041
Send websocket traffic to ws:// protocol
The console-stream URL needs to be talked to over ws:// to make sure the
WebSocket upgrade headers are sent.

Change-Id: I38f1f0b0f05403b9a47acdf88d5d8059f9af6295
2017-07-20 05:58:59 +09:00
Monty Taylor 0ef5671c85
Add web proxying to non-SSL vhost too
The previous add only added to the SSL site. Whoops.

Change-Id: I9c4f63b9cb981e9ca68d63c5021dee151769ce3e
2017-07-20 02:48:39 +09:00
Monty Taylor 3f4f1d9820
Add support for zuul-web and scheduler connections
Zuul v3 has a zuul-web service that provides web streaming and
eventually other web content. Add support for running it. Also add it to
the Apache mod_rewrite proxy info.

While we're adding things, add proxy info for plugin connections, which
should allow receiving github webhook events.

Change-Id: Ic79681b287dbd1a44469da70c680060940734f40
2017-07-12 05:26:04 -05:00
James E. Blair faa53c009c Update zuulv3.conf
Several config options have recently changed, update the conf
file template to match.

Change-Id: I084417b6d622bbb34377c973cc73c4cb0b5e84bb
2017-07-07 10:39:40 -07:00
James E. Blair d7a484b188 Add support for mapping dirs into bubblewrap
I9a8a74a338a8a837913db5e2effeef1bd949a49c introduced these options.

Change-Id: I2e457e05ccee8da33cddf11deb47144c556a024f
2017-06-27 14:26:02 -07:00
Paul Belanger 7028052386
Fix file path to SSL files
The previous patch incorrect add the SSL contents to zuul.conf, we
actually want the file path.

Change-Id: If1f9e9d333d23d22a9f3e2aa320b1b5cd1642d91
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-06-16 20:34:36 -04:00