Commit Graph

51 Commits

Author SHA1 Message Date
James E. Blair 41d8e478a5 Remove "sql connection" backwards compatability for database
In 4.0 we deprecated connections using the "sql" driver in favor of
using the new "database" config file section.  Remove the backwards
compatible handling of that so that "sql" connections or lack of
"database" section report an error.

Change-Id: I7e592cf5ff63f73f487e41bb6e3e4a4ae523e3fc
2022-01-25 16:07:08 -08:00
James E. Blair 215c96f500 Remove gearman server
The gearman server is no longer required.  Remove it from tests and
the scheduler.

Change-Id: I34eda003889305dadec471930ab277e31d78d9fe
2022-01-25 06:44:17 -08:00
James E. Blair 7d7d2f9f2a Remove time database
We can obtain the same information from the SQL database now, so
do that and remove the filesystem-based time database.  This will
help support multiple schedulers (as they will all have access to
the same data).

Nothing in the scheduler uses the state directory anymore, so clean
up the docs around that.  The executor still has a state dir where
it may install ansible-related files.

The SQL query was rather slow in practice because it created a
temporary table since it was filtering mostly by buildset fields
then sorting by build.id.  We can sort by buildset.id and get nearly
the same results (equally valid from our perspective) much faster.

In some configurations under postgres, we may see a performance
variation in the run-time of the query.  In order to keep the time
estimation out of the critical path of job launches, we perform
the SQL query asynchronously.  We may be able to remove this added
bit of complexity once the scale-out-scheduler work is finished
(and/or we we further define/restrict our database requirements).

Change-Id: Id3c64be7a05c9edc849e698200411ad436a1334d
2021-09-27 11:54:33 -07:00
Tristan Cacqueray 0dbd8c0784 prometheus: add options to start the server and process collector
This change adds a new prometheus_port option to start a metric server
to be scrapped by a prometheus service. By default, the server exposes
process informations.

Change-Id: Ie329df6adc69768dfdb158d00283161f8b70f07a
2021-04-26 14:47:36 +00:00
James E. Blair 3647139920 Move key_store_password to keystore section in zuul.conf
This is likely to be needed by executors as well since passing
decrypted secrets to the executors via zookeeper has the same
encrypted-at-rest concerns as they keystore itself.  To avoid
confusion around executors needing a zuul.conf with a scheduler
section, start a new keystore section which we can later indicate
is used by schedulers and executors.  It also makes it convenient
to add new options (like those dealing with rotation, or even
using an external keystore).

Also change some log levels from debug to info where it's useful
for the operator to know that the backup keystore was used (or
a key was generated).

Change-Id: If2491bbe4eb80b76435a274cf5354a4918315e65
2021-04-14 06:42:44 -07:00
Simon Westphahl dd2d7fee4c Store secrets keys and SSH keys in Zookeeper
As a preparation for the HA scheduler, project secrets keys and SSH keys
will now also be stored in Zookeeper. All private data in Zookeeper will
be encrypted at rest.

Existing keys on the filesystem will be automatically imported into
Zookeeper and new keys will still be available as files for backup.

Change-Id: I2a7d1a555f1db1f2178d3bb2f06756ecc8bc7a81
2021-04-14 08:22:07 +02:00
Matthieu Huin b599c7249d authentication config: add optional max_validity_time, skew
The Zuul admin can configure authenticators with an optional
"max_validity_time" field, which is the maximum age in seconds
for a valid authentication token. By default there is no
maximum age set for tokens, except the one deduced from
the token's "exp" claim.
If "max_validity" is set, tokens without an "iat" claim will
be rejected.

This is meant as an extra security to avoid accidentally issueing
very long lived tokens through the CLI.

The "skew" field can be used to mitigate clocks discrepancies
between Zuul and a JWT emitter.

Change-Id: I9351ca016b60050b5f3b3950b840d5f719e919ce
2019-12-10 16:39:29 +01:00
mhuin 19474fb62f Web: plug the authorization engine
Add an "authorize_user" RPC call allowing to test a set of claims
against the rules of a given tenant. Make zuul-web use this call
to authorize access to tenant-scoped privileged actions.

Change-Id: I50575f25b6db06f56b231bb47f8ad675febb9d82
2019-07-30 15:32:31 +00:00
mhuin 7a622a5823 Add Authorization Rules configuration
Allow an operator to define authorization rules.
Allow an operator to add authorization rules to a tenant.
Add a rule parser and a rule registry.

The authZ engine is not plugged in yet.

Change-Id: I3a86c6c7d62ad2bce68a98dbd2fff18549b94fb9
2019-07-30 15:32:15 +00:00
mhuin 9d86c00111 Zuul CLI: allow access via REST
Users can set the [webclient] section in their zuul.conf file so that the CLI
relies on REST calls rather than RPC. The CLI accepts a new --auth-token
argument allowing remote users to use privileged REST endpoints.

Change-Id: I5a07fccfd787246c4c494db592b575fbdf90ddb1
2019-07-30 10:46:17 +02:00
Matthieu Huin 6a7235fb50 web: add tenant and project scoped, JWT-protected actions
A user with the right JSON Web Token (JWT) can trigger a autohold,
reenqueue or dequeue a buildset from the web API.

The Token is expected to include a key called "zuul.admin" that
contains a list of the tenants the user is allowed to perform
these actions on.

The Token must be passed as a bearer token in an Authorization header.

The Token is validated thanks to authenticator declarations in Zuul's
configuration file.

Change-Id: Ief9088812f44368f14234ddfa25ba872526b8735
2019-07-10 12:11:14 +02:00
James E. Blair 46e48d7b97 Move SQL web handler to driver
The only rest API endpoint that uses sql queries is
/api/tenant/{tenant}/builds.  There's no connection in there, which
means it doesn't make sense for that to be attached to a sql connection
(which is currently the case).  Moreover, it doesn't make sense for
*every* tenant's endpoint to be attached to the *same* connection.

In other words, the current situation only allows for a single sql
connection system-wide, even if someone is using different connections
per tenant.

Moving the handler for the endpoint into the sql driver means that it
can dispatch the query to the appropriate connection for a given tenant
(since a tenant is always implied by the REST endpoint).

Moreover, the *rest* of the system actually allows multiple connections
within a single tenant, and we should support that here, but I don't
immediately have a solution of how to handle pagination across queries
that span multiple connections.  This is an improvement in that it is
now tenant-scoped, but it's not ideal.

This also removes the (undocumented!) sql_connection_name config file
option.

It also uses the tenant name from the path to constructe the query so
that it always includes the correct tenant (this eliminates the
inadvertant ability for one tenant to query another tenant's builds).

The internal API here isn't great, but it will get cleaned up in the
next patch which converts to cherrypy.

Change-Id: Ie1f19f0b392d4c010ef43dc6220ff1c8667f5a4a
2018-05-31 09:08:53 -07:00
Tristan Cacqueray 531a880a5d mqtt: add basic reporter
This change adds a MQTT reporter to publish build results message.

Change-Id: I5a9937a7952beac5c77d83ab791d48ff000b447b
2018-05-02 07:20:46 +00:00
Tobias Henkel e0bad8dc05
Remove webapp
The webapp has been superseeded by zuul-web now so remove it
completely.

Change-Id: I8125a0d7f3aef8fa7982c75d4650776b6906a612
2018-01-29 21:21:00 +01:00
Tobias Henkel 940da00e9b
Move status_url from webapp to web section
The webapp will be fully replaced by zuul-web so also move the
status_url setting there.

Change-Id: I8278d9ca81ed7b0a2a2189d42b8b69c5eea2bab5
2018-01-29 14:16:28 +01:00
Tristan Cacqueray daa95de3ac web: add /{tenant}/builds route
This change adds a SqlHandler to query the sql reporter database from
zuul-web through the /{tenant}/builds.json controller.

This change also adds a /{tenant}/builds.html basic web interface.

Change-Id: I423a37365316cc96ed07ad0895c7198d9cff8be5
2017-12-06 12:51:39 +00:00
Tristan Cacqueray 3c2d39dd86 web: add Cache-Control to static files
This change add the Cache-Control header to static files' response.

Change-Id: Ibdf1c35bad378507162d807cf5acdf13fc3fab88
2017-11-29 06:22:27 +00:00
James E. Blair ded241e598 Switch statsd config to zuul.conf
The automatic statsd configuration based on env variables has
proven cumbersome and counter-intuitive.  Move its configuration
into zuul.conf in preparation for other components emitting stats.

Change-Id: I3f6b5010d31c05e295f3d70925cac8460d334283
2017-10-13 14:04:42 -07:00
Tristan Cacqueray a7586c96a7 Add gearman server port configuration
This change adds the port configuration option to set a custom port
for the gearman server.

Change-Id: I1b65f93fa0403ff10e00a97afcdb4a3b512eb372
2017-08-29 11:08:39 +00:00
Tobias Henkel 34ee088603 Remove zuul_url from merger config
Currently the zuul_url is not used anywhere but still a required
merger setting. This removes it.

Change-Id: I627c8a18015f4c148c28d2f7e735b30cc1ef3862
2017-07-31 22:28:35 +02:00
Monty Taylor 01380dd885
Change name and document the bind_mount config paths
The content in these can be a file or a directory - so _dirs is
confusing. Change it to _paths and document it.

Change-Id: Ida38766cd3d440d75a6dc55035a54e0804e03760
2017-07-28 17:30:45 -05:00
Tristan Cacqueray 7d04c4d738 Fix status_url section in zuul.conf-sample
The status_url needs to be in the [webapp] section

Change-Id: I1568d7e6e62682bfca87979fc71db5648f1ed72e
2017-07-13 05:09:59 +00:00
Jenkins e128b517d0 Merge "Add web-based console log streaming" into feature/zuulv3 2017-07-10 20:09:21 +00:00
Monty Taylor 51139a0682 Add web-based console log streaming
zuul now provides socket-based console streaming, which is super cool.
In order to have jenkins parity with web streaming, we need to provide a
websocket (javascript in browsers can't really connect to random ports
on servers)

After surveying the existing python websocket options, basically all of
them are based around twisted, eventlet, gevent or asyncio. It's not
just a thing we can easily deal with from our current webob/paste
structure, because it is a change to the fundamental HTTP handling.
While we could write our own websocket server implementation that was
threaded like the rest of zuul, that's a pretty giant amount of work.

Instead, we can run an async-based server that's just for the
websockets, so that we're not all of a sudden putting async code into
the rest of zuul and winding up frankensteined. Since this is new code,
using asyncio and python3 seems like an excellent starting place.

aiohttp supports running a websocket server in a thread. It also
supports doing other HTTP/REST calls, so by going aiohttp we can set
ourselves up for a single answer for the HTTP tier.

In order to keep us from being an open socket relay, we'll expect two
parameters as the first message on the websocket - what's the zuul build
uuid, and what log file do we want to stream. (the second thing,
multiple log files, isn't supported yet by the rest of zuul, but one can
imagine a future where we'd like to support that too, so it's in the
protocol) The websocket server will then ask zuul over gearman for the
IP and port associated with the build and logfile and will start
streaming it to the socket.

Ultimately we'll want the status page to make links of the form:

  /console.html?uuid=<uuid>&logfile=console.log

and we'll want to have apache map the websocket server to something like
/console.

Co-Authored-By: Monty Taylor <mordred@inaugust.com>

Change-Id: Idd0d3f9259e81fa9a60d7540664ce8d5ad2c298f
2017-07-10 10:32:28 -04:00
James E. Blair fdb111d7a5 Move zookeeper_hosts to zookeeper section
To anticipate future use of zookeeper by more than just the scheduler,
create a new 'zookeeper' section in zuul.conf similar to the 'gearman'
section.  The 'hosts' entry is the only setting there now, but others
may be added later.

Change-Id: Ib0bb2dfdc96fa0eb4aad2b4700de2f2004ea15f2
2017-07-05 14:42:47 -07:00
James E. Blair edc4016b4b Correct sample zuul.conf
And add missing zuul.conf config entries.

Change-Id: I776826e7ff1fff2edd6f6d332c20a24af1a4eb58
2017-07-05 14:38:53 -07:00
Tristan Cacqueray 44aef15d6e executor: run trusted playbook in a bubblewrap
This change renames untrusted_wrapper to execution_wrapper and uses
bubblewrap for both trusted and untrusted playbooks by default.

This change adds new options to the zuul.conf executor section to let
operators define what directories to mount ro or rw for both context:
* trusted_ro_dirs/trusted_rw_dirs, and
* untrusted_ro_dirs/untrusted_rw_dirs

Change-Id: I9a8a74a338a8a837913db5e2effeef1bd949a49c
Story: 2001070
Task: 4687
2017-06-17 02:43:19 +00:00
Paul Belanger 0a21f0a1d5
Add ssl support to gearman / gearman_server
Enable SSL support for gearman. We also created an new SSLZuulBaseTest
class to provide a simple way to use SSL end to end where possible. A
future patch will enable support in zookeeper.

Change-Id: Ia8b89bab475d758cc6a021988f8d79ead8836a9d
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-06-14 10:10:45 -04:00
Jamie Lennox f12b1d317a Re-add the ability to set username on zuul-executor
Sometimes we need to log in to a nodepool node using a username of
something other than zuul. This used to be possible by setting that
[launcher] username= property. Re-enable it with the [executor]
default_username= property.

default_username is used instead of simply username as it is likely in
future this information will be able to be supplied by nodepool or other
in a node or image specific way. At which time that information will be
used in priority to the default specified in zuul.

Change-Id: Icf657b4f0bbe34e182307b9eea0cd64a8d813464
2017-04-19 13:59:45 +10:00
James E. Blair a22c541c40 Remove more swift configurations
These slipped through the earlier swift removal.

Change-Id: Iecd71563e1904f3bb8028fdafb00b10d95bc8ab2
2017-03-17 16:18:03 -07:00
Joshua Hesketh 25695cbb51 Merge branch 'master' into feature/zuulv3
Change-Id: I37a3c5d4f12917b111b7eb624f8b68689687ebc4
2017-03-06 09:40:04 -08:00
Paul Belanger bbb4875072 Plumb zookeeper config settings into zuul-scheduler
We need a way to configure zuul to connect to zookeeper.  Add a new
setting into the zuul section of zuul.conf. Multiple zookeeper hosts
would be configured as:

  [zuul]
  zookeeper_hosts=nodepool.openstack.org:2181,foo.openstack.org:1337

Change-Id: I4640c4af5ba925e099978b64c59ed29d723b1230
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-02-21 11:12:04 -05:00
Jenkins 4da3416c3e Merge "Add support for sqlalchemy reporter" 2017-02-16 00:42:21 +00:00
Jenkins 10b71820c4 Merge "Set keepalives for gerrit connections" 2017-02-09 14:49:09 +00:00
Joshua Hesketh d78b44878c Add support for sqlalchemy reporter
This will allow us to enter results from all jobs for
use with the openstack-health dashboard.

Depends-On: I08dbbb64b3daba915a94e455f75eef61ab392852
Change-Id: I28056d84a3f6abcd8d9038a91a6c9a3902142f90
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2017-02-06 12:47:29 -05:00
Evgeny Antyshev 9f0a3229f7 Set keepalives for gerrit connections
Some gerrit clients connect through stateful firewalls,
which are occasionally "flushed", which leads to hanging of the connection.
This could be avoided by tcp keepalives.
To turn keepalives on, set "keepalive" in gerrit connection attributes to non-zero
(0 means no keepalives sent)
The default is turn it on with timeout of 60 seconds.

Change-Id: Ic3d11e0ac2108f02832e2114ba8bc1adcc87cb4f
2016-08-17 08:59:08 +00:00
Paul Belanger 88ef0ea8ab Expose webapp listen_address and port
Move the hardcoded settings into the zuul.conf file to allow a user to
better customize them.

Change-Id: I9c817efc615ac3e8f8a7f4680dad14ef6cf3cc3b
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2015-12-23 11:57:02 -05:00
Joshua Hesketh fe485c6091 Document the new connections in zuul
Add in the new documentation for 'connections'.

Change-Id: Iafb2412e47e12e2bf9c92512ffbb00c1f9a19a57
2015-12-06 14:48:32 +11:00
Joshua Hesketh 36c3fa5749 Send swift upload instructions to workers
Have zuul send signed credentials as part of the job for workers to
consume and upload assets to a defined location.

Zuul currently doesn't care about logs however this change will
suggest a LOG_PATH to builders as a unqiue destination prefix
allowing zuul to know preemptively the destination.
The workers are still required to send a URL of the final location.

Change-Id: I042cdd2dd2407f381cafcabc5c6b83d9b9a9eb00
2014-03-25 10:46:06 +11:00
James E. Blair 4076e2b432 Split the merger into a separate process
Connect it to Zuul via Gearman.  Any number of mergers may be
deployed.

Directly find the pipeline for a build when processing a result,
so that the procedure is roughly the same for build and merge
results.

The timer trigger currently requires the gerrit trigger also be
configured.  Make that explicit inside of the timer trigger so
that the scheduler API interaction with triggers is cleaner.

Change-Id: I69498813764753c97c426e42d17596c2ef1d87cf
2014-02-17 11:47:15 -08:00
Arx Cruz b1b010d393 Add the possibility to pass the zuul url
This patch adds the zuul_url option in zuul conf file
in order to pass ZUUL_URL to Jenkins, which will be used
by devstack-vm-gate-wrap.sh.
Documentation added in launchers.rst and zuul.rst
explaining how this new option works.

Change-Id: I840423cc06fdfdacd301d30be3e0b3e589e563e9
2013-11-26 18:57:17 -02:00
Joshua Hesketh 5fea867c70 Add support for emailing results via SMTP
Utilises the new reporter plugin architecture to add support for
emailing success/failure messages based on layout.yaml.

This will assist in testing new gates as currently after a job has
finished if no report is sent back to gerrit then only the workers
logs can be consulted to see if it was successful. This will allow
developers to see exactly what zuul will return if they turn on
gerrit reporting.

Change-Id: I47ac038bbdffb0a0c75f8e63ff6978fd4b4d0a52
2013-09-02 09:55:49 +10:00
Joshua Hesketh 1879cf721c Separate reporters from triggers
Allows multiple reports per a patchset to be sent to pluggable
destinations. These are configurable per pipeline and, if not
specified, defaults to the legacy behaviour of reporting back only
to gerrit.

Having multiple reporting methods means only certain success/failure
/start parameters will apply to certain reporters. Reporters are
listed as keys under each of those actions.

This means that each key under success/failure/start is a reporter and the
dictionaries under those are sent to the reporter to deal with.

Change-Id: I80d7539772e1485d5880132f22e55751b25ec198
2013-08-29 13:17:12 +10:00
James E. Blair 77cc7b847e Include internal gearman server in docs
Change-Id: I668206d7a07f665309ca392a8e5f1702c278c47e
2013-07-15 13:22:37 -07:00
Paul Belanger b67aba1d94 Add the ability to set git user.email and user.name
It is possible the host system does not have git properly configured,
which results in merge failures because the git client is complain. For
example:

  GitCommandError: 'git merge FETCH_HEAD' returned exit status 128:
  *** Please tell me who you are.

  Run

    git config --global user.email "you@example.com"
    git config --global user.name "Your Name"

  to set your account's default identity.
  Omit --global to set the identity only in this repository.

Now we can pass user.name and user.email settings to git, if configured
to do so.

Change-Id: I896194d8d1f5334026954b02f3a1a8dd82bed2ac
Signed-off-by: Paul Belanger <paul.belanger@polybeacon.com>
Reviewed-on: https://review.openstack.org/29015
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Tested-by: Jenkins
2013-05-15 18:56:01 +00:00
Antoine Musso 2747501dee let us specify Gerrit root URL
Our code was expecting the Gerrit web interface to be available at the
root of the Gerrit server which is not always the case since people
could publish the Gerrit web interface under a subdirectory. In such
configuration, the existing assumption prevents Zuul from getting
references information.

This patch adds a new optional configuration variable 'baseurl' to the
'gerrit' section. It lets one override the default:

 [gerrit]
 server=gerrit.wikimedia.org
 baseurl=https://gerrit.wikimedia.org/r

Updated Gerrit._getInfoRefs() to use 'baseurl' instead of 'server' when
creating the url to fetch references from. This is still falling back to
the old default whenever gerrit.baseurl is not set.

Change-Id: Ia9910acf46fe82c425911784412356598b3c7eac
Reviewed-on: https://review.openstack.org/16884
Reviewed-by: Jeremy Stanley <fungi@yuggoth.org>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Approved: Clark Boylan <clark.boylan@gmail.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Tested-by: Jenkins
2012-11-26 19:57:55 +00:00
Paul Belanger f9efd0ba52 Fix typo with log_config in zuul.conf-sample
Change-Id: Ieda682233843b4500e8d59d91ac24b36dfd172dd
Signed-off-by: Paul Belanger <paul.belanger@polybeacon.com>
Reviewed-on: https://review.openstack.org/14433
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
2012-10-13 23:45:02 +00:00
Clark Boylan 9b670905d9 Provide link to status page in Gerrit comments.
When commenting on Gerrit that jobs are beginning include a link to the
zuul status page if one is configured.

Change-Id: Iba5f315f4baa006939dbdcb07b0758e570242ca8
Reviewed-on: https://review.openstack.org/13858
Reviewed-by: James E. Blair <corvus@inaugust.com>
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Approved: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
2012-10-01 17:04:14 +00:00
Antoine Musso 40b9907cd3 `git_dir` parameter in zuul.conf sample
Change-Id: I95ef3dde919f4ab5ce471295fb7cc1568e42402a
Reviewed-on: https://review.openstack.org/13526
Reviewed-by: Monty Taylor <mordred@inaugust.com>
Reviewed-by: Clark Boylan <clark.boylan@gmail.com>
Approved: James E. Blair <corvus@inaugust.com>
Reviewed-by: James E. Blair <corvus@inaugust.com>
Tested-by: Jenkins
2012-09-26 14:20:22 +00:00
James E. Blair 5d5bc2b92e Add graceful exit.
A SIGUSR1 will cause zuul to queue new events, wait for existing
jobs to finish, then save the queue and exit.

It will likely take quite a while to complete (perhaps an hour),
so it's not implemented as a SIGTERM handler.

Can be used in an init script to implement a graceful restart.

Change-Id: I09fce571e971f16b5d20c5d69d595a05c7f6a4ba
2012-07-06 10:31:57 -07:00