Commit Graph

214 Commits

Author SHA1 Message Date
Paul Belanger d38f969429 Add zuul-registry support
Depends-On: https://review.opendev.org/763101
Change-Id: I67c08418df38d31a01c2ecf114aa52daf8af185c
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2020-11-21 14:41:40 -05:00
Paul Belanger d21c7c01a6 Fix CI issues
This is a catch all to get CI working again. In some cases, like
ansible-lint we will just ignore the warnings for now, and loop back to
clean them up.

Change-Id: I3e09f8c2b470062634d981f2667b913820c58f7f
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2020-11-21 14:41:39 -05:00
Gonéri Le Bouder 1c14b66462
add AWS support
Depends on ansible-role-boto3 to install and configure the lib.

Depends-On: Id2cd819def3e83f92fcd8126a74aa19690dcf4c8

Change-Id: If66d975dedac16ba46522ab8183a7d6cff35d3d3
2020-05-04 12:24:21 -04:00
Paul Belanger a67474e7e1 Switch to windmill for roles
Continue with the change from openstack to opendev for hosting.

Change-Id: I6d3ead6a49c3460484b18f672ead0022e1dc2858
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2019-04-20 15:55:18 -04:00
Paul Belanger bdc5925121 Add additional checks for nginx
Validate nginx service is running and also collect some logs.

Change-Id: I3d3daf0344d978cf72ea171e84e3f1439725d990
Depends-On: https://review.openstack.org/648850
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2019-03-31 15:41:05 -04:00
Paul Belanger 1718c834c5 Recursive delete nodepool elements
We seem to be leaking old files when removed from git.

Change-Id: Ifb380d22b8300cd40f8fbcf92fcc68256d158d69
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2019-03-28 15:50:43 -04:00
Paul Belanger 0829dcf7bb Multi ansible support for zuul-executor
Change-Id: Ib9e07f2f7c8bb34ba2a5d64680cf4c4daa1b0ce7
Depends-On: https://review.openstack.org/646020
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2019-03-23 19:06:16 -04:00
Paul Belanger 1cbd015aff Manually pass secure.conf to nodepool command
This is for when we move our zookeeper information into secure.conf.

Change-Id: Ic68661933edd0738118c58c6a6c38e1c4afb0a04
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2019-03-22 22:13:11 -04:00
Paul Belanger 017053a93a Refactor our testing playbooks
As we try to run a windmill in zuul, it doesn't make sense to also run
our testing roles.  For now, we can move these into our prove playbook,
however long term I think we want to move to testinfra. Then people can
decide to also run testinfra against their production control plane.

Change-Id: I783e2e3b5f6fba224d761446c66aa9a58ecec9ca
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2019-03-20 16:04:58 -04:00
Paul Belanger 313df89a27 Fix gear virtualenv testing
Change-Id: I22fb917e5655d06e129b2cfc8d366109dc884529
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2019-03-20 15:34:05 -04:00
Paul Belanger 99e919a44c Fix assertion tests for nodepool
We didn't look in the right directory, if nodepool was installed into
something other then /opt/venv/nodepool.

Change-Id: Id06e2a807e4a798c9fefd552882e66ba74e242db
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2019-03-18 17:07:07 -04:00
Paul Belanger 401d3f6bbd Remove hardcoded statsd01 host
Since the statsd client only support a single server, we have to use the
first server in the statsd group.

Change-Id: Iebc7a89e7ce7fcb8d30208dd55a7013007cadf93
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2019-03-18 15:08:34 -04:00
Paul Belanger fb4ee9320e Remove statsd configuration
This has moved into windmill-config repo.

Change-Id: Ica10636cf6f5cf20c2a620b2e13a22222e03b9cc
Depends-On: https://review.openstack.org/643948
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2019-03-18 10:03:04 -04:00
Zuul 677bf183f8 Merge "Bootstrap nodes using windmill-ops" 2019-02-28 00:33:15 +00:00
Zuul a0fd5a0d2a Merge "Move sudoers include role into nodepool-builder only" 2019-02-27 16:25:32 +00:00
Paul Belanger 7c173d9ea5 Move playbooks/bastion.yaml into tests folder
This really isn't needed for production deployments, it is more to make
sure our testing in the gate is happy.

Change-Id: Ic8e367892fbb54a57d9864e9cdb8579783a5d83f
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2019-02-27 09:01:08 -05:00
Paul Belanger c77a8dee8f Bootstrap nodes using windmill-ops
Change-Id: I1078056dcf30529da8da86217417dae84428ee3b
Depends-On: https://review.openstack.org/639109
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2019-02-26 21:38:56 -05:00
Paul Belanger 267dc5ac5b Move sudoers include role into nodepool-builder only
Currently, we only need to give nodepool-builder sudoers access, so move
the code into that play only.

Change-Id: I0a39c8ec476e89c11bc758f000b89267d37183e3
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2019-02-26 16:36:58 -05:00
Paul Belanger 0fa7bd5de3 Update nodepool test roles to use nodepool_pip_virtualenv
This is the same as we did for zuul tests.

Change-Id: Ib474d2c2eb771032de4f1cf3ea39ee6b8545e2ce
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2019-01-21 00:35:48 -05:00
Paul Belanger 93f3880da8 Only validate nodepool system services if state is properly set
Like we did with zuul, we also need to do for nodepool.

Change-Id: Id065a1e56484973e9dd9e7cee1026639abbd7c68
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2019-01-20 22:24:39 -05:00
Paul Belanger 25a5b1d00d Update zuul test roles to use zuul_pip_virtualenv
We cannot expect users to use /opt/venv/zuul by default, instead use
zuul_pip_virtualenv setting.

Change-Id: I7ab71ed89248796936ae129c6da92eced9730108
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2019-01-20 18:45:41 -05:00
Paul Belanger ecd50da0f0 Only validate zuul system services if state is properly set
Now that we allow users to set zuul service state to false, don't try to
validate services are enabled.

Change-Id: Id7e4cde03f1148761c0fe16937e286fdfd446ca2
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2019-01-19 21:28:39 -05:00
Paul Belanger 14c6144ee9 Add the ability to disable hosts in inventory
It is possible, you don't want to run windmill on a specific host.
Rather then removing the host from the inventory file, you can now place
it in the disabled inventory group and our playbooks will skip it.

This logic is used by the opendev team when running ansible-playbooks
against their control plane.

Change-Id: I717fc4b951c16c5df6286bfe13f0fe5701cf7abb
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2019-01-18 13:30:03 -05:00
Paul Belanger eb2d859e87 Use False for zuul_connections_ssh items
This is because the openstack.ssh role uses False with when statements,
otherwise we get errors with these keys are missing.

Change-Id: Iad35362b0c7f5d5862f17a919e23a8e291560c7d
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2019-01-17 15:23:53 -05:00
Paul Belanger 3503f51467 Add ssh_known_hosts_content to zuul-connections
It is possible a user wants to update known_hosts for zuul-connections.

Change-Id: Ia74e1cccda7deb395eb71ef647c7dd3464d6e579
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2019-01-07 09:16:19 -05:00
Paul Belanger 61c0df4078 Create zuul-connections host group to manage SSH keys
This group will be used to manage SSH keys needed for github / gerrit
connections.

Change-Id: I17364e75e107cfb1bbe8fed7c3f0fff6c0eb5771
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2019-01-04 23:58:52 +00:00
Zuul b8821015c6 Merge "Fix deprecation warning" 2018-12-21 00:10:35 +00:00
Zuul e9f710714b Merge "Add nginx support to zuul-web" 2018-12-20 23:47:03 +00:00
Paul Belanger 26023ac7e5 Fix deprecation warning
Change-Id: Ia5b5e94a52ae450556202d05ac13eb35f8a149f1
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-12-20 18:25:17 -05:00
Paul Belanger cb3ab2d262 Add nginx support to zuul-web
Our zuul-web nodes will actually require nginx to help serve the static
content from zuul.

Change-Id: I6afaa78968fef59286412c7410a26605d3c971a3
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-12-20 17:59:55 -05:00
Paul Belanger a974997b47 Use -f1 (forks) over serial for gate testing
Because we us a single VM for all-in-one testing, switch to forks to
ensure we only run 1 task at a time.

Change-Id: Ia933ec0efc6d04ac02300f3d82faf26cd583d1d1
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-12-20 17:14:25 -05:00
Paul Belanger 3e7acd2f30 Simplify zuul playbooks
Based on how our inventory file is defined now, we can rework the
playbook process here and save some time.

Change-Id: I0852b3cd6b49c7ac57e1db2c1f3114d563515d84
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-12-18 16:13:23 -05:00
Paul Belanger e34fc20382 Collapse nodepool plays to single file
We don't actually need different plays for these servers, we can just
group them differently.  This should save some time off job runs.

Change-Id: I5e7763e7b406569ed39f6bdd0e381eee1eaabee3
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-12-18 12:16:47 -05:00
Paul Belanger ec7a464e2a Add yamllint to tox -elinters
This adds and enabled strict yamllinting on our playbooks / roles.

Change-Id: Ie66ab53c7d6ed4dee1d18a30bdadd9c679549d47
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-12-12 20:59:38 -05:00
Paul Belanger f56f95f39d Remove left over code from ubuntu-xenial
We no longer support ubuntu-xenial, so we can remove this package repo.

Change-Id: I2b06e281fce4bf91f5aa78f7f280e1fce9a72c33
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-12-07 20:59:36 -05:00
Paul Belanger 5037a1a47a More places for statsd to be optional
Again, if statsd group doesn't have any hosts, don't enable it for our
services.

Change-Id: Ideea60580b37365c006d344ef8d8d2286181f0bf
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-12-07 20:37:44 -05:00
Paul Belanger 74fae84e26 Have openstacksdk manage our clouds.yaml file
We can move this logic into the role now.

Change-Id: I7bcc2b700a3d06ce662dc5a7fb3c938657a7fbf1
Depends-On: https://review.openstack.org/623532
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-12-07 15:23:35 -05:00
Paul Belanger 203ef5a03a Move openstacksdk role into nodepool group
There is no need to do this in both groups, we can move up to the
top-level nodepool one.

Change-Id: I038062eb2409bacfb3e25a69eaa8f1b27c75665f
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-12-06 22:58:15 -05:00
Paul Belanger 232b50dc36 Add specific play to bootstrap python dependencies
For example, zookeeper nodes don't actually need to install the
virtualenv role because zookeeper is java.

Change-Id: I71318029fb558a7087cc2f8db625b48592615ad4
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-12-04 20:50:21 -05:00
Paul Belanger 5cf20687c4 Make statsd optional for nodepool
It is possible a user doesn't want to install statsd, if so don't setup
systemd to use it.

Change-Id: I99c2995069ee70dfa1c24c5a4fb5e93c128e6313
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-12-03 20:00:47 -05:00
Paul Belanger 12f3699294 Switch to ubuntu-bionic for nodepool images
Bionic has been working for some time, lets switch to using it.

Change-Id: I33ee5e113bcd08da4216ca4880af480c0912272b
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-08-20 23:07:39 -04:00
Paul Belanger 8a0415d871
Replace shade with openstacksdk
Nodepool no longer depends on shade. It has been updated to use
openstacksdk.

Change-Id: I0b3b021eebd31e5fb176da95f6726ffbe2f5d625
Depends-On: https://review.openstack.org/588413
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-08-02 21:29:24 -04:00
Paul Belanger 50df1cb5b9
Remove old playbook
We nolonger use build-images.yaml.

Change-Id: I57a53dfc5d5a21bbcecd001c39512635a29afd6a
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-07-28 21:54:05 -04:00
Paul Belanger c61712377b
Switch to import_playbook
Fix the deprecation warning from include.

Change-Id: Ie6778690604c8306d8aa3abebeb9adda0b1cacfd
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-07-28 21:53:18 -04:00
Paul Belanger 0305da4cb9
Reorganize playbooks for site.yaml
Make site.yaml much simpler, incase somebody wants to use
nodepool.yaml or zuul.yaml directly.

Change-Id: I4337c962f7cb9353a73bc598c4b29d33f3fa828c
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-07-28 21:50:03 -04:00
Paul Belanger 5505639aee
Remove duplidate sudoers role
This is done in the bootstrap phase.

Change-Id: I6c14a8599c93a2b374e8a0a49508b5d159a77862
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-07-28 21:48:17 -04:00
Paul Belanger 496111154c
Fix ansible-lint errors
Change-Id: I1f9e95d7ccaab8b30ad2a7dd33d7d6a01cd8f0be
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-07-28 15:08:07 -04:00
Artem Goncharov 419e46ea0c
enable gear logging configuration
Geard logging configuration is not enabled by default. This results in
geard floods /var/log/messages with debug info.
Change 568234 provides default geard logging configuration file. With
this change this configuration is being enabled.

Also collect additional logs for gear.

Depends-On: https://review.openstack.org/568234/
Change-Id: I0bfdb398dd806ec92f15eaff67ca5f8251ec193f
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-05-16 17:13:47 -04:00
Paul Belanger dc74bf5a91
Fix up PATH issues with nodepool-builder / zuul-executor
Both services will launch external applications, but because we didn't
properly include the $PATH to each virtualenv, we had to create
symlinks (we missed ansible-playbook).

We can properly fix this by setting up the PATH variable in our
systemd unit file override.

Change-Id: Ice32334a75c6e7f3bc996e11ac7f4fd73844388e
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-04-16 23:14:00 -04:00
Paul Belanger 7fd5994937
Create users first before starting with nodepool / zuul tasks
It's possible we want to install things like ssh keys or clouds.yaml
files into nodepool / zuul hosts before even starting the installation
process.  This is mostly to workaround issues where nodepool / zuul
will fail to start properly if they are missing.

Change-Id: Ieada13b11678147e9a556f7e76ea17a53bff1010
Signed-off-by: Paul Belanger <pabelanger@redhat.com>
2018-04-15 23:27:25 -04:00