Commit Graph

1698 Commits

Author SHA1 Message Date
Monty Taylor d95144e43b Retire mordred as infra-root
It has been over two years since I stopped working on OpenDev as
part of my job, and in that time I haven't found enough time to
keep up with the project as much as I otherwise might have hoped.
As a result, it's really not appropriate to continue to hold
elevated privileges, as I no longer have sufficient context to
be helpful.

Best wishes to everyone! Maybe one day I'll be lucky enough to
be able to return.

Change-Id: If2be80520a0c121698c586e3fa93d94d58a41943
2022-12-06 11:04:08 -06:00
Clark Boylan 5cc6c14a6d Remove ethercalc config management
About a month ago we announced [0][1] that this server would be shutdown
and removed on May 31, 2022. Before we can shutdown the server we need
to remove it from config management to prevent Ansible errors. This
change is safe to land now, then on the 31st we can shutdown, snapshot,
and delete the server.

[0] https://lists.opendev.org/pipermail/service-announce/2022-May/000038.html
[1] https://lists.openstack.org/pipermail/openstack-discuss/2022-May/028408.html

Change-Id: Ic44bed01384845e5b6322eeed02dd0932501cdb3
2022-05-30 12:57:48 -07:00
Jeremy Stanley d185aedd7d Decommission status.openstack.org and services
The status.openstack.org server is offline now that it no longer
hosts any working services. Remove all configuration for it in
preparation for retiring related Git repositories.

Also roll some related cleanup into this for the already retired
puppet-kibana module.

Change-Id: I3cfcc129983e3641dfbe55d5ecc208c554e97de4
2022-04-29 16:34:51 +00:00
Clark Boylan 4279e20293 Remove configuration management for ELK stack
We indicated to the OpenStack TC that this service would be going away
after the Yoga cycle if no one stepped up to start maintaining it. That
help didn't arrive in the form of OpenDev assistance (there is effort
to use OpenSearch external to OpenDev) and Yoga has released. This means
we are now clear to retire and shutdown this service.

This change attempts to remove our configuration management for these
services so that we can shutdown the servers afterwards. It was a good
run. Sad to see it go but it wasn't sustainable anymore.

Note a follow-up will clean up elastic-recheck which runs on the status
server.

Depends-On: https://review.opendev.org/c/opendev/base-jobs/+/837619
Change-Id: I5f7f73affe7b97c74680d182e68eb4bfebbe23e1
2022-04-18 10:04:06 -07:00
Clark Boylan a5f06418b6 Remove config management for subunit2sql workers
The openstack health service is being shutdown and retired. That
services was the only service that relied on the subunit2sql workers.
This means we can shutdown and retire the subunit2sql workers. This is
one step of that process.

Change-Id: Ibd02faaeba888dfcd1f512f4dd3a7d768497fc16
2022-04-18 10:01:57 -07:00
Ghanshyam Mann d37b9ee005 Retire opendev/puppet-openstack_health: remove from system-config
opendev/puppet-openstack_health is retiring(needed-by) so we need to
remove this puppet module from system cofig.

Needed-By: https://review.opendev.org/c/opendev/puppet-openstack_health/+/836711
Change-Id: I2ea259aaec0d2d14447dcd55931b3fa51cd04831
2022-04-06 13:44:49 -05:00
Jeremy Stanley 89c4fd9b3d Remove configuration management for wiki servers
We never finished puppeting the OpenStack wiki, and if we do manage
to get it under configuration management in the future it will
likely not use Puppet anyway. The dev server is already gone, and
deployment has been explicitly disabled for the other, so let's go
ahead and remove the references here and then we should be able to
retire the separate Puppet module we've been hosting.

Change-Id: I3f9ada3eb3d6f16545270135fab994ac460be94b
2022-02-14 22:32:18 +00:00
Jeremy Stanley e9770ef33f Switch translate's IDP to OpenInfraID
The OpenStackID project has been rebranded, and the old
openstackid.org deployment is being retained temporarily in order to
ease transition, but id.openinfra.dev is in place now and intended
as its successor.

Note that when this merges, a manual database edit will be required
to associate every user's new ID with their existing accounts, so
this should only be merged when we're ready to do that part just
prior to deploying and then check it again after to make sure we
didn't race any user additions.

Change-Id: I05a0371e45dacc9da2c1b20cb469647206afe137
2022-01-10 21:21:28 +00:00
Jeremy Stanley 2fc2d4be4e Switch translate-dev's IDP to OpenInfraID
The OpenStackID project has been rebranded, and the old
openstackid.org deployment is being retained temporarily in order to
ease transition, but id.openinfra.dev is in place now and intended
as its successor.

This will be used to test the IDP transition before applying a
similar change and scripted database edits to production.

Change-Id: Ia79f10d34d829784733ae43c9266241a57af9c23
2022-01-10 21:21:28 +00:00
Jeremy Stanley 2fbf6d9e7a Stop managing OpenStackID servers
The Open Infrastructure Foundation's developers who maintain the
OpenStackID software are taking over management of the site itself,
and have deployed it on new servers. DNS records have already been
updated to the new IP address, so it's time to clean up our end in
preparation for deleting the old servers we've been running.

OpenStackID is still used by some services we run, like RefStack and
Zanata, and we're still hosting the OpenStackID Git repository and
documentation, so this does not get rid of all references to it.

Change-Id: I1d625d5204f1e9e3a85ba9605465f6ebb9433021
2021-08-31 19:53:13 +00:00
Ian Wienand 5e52befdfa Remove paste01.openstack.org
This has been replaced by paste01.opendev.org and Ansible deployment.

Change-Id: I0f8f5374a3f5d269b317bde4ae2b37435e0871d5
2021-07-15 23:25:10 +00:00
Zuul fe6581f89f Merge "Cleanup eavesdrop puppet references" 2021-06-11 07:45:46 +00:00
Ian Wienand 8a1f6d9764 Cleanup eavesdrop puppet references
Cleanup documenation, puppet references and the eavesdrop_opendev
group.

Change-Id: I67096d8eced0be54db9b1ee277b24602d8c20f00
2021-06-10 09:02:23 +10:00
Ian Wienand 7de885b5ee Cleanup ask.openstack.org
This was retired with I8a31f8fcf9b3064c0ae58e463a6014dc14b518a7

Change-Id: Ieafac856b0feb91f41f05084aa669e2ccb92569d
2021-06-08 14:35:28 +10:00
Clark Boylan 0116b2d025 Assort IRC TODOs
We're moving to OFTC and this tries to capture the various types of
updates for bots and docs we'll need to do. I don't expect this to
be complete, but adds some good reminder for a few things we don't
want to miss.

Change-Id: I09f4c7aa1a2eb8cd167439d58ab4222f5e63a4b1
2021-05-29 17:18:55 +00:00
Clark Boylan 399ade787b More puppetry and inventory cleanups
This cleans up ask-staging which hasn't been a thing in a log time.
We remove some puppet stubs for nodepool builders (they are all ansible
now).

We also cleanup the inventory file to remove corvustest, lists-dev,
pbx, mirror-update*.openstack.org (is opendev.org now), and sort the
LE list.

Change-Id: I8da025640e16bf6e8aca1eb6ec7799d26bd03f12
2021-05-27 14:49:39 -07:00
Clark Boylan 9a085ab46e Switch openstackid to LE certs
The previous change should provision the certs for us. If we are happy
with the results then we can land this to swap production over.

Change-Id: I5b0de65a245c20763eca3165ca7076e5fb2d69a6
2021-05-26 13:28:28 -07:00
Clark Boylan 1fae3234eb Switch translate to LE cert
Once we are happy with the results of the parent change we can land this
one to switch translate's apache over to using the LE cert that was
provisioned.

Change-Id: I09ab944156d974a5cc45c4ab3e3c56cdd6fe0d36
2021-05-24 12:48:53 -07:00
Clark Boylan a36b76bb51 Switch storyboard to LE cert
Once we are happy with the newly provisioned LE cert for storyboard we
can land this change to swap apache2 over to it.

Change-Id: Ib77ce8c0b6927a85f09b857ca67ad56059898a84
2021-05-24 12:41:11 -07:00
Clark Boylan 2fb8998088 Switch ethercalc to the new LE cert
Once we are happy with the ethercalc LE cert we can land this change to
update the apache config to use the LE cert.

Change-Id: Ic35031fb03c928ba4089f292c4d714d4844f29fe
2021-05-24 08:26:10 -07:00
Zuul 9fbd1ccf2c Merge "Ansible mailman configs" 2021-05-19 15:55:09 +00:00
Zuul 4403289ef7 Merge "Cleanup ssl_cert_check puppet components" 2021-05-12 06:02:37 +00:00
Clark Boylan 4c4e27cb3a Ansible mailman configs
This converts our existing puppeted mailman configuration into a set of
ansible roles and a new playbook. We don't try to do anything new and
instead do our best to map from puppet to ansible as closely as
possible. This helps reduce churn and will help us find problems more
quickly if they happen.

Followups will further cleanup the puppetry.

Change-Id: If8cdb1164c9000438d1977d8965a92ca8eebe4df
2021-05-11 08:40:01 -07:00
Ian Wienand e0acf4a68d Retire Asterisk service
As announced in [1], retire the Asterisk PBX service

[1] http://lists.opendev.org/pipermail/service-discuss/2021-March/000198.html

Change-Id: I527eb3423831c6a155228b6d79428681f60a3273
2021-05-07 09:53:17 +10:00
Ian Wienand 159ada0e7c Cleanup ssl_cert_check puppet components
This migrated to Ansible with
Idbe084f13f3684021e8efd9ac69b63fe31484606.  Remove the now unused
puppet components.

Change-Id: I500d6eefcb64f4941e216b8590f4cd60ceec0811
2021-05-05 10:22:01 +10:00
Jeremy Stanley 1df1001cb4 Deprovision Limesurvey config management and docs
The Limesurvey service hosted at survey.openstack.org was a beta
which saw limited use. The platform it runs on, Xenial, is now EOL
from Ubuntu/Canonical and in order to upgrade to a newer
distribution release we would need to rewrite all the configuration
management (the version of Puppet supported by newer Ubuntu is not
backward-compatible with what we've been running).

If a similar service becomes interesting to users of our
collaboratory in the future, it will need to be reintroduced with
freshly written configuration management anyway. The old configs and
documentation remain in our Git history should anyone wish to use
them as inspiration.

Change-Id: I59b419cf112d32f20084ab93eb6f2417a7f93fdb
2021-05-01 15:12:00 +00:00
Zuul cb5898ae0a Merge "Remove firehose.openstack.org" 2021-04-14 18:50:16 +00:00
Clark Boylan 2eebb858af Remove firehose.openstack.org
Once we are satisfied that we have disabled the inputs to firehose we
can land this change to stop managing it in config management. Once that
is complete the server can be removed.

Change-Id: I7ebd54f566f8d6f940a921b38139b54a9c4569d8
2021-04-13 13:51:48 -07:00
Ian Wienand db76061c71 Stop managing planet01.openstack.org
This server has been retired.
If141aca5efbdbe60c91ceefaa4e05c98cd0ba5bb has redirected this.

Change-Id: I8d3c089e6e845d98a46ae39c0b32b1c845436add
2021-04-13 16:17:14 +10:00
Zuul df9a85e45c Merge "kerberos: switch servers to Ansible control" 2021-03-17 04:03:03 +00:00
Ian Wienand 2254b6e43d kerberos: switch servers to Ansible control
This is a follow-on to I60b40897486b29beafc76025790c501b5055313d to
switch the KDC servers to Ansible control and remove any related
puppet configuration.

Change-Id: Ib8f6ec657ca10a3ba648bd154a035fc3d8da4be5
2021-03-17 08:30:52 +11:00
Ian Wienand 018a14e34f refstack: cleanup old puppet
Remove old puppet configuration for the restack service, which is now
managed by Ansible.

Change-Id: I6b6dfd0f8ef89a5362f64cfbc8016ba5b1a346b3
2021-03-17 07:06:53 +11:00
Ian Wienand 39ffc685d6 backups: remove all bup
All hosts are now running thier backups via borg to servers in
vexxhost and rax.ord.

For reference, the servers being backed up at this time are:

 borg-ask01
 borg-ethercalc02
 borg-etherpad01
 borg-gitea01
 borg-lists
 borg-review-dev01
 borg-review01
 borg-storyboard01
 borg-translate01
 borg-wiki-update-test
 borg-zuul01

This removes the old bup backup hosts, the no-longer used ansible
roles for the bup backup server and client roles, and any remaining
bup related configuration.

For simplicity, we will remove any remaining bup cron jobs on the
above servers manually after this merges.

Change-Id: I32554ca857a81ae8a250ce082421a7ede460ea3c
2021-02-16 16:00:28 +11:00
Ian Wienand 61e9d0948a Remove AFS puppet
This has all been replaced by Ansible roles and is no longer used

Change-Id: Ic807498ad3ca4f305b168464b86fe197a61b4d13
2021-01-21 07:08:37 +11:00
Ian Wienand 368466730c Migrate codesearch site to container
The hound project has undergone a small re-birth and moved to

 https://github.com/hound-search/hound

which has broken our deployment.  We've talked about leaving
codesearch up to gitea, but it's not quite there yet.  There seems to
be no point working on the puppet now.

This builds a container than runs houndd.  It's an opendev specific
container; the config is pulled from project-config directly.

There's some custom scripts that drive things.  Some points for
reviewers:

 - update-hound-config.sh uses "create-hound-config" (which is in
   jeepyb for historical reasons) to generate the config file.  It
   grabs the latest projects.yaml from project-config and exits with a
   return code to indicate if things changed.

 - when the container starts, it runs update-hound-config.sh to
   populate the initial config.  There is a testing environment flag
   and small config so it doesn't have to clone the entire opendev for
   functional testing.

 - it runs under supervisord so we can restart the daemon when
   projects are updated.  Unlike earlier versions that didn't start
   listening till indexing was done, this version now puts up a "Hound
   is not ready yet" message when while it is working; so we can drop
   all the magic we were doing to probe if hound is listening via
   netstat and making Apache redirect to a status page.

 - resync-hound.sh is run from an external cron job daily, and does
   this update and restart check.  Since it only reloads if changes
   are made, this should be relatively rare anyway.

 - There is a PR to monitor the config file
   (https://github.com/hound-search/hound/pull/357) which would mean
   the restart is unnecessary.  This would be good in the near and we
   could remove the cron job.

 - playbooks/roles/codesearch is unexciting and deploys the container,
   certificates and an apache proxy back to localhost:6080 where hound
   is listening.

I've combined removal of the old puppet bits here as the "-codesearch"
namespace was already being used.

Change-Id: I8c773b5ea6b87e8f7dfd8db2556626f7b2500473
2020-11-20 07:41:12 +11:00
Zuul d3a53e8ec0 Merge "Remove mirror-update server and related puppet" 2020-11-09 21:07:11 +00:00
Ian Wienand c49ece9204 Cleanup grafana.openstack.org
The opendev.org server is in production, cleanup the old puppet-based
host.

Change-Id: I6db3ce929226a23b96234b52ece8b17f4c6a326a
2020-10-29 07:59:42 +11:00
Ian Wienand f8852b76fb Remove mirror-update server and related puppet
This has all transitioned to Ansible and the mirror-update.opendev.org
server now.

Change-Id: I5f82139c981c2716f568b15b118690e943b02d52
2020-10-28 11:39:54 +11:00
Ian Wienand 1b4006757a Cleanup graphite01
Server is replaced with graphite02.opendev.org

Change-Id: Ie6099e935a6a7e10c818d1d3003e44bca11dd13a
2020-09-30 11:55:24 +10:00
smarcet 2f970563c0 OpenstackId config updates
Added cloud storage config

Change-Id: I39cefce0c1910df0fc051817193e14e5a38c3a1e
Signed-off-by: smarcet <smarcet@gmail.com>
2020-09-21 17:40:19 -03:00
smarcet d7a418c024 Updated openstack id to include
message broker configuration

Change-Id: Ia3fe6ddbe92b354b81f5572ba3f6fba60ac3ce31
Signed-off-by: smarcet <smarcet@gmail.com>
2020-09-21 09:02:09 -03:00
Clark Boylan 32ff621637 Cleanup old puppet management of release-volumes.py
This script has been moved into management done by ansible and is
executing on mirror-update not afsdb01. Cleanup the unused dead code.

Change-Id: Idc1c10cc968eef5ec1aeece70bad7606a7607269
2020-06-09 15:03:44 -07:00
Monty Taylor 8c9b4af143 Stop cloning more puppet modules
Previous review pointed out some additional modules we probably
aren't using any longer.

Remove the openafs::client section from openstack_project::server
because we're doing this with ansible now.

Depends-On: https://review.opendev.org/733890
Change-Id: Ib5104da9cf7d53b77191f48ec185f5d667d51944
2020-06-05 12:09:30 -05:00
Ian Wienand c9215801f0 Generate ssl check list directly from letsencrypt variables
This autogenerates the list of ssl domains for the ssl-cert-check tool
directly from the letsencrypt list.

The first step is the install-certcheck role that replaces the
puppet-ssl_cert_check module that does the same.  The reason for this
is so that during gate testing we can test this on the test
bridge.openstack.org server, and avoid adding another node as a
requirement for this test.

letsencrypt-request-certs is updated to set a fact
letsencrypt_certcheck_domains for each host that is generating a
certificate.  As described in the comments, this defaults to the first
host specified for the certificate and the listening port can be
indicated (if set, this new port value is stripped when generating
certs as is not necessary for certificate generation).

The new letsencrypt-config-certcheck role runs and iterates all
letsencrypt hosts to build the final list of domains that should be
checked.  This is then extended with the
letsencrypt_certcheck_additional_domains value that covers any hosts
using certificates not provisioned by letsencrypt using this
mechanism.

These additional domains are pre-populated from the openstack.org
domains in the extant check file, minus those openstack.org domain
certificates we are generating via letsencrypt (see
letsencrypt-create-certs/handlers/main.yaml).  Additionally, we
update some of the certificate variables in host_vars that are
listening on port !443.

As mentioned, bridge.openstack.org is placed in the new certcheck
group for gate testing, so the tool and config file will be deployed
to it.  For production, cacti is added to the group, which is where
the tool currently runs.  The extant puppet installation is disabled,
pending removal in a follow-on change.

Change-Id: Idbe084f13f3684021e8efd9ac69b63fe31484606
2020-05-20 14:27:14 +10:00
Ian Wienand 45201f3d66 Remove puppet mirror support
Remove the separate "mirror_opendev" group and rename it to just
"mirror".  Update various parts to reflect that change.

We no longer deploy any mirror hosts with puppet, remove the various
configuration files.

Depends-On: https://review.opendev.org/728345
Change-Id: Ia982fe9cb4357447989664f033df976b528aaf84
2020-05-16 10:14:25 +10:00
Monty Taylor e0619f17f1 Run nodepool launchers with ansible and containers
We don't run start in prod normally but we do need to run
it in the gate.

Change-Id: Iec50684280409eb978bf5638bf74ae16fad8aa26
2020-04-30 17:37:22 +00:00
Zuul b21a8e58cf Merge "Run Zuul using Ansible and Containers" 2020-04-24 16:31:42 +00:00
Monty Taylor f0b77485ec Run Zuul using Ansible and Containers
Zuul is publishing lovely container images, so we should
go ahead and start using them.

We can't use containers for zuul-executor because of the
docker->bubblewrap->AFS issue, so install from pip there.

Don't start any of the containers by default, which should
let us safely roll this out and then do a rolling restart.
For things (like web or mergers) where it's safe to do so,
a followup change will swap the flag.

Change-Id: I37dcce3a67477ad3b2c36f2fd3657af18bc25c40
2020-04-24 09:18:44 -05:00
Monty Taylor 9fd2135a46 Split eavesdrop into its own playbook
Extract eavedrop into its own service playbook and
puppet manifest. While doing that, stop using jenkinsuser
on eavesdrop in favor of zuul-user.

Add the ability to override the keys for the zuul user.

Remove openstack_project::server, it doesn't do anything.

Containerize and anisblize accessbot. The structure of
how we're doing it in puppet makes it hard to actually
run the puppet in the gate. Run the script in its own
playbook so that we can avoid running it in the gate.

Change-Id: I53cb63ffa4ae50575d4fa37b24323ad13ec1bac3
2020-04-23 14:34:28 -05:00
Monty Taylor d5c68c5131 Split codesearch into its own playbook
Make a service playbook, manifest and jobs for codesearch.

Remove openstack_project::server - it doesn't do anything.

Change-Id: I44c140de4ae0b283940f8e23e8c47af983934471
2020-04-21 13:18:28 -05:00