Commit Graph

3129 Commits

Author SHA1 Message Date
Ian Wienand 8739e1448d
translate: fix dump with MySQL 5.7
Recently the hosted database that translate.openstack.org uses was
upgraded to MySQL 5.7.  It turns out there was a breaking
security-related change to MySQL that means you require PROCESS
privileges to dump the DB.  I guess the migration didn't do this, and
via the RAX hosting UI there's no easy way to add this (there's no
root user available by default; perhaps this is a corner case of the
upgrade process).

Rather than trying to figure out the backend, to work around this use
--no-tablespaces to the dump command.  We just need a dump of the
zanata tables as backup, so this is fine.

Change-Id: Idaa35249fe15867103fa7b70dbdc7d135c5e1396
2022-09-13 09:27:10 +10:00
Clark Boylan 5cc6c14a6d Remove ethercalc config management
About a month ago we announced [0][1] that this server would be shutdown
and removed on May 31, 2022. Before we can shutdown the server we need
to remove it from config management to prevent Ansible errors. This
change is safe to land now, then on the 31st we can shutdown, snapshot,
and delete the server.

[0] https://lists.opendev.org/pipermail/service-announce/2022-May/000038.html
[1] https://lists.openstack.org/pipermail/openstack-discuss/2022-May/028408.html

Change-Id: Ic44bed01384845e5b6322eeed02dd0932501cdb3
2022-05-30 12:57:48 -07:00
Jeremy Stanley d185aedd7d Decommission status.openstack.org and services
The status.openstack.org server is offline now that it no longer
hosts any working services. Remove all configuration for it in
preparation for retiring related Git repositories.

Also roll some related cleanup into this for the already retired
puppet-kibana module.

Change-Id: I3cfcc129983e3641dfbe55d5ecc208c554e97de4
2022-04-29 16:34:51 +00:00
Clark Boylan 4279e20293 Remove configuration management for ELK stack
We indicated to the OpenStack TC that this service would be going away
after the Yoga cycle if no one stepped up to start maintaining it. That
help didn't arrive in the form of OpenDev assistance (there is effort
to use OpenSearch external to OpenDev) and Yoga has released. This means
we are now clear to retire and shutdown this service.

This change attempts to remove our configuration management for these
services so that we can shutdown the servers afterwards. It was a good
run. Sad to see it go but it wasn't sustainable anymore.

Note a follow-up will clean up elastic-recheck which runs on the status
server.

Depends-On: https://review.opendev.org/c/opendev/base-jobs/+/837619
Change-Id: I5f7f73affe7b97c74680d182e68eb4bfebbe23e1
2022-04-18 10:04:06 -07:00
Clark Boylan a5f06418b6 Remove config management for subunit2sql workers
The openstack health service is being shutdown and retired. That
services was the only service that relied on the subunit2sql workers.
This means we can shutdown and retire the subunit2sql workers. This is
one step of that process.

Change-Id: Ibd02faaeba888dfcd1f512f4dd3a7d768497fc16
2022-04-18 10:01:57 -07:00
Ghanshyam Mann d37b9ee005 Retire opendev/puppet-openstack_health: remove from system-config
opendev/puppet-openstack_health is retiring(needed-by) so we need to
remove this puppet module from system cofig.

Needed-By: https://review.opendev.org/c/opendev/puppet-openstack_health/+/836711
Change-Id: I2ea259aaec0d2d14447dcd55931b3fa51cd04831
2022-04-06 13:44:49 -05:00
Jeremy Stanley 89c4fd9b3d Remove configuration management for wiki servers
We never finished puppeting the OpenStack wiki, and if we do manage
to get it under configuration management in the future it will
likely not use Puppet anyway. The dev server is already gone, and
deployment has been explicitly disabled for the other, so let's go
ahead and remove the references here and then we should be able to
retire the separate Puppet module we've been hosting.

Change-Id: I3f9ada3eb3d6f16545270135fab994ac460be94b
2022-02-14 22:32:18 +00:00
Jeremy Stanley 2fbf6d9e7a Stop managing OpenStackID servers
The Open Infrastructure Foundation's developers who maintain the
OpenStackID software are taking over management of the site itself,
and have deployed it on new servers. DNS records have already been
updated to the new IP address, so it's time to clean up our end in
preparation for deleting the old servers we've been running.

OpenStackID is still used by some services we run, like RefStack and
Zanata, and we're still hosting the OpenStackID Git repository and
documentation, so this does not get rid of all references to it.

Change-Id: I1d625d5204f1e9e3a85ba9605465f6ebb9433021
2021-08-31 19:53:13 +00:00
Ian Wienand 5e52befdfa Remove paste01.openstack.org
This has been replaced by paste01.opendev.org and Ansible deployment.

Change-Id: I0f8f5374a3f5d269b317bde4ae2b37435e0871d5
2021-07-15 23:25:10 +00:00
Zuul fe6581f89f Merge "Cleanup eavesdrop puppet references" 2021-06-11 07:45:46 +00:00
Ian Wienand 8a1f6d9764 Cleanup eavesdrop puppet references
Cleanup documenation, puppet references and the eavesdrop_opendev
group.

Change-Id: I67096d8eced0be54db9b1ee277b24602d8c20f00
2021-06-10 09:02:23 +10:00
Ian Wienand 7de885b5ee Cleanup ask.openstack.org
This was retired with I8a31f8fcf9b3064c0ae58e463a6014dc14b518a7

Change-Id: Ieafac856b0feb91f41f05084aa669e2ccb92569d
2021-06-08 14:35:28 +10:00
Clark Boylan 3a776f0c30 Swap meetbot networks
This will update meetbot to connect to OFTC using the new opendevmeet
nick. We keep the site name the same for simplicity. However, the
network name updates which causes irclogs to be written to a new
location. We have already copied the old logs from the FreeNode location
to the OFTC location so this should be a noop.

Change-Id: Ie72280ad2129418d7df549f2ba629a891f172496
2021-05-29 08:25:29 -07:00
Clark Boylan 399ade787b More puppetry and inventory cleanups
This cleans up ask-staging which hasn't been a thing in a log time.
We remove some puppet stubs for nodepool builders (they are all ansible
now).

We also cleanup the inventory file to remove corvustest, lists-dev,
pbx, mirror-update*.openstack.org (is opendev.org now), and sort the
LE list.

Change-Id: I8da025640e16bf6e8aca1eb6ec7799d26bd03f12
2021-05-27 14:49:39 -07:00
Clark Boylan 9a085ab46e Switch openstackid to LE certs
The previous change should provision the certs for us. If we are happy
with the results then we can land this to swap production over.

Change-Id: I5b0de65a245c20763eca3165ca7076e5fb2d69a6
2021-05-26 13:28:28 -07:00
Clark Boylan a36b76bb51 Switch storyboard to LE cert
Once we are happy with the newly provisioned LE cert for storyboard we
can land this change to swap apache2 over to it.

Change-Id: Ib77ce8c0b6927a85f09b857ca67ad56059898a84
2021-05-24 12:41:11 -07:00
Clark Boylan 5efc72a33e Cleanup mailman puppetry
We have shifted over to using ansible for managing the listservs.

This also updates our service docs to point at the corret ansible and
not puppet.

Change-Id: I76f01ff1479c5af0a502a060aac2baa1ab622b21
2021-05-19 12:13:16 -07:00
Zuul 4403289ef7 Merge "Cleanup ssl_cert_check puppet components" 2021-05-12 06:02:37 +00:00
Ian Wienand e0acf4a68d Retire Asterisk service
As announced in [1], retire the Asterisk PBX service

[1] http://lists.opendev.org/pipermail/service-discuss/2021-March/000198.html

Change-Id: I527eb3423831c6a155228b6d79428681f60a3273
2021-05-07 09:53:17 +10:00
Ian Wienand 159ada0e7c Cleanup ssl_cert_check puppet components
This migrated to Ansible with
Idbe084f13f3684021e8efd9ac69b63fe31484606.  Remove the now unused
puppet components.

Change-Id: I500d6eefcb64f4941e216b8590f4cd60ceec0811
2021-05-05 10:22:01 +10:00
Jeremy Stanley 1df1001cb4 Deprovision Limesurvey config management and docs
The Limesurvey service hosted at survey.openstack.org was a beta
which saw limited use. The platform it runs on, Xenial, is now EOL
from Ubuntu/Canonical and in order to upgrade to a newer
distribution release we would need to rewrite all the configuration
management (the version of Puppet supported by newer Ubuntu is not
backward-compatible with what we've been running).

If a similar service becomes interesting to users of our
collaboratory in the future, it will need to be reintroduced with
freshly written configuration management anyway. The old configs and
documentation remain in our Git history should anyone wish to use
them as inspiration.

Change-Id: I59b419cf112d32f20084ab93eb6f2417a7f93fdb
2021-05-01 15:12:00 +00:00
Zuul cb5898ae0a Merge "Remove firehose.openstack.org" 2021-04-14 18:50:16 +00:00
Clark Boylan 2eebb858af Remove firehose.openstack.org
Once we are satisfied that we have disabled the inputs to firehose we
can land this change to stop managing it in config management. Once that
is complete the server can be removed.

Change-Id: I7ebd54f566f8d6f940a921b38139b54a9c4569d8
2021-04-13 13:51:48 -07:00
Ian Wienand db76061c71 Stop managing planet01.openstack.org
This server has been retired.
If141aca5efbdbe60c91ceefaa4e05c98cd0ba5bb has redirected this.

Change-Id: I8d3c089e6e845d98a46ae39c0b32b1c845436add
2021-04-13 16:17:14 +10:00
Ian Wienand 32c6ba7c2b Remove gem mirroring puppet
It looks like we missed these in cleanups for the old puppet-managed
mirror-update server (I5f82139c981c2716f568b15b118690e943b02d52).
These are unused.

Change-Id: Ia79920a7567d73d311f37d73e10c1396d09ddf93
2021-03-29 14:47:45 +11:00
Ian Wienand 2254b6e43d kerberos: switch servers to Ansible control
This is a follow-on to I60b40897486b29beafc76025790c501b5055313d to
switch the KDC servers to Ansible control and remove any related
puppet configuration.

Change-Id: Ib8f6ec657ca10a3ba648bd154a035fc3d8da4be5
2021-03-17 08:30:52 +11:00
Ian Wienand 39ffc685d6 backups: remove all bup
All hosts are now running thier backups via borg to servers in
vexxhost and rax.ord.

For reference, the servers being backed up at this time are:

 borg-ask01
 borg-ethercalc02
 borg-etherpad01
 borg-gitea01
 borg-lists
 borg-review-dev01
 borg-review01
 borg-storyboard01
 borg-translate01
 borg-wiki-update-test
 borg-zuul01

This removes the old bup backup hosts, the no-longer used ansible
roles for the bup backup server and client roles, and any remaining
bup related configuration.

For simplicity, we will remove any remaining bup cron jobs on the
above servers manually after this merges.

Change-Id: I32554ca857a81ae8a250ce082421a7ede460ea3c
2021-02-16 16:00:28 +11:00
Zuul e04a13b3ba Merge "Correct path in mk-archives-index cronjob on lists" 2021-02-16 01:16:03 +00:00
Thierry Carrez 75e15d765d PTGBot is now openinfraptg on #openinfra-events
Channel for PTG discussion was moved to #openinfra-events, and
the bot was renames "openinfraptg" to match the extended scope
of the event.

Change-Id: I52718358ddb4a199d24fc6da6e71f81c646da1f2
2021-02-10 14:33:55 +01:00
Ian Wienand 01990670c9 translate: backup zanata db directly to borg
As noted inline, a recent mysql client update has broken the
"--all-databases" flag, at least for the client version and very old
server version we use.

Emperically, dumping individual databases still works with this
client.  Switch this to stream the db directly into borg.

Ignore the old backups and remove the bup backup while we are here,
since this is all borg now.

Change-Id: I5fe762a003ce2c2ba4830367be87598f67f7e763
2021-02-05 14:05:24 +11:00
Ian Wienand f9184ce323 ask: stream db backup
Despite be deprecated, the ask server is our 3rd biggest backup.  Even
though the site is R/O we're still backing up the fresh rotations of
the gzipped backups every day.

To reduce the incremental space requirements, move to our plain-text
streaming for the db backup.  This just needs a file dropped in /etc;
see the backup-borg role README documentation.  We do this in puppet
to avoid complexity adding this deprecated service to ansible.  This
then excludes the on-disk db backup dir.

Drop the bup backups while we are here.

Change-Id: Icfd81aca58b9a0dc3a3b74de04c1b00f03160327
2021-02-05 13:24:57 +11:00
Zuul e3e555d9fd Merge "Remove AFS puppet" 2021-02-04 03:52:44 +00:00
smarcet 1ff4ccadaa OpenstackId v3.0.18
* updated user rest api for CRUD
* improved session managing ( iframe)
* fixed admin UI issues
* fixed CORS issues ( upgraded middleware)
* bug fixing

Change-Id: I1344b5a5b0df8c24ec4e13ab2aae0dcf5339b6ae
Signed-off-by: smarcet <smarcet@gmail.com>
2021-01-26 17:16:27 -03:00
smarcet fcdb1762d6 OpenstackId v3.0.17
* updated user rest api for CRUD
* improved session managing ( iframe)
* fixed admin UI issues
* fixed CORS issues ( upgraded middleware)
* bug fixing

Change-Id: I0ed6092f62eb5ee6e80cbaabc03d79633dee9aaa
Signed-off-by: smarcet <smarcet@gmail.com>
2021-01-25 11:38:35 -03:00
Ian Wienand 61e9d0948a Remove AFS puppet
This has all been replaced by Ansible roles and is no longer used

Change-Id: Ic807498ad3ca4f305b168464b86fe197a61b4d13
2021-01-21 07:08:37 +11:00
Jeremy Stanley 25697d62bb Correct path in mk-archives-index cronjob on lists
The mk-archives-index command is installed in /usr/local/sbin, so
add that to the path of the cronjob which calls it. Otherwise,
http://lists.opendev.org/archives.yaml is empty and engagement
statistics cannot be generated.

Change-Id: Ib49e8a7b78f8cb9cb385ba09b39e3f940cd17ad6
2021-01-16 15:27:02 +00:00
Zuul c818a82376 Merge "Publish structured data listing our ML archives" 2021-01-14 03:38:55 +00:00
Ian Wienand 368466730c Migrate codesearch site to container
The hound project has undergone a small re-birth and moved to

 https://github.com/hound-search/hound

which has broken our deployment.  We've talked about leaving
codesearch up to gitea, but it's not quite there yet.  There seems to
be no point working on the puppet now.

This builds a container than runs houndd.  It's an opendev specific
container; the config is pulled from project-config directly.

There's some custom scripts that drive things.  Some points for
reviewers:

 - update-hound-config.sh uses "create-hound-config" (which is in
   jeepyb for historical reasons) to generate the config file.  It
   grabs the latest projects.yaml from project-config and exits with a
   return code to indicate if things changed.

 - when the container starts, it runs update-hound-config.sh to
   populate the initial config.  There is a testing environment flag
   and small config so it doesn't have to clone the entire opendev for
   functional testing.

 - it runs under supervisord so we can restart the daemon when
   projects are updated.  Unlike earlier versions that didn't start
   listening till indexing was done, this version now puts up a "Hound
   is not ready yet" message when while it is working; so we can drop
   all the magic we were doing to probe if hound is listening via
   netstat and making Apache redirect to a status page.

 - resync-hound.sh is run from an external cron job daily, and does
   this update and restart check.  Since it only reloads if changes
   are made, this should be relatively rare anyway.

 - There is a PR to monitor the config file
   (https://github.com/hound-search/hound/pull/357) which would mean
   the restart is unnecessary.  This would be good in the near and we
   could remove the cron job.

 - playbooks/roles/codesearch is unexciting and deploys the container,
   certificates and an apache proxy back to localhost:6080 where hound
   is listening.

I've combined removal of the old puppet bits here as the "-codesearch"
namespace was already being used.

Change-Id: I8c773b5ea6b87e8f7dfd8db2556626f7b2500473
2020-11-20 07:41:12 +11:00
Zuul d3a53e8ec0 Merge "Remove mirror-update server and related puppet" 2020-11-09 21:07:11 +00:00
Zuul 00c496e879 Merge "Add service-incident@opendev mailing list" 2020-10-30 15:37:51 +00:00
Zuul b72845c274 Merge "Cleanup grafana.openstack.org" 2020-10-29 05:15:00 +00:00
Ian Wienand c49ece9204 Cleanup grafana.openstack.org
The opendev.org server is in production, cleanup the old puppet-based
host.

Change-Id: I6db3ce929226a23b96234b52ece8b17f4c6a326a
2020-10-29 07:59:42 +11:00
Ian Wienand f8852b76fb Remove mirror-update server and related puppet
This has all transitioned to Ansible and the mirror-update.opendev.org
server now.

Change-Id: I5f82139c981c2716f568b15b118690e943b02d52
2020-10-28 11:39:54 +11:00
Ian Wienand 10b2cd5fed reprepo: enable cron jobs
Enable the Ansible based cron jobs, and disable the puppet host
versions to cut over the mirroring to the new server.

Change-Id: I0ffb1c484e64e67f5a5017dc3c3c8ebcdc3845c8
2020-10-28 11:29:26 +11:00
Zuul 89a1edce3d Merge "Remove old debian-ceph mirrors" 2020-10-27 02:57:44 +00:00
Jeremy Stanley abc66ed38c Add service-incident@opendev mailing list
Create a mailing list for private coordination of security incidents
for the OpenDev Collaboratory. The intent is that this can be used
to share sensitive information between sysadmins and council members
in the event of any suspected breach. For the sake of transparency,
all information discussed on this list which can safely be made
public should also be communicated to the service-announce or
service-discuss mailing lists at the earliest opportunity.

Change-Id: I32bef68eb7019261471c167d19eee733457078a2
2020-10-22 16:16:04 +00:00
Ian Wienand 961bab63d9 Remove old debian-ceph mirrors
The active releases according to [1] are octopus and nautlius.  Remove
the old releases from our mirroring.  This needs manual cleanup of the
jobs and volumes -- I will do this manually as this is mostly about
clearing out old things before moving the mirroring to Ansible.

[1] https://docs.ceph.com/en/latest/releases/

Change-Id: I050f737521fa6837f3b6b52b8028a839a29f7bd2
2020-10-16 14:16:19 +11:00
smarcet 807ea2608e OpenstackId v3.0.16
added new search criteria for endpoint
GET /api/v1/users

primary_email (==,@=)

Change-Id: Ib643a8c1ba4e79444463777197fc86a64a1912be
Signed-off-by: smarcet <smarcet@gmail.com>
2020-10-15 00:23:08 -03:00
smarcet e394198d03 OpenstackID v3.0.15
* updated registration emails
* updated registration form
* updated password policies
* bug fixing

Change-Id: Ibd644e9daa9bd345cf883db3dfa75b58b4ad7a18
Signed-off-by: smarcet <smarcet@gmail.com>
2020-10-12 17:20:54 -03:00
smarcet 2f970563c0 OpenstackId config updates
Added cloud storage config

Change-Id: I39cefce0c1910df0fc051817193e14e5a38c3a1e
Signed-off-by: smarcet <smarcet@gmail.com>
2020-09-21 17:40:19 -03:00