Commit Graph

502 Commits

Author SHA1 Message Date
Clark Boylan 9e9e980f09 Update gitea to 1.21.9
Almost immediately after we upgraded to 1.21.8 a new 1.21.9 release
became available. Again this appears to largely be a bugfix release with
no super important changes for us. However, there are performance
improvements which are always nice to see. The template files that we
override have not changed between 1.21.8 and 1.21.9.

Full change log can be found here:

  https://github.com/go-gitea/gitea/blob/v1.21.9/CHANGELOG.md

Change-Id: Ica763081203d9be44c9de0923a261afa820c891b
2024-03-22 09:10:20 -07:00
Clark Boylan 5a2bd42a4d Update gitea to 1.21.8
This is a bugfix release with no template updates and no other impactful
deployment changes that I can see. Full changelog notes can be found
here:

  https://github.com/go-gitea/gitea/blob/v1.21.8/CHANGELOG.md

Change-Id: I6009bbebc261e87702b7f603bf179be89d31edb9
2024-03-19 07:40:38 -07:00
Zuul a44a354e53 Merge "Upgrade gitea to 1.21.7" 2024-02-28 18:01:58 +00:00
Clark Boylan 31ea71655c Upgrade gitea to 1.21.7
This upgrades our gitea container image and, thus deployment, to version
1.21.7 from 1.21.5. There are no updates to the three template files we
override upstream according to git diff in the gitea repo.

A full changelog can be found here:

  https://github.com/go-gitea/gitea/blob/v1.21.7/CHANGELOG.md

Change-Id: I95d92f47085532275bf0f2508f9026e9394aebc7
2024-02-26 08:20:18 -08:00
Clark Boylan d720d58e70 Update gerrit image to 3.8.4
There is at least one Gerrit bugfix for an NPE that we should pick up by
this update. There are also improvements to the MINA SSHD server that
gerrit runs.

Full changelogs can be found here:
  https://www.gerritcodereview.com/3.8.html#384

Change-Id: Icba387496457c5a60fd914a6ee689104d3a52c1d
2024-02-26 08:17:52 -08:00
Clark Boylan a53dcc8a7d Update etherpad to 1.9.7
This change updates etherpad to version 1.9.7 from 1.9.6. The
changelog [0] is minimal, but does indicate there are changes to plugin
installations. Looking at the upstream Dockerfile, which we based our
Dockerfile on, there are no changes between 1.9.6 and 1.9.7 implying
this plugin installation update is transparent to us. That said we
should hold a node and test that our plugins are working as expected.

[0] https://github.com/ether/etherpad-lite/blob/v1.9.7/CHANGELOG.md

Change-Id: Ie708299fae39549f048f37938daa60668189be67
2024-02-07 09:34:46 -08:00
Zuul f805502cf7 Merge "Upgrade gitea to 1.21.5" 2024-02-06 22:38:24 +00:00
Clark Boylan dcf5dbb115 Rebuild python base images
This is our semi regular rebuild of these images to catch up to updates
in packages and python.

Change-Id: I9239fb5e688b38896aa9613f26dd14df694d7845
2024-02-04 08:05:01 -08:00
Clark Boylan 3fd045aec3 Upgrade gitea to 1.21.5
This update includes a number of bugfixes. The changelog can be found
here: https://github.com/go-gitea/gitea/blob/v1.21.5/CHANGELOG.md.

There is a security fix for inappropriate access to non public container
images. We don't how private data and we don't use the container
registry in gitea so this doesn't affect us.

There are no changes to template files that we override.

Change-Id: I9419a22736de82e135a25fca22aef1ed10c19e1a
2024-02-01 10:14:43 -08:00
Clark Boylan 3e6c282d54 Update to etherpad 1.9.6
The changelog [0] indicates this is largely a bugfix and image build
update. We update our image build accordingly.

[0] https://github.com/ether/etherpad-lite/blob/v1.9.6/CHANGELOG.md

Change-Id: I439aa41eaee1dd7825d41ea3da9b1903fa27fa44
2024-01-31 10:55:17 -08:00
Clark Boylan fb531dae6f Update gitea to 1.21.4
We are currently running 1.21.3 so this shouldn't be a huge upgrade for
us. Full changelog can be found here:

  https://github.com/go-gitea/gitea/blob/v1.21.4/CHANGELOG.md

Two template files are removed from our custom template overrides. They
were both included for the 1.21.3 so that we could manually patch a bug
that resulted in HTTP 500 errors when using gitea's code search
functionality. Upstream included these fixes in the 1.21.4 release so we
don't need to override to fix this any longer. This should be covered by
a testinfra test case now too.

Change-Id: I221e5cd185631751c082bdf5e2902057e5200dc0
2024-01-18 12:59:04 -08:00
Jeremy Stanley 79103e1a35 Update our Gitea robots.txt from gitea.com's
We've experienced some runaway growth of Gitea archive cache files
on one of our backends, which according to upstream is often caused
by web crawlers indexing the archive URLs. They recommended updating
our robots.txt to the current state of https://gitea.com/robots.txt
in order to help mitigate the issue.

I've kept things we expressly commented out before still commented
out, or anything that seems similar to what we commented out on the
assumption that the reasons would carry over.

After some discussion in IRC, we also decided it would make sense to
disallow /avatars and /user/* like they do.

Change-Id: I2b43b89de08c9a9d170e1ecbd14b1e6336fd2c84
2024-01-05 17:14:20 +00:00
Zuul e5a71ece6b Merge "Update gitea to 1.21.3" 2024-01-04 18:43:46 +00:00
Clark Boylan 30279610b6 Update gitea to 1.21.3
Upgrade Gitea to 1.21.3. The changelogs for this release can be found
here:

  https://github.com/go-gitea/gitea/blob/v1.21.3/CHANGELOG.md

I have attempted to collect the interesting bits in this commit message
as well as information on why we do or don't make changes to address
these items.

1.21.0
 * BREAKING
   * Restrict certificate type for builtin SSH server (https://github.com/go-gitea/gitea/pull/26789)
     * We don't use the builtin SSH server and don't use certificates
       for auth. Nothing to do here.
   * Refactor to use urfave/cli/v2 (https://github.com/go-gitea/gitea/pull/25959)
     * The major change here updated `gitea` to stop accepting
       `gitea web`'s command options. Our dockerfile is set up to use
       `CMD ["/usr/local/bin/gitea", "web"]` so we are not affected.
   * Move public asset files to the proper directory (https://github.com/go-gitea/gitea/pull/25907)
     * We update the testinfra test for robots.txt to more robustly
       check file contents. Previously it checked a very generic
       prefix which may indicate a generic file being served.
     * We move custom/public/img into custom/public/assets/img.
       Screenshots should be used to confirm this works as expected.
   * Remove commit status running and warning to align GitHub (https://github.com/go-gitea/gitea/pull/25839)
     (partially reverted: Restore warning commit status (https://github.com/go-gitea/gitea/pull/27504) (https://github.com/go-gitea/gitea/pull/27529))
     * We don't rely on commit statuses as this is a read only replica
       of Gerrit.
   * Remove "CHARSET" config option for MySQL, always use "utf8mb4" (https://github.com/go-gitea/gitea/pull/25413)
     * We don't set [database].CHARSET. Doesn't affect us.
   * Set SSH_AUTHORIZED_KEYS_BACKUP to false (https://github.com/go-gitea/gitea/pull/25412)
     * We don't set this value explicitly so the default will flip from
       true to false for us. I don't think this is an issue because we
       keep track of our pubkeys in git.

 * SECURITY
   * Dont leak private users via extensions (https://github.com/go-gitea/gitea/pull/28023) (https://github.com/go-gitea/gitea/pull/28029)
     * We don't use private users.
   * Expanded minimum RSA Keylength to 3072 (https://github.com/go-gitea/gitea/pull/26604)
     * We have rotated keys used to replicate from gerrit to gitea to
       work around this. Now are keys are long enough to make gitea
       happy.

 * BUILD
   * Dockerfile small refactor (https://github.com/go-gitea/gitea/pull/27757) (https://github.com/go-gitea/gitea/pull/27826)
     * I've updated our Dockerfile to mimic these changes. Comment
       whitespace as well as how things are copied and chmoded in the
       build image have been updated.
     * TODO the file copies aren't working for us. I think due to how we
       ultimately clone the git repo. We use RUN but upstream is using
       COPY against the local build dir. I've aligned as best as I can,
       but we should see if we can do a similar COPY on our end.
   * Fix build errors on BSD (in BSDMakefile) (#27594) (#27608)
     * We don't run on BSD.
   * Fully replace drone with actions (#27556) (#27575)
     * This is how upstream builds their images. Doesn't affect our
       builds.
   * Enable markdownlint no-duplicate-header (#27500) (#27506)
     * Build time linters are somethign we don't care too much about on
       our end.
   * Enable production source maps for index.js, fix CSS sourcemaps (https://github.com/go-gitea/gitea/pull/27291) (https://github.com/go-gitea/gitea/pull/27295)
     * This emits a source map for index.js which can be used for in
       browser debugging. Don't think this is anything we need to take
       action on.
   * Update snap package (#27021)
     * We don't use a snap package.
   * Bump go to 1.21 (https://github.com/go-gitea/gitea/pull/26608)
     * Our go version is updated in the Dockerfile.
   * Bump xgo to go-1.21.x and node to 20 in release-version (https://github.com/go-gitea/gitea/pull/26589)
     * Our node version is updated in the Dockerfile.
   * Add template linting via djlint (#25212)
     * Build time linters are somethign we don't care too much about on
       our end.

1.21.1
 * SECURITY
   * Fix comment permissions (https://github.com/go-gitea/gitea/pull/28213) (https://github.com/go-gitea/gitea/pull/28216)
     * This affects disclosure of private repo content. We don't have
       private repos so shouldn't be affected.

1.21.2
 * SECURITY
   * Rebuild with recently released golang version
     * We'll automatically rebuild with newer golang too.
   * Fix missing check (https://github.com/go-gitea/gitea/pull/28406) (https://github.com/go-gitea/gitea/pull/28411)
     * There is minimal info here but it appears to be related to
       issues. We don't use issues so shouldn't affect us.
   * Do some missing checks (https://github.com/go-gitea/gitea/pull/28423) (https://github.com/go-gitea/gitea/pull/28432)
     * There is minimal info here but it appears to be related to
       checks around private repos. We don't use private repos so this
       shouldn't affect us.

1.21.3
 * SECURITY
   * Update golang.org/x/crypto (https://github.com/go-gitea/gitea/pull/28519)
     * This addresses recent concerns found in ssh for gitea's built in
       ssh implementation. We use openssh as provided by debian so will
       rely on our distro to provide fixes.

Finally 1.21.x broke rendering of code search templates. The issue is
here: https://github.com/go-gitea/gitea/issues/28607. To address this
I've vendored the two fixed template files
(https://github.com/go-gitea/gitea/pull/28576/files)into our custom
template dirs. Once upstream makes a release with these fixes we can
drop the custom files entirely as we don't override anything special in
them.

Change-Id: Id714826a9bc7682403afcf90f2761db8c84eacbf
2024-01-03 16:36:17 -08:00
Clark Boylan a0089cfac6 Upgrade to etherpad 1.9.5
This bumps etherpad to 1.9.5. The changelog is minimal for this update,
but upstream switches to nodejs 20 by default so we make the same update
here. We also remove TidyHTML configs from our configs to match upstream
updates that did the same thing. Complete release notes can be found
here:

  https://github.com/ether/etherpad-lite/blob/v1.9.5/CHANGELOG.md

We should hold a node and test functionality before merging this change.

Change-Id: Ib6cd888f35624490f630e091f184946e9c4e48aa
2024-01-02 08:41:39 -08:00
Zuul aefc69c9ab Merge "Make bookworm the python Dockerfile parent default image" 2023-12-05 20:09:51 +00:00
Clark Boylan 6cf8b63bc8 Upgrade gitea to 1.20.6
This is a bugfix release with some security updates that while maybe not
critical due to our use of gitea as a read only mirror would be good to
get in anyway. Additionally we'll want to be on the latest 1.20 release
before updating to 1.21.

The changelog can be found here:

  https://github.com/go-gitea/gitea/blob/v1.20.6/CHANGELOG.md

Git diff reports no template updates between 1.20.5 and 1.20.6 in the
templates that we override.

Change-Id: Idd38660dce53b5765c1ab4bc021544bd105df138
2023-11-28 08:23:17 -08:00
Clark Boylan 526501db05 Make bookworm the python Dockerfile parent default image
This was still set to bullseye which isn't a problem for our Zuul jobs
as they always specify what version to use. However, local builds would
build bullseye by default which isn't super useful now that the vast
majority of images are built on top of bookworm. Swap things around to
avoid potential confusion.

Change-Id: If68e32a358268a423e35e44e3150115cd1da6f8c
2023-11-21 09:04:16 -08:00
Zuul a93ad36865 Merge "Upgrade Etherpad to 1.9.4" 2023-11-02 17:34:14 +00:00
Zuul bd844f01fb Merge "Update Etherpad settings from upstream" 2023-11-02 16:51:15 +00:00
Jeremy Stanley b5d32d39cd Upgrade Etherpad to 1.9.4
The changelogs can be found here:

https://github.com/ether/etherpad-lite/blob/v1.9.3/CHANGELOG.md#193
https://github.com/ether/etherpad-lite/blob/v1.9.4/CHANGELOG.md#194

There doesn't appear to be anything relevant to our deployment in
these updates other than bug fixes and library version increases,
but upgrading now will reduce future deltas.

Change-Id: Ic1629bf8cb140c33a641e1c613d43e8a9d4d0f1e
2023-11-01 18:53:42 +00:00
Jeremy Stanley 09d89298e3 Update Etherpad settings from upstream
Refresh our versions of settings.json.docker and
settings.json.template from upstream, incorporating our local
preferences as edits to the latter (the former is included in the
container image we publish but the latter gets mapped over it during
deployment).

Changes to the required version of node-log4js in Etherpad 1.9.4
will invalidate our custom logging configuration and error out,
preventing the service from starting, so go ahead and remove it now.

Change-Id: Ic05ed9be7b6900ba9cdfa09b28600bcd55b770fd
2023-11-01 18:49:05 +00:00
Jeremy Stanley b312e15b57 Upgrade to latest Mailman 3 releases
New releases info:

https://lists.mailman3.org/archives/list/mailman-users@mailman3.org/thread/4U5AP7GZ76NYQONACUVPDHSJBLLBSENL/

Sync all our forked files from mailman-docker to the current
upstream state, except for our overridden hyperkitty->archives and
postorius->mailman3 URL routes in
docker/mailman/web/mailman-web/urls.py.

Change-Id: I3b3955c8b2b91f167510c8a1122d9d8e2d620082
2023-10-29 16:28:43 +00:00
Zuul 51003247f8 Merge "Update Gerrit to 3.7.5" 2023-10-11 23:31:19 +00:00
Clark Boylan 9f024b5fea Rebuild gitea 1.20.5 on latest golang 1.20.10
Upstream golang updates are worth recompiling gitea under. Details can
be found in the golang 1.20 release notes:

  https://go.dev/doc/devel/release#go1.20.minor

Change-Id: I6ddeaa23d5aee23928d6f448095bb69fe82d94a9
2023-10-11 08:53:53 -07:00
James E. Blair e85ab6f746 Add a debug python base image
This adds a python-base:3.11-bookworm-debug image, which is built
on the normal python:3.11-bookworm upstream image instead of the
slim upstream image.  The normal image includes debug symbols for
the python interpreter which is compiled during its build phase,
so this is the best way to get an opendev python-base image with
debug symbols.

Change-Id: I1d89ac947cd3bea8a468f3ee022fb4cc93bece1f
2023-10-10 08:30:32 -07:00
Zuul cac37a7a3c Merge "Update gerrit image to bookworm" 2023-10-09 16:19:47 +00:00
Zuul 8a9e9ffe1b Merge "Upgrade to gitea 1.20.5" 2023-10-06 16:54:55 +00:00
Clark Boylan f1cc7d4f8e Update gitea base OS during image builds
This ensures we're always up to date with our packages even if the
upstream container images lag behind debian proper. Useful for pulling
in bugfixes more quickly than upstream seems to think we want them.

Change-Id: Ia7ec97ca17ad1175c8ddd4c5d037f516dcdd891a
2023-10-05 10:59:24 -07:00
Clark Boylan dd48296edd Fix python-builder container image system updates
There was a small issue in the recent change to perform system updates
when building the python-builder and python-base images. I didn't
realize that python-builder is a two stage build and we need to do the
update in both stages.

Ultimately this has minimal impact on the final images we produce as
those are all build on python-base not python-builder. But to ensure
some difference during python wheel build time on builder doesn't affect
the install location on the base image we should keep these in sync.

Change-Id: I16159fbb490b0ec2e179381a50b9570c9aacd18f
2023-10-04 08:32:34 -07:00
Clark Boylan e81e37ad43 Update our base container images
There are new pythons and other things we should update to pull in.

Change-Id: I5430e4b6e17861049d2dd60e88bf330595388e23
2023-10-03 17:04:50 -07:00
Clark Boylan 267e0cb6f0 Upgrade to gitea 1.20.5
This looks like a straightforward bugfix release according to the
release notes [0]. There are also no template changes in the three
templates we override.

[0] https://github.com/go-gitea/gitea/blob/v1.20.5/CHANGELOG.md

Change-Id: Id5521289daeb974ac1ec73ffb85d5adb5780fae8
2023-10-03 09:39:23 -07:00
Clark Boylan 00c098bcad Bump refstack to Bookworm and python3.11
Refstack doesn't rely on much in the base operating system as far as I
can tell. That said refstack seems to test with python3.10 and not 3.11
so this may not work, but our testing should give us a good idea. Bump
these things up as we are updating all the services we can in order to
eventually cleanup image builds for old debian and old python.

Change-Id: Id39027691484e8f81bd097c174f0a4a1e81463af
2023-09-19 10:22:56 -07:00
Clark Boylan da41905b79 Update Gerrit to 3.7.5
Gerrit just released new point versions for all supported branches
including an update of 3.7 to 3.7.5. We also bump our 3.8 image up to
3.8.2 (note this isn't used in production, only for testing of the
upgrade to 3.8 currently).

Release notes can be found here:

  https://www.gerritcodereview.com/3.7.html#375
  https://www.gerritcodereview.com/3.8.html#382

The updates look minor for us, but there are some bug fixes so worth
updating. We might want to land this change as well as the bookworm
update together in order to do a single short gerrit downtime to get all
of these updates deployed in production.

Change-Id: Ib4ccfe12db94d032fc4743a7aafdf90735aecfa3
2023-09-15 15:39:34 -07:00
Clark Boylan 2888408833 Convert haproxy-statsd to bookworm base image
We keep the same python 3.11 version we had on bullseye but switch the
rest of the userspace to bookworm. Since the python version doesn't
change this should have minimal impact.

Change-Id: I59ad8c2a92159f51d567dd0212e2ab8bec1b45b1
2023-09-14 13:36:18 -07:00
James E. Blair fec1277185 Update gerrit image to bookworm
And upgrade to Python 3.11, and JDK to 17.

Change-Id: I7c9415e6706141db6cd9cab056a439da81469def
2023-09-08 08:28:22 -07:00
Clark Boylan c06b31df3a Update to gitea 1.20.4
This is a small update from what we are currently running (1.20.4). The
full changelog can be found here:

  https://github.com/go-gitea/gitea/blob/v1.20.4/CHANGELOG.md

There is one small template update in 1.20.4, but it is to documentation
urls which we are already overriding with our own documentation links so
doesn't affect us.

Change-Id: I5ed374e2e6e0056397e05404e0bf42ffd3906469
2023-09-08 08:10:29 -07:00
Zuul c91d217681 Merge "Update Gitea images to bookworm" 2023-09-01 21:29:18 +00:00
Zuul 90620b284b Merge "Rebuild gitea on bullseye" 2023-09-01 19:35:49 +00:00
Clark Boylan dea0930962 Update Gitea images to bookworm
There are two main components that I expect will be affected by this
bookworm update. The first is git. Git is updated from 2.30.2-1+deb11u2
to 2.39.2-1.1. In general git has been very good about maintaining
compatibility, but there is some risk of a behavior change impacting us.

Second is openssh-server. In particular we upgrade from 8.4p1-5+deb11u1
to 9.2p1-2 which crosses the 8.8 release threshold. 8.8 changed how RSA
keys are handled. Now by default only RSA + SHA2 is negotiated by
default and RSA + SHA1 is not allowed. Gerrit currently uses RSA keys
for replication. This should all be fine because MINA added support for
negotiating RSA + SHA2 as both a client and server in version prior to
that running in Gerrit 3.7, but there is still some risk this will
break.

We can test this with held nodes, or we can assume it will work given
the fixes in MINA. Then if it breaks we can switched to ed25519 keys or
update config to openssh to reenable SHA1 or we can revert to bullseye.

Note, we make a small update to the image to set `ENV USER git` in both
the web and ssh iamges as the ssh image uses this env var to dynamically
set sshd_config's AllowUsers value. We weren't setting this value
previously which older sshd seems to ignore. Bookworm sshd gets angry
about this directly being set without a value in its config.

Change-Id: I5a923798e90be4dcd9486a97014180ed1790fab1
2023-09-01 09:44:52 -07:00
Clark Boylan a63f8c6dd7 Rebuild gitea on bullseye
We want to rebuild gitea on bullseye before upgrading to bookworm. The
reason for this is we only prune images that are more than 72 hours old.
Deploying a new bullseye image then deploying the bookworm image ensures
we have 72 hours before that bullseye image is pruned allowing us to
easily revert if necessary.

Change-Id: I5cc8078e0c5f6e55215e9419ac3569a686060b05
2023-09-01 09:44:51 -07:00
Zuul af6c43bce3 Merge "Update jinja-init image to bookworm" 2023-08-30 16:13:58 +00:00
Zuul 52b865a9eb Merge "Update zookeeper-statsd image to bookworm" 2023-08-25 21:21:16 +00:00
Jeremy Stanley c9c8febd84 Trigger mm3 deployment when containers change
Add the docker/mailman tree to the infra-prod-service-lists3 job so
that we deploy new versions whenever we make changes to the
container images.

Change-Id: Ife5e878b1f81c2879c2959fe6d4de22fe841583b
2023-08-25 16:35:46 +00:00
Zuul aa4afe593b Merge "Upgrade to latest Mailman 3 releases" 2023-08-25 16:27:25 +00:00
Zuul 1c5bee394c Merge "Update hound image to bookworm" 2023-08-24 21:02:41 +00:00
Zuul c2ea42a86e Merge "Update to Gitea 1.20" 2023-08-24 18:58:57 +00:00
James E. Blair 55abbf33bf Update hound image to bookworm
Change-Id: I51b8630b27149aa96c63d13d6fae240c5c81cd96
2023-08-24 11:31:46 -07:00
James E. Blair d39162a79a Update zookeeper-statsd image to bookworm
And upgrade to Python 3.11.

Change-Id: I863aabd21e0518879604b187509aa9b46b15bace
2023-08-24 11:19:23 -07:00
James E. Blair 1354a1e6da Update jinja-init image to bookworm
And upgrade to python 3.11.

Change-Id: I47a99263b93be027ffe251fce24d677d07f519d7
2023-08-24 11:18:11 -07:00