Commit Graph

105 Commits

Author SHA1 Message Date
Clark Boylan 45b7bcaa85 Retire this repo
The opendev project has been moving away from puppet and this is one of
the puppet modules that is no longer used. To simplify things for us we
are taking the extra step of retiring this repo.

Change-Id: I1e1d3a52022f996e3b28362a6bd97114e494d378
2022-02-17 11:55:50 -08:00
Jeremy Stanley d476171f72 Add missing extensions
Our configuration references the Cite, Gadgets, Nuke,
ParserFunctions, Renameuser, SpamBlacklist, SyntaxHighlight_GeSHi,
and WikiEditor extensions. These are not shipped in the MW source
tree, so we have to add them to the extensions list in the manifest
to make sure they get downloaded.

Change-Id: I27ab06adcd1bee69bcc196a44dd401be6cff574f
2020-01-29 19:27:03 +00:00
Jeremy Stanley 5e1865d1d1 Sort extensions list for improved manageability
The extensions list is already somewhat long, and will be getting
longer as missing extensions from our config are identified and
added. Sort the entries in alphanumeric ASCII ordering to simplify
further insertions.

Change-Id: I9333f99a23bd3304722bf6363c2a0719a5e20711
2020-01-29 19:26:36 +00:00
Jeremy Stanley d590306256 Update default OpenID provider to ubuntu.com
The old login.launchpad.net OpenID provider has been replaced by
login.ubuntu.com for years, and now no longer maintains a usable
x.509 cert. Update the default wg_openidforcedprovider class
parameter to reflect the newer name.

Change-Id: I4542fd5de9907e36460893be8e9f0663ff98646b
2020-01-29 18:53:05 +00:00
Jeremy Stanley f242561761 puppet-lint: class included by absolute name
Recent puppet-lint has decided it looks better not to include
leading :: on class names, so roll with it.

Change-Id: Ib4ffa2790a20ab074597dc26c7df1f1c08dc66f9
2020-01-29 18:51:45 +00:00
Jeremy Stanley b321a9229e Fetch external dependencies
The "vendor" subdirectory of the wiki tree is expected to contain
all MediaWiki's external dependencies which are no longer carried
within its core Git repository. The documentation on how to populate
it can be found here:

https://www.mediawiki.org/wiki/Download_from_Git#Fetch_external_libraries

I'm not thrilled by the idea of running an application which creates
a directory within the Git worktree, but at least it and the other
trashfiles it creates are tracked in the .gitignore so shouldn't
pose a problem for updating the repository state.

Change-Id: I81ac4ff371346c217d760390a02139dfae82332f
2019-10-15 17:37:10 +00:00
Jeremy Stanley acf82e4407 Make sure the images tree is owned by www-data
This is mostly a belt-and-braces measure, as the images tree is
stateful and copied from deployment to deployment, but just in case
permissions on those files weren't set correctly when
copying/unpacking, have Puppet take care of correcting their
ownership.

Change-Id: Ib682e04e9f324f22ad4e1085946d8100949b0e3f
2019-10-15 17:17:01 +00:00
Jeremy Stanley 08f1087ece Move cache directory out of mediawiki Git worktree
For improved sanitation, have the application write its cache files
in a directory parallel to the Git worktree rather than inside it.

Change-Id: Idcfcf0e3709a03696966aa8c27b897d5be7a1970
2019-10-15 17:14:18 +00:00
Jeremy Stanley 87af3d00e6 Update to 1.28.x branch
The OpenStack wiki is running a MediaWiki 1.28 point release, so
upgrade to a compatible source branch in preparation for a new data
import on the dev server currently managed by this module.

Change-Id: I04ca733dd8da0dc43a90b77d9a1356888e9f0f69
2019-08-10 14:01:02 +00:00
Jeremy Stanley 8cfec53993 Put image data in a parallel path to source code
To avoid co-mingling configuration-managed files with state data,
place them in parallel paths so that the latter can be mounted from
an external volume.

Change-Id: Ibfedb88bbcf7c816b200b978c987a7c2f3e9f0d4
2019-08-10 00:24:26 +00:00
Jeremy Stanley cd0f868884 Disable EmbedVideo extension
The EmbedVideo extension referenced in the manifest is no longer
compatible with MediaWiki<1.29.0 so remove it from the active
extensions for now (we can add it back after we move to a newer MW
release).

Change-Id: I8865f143019553062f610c4004147150cff438cd
2019-08-09 18:06:08 +00:00
Jeremy Stanley af5a993a08 Add php-mbstring and php-xml distro packages
The system packages for php-mbstring and php-xml are required by
MediaWiki, so add them. While we're here, alphabetize the list.

Change-Id: I61d7af1dd4906ae724c2fe850290714e3b092a08
2019-08-09 18:06:00 +00:00
Clark Boylan 78c7ec34c9 Stop managing one more mediawiki git submodule
The mediwiki vendor repo is managed as a submodule and is already
checked out for us. We don't need to manage it directly with vcsrepo.
Doing so fails as the dir exists and isn't a full git repo.

Change-Id: Ibd74f66a90ea4300e27a0e5a14c58f0c209563a5
2019-07-29 10:02:07 -07:00
Clark Boylan 99faf592da Remove extension which are git submodules
We cannot clone in these git repos because they are already git
submodules which are checked out for us and you can't clone a git repo
into a dir that exists. It seems that at some point the extensions that
have been removed from our list here have made it into mediawiki's
default extension list.

Change-Id: Ibdf565506bef68a7d297fc73935ef817ef9d24e5
2019-07-24 13:50:39 -07:00
Jeremy Stanley 9b9645c1cb Canonicalize clone URLs
The recent Gerrit upgrade at Wikimedia made it necessary to redirect
/r/p/ URLs to /r/, so the p/ can be safely dropped from all
occurrences.

Depends-On: https://review.opendev.org/668761
Change-Id: Ic5d63376c50b5562b43805c89b7255c93fa51ce8
2019-07-02 21:59:57 +00:00
Jeremy Stanley 3821055e0d More transitional Xenial package name replacements
Some additional packages specified in the manifest have slightly
different names on Xenial than their Trusty era counterparts. Update
them accordingly.

Change-Id: I9af30b01ee1cd9c28a408de14d6130a201cc6385
2019-05-28 21:18:36 +00:00
Zuul 1d39e14b77 Merge "Replace transitional package names for Xenial" 2019-05-21 20:17:11 +00:00
OpenDev Sysadmins 243fb4e29c OpenDev Migration Patch
This commit was bulk generated and pushed by the OpenDev sysadmins
as a part of the Git hosting and code review systems migration
detailed in these mailing list posts:

http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003603.html
http://lists.openstack.org/pipermail/openstack-discuss/2019-April/004920.html

Attempts have been made to correct repository namespaces and
hostnames based on simple pattern matching, but it's possible some
were updated incorrectly or missed entirely. Please reach out to us
via the contact information listed at https://opendev.org/ with any
questions you may have.
2019-04-19 19:25:36 +00:00
Jeremy Stanley dbfc8381c7 Replace transitional package names for Xenial
Some of the packages specified in the manifest have slightly
different names on Xenial than their Trusty era counterparts. In
some cases these were already dependencies of other packages we're
installing and were specified unnecessarily so we can just remove
them from the manifest. In some cases there are virtual packages
which match the old and new names so we can just use those instead
and ease future maintenance as well. In some cases the old package
name was actually a dummy transitional package already so using the
newer name works for both platforms.

Change-Id: Ib80fa6becffe4f76a9d14777d5c4a634ebaadb87
2019-04-09 18:46:28 +00:00
Ian Wienand 36e8b6930e Replace openstack.org git:// URLs with https://
This is a mechanically generated change to replace openstack.org
git:// URLs with https:// equivalents.

This is in aid of a planned future move of the git hosting
infrastructure to a self-hosted instance of gitea (https://gitea.io),
which does not support the git wire protocol at this stage.

This update should result in no functional change.

For more information see the thread at

 http://lists.openstack.org/pipermail/openstack-discuss/2019-March/003825.html

Change-Id: I019dd6a8a66810c0e9813d5b18d6f743a56b2b90
2019-03-24 20:35:41 +00:00
Jeremy Stanley a7956cd41d Switch from PHP5 to default PHP (PHP7 on Xenial)
In working out upgrading our Mediawiki deployment to Ubuntu 16.04
LTS (Xenial Xerus), it became apparent that libapache2-mod-php5 is
not available there because the distro has moved on to
libapache2-mod-php7.0 instead. By using the libapache2-mod-php
virtual package name instead, we can support both and also future
version bumps as well.

Change-Id: I5d1a5e51beb09a1cb26b1de6dba1b7d05781ee03
2019-03-14 22:09:53 +00:00
Zuul 047a444178 Merge "Optionally alias to a favicon.ico file if provided" 2018-12-17 16:33:14 +00:00
Colleen Murphy 67deb6fa07 Update Gemfile for Zuulv3
The logic in the Gemfile was relying on Zuulv2 variables to find out
whether the spec helper gem was already available on disk, and since
Zuulv3 has changed things it was failing to find it and downloading the
master version instead. This patch ensures the Gemfile looks for the gem
in the right place when running in CI.

Change-Id: If995cd15de1fd2a7a2c37bc75e7ebbf206e40d06
2018-07-12 09:57:45 +02:00
Colleen Murphy adec8d34f2 Depend on helper gem for spec_helper_acceptance
Instead of keeping a local copy of spec_helper_acceptance.rb and
requiring updates to all modules for any change, we can move it into the
common helper gem and require it from there. This will make it easier to
create and review changes that affect all puppet modules. Also change
the Gemfile to look for the gem in the local workspace if running in a
zuul environment.

Change-Id: I41568a6122754097a5d78f0ae0f9b6b61504f551
2017-08-18 10:41:44 +02:00
Colleen Murphy 89d969b05c Update beaker setup for xenial
Add a xenial nodeset and update the spec helper to install puppet 3 from
the Ubuntu repos instead of from puppetlabs.

Change-Id: I907648e5f09710b4bcc78ab8287fc8376b8429c4
2017-06-11 20:21:08 +02:00
Jeremy Stanley dc007b3473 Optionally alias to a favicon.ico file if provided
Make it possible for calling manifests to pass in the path to a
static favicon.ico shortcut icon file on the filesystem, and if
there is one then alias /favicon.ico to it in the Apache vhost
configuration.

Change-Id: Iba36d169335b2b8ee278f3f4500893a1641e4b28
2017-02-28 23:03:16 +00:00
Tom Fifield dca11a634c Add a conservative page creation rate limit
This patch limits users who are not in the known-good users
list ('autopatrol') to creating 5 pages per day.

The vast majority of our spam comes from the creation of new
pages. This patch will limit the usefulness of each spammer
account, by limiting the number of pages each can create.

By setting the number at 5 initially, it is not expected to
have an impact on legitimate new users playing with the wiki
for the first time. Neither will it have an impact for many spam
accounts, but at least it provides and upper limit for cleanup.

Relevant references:
* Rate limts https://www.mediawiki.org/wiki/Manual:$wgRateLimits
* User rights https://www.mediawiki.org/wiki/Manual:User_rights

Change-Id: Iabe785fedb121a32d8adac5a490d9b5b9f40219f
2016-11-20 14:44:14 +00:00
Tom Fifield d7dc1746d6 Restrict File Uploads, Page Moves to Known Users
Since https://review.openstack.org/#/c/368114/
introduced the 'autopatrol' group, a team has been proactively
verifying and adding known-good users to the group as edits are
made.

The group currently contains almost 300 members, and most non-spam
edits made on the wiki today are made by users with membership of
this group.

To date, group membership allowed bypassing the CAPTCHA for edits.
This patch proposes 3 additional, low impact, changes to further
help with our ongoing spam problem.

1) Restrict File Uploads to members of the autopatrol group

Non-Spam file uploads on the OpenStack wiki are relatively rare,
particularly for new users, and anyone who has uploaded a file in the
past is already covered by the autopatrol group.
Restricting this ability to verified users would completely remove
a major spam avenue from the current attack.

2) Restrict Page Moves to members of the autopatrol group

Page moves tend to only be conducted by very experienced users of
the OpenStack wiki, who have all been around a long time and are
members of the autopatrol group. They are also very rare outside of
spam situations.

In spam situations, page moves make clean-up doubly difficult and
allow spammers to work around new page creation restrictions.
Restricting this ability assists somewhat with the spam attack

3) Restrict Write API access to members of the autopatrol group
Author is unaware of anyone in the OpenStack community using the
MediaWiki write API, however the fact that it is open for all users
seems to make it a very tempting attack avenue for spammers.

Change-Id: I8a59e2a0d0fcddc0f2774a8a704c1ac57003d6a8
2016-11-08 01:04:56 +08:00
Jeremy Stanley 3b55a5b98d Allow sufficient access to serve image files
Add a grant for all clients to read files from the images directory.
Without this, all requests for images are denied.

Change-Id: I50817ce3170453b6522e01ad3e1a3d503f59f9fb
2016-09-10 17:45:13 +00:00
Jeremy Stanley 35f57bcb4f Make robots.txt reachable
If disallow_robots is enabled, make sure we not only install and
alias the robots.txt file but also grant permission to read it and
omit it from our general article URL rewrites.

Change-Id: I9532dd8fd18010aaad388e8fdcbc3051fc653234
2016-09-10 14:37:29 +00:00
Tom Fifield f5a6ac32a7 Skip CAPTCHA for autopatrol users
autopatrol users have been human-verified to be legit.
Therefore, they don't need to go through the pain of CAPTCHA

This patch adds the necessary permissions.

Change-Id: I501eae00f471ee13525073d39c8e51a7ccb67fdc
2016-09-10 13:39:41 +00:00
Jeremy Stanley d79c672d41 Support disallowing robots
Add a disallow_robots parameter which can be used to disallow
indexing of all site content by robots. This is particularly useful
for test deployments where you don't want extra (often stale) copies
of your content to show up in search engines.

Change-Id: Ic62a72555315bd344db338809920a3605f17c8c6
2016-09-07 20:45:25 +00:00
Jeremy Stanley 43b131e06d Conditionally define the default extension source
Since $type and $source are both class parameters, we can't
interpolate $type in the default definition for $source. Instead
redefine $source inside the resource if it's unchanged from the
resource default.

Also make the contents of /src/mediawiki/w require the vcsrepo
resource for it. Early creation of the directory causes vcsrepo to
fail to clone into it.

Change-Id: I9b447173d4966f03a195825047cf25afc1a4afd6
2016-09-07 19:29:00 +00:00
Jeremy Stanley ea9b6df992 Clean up OpenStack references and genericize
To make this module more downstream-friendly, abstract out any
references to OpenStack and turn them into classvars.

Change-Id: Ie005d5629623a14f4ebd8aef21294f222249232f
2016-08-24 18:18:20 +00:00
Jeremy Stanley 0af7e6addd Clean up old recaptcha parameters
Now that the openstack_project::wiki class no longer sets the old,
unused recaptcha parameters, remove them from the module.

Change-Id: I468b4936582701cf308be269bfc06801276efd18
Depends-On: I9f9ba63399a8885e3694cef37ec987f223ff6dca
2016-08-23 18:23:26 +00:00
Jeremy Stanley 28767a5486 Switch from old recaptcha to recaptcha-nocaptcha
The newer recaptcha-nocaptcha implementation is a little better at
thwarting script-driven spammers and vandals, so switch to it. Leave
the parameter names for the old recaptcha implementation in place
until our consumer in the openstack_project::wiki class no longer
attempts to set it.

Change-Id: I435484cd65a028d774dfd920ca19f3077d4e03fb
2016-08-23 18:23:26 +00:00
Jeremy Stanley e91131f664 Parameterize database connection settings
So that we can have non-local (e.g., Trove based) databases, and for
general downstream flexibility, allow the database server hostname,
database name and database username to be passed into this module by
the caller.

Change-Id: I26a03e2ce9a6bf8bcd6acbad9e8eaaa98e0f26ab
2016-08-23 18:23:21 +00:00
Jeremy Stanley c4044a9766 Update scope.lookupvar() calls to shorter @ lookup
Consistent with the vhost template, in Settings.php.erb use the
shorter @ lookups rather than calling the scope.lookupvar()
function.

Change-Id: Ic03dbb55e4931d334a2cf5d4fae6a07cf53dd807
2016-08-23 18:21:18 +00:00
Jeremy Stanley 4cbe39375c Standardize HTTPS and vhost configuration
Adjust the SSL/TLS X.509 file handling to match our flexible
standard, which allows the deployer to choose between managing the
contents in or outside Puppet while still providing for a sane
snakeoil fallback to ease testing and development. Also tune the SSL
configuration to our typical stricter defaults, unconditionally
redirect all HTTP requests to HTTPS, and reformat the vhost
configuration for consistency and ease of future maintenance.

Change-Id: Id5241377665845f8c453cbb833bc40d3a5915d76
2016-08-22 23:21:51 +00:00
Jeremy Stanley 670312c04a Drop Ubuntu Precise compatibility
Since we've already manually upgraded production to Ubuntu Trusty,
let's start with a clean slate for now and just target latest
Mediawiki on a fresh Trusty deployment. This simplifies or removes
some of the conditional logic in this module.

Change-Id: Ia8e74c85667719178c2f50b1458a9719582e8502
2016-08-20 00:36:23 +00:00
Jenkins 742d9c09a9 Merge "Ensure cache directory is owned by the right user/group" 2016-08-19 17:53:59 +00:00
Jenkins 08b779ae7f Merge "Also require php5-memcached" 2016-08-19 17:41:12 +00:00
Jenkins c36606bd78 Merge "image_scaler: Partially separate packages for Trusty" 2016-08-19 17:39:09 +00:00
Jenkins fc55c69401 Merge "Puppetise extension repositories" 2016-08-19 07:55:59 +00:00
Jenkins f7fc800421 Merge "Stick to REL1_27 on Trusty, not master" 2016-08-19 03:38:51 +00:00
Jenkins bbf120e47e Merge "Also clone the vendor repository for newer MW versions" 2016-08-19 03:24:56 +00:00
Jenkins 864955904c Merge "Puppetise LocalSettings.php -> ../Settings.php symlink" 2016-08-19 02:37:08 +00:00
Jenkins 53885449ef Merge "Add 'autopatrol' group" 2016-08-19 02:00:12 +00:00
Tom Fifield a03b466574 Add 'autopatrol' group
Patrolled Edits is a feature on mediawiki that allows marking edits as
"known good" (i.e. ham, as opposed to spam). This can be used to
list edits that have not been assessed, which makes spam easier to
identify and deal with.

Automatic patrolling is where this happens without human intervention.
In our version of mediawiki, there is the ability to have defined users
whose edits are automatically marked as 'patrolled'.

This patch makes a new group, so that bureaucrats of the wiki can
assign "known good" users to this group. The group is given the permission
of 'autopatrol', which means users added to this group will have their
edits automatically marked as 'patrolled'.

Further information about this feature and how to verify the syntax for
this patch is available at:
https://www.mediawiki.org/wiki/Help:Patrolled_edits
https://www.mediawiki.org/wiki/Manual:User_rights

Change-Id: I798f3909bc95fb28c3a0f704fb89a490e10b015f
2016-08-18 08:42:32 +08:00
Alex Monk caf009dc7a Ensure cache directory is owned by the right user/group
Change-Id: I72ac640b70675561ef32f906b47f6d900e76a028
2016-08-10 18:45:13 +01:00