The RedHat-based operating systems such as CentOS have a pretty
stable list of packages, therefore, we don't need to pin it by
version and we can instead move to a much more generic redhat.yml
which will support a bigger range of systems.
This patch also sets up the mode for the /etc/mysql directory
to be 755 for things to work. We should lock that down eventually
for only the MySQL user.
Change-Id: I2e9ee987a353a28da00cd37cbdb82ec46ccc414c
We're currently deploying 10.2.17 which is quite old and we seem
to be having issues in the gate with some database sync's causing
MariaDB to crash, as well as personal experience with similar
crashes around Cinder database syncs
In addition, this change implements the usage of mariabackup for
SST transfers which eliminates the need for the Percona repos.
The compression is no longer recommended by upstream now, therefore,
we remove it.
Change-Id: I59a0e225205be43b5bfc76c4bc3381b6e9c54cfd
This patch adds /etc/pki/rpm-gpg/PERCONA-PACKAGING-KEY which
comes from installing [1] and is required in order to install
the percona packages on CentOS packages. Without it we get
the error:
Public key for percona-toolkit-3.0.13-1.el7.x86_64.rpm is not installed
We also rename RPM-GPG-KEY-percona to RPM-GPG-KEY-Percona in
order to match what is done by [1].
[1] https://repo.percona.com/yum/percona-release-latest.noarch.rpm
Change-Id: I2eeac8ea21c52fac3b1f32036fedcf7c62aac37f
To ensure that we have a consistent implementation
between the galera_client and galera_server roles,
we change the galera_server role to match galera_client
as was done in I520ccbadf3320b0d07fc83e3dbec9ea2bd16ec83
This updates it to a mechanism which will be easier to
maintain.
Change-Id: I7ac1a5e3a05aa3d0b4fae86c4a325ef147a9a528
In https://review.openstack.org/534819 we introduced the removal
of conflicting packages from the targeted host, but then we deleted
the list for RedHat in https://review.openstack.org/578844 because
yum kept removing and re-adding the same packages.
In https://review.openstack.org/603205 we solve the issue properly
given that yum is case-insensitive, and the root cause of the repeat
remove/install.
As such, in this patch, we restore the removal of conflicting
packages for RedHat in a different way. Each of the package removal
tasks are moved into the tasks specific to each package manager so
that each can be handled differently.
Change-Id: I70fbfa6eff8796713c6bec32319382273f8281f8
Related-Bug: #1762421
Related-Bug: #1742206
mariadb 10.2 was first released in April 2016 and OpenStack services
should be ready to use it by now, so lets switch to 10.2. For mariadb
10.2 we need to use xtrabackup-24. This version is not available in Leap
42.3 so we add an extra OBS repository for it.
Change-Id: I5f1aaf1f8608ad085acfebc8458910391f280193
Removing mariadb-server and mariadb-libs leads to
removing MariaDB-server and MariaDB-shared and its
dependencies
rpmquery --whatprovides mariadb-server
MariaDB-server-10.1.30-1.el7.centos.x86_64
rpmquery --whatprovides mariadb-libs
MariaDB-shared-10.1.30-1.el7.centos.x86_64
Which in turn shuts down the server.
Change-Id: I89df8f6b57b1ebe0ce96d63f193da35107d2bad9
Closes-Bug: #1762421
Related-Bug: #1742206
We have mirrored provided by the OpenStack infrastructure which
should reduce the amount of network failures with MariaDB.
Depends-On: I49a744ee5c41279f4a17bc7d2339d12c3a132026
Change-Id: I205e471dba844a98a5c81e0e387704bef648069e
We don't need python cryptography bindings in order to deploy a Galera
server therefore drop it from our dependencies. It also is no longer
used inside OpenStack world for lack of security.
Also, we make remote network hits to get the GPG keys which are quite
unreliable so let's store them inside the role and use them.
Change-Id: I9443f10e8c803599cbebfc2a53cb9c432bfa60d1
In order to allow using mirrors, this introduces a new variable which
can control the host for the percona repo: galera_percona_xtrabackup_repo_host
Change-Id: I4d9d408f99608f1625dac9e0802e8d8d77b0f48b
Due to the security role adding the `localpkg_gpgcheck` option and
the fact that the Percona release RPM is not signed, it fails to
install leading to failing gate for CentOS.
The behaviour for Debian-based operating systems is to manually
install the repositories so let's switch CentOS to do the same.
Change-Id: I71840bd209cd398e0ce46d61ac895dea68ba193c
Related-Bug: #1675918
The hosts may end up in a rather messed up state when mixing
distribution and pip packages. The python MySQL bindings are already
provided in the distro repositories so we can simply use these in order
to keep the host as clean as possible.
Implements: blueprint openstack-distribution-packages
Change-Id: Ic8c7f642378536d79d763625451ec267c2a108fd
In https://review.openstack.org/535252 the installation
process for the role was simplified, but an unintentional
side-effect was to removed the previously included support
for installing the extra percona packages when installing
on the ppc64le platform.
This patch re-introduces that ability, but scopes it to
only execute on that hardware platform, and only for Ubuntu.
The download is, by default, facilitated through the deploy
node (rather than the target nodes) so that the download
is done once, then pushed to the targets. This can be
adjusted with the right parameters to download from the
targets instead.
Also, in https://review.openstack.org/543888 adjustments
were made to disable compression/qpress on architectures
other than x86_64, and to fail the role execution if it
was enabled on any other architecture. This has been
corrected to ensure that compression is enabled by default
for ppc64le on Ubuntu, and enabled by default for x86_64,
but disabled by default for all other combinations. The
fail task is adjusted appropriately and moved to the main
task file so that it executes and fails out before any
changes are made.
Change-Id: I850a37b465a427a827e357111942973457fafa0d
The 'use_percona_upstream' variable had quite different and conflicting
meanings in the tasks. It's used to either denote that the upstream
packages will be used directly or that the upstream repositories will
be configured on the hosts and then use the packages from these
repositories. However, the packages in both cases are the same so the
variable was pointless in this form. The variable was meant to be used
to select between upstream and downstream repositories for the Percona
packages. Moreover, upstream does not provide packages for SUSE so we
have to exit the deployment if that variable is set.
Partial-Bug: #1654004
Change-Id: I18853707c29955186da5a1f4d6e80bc3012c04f9
CentOS has a repo configured for MariaDB that uses HTTPS and this
prevents the repo server from caching the MariaDB packages. That
causes significant delays during CentOS 7 deployments/gates since
those packages are quite large.
This patch changes the MariaDB mirror URL for CentOS to use HTTP.
Closes-Bug: 1750665
Change-Id: I2192ade6eb204404fe67d47af063e0e57fbe7d62
The qpress library is only needed for xtrabackup compression. The
compression only occurs when a new node is joining the cluster
which shouldn't happen very often. The variable is set to true only
when the architecture is x86_64 since that's the only arch that's
supported by the upstream repository and CentOS and Ubuntu do not
provide qpress in the downstream distributions at all.
Change-Id: Iae8b5d0e0998c85ee453efc6620940c80243540e
When the galera_server role is deployed along with other roles, some
of them may pull mariadb packages as a dependency for the services
they deploy. However, this role uses MariaDB packages from the upstream
repository which conflict with those provided by the EPEL or the RDO
ones so we need to ensure that they are gone before we install the
upstream ones.
Change-Id: Ie800c1f4628eff24954f392fe6318206adfabddb
Closes-Bug: #1742206
At the moment, RedHat currently installs and listens on the
/var/lib/mysql/mysql.sock path, this can be seen by the following:
mysqld 13627 mysql 33u unix 0xffff88079261cc00 0t0 3312115 /var/lib/mysql/mysql.sock
This patch resolves it which will allow the health checks to
function as they rely on the [client] section which contains
an incorrect path.
Change-Id: Iec88e911c6cf207bcea5bd067f9743b3eb50f60e
This is the time of the cycle where we update all the static
elements in the roles.
This version is defined staticly, so we bump it.
QPress has no updated package.
Suse has no definition for those packages.
Change-Id: I0af27722a0d7f2a3c3c0a456a0c60512e36a5728
The galera server role has quite a bit going on within it and because of
recent improvements in Ansible we can make better use of tasks, blocks,
facts, local facts, and organization. This change tunes the role up
following some of our better/more modern patterns allowing the role to
not only be more efficient but also easier to understand and improves
the roles idempotency.
Change-Id: If189a8192f22aafb168587361ca8e6903c918697
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
The galera cluster rely on WSREP for cluster consistency. While the
default MySQL monitor will allow us to know when the database node is
minimally functional it does not provide the ability to query the node
state allowing loadbalancers, operators, and deployers to know a node
is healthy prior to being allowed to accept connections. This change
implements the checkcluster script as provided by the fine folks at
Percona. The implementation of this check follows the guild-lines noted
here [0]. With this in-place, we'll be able to convert our haproxy check
for the galera cluster nodes to use an HTTP check on port 9200 instead
of the default MySQL login which will provide for a more robust and
fault tolerant cluster.
[0] https://www.percona.com/doc/percona-xtradb-cluster/LATEST/howtos/virt_sandbox.html
Closes-Bug: #1665667
Change-Id: Ie1b3b9724dd33de1d90634166e585ecceb1f4c96
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
openSUSE uses /var/run/mysql/mysql.sock instead of
/var/run/mysqld/mysqld.sock so make this option distro configurable.
Change-Id: I5a42bd5bdab1a89bfc1e39b6764a1380d8d98918
Setting a custom MariaDB mirror URL is very tedious to maintain
because operators must ensure they update the overridden repo
URL every time the major version changes within the galera role.
This will allow operators to set overrides for their local mirror
like:
galera_repo_url: "http://mirror.mydomain.com/mariadb/repo/{{ galera_major_version }}/ubuntu"
Change-Id: Ie5dd27b8cffff5245724ecc79e79272086bf3b30
Currently overriding this setting requires the
use of extra-vars. This change allows it to be
done via any other variable override.
Change-Id: I842fecfcf851b026c008f4fd1296c790bd00c5cc
It may be required to override the repo information
in its entirety, rather than just the URL. This
patch allows that to be done.
Change-Id: I9cfd0ae520b9b584ddc50933311c3fdb555d5029
Add support for the openSUSE Leap distributions. For openSUSE we use the
packages provided by the Open Build Service repository which contains the
Galera clustering support. These packages are different compared to the
CentOS7 and Ubuntu ones so the way the cluster is being bootstrapped is
also different. The systemd service file can't be used for that, so we
need to modify the MariaDB configuration file on the boostrap node to
initiate the cluster. Moreover, files are installed in different places
so we need to modify the distribution files and templates to take that
into consideration as well.
Change-Id: I1ac31fbc06152da7f93e57911d4a952f0dd83849
When trying to execute an upgrade the removal fails
if packages in the list are included that cannot be
found in one of the repositories configured.
This patch endeavours to only remove packages that
will be found.
Change-Id: I67cc225b25fca005cdba3bf67793b4f23d6ae82a
This patch bumps the version of the percona xtrabackup tool, and
ensures that CentOS/ubuntu are running the same version.
Change-Id: I3e82c7be72df67604dbf43a5123a08bdbc1ed8a9
This patch ensure the local host's ansible facts are used for the
"ansible_architecture" fact, rather than the "physical_host". Since we
don't explicitly collect facts for "physical_host", and the
ansible_architecture fact is set for the local host container, we
don't need to try to retrieve the "physical_host" facts.
A new var "galera_package_arch" is created so that the architecture can
be overriden if it isn't being calculated correctly. Additionally, the
duplicate vars that are not specific to distributions have been moved
into defaults, and out of the specific distribution vars files.
Change-Id: I5124c061faf124388a88eefee93d96e9a9e90d8f
Closes-Bug: 1650384
Update repos and packages to install the server for MariaDB 10.1, the
current stable release.
Several improvements have been made to improve the upgrade process and
an upgrade from 10.0 has been included in the test playbooks.
Make use of the yum_repository module for installing on yum based
systems and give the apt repository files consistent names, 'MariaDB'
and 'Percona' for easier maintenance and handling of upgrades going
forward.
Depends-On: I8939703f26e5d8adc393b984266f4cad7a6e0b4c
Change-Id: Ib6409f1fcf4a664b65dbe01372a19509d02d70a4
Percona has started signing their Ubuntu packages with a new key.
Update the galera_gpg_keys var to add it to the keyring.
https://www.percona.com/blog/2016/10/13/new-signing-key-for-percona-debian-and-ubuntu-packages/
To resolve a reno dependency required to pass gate testing, also include
the libjpeg-dev package in bindep requirements.
Also, due to an expired certificate being used for
https://yum.mariadb.org the GPG key is set to be
fetched via http.
Related-Bug: #388205
Change-Id: I1dc3a78b8e0b759cc98f3de2bd33774db3b477da
physical_host is a host variable created by openstack-ansible's dynamic
inventory. When using this role outside of an integrated OSA deployment
it wouldn't be expected to be set. Fallback on using inventory_hostname
when physical_host is undefined.
Closes-Bug: 1631158
Change-Id: I3364aac2e707d10ac9b4020dfcc0de7d4d158ff9
Add architecture-specific location support for percona-xtrabackup
and qpress packages to fix limitation of hardcoded amd64 packages.
Change-Id: Ifa9d53a91f6ae7d497045fa4e7558c1b7c4d53a8
In order to make it easier to differentiate between the lists of
python packages, distribution packages, downloaded packages,
package pins and other similar variables the variable names are
being changed to ensure that they have a more explicit suffix
that defines the purpose and makes the naming more consistent.
This is to facilitate a lookup plugin which will be able to look
up all the package lists and present them as a consolidated piece
of data which may be used for artifact preparation.
Change-Id: Iba040bf8da6ec0552fcf7d10022094dc57d312bc
Updates the multi-distro framework to add CentOS7 support.
Depends-On: Ieadcdd70e9b8271aee3880896255a5037f56567e
Change-Id: I7e889ea4653a6972ea7f8208416fc1fc3db45e5c
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
Mohammed Naser