Since we've switched to ansible 2.10 we should explicitly use modules
provided by mysql collection.
Change-Id: Ic0e96e219792638b24a8e62517d98e21c3150f46
This is a merge of two previous patches [1,2] which require each
other in order to merge in particular due to unpredictable behaviour
of MariaDB 10.4 during cluster status reporting which is addressed
through [2].
[1] https://review.opendev.org/735204
[2] https://review.opendev.org/735079
Co-Authored-By: Dmitriy Rabotyagov <drabotyagov@vexxhost.com>
Co-Authored-By: Marc Gariepy <gariepy.marc@gmail.com>
Change-Id: Id04fa994f527e2a15a87ba91d56d10eaadf8f58f
Patch aims to use next stable release of MariaDB
Remove no_log from functional tests, the only credentials used here
are for the purposes of the test and debug output is valuable.
Change-Id: Ic4cdc26ddf1cb0591f0a2218355b7cca7af0b0fb
The connection plugin no longer falls back on using the
inventory_hostname as the container_name. Set container_name as a host
var for each container in the test inventory.
Change-Id: Ib1bac8c596e97505e537095474aa9aca208eea41
Now that we're using the general templates, we can slim down
the role test definitions. We can also remove tests which are
not being watched, or which are fundamentally broken. With
this we can also remove unused scripts/plays.
We do the following:
1. We remove the 'upgrade' job, given that it's been broken
all cycle due to the way the job executes and we need to
regroup to figure out another way to do it.
Change-Id: I7d1f07878dba7d21aecd59fa75e9117b50b886e6
Now that run_tests.sh handles the tests repo clone, we can
remove the use of the older tests-repo-clone.sh script.
Change-Id: Ib127aa9cbf8672347f3734da7c363f5d65cc3144
Ansible 2.4+ suffers from a precedence bug when including tasks in
roles. It appears that a file in a relative tasks/ path inside the
current working directory is preferred over an identical file
from the role itself. For example:
task path: /home/zuul/.ansible/roles/previous_galera_server/tasks/galera_install.yml:20
included: /home/zuul/src/git.openstack.org/openstack/openstack-ansible-galera_server/tasks/galera_install_apt.yml for container1
As one can see, instead of including the galera_install_apt.yml from the
/home/zuul/.ansible/roles/previous_galera_server/tasks/ directory, the
one which is relative to the current working directory
(/home/zuul/src/git.openstack.org/openstack/openstack-ansible-galera_server/)
is preferred. We can simply workaround that by entering a different
directory which doesn't look like a role directory structure before executing
that part of the upgrade job.
Change-Id: If208e72d66ed441e8fcdee4882637301527cb231
Link: https://github.com/ansible/ansible/pull/34790
The meta-play test-setup-host.yml already includes
test-prepare-containers.yml so there's no need to
re-run it.
Change-Id: I04d195e579796cad38fb26ee07bb000cb8b4bed8
We are already collecting logs as part of the post-run playbook so we
don't need to do that as part of the exit trap. This avoid collecting
and compressing the logs twice.
Change-Id: If6f0adb93bcc6e2cae3885017c58159f3dca1772
When doing the role upgrade, we run every playbook with current's
branch ansible. This is normal for the infrastructure bits (which
are running on current branch), but not for the installation of
the previous branch's role, which should use previous branch's
environment.
Related-Bug: #1741471
Depends-On: I7ebb045885dd645b820de2b7f75b46c755c015f6
Change-Id: I7bc4dd644c48b403e9e669cb0ea460d9d1149410
The upgrade tests are very noisy since Ansible verbosity is set too
high. This makes debugging problems very painful so drop verbosity
by default and let users set it if desired.
Change-Id: I44b7f13f762807946ad9fdf0078e24c4c52b9da1
The galera cluster rely on WSREP for cluster consistency. While the
default MySQL monitor will allow us to know when the database node is
minimally functional it does not provide the ability to query the node
state allowing loadbalancers, operators, and deployers to know a node
is healthy prior to being allowed to accept connections. This change
implements the checkcluster script as provided by the fine folks at
Percona. The implementation of this check follows the guild-lines noted
here [0]. With this in-place, we'll be able to convert our haproxy check
for the galera cluster nodes to use an HTTP check on port 9200 instead
of the default MySQL login which will provide for a more robust and
fault tolerant cluster.
[0] https://www.percona.com/doc/percona-xtradb-cluster/LATEST/howtos/virt_sandbox.html
Closes-Bug: #1665667
Change-Id: Ie1b3b9724dd33de1d90634166e585ecceb1f4c96
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
The openSUSE repository bumped the mariadb package to 10.2. However,
there is an upstream bug that prevents nodes from joining the cluster.
The way to workaround it is to export WSREP_SST_OPT_PORT=4444 in the
systemd service file. Moreover, we pull the mariadb-galera subpackage
which contains some necessary tools for galera clusters. Finally,
we drop the default configuration files which are being installed by
the packages because they conflict with the ones installed by this role.
[hwoarang: This also applies the linter fixes from
https://review.openstack.org/#/c/523080/ in order to make the
gates happy]
Depends-On: Ia4856f36b2d106d987e3c774f31493e25a23d4b5
Link: https://jira.mariadb.org/browse/MDEV-14256
Change-Id: I97cf1585b2fed08f53f62a547547e422bc34fa53
This patch implements an initial set of jobs intended to match
the current job execution method. It does not intend to improve
how the jobs are executed - only to replicate what is currently
in openstack-infra/openstack-zuul-jobs and provide the platform
to iterate on.
Change-Id: I1911542e61df7bed475eac199669773ae4e1a205
When executing the upgrade test, the designated source
role is used for the initial build. This patch ensures
that the source for testing in master is stable/pike.
Change-Id: I7714a1145c21602fd6beaa53880e5345ea7f51eb
In order to independently test a greenfield deploy
and an upgrade, with the upgrade's initial deploy
using the previous stable branch role, we seperate
the two tests.
This makes it a more accurate test without having
to do funky workarounds in the role required to
cater for two versions of galera.
Depends-On: Ic881bf3acf6fb6daf71611c26322f02fb8b0b9be
Depends-On: I5c88b2019c01b44193a5d0df9299ecce6de52f01
Depends-On: I8b09c2520c84bb43353b4c56bac157259f71b041
Needed-By: I2aca4721e4e023d315f3fcdae0f931dc960ef2be
Change-Id: I0c74e511cec01ed51101b873784c4a90a7302da6
The .cnf files can have valueless options present.
This commit ensures that these valueless options are not ignored
Addtionally, my.cnf overrides are added to the variable overrides
test file to ensure that the my.cnf file can run through the
config_template engine properly.
Change-Id: I8b09c2520c84bb43353b4c56bac157259f71b041
Closes-Bug: #1693234
Add support for the openSUSE Leap distributions. For openSUSE we use the
packages provided by the Open Build Service repository which contains the
Galera clustering support. These packages are different compared to the
CentOS7 and Ubuntu ones so the way the cluster is being bootstrapped is
also different. The systemd service file can't be used for that, so we
need to modify the MariaDB configuration file on the boostrap node to
initiate the cluster. Moreover, files are installed in different places
so we need to modify the distribution files and templates to take that
into consideration as well.
Change-Id: I1ac31fbc06152da7f93e57911d4a952f0dd83849
In preparation for adding support for SUSE distributions, we sync the
test files from the openstack-ansible-tests repository so the upcoming
commits can be tested on all supported distributions.
Change-Id: I2244f7cf0de908a10eec244122f8be32c3ade459
This patch adds the `galera_disable_privatedevices` variable that
allows deployers to disable PrivateDevices in the systemd unit file
shipped with MariaDB 10.1+ on CentOS 7 systems.
This is a workaround to fix the systemd/LXC issues with bind
mounting an already bind mounted `/dev/ptmx` inside the LXC
container.
See Launchpad bug, lxc/lxc#1623, or systemd/systemd#6121 for more
details.
Co-Authored-By: Major Hayden <major@mhtx.net>
Closes-bug: 1697531
Change-Id: I8a74113bd16a768a4754fb1f6ee04caf1ac82920
When executing the tests repo clone in OpenStack-CI,
use zuul-cloner instead of git to enable cross-repo
testing. This ensures that if a dependent patch from
the tests repo is noted using 'Depends-On: <change-id>'
in the commit message, that patch will be included.
Change-Id: Id40ba2e8093741feb795959a9d8292fd73af887b
Depends-On: Idce7abebf32f24c356a27e099fbca954d917402b
Move test host vars from the inventory to individual files for each
host. 'ansible_become' has been removed from localhost's vars since it
should be handled on a playbook basis.
Change-Id: I57f17002f6e8877403d3fc00f6eb148f50b80241
During an upgrade, we run the upgrade in serial, and we attempt to
bootstrap the cluster from the "galera_server_bootstrap_node". This
causes issues when there is an existing cluster that is still up.
To avoid this, we can simply attempt to start the cluster normally, if
it fails to start, and join the existing cluster, we can bootstrap as
usual.
Additionally, we can make the bootstrap slightly more efficient by only
running it against the bootstrap host and splitting out the
"galera_upgrade_post.yml" tasks.
Finally - run the upgrade in serial which mirrors our approach in the
integrated repo.
Change-Id: Ic4d69f0fa75c1eea81d10a76cca9a8d9c3822094
Closes-Bug: #1667103
Update repos and packages to install the server for MariaDB 10.1, the
current stable release.
Several improvements have been made to improve the upgrade process and
an upgrade from 10.0 has been included in the test playbooks.
Make use of the yum_repository module for installing on yum based
systems and give the apt repository files consistent names, 'MariaDB'
and 'Percona' for easier maintenance and handling of upgrades going
forward.
Depends-On: I8939703f26e5d8adc393b984266f4cad7a6e0b4c
Change-Id: Ib6409f1fcf4a664b65dbe01372a19509d02d70a4
This standardizes the IP range and bridge usage for testing. Which lines
up with the various other roles and how they are setup.
Change-Id: I835d7174f94f2c96a9554fddd4bd58a8dd75647f
This changes 'ansible_ssh_host' to 'ansible_host'. The 'ansible_ssh_host'
variable has been deprecated as noted here: [0].
[0] - http://docs.ansible.com/ansible/intro_inventory.html#hosts-and-groups
Change-Id: If133068625584b8794615bf212c700b6253dce19
Related-Bug: #1636606
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
This patch consumes the test scripts implemented by
https://review.openstack.org/375061 to ensure that
the tests and test preparation is consistent and
more maintainable.
Change-Id: I635914f58a095362788f4430b578e8d5f9009842
Ansible 2.1.1 introduces a regression in the way conditional
includes are handled which results in every task in the
included file being evaluated even if the condition for the
include is not met. This extends the run time significantly
for a deployment.
This patch forces all conditional includes to be dynamic.
Change-Id: I18d8c2119e00aeec3c53cb7e39c8cf09cf5135f8
Related-Bug: https://github.com/ansible/ansible/issues/17687
Replacing usage of ansible_ssh_host, ansible_ssh_user,
ansible_ssh_port with ansible_host, ansible_user and ansible_port
Change-Id: I35371879a87c115f219cd181245fe59f6d7d3f77
Updates the multi-distro framework to add CentOS7 support.
Depends-On: Ieadcdd70e9b8271aee3880896255a5037f56567e
Change-Id: I7e889ea4653a6972ea7f8208416fc1fc3db45e5c
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
Previous versions of OpenStack-Ansible allowed
for upgrading Mariadb from version 5.5 to 10.0.
Mitaka assumes Mariadb version 10.0, thus the
upgrade check is no longer needed for gate
testing.
Change-Id: I589b3849e52ab4d6810f16588c733aed4d17b6b2
The following changes have been made to reorganize the structure of this
role's test playbooks, bringing them into line with other
OpenStack-Ansible roles:
- move each playbook to an individual file
- rename playbooks descriptively
- define hosts and groups directly in the inventory file
- include group vars required by containers
- store extra variables in a single shared test-vars file
- remove use of the openstack_hosts role
- include callback plugin for human readable logging of test tasks
Change-Id: I4829e368003f58685cb96b7ee743255d00542187
The git source for python2_lxc was necessary before the package was
available on pypi, which it now is:
- https://pypi.python.org/pypi/lxc-python2
This patch removes the py_from_git role from the functional testing
playbook.
Change-Id: Icdee44fc00546c2e565c7d4233af60d0f1d8adb6
This patch sets the test to check if the test is running in
OpenStack-CI, and if it is - then make the containers use
the host's pip.conf and some of the apt configuration.
All containers should make use of the hosts's apt sources
configuration. The tasks detect and implement this.
The intent is to make use of OpenStack-CI resources better
and consequently reduce the test run time.
Change-Id: I56012c4dd79477180f87cc25c5175ac08baf470e
Create a new variable called galera_cluster_members which is a list of
the hosts that make up the galera cluster. Set the default value to
groups['galera_all'] which is the group used by openstack-ansible to
refer to the members of the cluster. Replace all other references to
groups['galera_all'] with the new variable. This allows other consumers
of this role to use their own group naming scheme.
Add a new task to verify that current host is a member of
galera_cluster_members.
Add the var galera_client_drop_config_file=true to the role dependency
galera_client, this is required because the default value for that var
in the galera_client role is based on membership of the galera_all
group.
Modify the functional testing to override galera_cluster_members, this
aims to prevent the return of any hard-coded role references.
Change-Id: I59af07217114a001cbebaa95a651919d53c9ec21
The change adds in the ability for the role to take care of a major upgrade
in a version of an installed mariadb galera cluster.
The change adds a new task file that checks if the installed version of the
galera cluster matches that of the specified major version. The role will
hard stop if there is a version mismatch and the option "galera_upgrade=true"
is not passed.
Change-Id: I26560668325d45f670c8b946c978c48559f58419
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
This patch updates the tox.ini the same bashate exceptions as are
currently in the OpenStack-Ansible playbook repo.
It also ensures that the linters and all lint targets work
appropriately and normalises the tox.ini configuration to use
uniform formatting.
The use of ansible.cfg is removed as there is no way of being
certain which paths can be used without reverting to an ugly
sed hack in the commands. This is why it is preferred to make
use of environment variables which make use of tox's default
substitutions instead. It's a more reliable way of achieving
the goal for the purpose of gating and testing.
The switch to using a git clone instead of ansible-galaxy to
download the plugins is due to the path spec not being able to
work in Ansible 2.x. [1]
[1] https://github.com/ansible/ansible/issues/13563
Change-Id: I077c5b79a95e78fd81c33382843273893d48bbb6
The role was changed to make it compatible with the OpenStack
CI. The changes effect defaults, handlers, and the tests for
the role and adds gitignore/review files.
The changes essentially get the role to a state where its passing
the tests which are spinning up a galera cluster, adding users
and databases, and then testing integrity from every node. The
tests specifically ensure we're able to guarantee that after the
role runs everything works. Previously to these changes the role
assumed everything was working and nothing had been done to
guarantee cluster state.
In the handler changes, the temporary "sst" directory is cleaned up
should the handler restart fail. This ensure that a node is in a clean
state if a leftover sst directory was on the disk which would cause
a node to fail to join a cluster or bootstrap. Additionally the
environment variable "MYSQLD_STARTUP_TIMEOUT" is now being passed
into the init script because the defaults are not being sourced
at the init script runtime.
In the task changes a new configuration file, that is part of the
mariadb package, is being removed which has unforeseen options within
it causing no logs to be created.
the default option "galera_innodb_additional_mem_pool_size" was removed
because its no longer valid within MariaDB10 and we'd never caught that
error message until now.
The tests were updated to support running the role from a user which
was not root.
Change-Id: I16af30c660790656fc2d59f9943c172b88098905
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>