There were bunch of tasks for upgrading from lsyncd to the shared fs
mount, which can be safely cleaned-up now.
Change-Id: Ia0f5cbced196467007eafc61d3152ebea7559b84
At the moment there is a check, which ensures for race condition to
not happen between asking for a mount and mounting the point.
However, it tries to check for the mount when there is no mount
defined for the directory.
We add extra check to wait for the mount only when mounts are defined.
Change-Id: I900a55a6f4edce3d3fe419821c47cf56d641192f
With update of ansible-lint to version >=6.0.0 a lot of new
linters were added, that enabled by default. In order to comply
with linter rules we're applying changes to the role.
With that we also update metdata to reflect current state.
Change-Id: I74cefdfa885fa26dd7199fd0798527f511bf329d
By overriding the variable `repo_backend_ssl: True` HTTPS will
be enabled, disabling HTTP support on the repo_server backend.
The ansible-role-pki is used to generate the required TLS
certificates if this functionality is enabled.
Depends-On: https://review.opendev.org/c/openstack/openstack-ansible/+/879085
Change-Id: I5c5d3dd5689ac122781303ad21dacc8a1fa746eb
Include is deprecated, additionally include_tasks is now faster.
Signed-off-by: Kevin Carter <kevin@cloudnull.com>
Change-Id: I20f92b972b794513cb774983f3aa6044769987d7
The version of nginx on centos-8 appears to keep file handles open,
possibly the old /var/www/repo directory persistently. Once the old
content is removed and the new shared filesystem mount is created at
the web root, ensure that nginx is restarted to close any file handles
which are now stale.
Change-Id: I941359b1b42aa4a874230a32b438dcefddfb2acb
Currently we're trying to restore content for each host independently
This is not needed as we already storing content on shared FS,
that's been mounted.
We don't implement variable since that's only for migration purposes.
Change-Id: I0795fa1936aaeb9b9005a35685d85b6a4a619835
With current behavior of u-c file that is stored on repo server and
migration to shared FS instead of lsyncd, we don't have any reliable
path to check for.
Also we have issue, that with updated u-c SHA users catch "unrelated"
error in python_venv_build role.
As a solution, we can create a symlink to the u-c file, that will have
persistant name and be updated with repo_server role run.
That would give us both file to verify and it can be used as u-c URI.
Change-Id: Ie7bdb9137ed69be465f014fa811b750dbec4e428
Related-Bug: #1943978
These are no longer needed when a shared filesystem is used to
syncronise the repo server contents
Change-Id: I3109bd891d4c6b3522f5f741d9970093b1e882c8
This patch checks if the repo server content directory is a mountpoint,
and if it is not, creates an archive of the repo server contents
before mounting the shared filesystem. After the mount completes, the
archive of repo server contents is copied to the shared filesystem.
This runs on every repo server in the deployment to include corner
cases of repo servers running different OS versions or CPU architecures
and will gather all of the content onto a single shared mount.
Change-Id: I976a5ea5f6b6ebd65c22e89657763fef87cf4b23
If insist is not set to true then lsyncd will exit if it cannot
perform an initial rsync to the target hosts.
Due to the order in which the repo servers are configured, lsyncd
may be installed and started on the first host in the repo_servers
group before the ssh keys and other necessary configuration have
been placed on the remaining hosts. This leads to a failure to
start lsyncd.
This patch moves the setting of insist into the lua config file
for all operating systems, and removes the need to template a
defaults file on debian derivatives.
Change-Id: I26bb0e21d797c2bfbe67e03003da01c355c27561
Use a first_found lookup instead of a with_first_found loop so that
the 'paths' parameter can be used.
This ensures that only vars from the role are included, and not vars
from a parent calling role. This can happen when a parent role has
a higher priority vars file available for inclusion than the role
it calls.
Change-Id: I14afe3040d8ecd8702edca19877ab50b6e57007f
All references to Gentoo, SUSE, Debian stretch and Centos-7 are removed.
Conditional tasks, ternary operators and variables are simplified where possible
OS specific variables files are generalised where possible
Change-Id: Ibe914d0d547b930b8167bfa475cc838df8d7ae25
If a repo container and its data are deleted and recreated then it is
not currently possible for a loadbalancer healthcheck to differentiate
between an empty repo server and a correctly synchronised one.
This patch creates a file 'repo-sync-complete' as part of the process
of synchronising repo contents from master repo servers to slaves. The
presence of this file on the slave can then be used as the loadbalancer
healthcheck to ensure that repo contents are only served once sync has
completed.
In addition, this patch ensures that synchronisation occurs from the
master to a reprovisioned slave by triggering a master repo server lsyncd
restart handler during the initial setup of the slave repo server.
Currently, a freshly provisioned repo server will remain empty
for an indeterminate amount of time, this patch forces a complete re-sync
to occur.
Change-Id: I6913341674dbde5524c2270e824bda4544211eca
Reverting patch https://review.opendev.org/#/c/709532/ which is installing lsyncd using source
but now el8 epel repo providing distro rpm for lsyncd so we don't need to install from source.
also changing lsyncd.conf.lua path to /etc/lsyncd/ for centos8 support.
adding rsync-daemon rpm because centos8 islocated daemon and client in seperate package.
Change-Id: Ia8c1fad5728bd6adec510f8bc1d7bb754393de93
Previously the handler for restarting rsyncd was never called.
Split the config file template task out from the nginx task and
notify the rsyncd handler when the config file is changed.
Change-Id: I47982b5974c686a6dec7bd1a789daa54ea9652c7
This avoids the following warning:
[WARNING]: The value 0 (type int) in a string field was converted to u'0' (type
If this does not look like what you expect, quote the entire value to
ensure it does not change.
Change-Id: Ifaf4035b2101f6921182c3d3275554c7e5f4cfa2
The apt-cache was removed a long time ago and these tasks should
already have been removed in the Train release. Remove them now.
There is no change for existing deployments as these tasks are no
longer required as the apt-cache component of the repo server is already
retired.
Change-Id: I42785ee153955eb6e72bef01c26791ae8eec2087
Caching git repositories has been deprecated in Queens, so it's high time
we removed this functionality.
This shouldn't influece OSA deployments in any way.
Change-Id: I35829aa35489f06dbb3b65f522f0a08318eccbfa
For Centos, we use by default public repo for nginx. You can change this
behaviour with these role-wide variables:
- repo_centos_nginx_mirror
- repo_centos_nginx_key
Or with these osa-wide variables:
- centos_nginx_mirror
- centos_nginx_key
Change-Id: I8dcb3c97e9593877a4a420bd32b50ae29d9d311c
When using a custom repo with repo_centos_epel_mirror, you maybe need to
change the gpg key url because offline env
You can use this variable: repo_centos_epel_key
Change-Id: I9bb305d866f3d65653b95a25a5b9f5ecde5af0b0
The newer packages havee moved rsyncd into its own package called
rsync-daemon so we'll install that for anything newer than EL7.
Change-Id: Id1e8cc503d71cf6e7d0d84b5a74f6c3d7bce9393
It is no longer needed because of how we are using python_venv_build
at the moment, so let's remove it.
Depends-On: https://review.openstack.org/648477
Change-Id: I56531388fb49a8c3d098fd762392299742b0e120
The repo build process does many git clone operations. This patch
ensures that the repo server git configuration allows git to detect
and make use of the number of CPU which are present, which
benefits operations such as check out which are highly threaded.
Change-Id: Ib01fc1c560dcb9261c328841d7472c87434edab0
In an environment with sudo and lsyncd not already installed the pre
install tasks try to set up config files before the config directories
exist, and the tasks fail.
This patch moves the sudo and lsyncd config tasks to run after the
distro packages have been installed.
In addition, sudo is added as a required distro package for the
repo server.
Change-Id: Iab09b732d63d007c1d638ec9dc35834e564e5ce5
The repo container's package cache causes quite a bit of confusion
given that it's a 'hidden' feature which catches deployers off-guard
when they already have their own cache configured. This is really
the kind of service which people should manage outside of OSA. It
also makes no sense if the deployer is using their own local mirror
which is a fairly common practise. Adding to that, it seems that it
is broken in bionic, causing massive delays in package installs.
Finally, it also adds to quite a bit of complexity due to the fact
that it's in a container - so in the playbooks prior to the container's
existence we have to detect whether it's there and add/remove the config
accordingly.
Let's just remove it and let deployers managing their own caching
infrastructure if they want it.
Change-Id: I829b9cfa16fbd1f9f4d33b5943f1e46623e1b157
The ssh service on ubuntu based systems is "ssh" which is established by
the service unit path `/lib/systemd/system/ssh.service`. When running
the service will respond to the name "sshd" however this is just an
alias. This change adds a variable to set the service unit name
based on the distro family which will allow the service to start should
it be masked.
Change-Id: I04b9f5ed761270c0bb76b607a6562b511c6f4773
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
The repo server relies on SSHD for the repo synchronisation,
so in this patch we ensure that it is installed, enabled and
running.
Change-Id: I0618e76125fc1489903b3518b9bef3999c8c7347
For upgrades, and some new build configurations, trying to
create a new venv results in the error:
Could not get output from /usr/bin/virtualenv --help: Traceback (most recent call last):
File "/usr/bin/virtualenv", line 6, in <module>
from pkg_resources import load_entry_point
ImportError: No module named pkg_resources
To work around this issue, we do the initial build of the
venv using the CLI with the argument '--never-download'
so that virtualenv just uses the versions of pip, setuptools
and wheel that it already has.
Change-Id: I639bc78d34b640a52c32fa175b12fa958518e999
Given that the openstack_hosts role installs pip and virtualenv,
we do not need this extra meta-dependency and extra task/var.
Change-Id: Iac9f72586f6b26bd31d59a4fa5055687ff77f78b
With the more recent versions of ansible, we should now use
"is" instead of the "|" sign for the tests.
This should fix it.
Change-Id: Ib73373d8801c9eedd7c6e92b994a2fba0349ed90
With the removal of EPEL, we are selectively adding packages that
we need. This patch adds EPEL repositories only for lsyncd to
avoid conflicts and the NGINX upstream repositories for the web
server.
Change-Id: I832ae27570c5f35d01728a9e387fc451447b38ce
Dmitriy Rabotyagov