At the moment we don't restart services if systemd unit file is changed.
We knowingly prevent systemd_service role handlers to execute
by providing `state: started` as otherwise service will be restarted twice.
With that now we ensure that role handlers will also listen for systemd
unit changes.
Change-Id: I4273d2fbcbff3028e693e3274093c1afebdcfca2
Keystone role was never migrated to usage of haproxy-endpoints role
and included task was used instead the whole time.
With that to reduce complexity and to have unified approach, all mention
of the role and handler are removed from the code.
Change-Id: I3693ee3a9a756161324e3a79464f9650fb7a9f1a
libvirtd.socket does monitor libvirtd.service and trigger service restart
when it spot that service is down.
However in order to enable tcp and tls sockets, we need libvirt
to be stopped.
Currently race condition can happen, when we stop libvirt, but it's
started by socket before we enable tls one.
To overcome this we stop socket along with service.
Change-Id: Iacc093311036fb8d6559a0e32252579303a639ba
Since all supported distros have libvirt version >= 5.7 there's
no reason to ensure that it is true.
So we remove corresponsive code and simplify logic.
Change-Id: I281829214df8affec7774a45a3ca0405a866b5c0
To secure communications from the proxy server to the compute
nodes using VeNCrypt authentication scheme.
In a previous patch a TLS server certificate was deployed to
compute nodes, this patch makes use of this same server cert for
securing VNC sessions on compute nodes. It is recommended that
this certificate be issued by a dedicated certificate authority
solely for the VNC service, as libvirt does not currently have a
mechanism to restrict what certificates can be presented by the
proxy server. This has not been implemented to reduce complexity.
In addition the noVNC proxy needs to present a client certificate
so only approved VNC proxy servers can connect to the Compute nodes.
The PKI role has been used to create a client certificate for the
nova console nodes.
Related Nova docs:
https://docs.openstack.org/nova/latest/admin/remote-console-access.html
To help with the transition from from unencrypted VNC to VeNCrypt,
initially compute nodes auth scheme allows for both encrypted and
unencrypted sessions using the variable `nova_vencrypt_auth_scheme`, this
will be removed in future releases.
Change-Id: Iafb788f80fd401c6ce6e4576bafd06c92431bd65
As per the community goal of migrating the policy file
the format from JSON to YAML[1], we need to replace policy.json to
policy.yaml and remove deprecated policy.json.
config_template has been choosen instead of the copy, since it can
properly handle content that has been lookuped.
We make a separate task not to restart service when it's not needed.
[1] https://governance.openstack.org/tc/goals/selected/wallaby/migrate-policy-format-from-json-to-yaml.html
Change-Id: I3ff3ce4f150854ece690f1bcbd7183f511fccf2e
Since libvirt 5.7 we should not use traditional mode. Instead systemd
mode should be choosen. Sockets are provided by libvirt package.
We just need to conditionally enable or disable them.
As addition we changed nova_libvirtd_listen_tls to 0 by default
because listen_tls requires certificate to be provided which is not
the case in the current role state. So we also fix behaviour of the role
when nova_libvirtd_listen_tls is 1, supposing that deployer has manually
distributed certificates across compute hosts.
Change-Id: Id73cb67de26c305908d0245551fa57a7e6448784
Closes-Bug: #1903846
The files and templates we carry are almost always in a state of
maintenance. The upstream services are maintaining these files and
there's really no reason we need to carry duplicate copies of them. This
change removes all of the files we expect to get from the upstream
service. while the focus of this change is to remove configuration file
maintenance burdens it also allows the role to execute faster.
* Source installs have the configuration files within the venv at
"<<VENV_PATH>>/etc/<<SERVICE_NAME>>". The role will now link the
default configuration path to this directory. When the service is
upgraded the link will move to the new venv path.
* Distro installs package all of the required configuration files.
To maintain our current capabilities to override configuration the
role will fetch files from the disk whenever an override is provided and
then push the fetched file back to the target using `config_template`.
Depends-On: https://review.openstack.org/636162
Change-Id: Ib7d8039513bc2581cf7bc0e2e73aa8ab5da82235
Signed-off-by: Kevin Carter <kevin@cloudnull.com>
In order to radically simplify how we prepare the service
venvs, we use a common role to do the wheel builds and the
venv preparation. This makes the process far simpler to
understand, because the role does its own building and
installing. It also reduces the code maintenance burden,
because instead of duplicating the build processes in the
repo_build role and the service role - we only have it all
done in a single place.
We also change the role venv tag var to use the integrated
build's common venv tag so that we can remove the role's
venv tag in group_vars in the integrated build. This reduces
memory consumption and also reduces the duplication.
This is by no means the final stop in the simplification
process, but it is a step forward. The will be work to follow
which:
1. Replaces 'developer mode' with an equivalent mechanism
that uses the common role and is simpler to understand.
We will also simplify the provisioning of pip install
arguments when doing this.
2. Simplifies the installation of optional pip packages.
Right now it's more complicated than it needs to be due
to us needing to keep the py_pkgs plugin working in the
integrated build.
3. Deduplicates the distro package installs. Right now the
role installs the distro packages twice - just before
building the venv, and during the python_venv_build role
execution.
Depends-On: https://review.openstack.org/598957
Change-Id: I182bde29c049a97bc2b55193aee0b5b3d8532916
Implements: blueprint python-build-install-simplification
Signed-off-by: Jesse Pretorius <jesse.pretorius@rackspace.co.uk>
In order to reduce the packages required to pip install on to the hosts,
we allow the service setup to be delegated to a specific host, defaulting
to the deploy host. We also switch as many tasks as possible to using the
built-in Ansible modules which make use of the shade library.
The 'virtualenv' package is now installed appropriately by the openstack_hosts
role, so there's no need to install it any more. The 'httplib2' package is a
legacy Ansible requirement for the get_url/get_uri module which is no longer
needed. The keystone client library is not required any more now that we're
using the upstream modules. As there are no required packages left, the task
to install them is also removed.
Unfortunately we need to use the openstack client to wait for a compute host
to register, so we add it into the nova venv and implement a change in the
way we do the wait so that openrc/clouds.yaml is only implemented on a single
compute host and the wait task is executed there.
Depends-On: https://review.openstack.org/582359
Change-Id: I702480a5188a583a03f66bb39609f7d25a996e4a
With the more recent versions of ansible, we should now use
"is" instead of the "|" sign for the tests.
This should fix it.
Change-Id: If3e4366c22e900557e4730a7e8838f55ffe30ecc
All operating systems supported by the role have systemd and these
conditionals are no longer needed.
Change-Id: I35500f7eec993b2bcdb245a995a05cacf2c596f8
The placement service is already setup to use UWsgi, we need
to move the other Nova services to follow suit as part of our community
goal for Pike.
Additionally, we need to clean up the nginx configuration as we are
moving away from fronting uWSGI with nginx inside the roles.
Depends-On: Ib66b9709fb88205eaf3f133c87357a4dbbdde5ae
Change-Id: If6c30e00c1c753692c970457b75e3ae7f5cc066c
Implements: blueprint goal-deploy-api-in-wsgi
In order to cater for artifact-based installed, and
rolling upgrades, this patch implements a set of local
facts to inform the online migrations task.
The 'nova_all_software_updated' variable will be
set by the playbook on each run to ensure that the
online migrations only happen once all venvs are
homogenous. This ensures that the playbook can be
executed in a serialised fashion and the data will
not be corrupted.
The ``upgrade_levels`` setting for ``compute`` is set
to ``auto`` to ensure that a mixed RPC version
deployment can operate properly when doing a rolling
upgrade as suggested by [1].
Additional changes are made to improve the role's
ability to be executed using serialised playbooks.
Finally, the nova-manage command references to the
config file location have been removed as they refer
to the default location.
[1] https://docs.openstack.org/developer/nova/upgrade.html
Change-Id: I08e5a7f0ce526b11aa52c35ee29c458954a5f22d
The policy.json file is currently read continually by the
services and is not only read on service start. We therefore
cannot template directly to the file read by the service
(if the service is already running) because the new policies
may not be valid until the service restarts. This is
particularly important during a major upgrade. We therefore
only put the policy file in place after the service restart.
This patch also tidies up the handlers and some of the install
tasks to simplify them and reduce the tasks/code a little.
Change-Id: Icba9df7be6012576eca0afb040a6953809cc9a5f
On CentOS the default is to have service disabled, this ensure nginx is enabled.
Closes-bug: 1681533
Change-Id: I98018fca9c277248b77b60081ea560c012b370af
During an upgrade new service files are added, but systemd is not
reloaded during restart of nova services to pick up these file
changes. This performs a daemon-reload when restarting nova
services.
Change-Id: I98b3f66429ee045f052ad491847cf82d2f5d4efc
Closes-Bug: #1673889
We don't need to restart nginx - we can instead just reload the
service. Additionally, as more services move to use nginx frontend, it
would be bad to restart all NGinx services at the same time.
For now we need to investigate the impact of reloads in UWsgi before
moving over to a "reload" on UWsgi.
Change-Id: I60e370e784a1ff3a0f5bf8551be804bf05d8bb43
A race condition is caused when nova-compute is started for the
first time because it takes a period of time for nova-compute
to spin up, register itself with nova API, and become available for
cell enrollment.
Prior to this there was no wait condition when nova-compute
restarts occurred, so the first time nova-compute started, often
the compute service was not registered in the database and available
for cell enrollment when the enrollment tasks ran.
Change-Id: I510f0a957f53d15affa1fc23f809abff52208438
This patch adds Nova requirements for Ocata:
* Nova Placement API running as uwsgi with Nginx.
* cell_v2 setup for cell0 and cell1
* All required settings for these services with sane defaults
It fixes up some ordering for DB operations:
* online_db_migrations should only happen after a full upgrade.
* Cell setup needs to happen after api_db sync but before db sync.
* Discover_hosts for cell_v2 needs to happen after compute is restarted
This adds functionality to allow uwsgi apps in the init scripts:
* Allowing the "--log" line to be adjusted.
* Setting the condition value so that only enabled services are deployed
* Fixes a bug for program_override which mean this value was never being
used.
Depends-On: I082f37bb3ce61a900e06a58f21c7882f83671355
Change-Id: I282d25988377d18257b708859f89a7ae4260ac07
Using ansible systemd module to daemon reload and service reload is the solution for the future.
Change-Id: I3f9142357379a548b1e1f4190e61157596f750fa
Co-Authored-By: Jean-Philippe Evrard <Jean-Philippe.Evrard@rackspace.co.uk>
* only kvm host are supported right now.
Depends-On: Iff4a5999be0263a2c1843d7ca29843468cbc0ccc
Depends-On: I78fb85d44b5b0e1643bd07af3e15462c02041c89
Change-Id: Ie05c243daa7d2d46b5e8779371a363d95cc990e9
Change the 'nova_service_names' from a list to a dictionary mapping
of services, groups that install those services. This brings the
method into line with that used in the os_neutron role in order to
implement a more standardised method.
The init tasks have been updated to run once and loop through this
mapping rather than being included multiple times and re-run against
each host. This may potentially reduce role run times.
Currently the reload of upstart/systemd scripts may not happen if
only one script changes as the task uses a loop with only one result
register. This patch implements handlers to reload upstart/systemd
scripts to ensure that this happens when any one of the scripts
change.
The handler to reload the services now only tries to restart the
service if the host is in the group for the service according to the
service group mapping. This allows us to ensure that handler
failures are no longer ignored and that no execution time is wasted
trying to restart services which do not exist on the host.
Finally:
- Common variables shared by each service's template files have
been updated to use the service namespaced variables.
- Unused handlers have been removed.
- Unused variables have been removed.
Change-Id: I53fb0ab1cc5762e3559d4ee2635d4cca532df7e3
When executing the role with Ansible 2.1, the following
deprecation warning is issued in the output for some tasks.
[DEPRECATION WARNING]: Using bare variables is deprecated.
This patch addresses the tasks to fix the behaviour appropriately.
Change-Id: I7ef4e446d6fc509420d5b297378f4fa91a519fc8
This change implements the blueprint to convert all roles and plays into
a more generic setup, following upstream ansible best practices.
Items Changed:
* All tasks have tags.
* All roles use namespaced variables.
* All redundant tasks within a given play and role have been removed.
* All of the repetitive plays have been removed in-favor of a more
simplistic approach. This change duplicates code within the roles but
ensures that the roles only ever run within their own scope.
* All roles have been built using an ansible galaxy syntax.
* The `*requirement.txt` files have been reformatted follow upstream
Openstack practices.
* Dynamically generated inventory is now more organized, this should assist
anyone who may want or need to dive into the JSON blob that is created.
In the inventory a properties field is used for items that customize containers
within the inventory.
* The environment map has been modified to support additional host groups to
enable the seperation of infrastructure pieces. While the old infra_hosts group
will still work this change allows for groups to be divided up into seperate
chunks; eg: deployment of a swift only stack.
* The LXC logic now exists within the plays.
* etc/openstack_deploy/user_variables.yml has all password/token
variables extracted into the separate file
etc/openstack_deploy/user_secrets.yml in order to allow seperate
security settings on that file.
Items Excised:
* All of the roles have had the LXC logic removed from within them which
should allow roles to be consumed outside of the `os-ansible-deployment`
reference architecture.
Note:
* the directory rpc_deployment still exists and is presently pointed at plays
containing a deprecation warning instructing the user to move to the standard
playbooks directory.
* While all of the rackspace specific components and variables have been removed
and or were refactored the repository still relies on an upstream mirror of
Openstack built python files and container images. This upstream mirror is hosted
at rackspace at "http://rpc-repo.rackspace.com" though this is
not locked to and or tied to rackspace specific installations. This repository
contains all of the needed code to create and/or clone your own mirror.
DocImpact
Co-Authored-By: Jesse Pretorius <jesse.pretorius@rackspace.co.uk>
Closes-Bug: #1403676
Implements: blueprint galaxy-roles
Change-Id: I03df3328b7655f0cc9e43ba83b02623d038d214e
Dmitriy Rabotyagov