This change combines the previous puppet and docker files into a single
file that performs the docker service installation and configuration.
With this patch the baremetal version of memcached services has been removed.
Depends-On: https://review.rdoproject.org/r/#/c/16994/
Change-Id: Ibb74d9e1673d079a6090efe4215c7ee041fce7d6
Related-Blueprint: services-yaml-flattening
We don't need upgrade_tasks that stop systemd services since all
services are now containerized.
However, we decided to keep the tasks that remove the rpms in case some
of deployments didn't cleanup them in previous releases, they can still
do it now.
Change-Id: I6abdc9e37966cd818306f7af473958fd4662ccb5
Related-Bug: #1806733
For all containers where restart=always is configured and that are not
managed by Pacemaker (this part will be handled later), we remove these
containers at step 1 of post_upgrade_tasks.
Change-Id: Id446dbf7b0a18bd1d4539856e6709d35c7cfa0f0
This has been unused for a while, and even deprecation was scheduled
(although the patch never merged [1]). So, in order to stop folks
getting confused with this, it's being removed.
[1] https://review.openstack.org/#/c/543871/
Change-Id: Iada64874432146ef311682f26af5990469790ed2
This has been unused for a while, and even deprecation was scheduled
(although the patch never merged [1]). So, in order to stop folks
getting confused with this, it's being removed.
[1] https://review.openstack.org/#/c/543871/
Change-Id: Icc6b51044ccc826f5b629eb1abd3342813ed84c0
Add block to step_0 for all services
Add block to step_6 for neutron-api.yaml
Add block to step_1 for nova-compute.yaml
Change-Id: Ib4c59302ad5ad64f23419cd69ee9b2a80333924e
Even though https://bugs.launchpad.net/tripleo/+bug/1776889 got fixed,
memcached is still configured to log to a file due to the way
puppet-memcached works:
- It would omit logging to a file when the logfile parameter (which has
a default value) is set to undef
- In puppet/hiera undeffing a parameter that has a default value does
not work: https://tickets.puppetlabs.com/browse/PUP-5295
In fact even after the fixes from https://bugs.launchpad.net/tripleo/+bug/1776889 get included we still see the following:
[root@rhosp-ctr01 ~]# ps -ef | grep memcached
42457 44366 44342 0 10:06 ? 00:00:00 /bin/bash -c source /etc/sysconfig/memcached; /usr/bin/memcached -p ${PORT} -u ${USER} -m ${CACHESIZE} -c ${MAXCONN} $OPTIONS
42457 44385 44366 0 10:06 ? 00:00:08 /usr/bin/memcached -p 11211 -u memcached -m 16005 -c 8192 -v -l 192.168.18.11 -U 0 -t 8 >> /var/log/memcached.log 2>&1
This change depends on the following memcached PR:
https://github.com/saz/puppet-memcached/pull/99/files to work correctly.
With this patch memcached logs to stdout/err correctly:
[root@overcloud-controller-0 etc]# ps auxwfw |grep memcache
42457 37207 0.0 0.0 11680 1396 ? Ss 19:23 0:00 | \_ /bin/bash -c source /etc/sysconfig/memcached; /usr/bin/memcached -p ${PORT} -u ${USER} -m ${CACHESIZE} -c ${MAXCONN} $OPTIONS
42457 37224 0.0 0.0 293160 3724 ? Sl 19:23 0:00 | \_ /usr/bin/memcached -p 11211 -u memcached -m 5877 -c 8192 -v -l 172.16.2.14 -U 0 -X -t 2
Change-Id: I8cec029bd1f64ec1daa76a89e949e7d245580918
Closes-Bug: #1782223
Memcached logging configuration is not friendly.
We cannot use configuration options as they seemed to be
ignored because of a bug. We cannot use redirection >>
as this breaks logrotate signalling SIGHUP to processes
and expecting them to handle the open file descriptors
associated with truncated/removed logfiles - this does
not work with redirected output into a file.
The only option is not log memcached into a file. There is
not much of actual value in the log. Keep it logging to
stdout/stderr in its container, which will get captured in
the docker logs (and journald via the logging backend).
Co-authored-by: Sergii Golovatiuk <sgolovat@redhat.com>
Co-authored-by: Alex Schultz <aschultz@redhat.com>
Closes-bug: #1776889
Change-Id: I3b5aec6b7f7c5fa4f84e3c3eb243c155a6189e2c
Signed-off-by: Bogdan Dobrelya <bdobreli@redhat.com>
To not to redefine variable multiple times in each service we
run check only once and we set fact. To increase readability of
generated playbook we add block per strep in services.
Change-Id: I2399a72709d240f84e3463c5c3b56942462d1e5c
The new master branch should point now to rocky.
So, HOT templates should specify that they might contain features
for rocky release [1]
Also, this submission updates the yaml validation to use only latest
heat_version alias. There are cases in which we will need to set
the version for specific templates i.e. mixed versions, so there
is added a variable to assign specific templates to specific heat_version
aliases, avoiding the introductions of error by bulk replacing the
the old version in new releases.
[1]: https://docs.openstack.org/heat/latest/template_guide/hot_spec.html#rocky
Change-Id: Ib17526d9cc453516d99d4659ee5fa51a5aa7fb4b
Instead of using host_prep_tasks (which are part of deployment tasks),
we'll use the upgrade tasks that are now well known and tested in
previous releases, when the we containerized the overcloud.
Depends-On: Id25e6280b4b4f060d5e3f78a50ff83aaca9e6b1a
Change-Id: Ic199c7d431e155e2d37996acd0d7b924d14af2b7
Using host_prep_tasks interface to handle undercloud teardown before we
run the undercloud install.
The reason of not using upgrade_tasks is because the existing tasks were
created for the overcloud upgrade first and there are too much logic
right now so we can easily re-use the bits for the undercloud. In the
future, we'll probably use upgrade_tasks for both the undercloud and
overcloud but right now this is not possible and a simple way to move
forward was to implement these tasks that work fine for the undercloud
containerization case.
Workflow will be:
- Services will be stopped and disabled (except mariadb)
- Neutron DB will be renamed, then mariadb stopped & disabled
- Remove cron jobs
- All packages will be upgraded with yum update.
Change-Id: I36be7f398dcd91e332687c6222b3ccbb9cd74ad2
This patch enables health check execution for memcached docker container.
Change-Id: Ie3634af9c62787ab4059a18778d3a3632dec2500
Depends-On: Ib82cb849540694106a869ec81694f1159967ee79
If we use variables defined in later step in conditional before
checking which step are we on we will fail.
Resolves: rhbz#1535457
Closes-Bug: #1743764
Change-Id: Ic21f6eb5c4101f230fa894cd0829a11e2f0ef39b
To prevent users walking the memcached keys, Add "-X"
to memcached in both containerized and puppet memcached
overcloud services.
Change-Id: I50eefdbdf7a7911f2ba6a7f3b4e739b8e67a7c1c
Partial-Bug: #1738835
This converts "tags: stepN" to "when: step|int == N" for the direct
execution as an ansible playbook, with a loop variable 'step'.
The tasks all include the explicit cast |int.
This also adds a set_fact task for handling of the package removal
with the UpgradeRemovePackages parameter (no change to the interface)
The yaml-validate also now checks for duplicate 'when:' statements
Q upgrade spec @ Ibde21e6efae3a7d311bee526d63c5692c4e27b28
Related Blueprint: major-upgrade-workflow
[0]: 394a92f761/tripleo_common/utils/config.py (L141)
Change-Id: I6adc5619a28099f4e241351b63377f1e96933810
This was supposed to be in I6b36db7b7ce8b3e4da566eb7828d0c3b8646a14f
but i missed it on the edit because memcached doesn't have its own
directory, so it didn't match the regex i created to edit the
patch. Apologies for the patch churn on this. This fix has been
directly incorporated into I6b36db7b7ce8b3e4da566eb7828d0c3b8646a14f
backport, so we at least won't need to backport this oneliner.
Change-Id: I31d8adf72ea6c3544e534dca57c5c415e2908674
Partial-Bug: #1730957
Step config is only required within the puppet_configs section
of docker/services/*. This patch drops the top level 'step_config'
and updates the unit tests accordingly.
Change-Id: I7dc7cfae3ef1965ec95b1d9ef23e7f162418c034
This should help operators find the new log files. We do have them
documented, but not everybody reads every word in the docs :)
The readme creation has ignore_errors: true so that if the directory
isn't present at all (e.g. on deployed server environments, which
don't have openstack packages installed), we don't fail the deployment
when we're not able to create the readme.
Change-Id: I6b36db7b7ce8b3e4da566eb7828d0c3b8646a14f
Partial-Bug: #1730957
Docker services are missing the pre-upgrade validation task
in the upgrade_tasks section which verifies if the service
is running before going on with the upgrade.
Change-Id: Iee10761330311d40825f0ad11b676d8f1f20355f
Partial-Bug: #1704389
I missed this in change
Iecce2820657c5ad6bd2b02849c0433e5dce8009b. Memcached seems to be the
only service which mounts the /var/log/container/<service-name>
directory directly under /var/log in the container.
Change-Id: I26928bc7ad5f2f1bed60d200e8f2f0c310a14ee5
Partial-Bug: #1730957
We were relying on the sysconfig options to set the memcached log file,
however, this is not happening, as the redirection is being taken as an
option and ends up being ignored by the memcached command. So instead,
we set the redirection in the container template.
Change-Id: Ic94e3fd7884d518eb9558c53acdc6b294823cd0a
Closes-Bug: #1720183
We used to bind-mount /var/log/memcached.log, but this resulted in the
file being createdin the memcached container as a directory, since this
file didn't exist.
This commit takes the approach of other containers and gets the logs to
a memcached directory in /var/log/containers.
Change-Id: I926b65fa557ad56b4faa2be34452b58f7b01247a
Closes-Bug: #1720183
The services that docker depends on, have logging_sources and logging_groups;
but those are not set on the docker outputs so they are not used when dockers
are deployed.
Added logging_source & logging_groups as docker optional parameters in
tools/yaml-validate.py
Closes-Bug: #1718110
Change-Id: I8795eaf4bd06051e9b94aa50450dee0d8761e526
This removes the default container names from all the templates
and uses a single environment file to specify the full container
name and registry from which to pull. Also does away with most
of DockerNamespace.
Change-Id: Ieaedac33f0a25a352ab432cdb00b5c888be4ba27
Depends-On: Ibc108871ebc2beb1baae437105b2da1d0123ba60
Co-Authored-By: Dan Prince <dprince@redhat.com>
Co-Authored-By: Steve Baker <sbaker@redhat.com>
Makes it possible to resolve network subnets within a service
template; the data is transported into a new property ServiceData
wired into every service which hopefully is generic enough to
be extended in the future and transport more data.
Data can be consumed in service templates to set config values
which need to know what is the subnet where a deamon operates (for
example the Ceph Public vs Cluster network).
Change-Id: I28e21c46f1ef609517175f7e7ee19e28d1c0cba2
This solves a problem with bind-mounts when the containers are holding
files descriptors open.
At the same time this makes the template more robust to puppet changes
since new config files will be available in the containers without
needing to update the templates.
Partial-Bug: #1698323
Change-Id: Ia4ad6d77387e3dc354cd131c2f9756939fb8f736
This commit consistently defines a heat template parameter in the form
of DockerXXXConfigImage where XXX represents the name of the
config_volume that is used by docker-puppet.
The goal is to mitigate hard to debug errors where the templates would
set different defaults for the image docker-puppet.py uses to run, for
the same config_volume name.
This fixes a couple of inconsistencies on the way.
Change-Id: I212020a76622a03521385a6cae4ce73e51ce5b6b
Closes-Bug: #1699791
In many occasions we had log directory initialization containers
without `detach: false`, which didn't guarantee that they'll finish
before the container depending on them will start using the log
directory.
This is now fixed by moving the initialization container one global
step earlier, so that we can keep the concurrency when creating the
log dirs. (Using `detach: false` makes paunch handle just one
container at a time, and as such it can have negative performance
impact.)
For services which have their container(s) starting in step_1,
initialization cannot be moved to an earlier step, so the solution
here was to just add `detach: false`.
As a minor related change, cinder DB sync container now mounts the log
directory from host to put cinder-manage.log into the expected
location.
Change-Id: I1340de4f68dd32c2412d9385cf3a8ca202b48556
This was forgotten in I72376a803ec6b2ed93903cc0c95a6ffce718b6dc and
broke containerized deployment.
Change-Id: I599a87bf06efbfefd3067c77ed6ca866505900f9
Closes-Bug: #1690870
When a service is enabled on multiple roles, the parameters for the
service will be global. This change enables an option to provide
role specific parameter to services and other templates.
Two new parameters - RoleName and RoleParameters, are added to the
service template. RoleName provides the role name of on which the
current instance of the service is being applied on. RoleParameters
provides the list of parameters which are configured specific to the
role in the environment file, like below:
parameters_default:
# Default value for applied to all roles
NovaReservedHostMemory: 2048
ComputeDpdkParameters:
# Applied only to ComputeDpdk role
NovaReservedHostMemory: 4096
In above sample, the cluster contains 2 roles - Compute, ComputeDpdk.
The values of ComputeDpdkParameters will be passed on to the templates
as RoleParameters while creating the stack for ComputeDpdk role. The
parameter which supports role specific configuration, should find the
parameter first in in the RoleParameters list, if not found, then the
default (for all roles) should be used.
Implements: blueprint tripleo-derive-parameters
Change-Id: I72376a803ec6b2ed93903cc0c95a6ffce718b6dc
Some containers are using the logs named volume for collecting logs
written to `/var/log`. We should make this consistent for all the
containers.
This patch also cleans up some mounts that weren't needed for some
services. For example, glance-api doesn't need `/run` to be mounted.
Other changes:
* Rework log volumes to hostpath mounts to omit slow COW writes.
* Add kolla_config's permission and host_prep_tasks create and
manage hostpath mounted log dirs permissions.
* Rework data owning init containers to kolla_config permissions
* When a step wants KOLLA_BOOTSTRAP or DB sync, use logs data owning
init containers to set permissions for logs. This is required
because kolla bootsrap and DB sync runs before the kolla config
stage and there is yet permissions set for logs.
* In order to address hybrid cases for host services vs containerized
ones to access logs having different UIDs, persist containerized
services' logs into separate directories (an upgrade impact)
* Ensure host prep tasks to create /var/log/containers/ and /var/lib/
sub-directories for services
* Fix missing /etc/httpd, /var/www config-data mounts for zaqar/ironic
* Fix YAML indentation and drop strings quotation.
Co-authored-by: Bogdan Dobrelya <bdobreli@redhat.com>
Partial blueprint containerized-services-logs
Change-Id: I53e737120bf0121bd28667f355b6f29f1b2a6b82
list_concat was introduced recently and is able to replace the yaql
calls for concatenating lists.
Change-Id: Id3a80a0e1e4c25b6d838898757c69ec99d0cd826
This enables common resources that the docker templates might need.
The initial resource only is common volumes, and two volumes are
introduced (localtime and hosts).
Change-Id: Ic55af32803f9493a61f9b57aff849bfc6187d992
We don't use docker_image for anything. It is a remant of the
pre-composable docker templates and we can now remove it.
This patch removes references to the 'docker_image' section
from docker/post.yaml and all of the docker/services* templates.
Change-Id: I208c1ef1550ab39ab0ee47ab282f9b1937379810
This aligns the docker based services with the new composable upgrades
architecture we landed for ocata, and does a first-pass adding upgrade_tasks
for the services (these may change, atm we only disable the service on
the host).
To run the upgrade workflow you basically do two steps:
openstack overcloud deploy --templates \
-e environments/major-upgrade-composable-steps-docker.yaml
This will run the ansible upgrade steps we define via upgrade_tasks
then run the normal docker PostDeploySteps to bring up the containers.
For the puppet workflow there's then an operator driven step where
compute nodes (and potentially storage nodes) are upgrades in batches
and finally you do:
openstack overcloud deploy --templates \
-e environments/major-upgrade-converge-docker.yaml
In the puppet case this re-applies puppet to unpin the nova RPC API
so I guess it'll restart the nova containers this affects but otherwise
will be a no-op (we also disable the ansible steps at this point.
Depends-On: I9057d47eea15c8ba92ca34717b6b5965d4425ab1
Change-Id: Ia50169819cb959025866348b11337728f8ed5c9e
This approach removes the need for the yaql zip to build the
docker-puppet data by building the data in a puppet_config dict.
This allows a future change to make docker-puppet.py only accept dict
data.
Currently the step_config is left where it is and referenced inside
puppet_config, but feedback is welcome whether this is necessary or
desirable.
Change-Id: I4a4d7a6fd2735cb841174af305dbb62e0b3d3e8c
David J Peacock