Commit Graph

33 Commits

Author SHA1 Message Date
Dan Prince 9b1bb23aa6 flatten the ec2 service configurations
This change combines the previous puppet and docker files into a single
file that performs the docker service installation and configuration
for the ec2 services.

Related-Blueprint: services-yaml-flattening

Change-Id: I401a1766222d45a4a584069d27cd880806cbab9f
2019-02-25 09:21:26 -05:00
Jose Luis Franco Arza 2e5de85cef Fix with_items indentation in logs readme ec2-api.
Wrongly indented with_items parameter for task 'ec2 logs readme' is
failing, however it wasn't caught because the task includes the
ignore_errors param.

Change-Id: Ib60de9b0df451273d1d81ba049b46b5214e09080
2019-02-08 11:21:13 +01:00
David J Peacock 1d3fe8cb27 flatten database service MySQL Client
This change combines the previous puppet and docker files into a single
file that performs the docker service installation and configuration.

With this patch the baremetal version of database service MySQL Client
has been removed.

Change-Id: I855524f30cfe3c8cdab6c52a67fba0dee157103d
Related-Blueprint: services-yaml-flattening
2019-01-30 09:38:32 -05:00
Alex Schultz 096fa87741 Explicitly manage http configs
Previously the kolla config is merging the existing apache configuration
files in the container with our generated ones. This can lead to extra
configurations in the containers that we are not expecting. This change
updates the kolla configs to not merge the httpd conf.d folder so we only
end up with our expected configurations.

Change-Id: Ibb9bbeb12e73b2cf8887554f461873e42532edd7
Related-Bug: 1813084
2019-01-24 11:08:28 -07:00
Rabi Mishra 87bc72a37c Use net=none for *_init_log(s) containers
It seems we're using the default docker0(docker)/cni0(podman) bridges
for these containers. It would be good to remove dependencies on these
default bridges.

Some other containers changed that have the same issue:
- ironic_inspector_init_dnsmasq_dhcp_hostsdir
- nova_statedir_owner
- swift_setup_srv
- swift_copy_rings
- horizon_fix_perms

Change-Id: Id3e60de7bb5a13e32544bed76e4970d53a9257d4
2019-01-21 13:43:07 +05:30
Carlos Camacho 1e318b569e Ensure logs folder is created in prep hosts tasks.
We have non fatal errors in the upgrade
jobs execution if the logs folder is not
created when adding the readme.txt file
to clarify the possible locations of
the logs.

Closes-Bug: 1811708
Change-Id: Ibc0a266bdc6630eaf34bfadeff21f7bd72fa75ad
2019-01-18 09:28:56 +01:00
Emilien Macchi e4ee042a2a upgrade: remove tasks that stop and disable services
We don't need upgrade_tasks that stop systemd services since all
services are now containerized.
However, we decided to keep the tasks that remove the rpms in case some
of deployments didn't cleanup them in previous releases, they can still
do it now.

Change-Id: I6abdc9e37966cd818306f7af473958fd4662ccb5
Related-Bug: #1806733
2018-12-10 09:19:59 -05:00
Emilien Macchi 47b75398d1 upgrade: remove Docker containers now managed by Podman
For all containers where restart=always is configured and that are not
managed by Pacemaker (this part will be handled later), we remove these
containers at step 1 of post_upgrade_tasks.

Change-Id: Id446dbf7b0a18bd1d4539856e6709d35c7cfa0f0
2018-11-27 00:20:31 +00:00
Cédric Jeanneret c76584c021 Set setype for log and persistant data directories
This allows to deploy and use the services on a selinux-enforcing host
with proper selinux separation.

Change-Id: Icde6c61a0b26741946d079b2b00475de34722bea
2018-11-12 02:04:07 +00:00
Juan Antonio Osorio Robles cb3c72f37d Remove references to logging_source
This has been unused for a while, and even deprecation was scheduled
(although the patch never merged [1]). So, in order to stop folks
getting confused with this, it's being removed.

[1] https://review.openstack.org/#/c/543871/

Change-Id: Iada64874432146ef311682f26af5990469790ed2
2018-10-08 13:43:47 +03:00
Juan Antonio Osorio Robles 90234f4f2a Remove references to logging_group
This has been unused for a while, and even deprecation was scheduled
(although the patch never merged [1]). So, in order to stop folks
getting confused with this, it's being removed.

[1] https://review.openstack.org/#/c/543871/

Change-Id: Icc6b51044ccc826f5b629eb1abd3342813ed84c0
2018-08-29 13:43:30 +03:00
Lukas Bezdicka 56bec75c02 Upgrades: Refactor playbooks to set facts
To not to redefine variable multiple times in each service we
run check only once and we set fact. To increase readability of
generated playbook we add block per strep in services.

Change-Id: I2399a72709d240f84e3463c5c3b56942462d1e5c
2018-06-08 11:46:12 +02:00
Cédric Jeanneret 3848713182 Correct log location in container environment for Fluentd
Since we moved services in containers, their logs aren't in the old location, but
in /var/log/containers/<service>. This patch fixes the generated Hiera hash used
by Fluentd for its configuration

Regarding Designate config service: some of the yaml doesn't use at all the
`service_config_settings` parameter - they will need to be updated accordingly
once it's supported

Co-Authored-By: Thomas Herve <therve@redhat.com>, Steven Hardy <shardy@redhat.com>
Change-Id: I1bc0930de4053dc1c34b50477e82d9ccdab7ae2e
Closes-Bug: 1769051
Related-Bug: 1674715
2018-05-18 10:18:43 +02:00
Carlos Camacho 44ef2a3ec1 Change template names to rocky
The new master branch should point now to rocky.

So, HOT templates should specify that they might contain features
for rocky release [1]

Also, this submission updates the yaml validation to use only latest
heat_version alias. There are cases in which we will need to set
the version for specific templates i.e. mixed versions, so there
is added a variable to assign specific templates to specific heat_version
aliases, avoiding the introductions of error by bulk replacing the
the old version in new releases.

[1]: https://docs.openstack.org/heat/latest/template_guide/hot_spec.html#rocky
Change-Id: Ib17526d9cc453516d99d4659ee5fa51a5aa7fb4b
2018-05-09 08:28:42 +02:00
Lukas Bezdicka 0cb5c847f3 Always evaluate step first in conditional
If we use variables defined in later step in conditional before
checking which step are we on we will fail.

Resolves: rhbz#1535457
Closes-Bug: #1743764
Change-Id: Ic21f6eb5c4101f230fa894cd0829a11e2f0ef39b
2018-02-09 17:12:29 +01:00
marios dec003def8 Convert tags to when statements for Q major upgrade workflow
This converts "tags: stepN" to "when: step|int == N" for the direct
execution as an ansible playbook, with a loop variable 'step'.
The tasks all include the explicit cast |int.

This also adds a set_fact task for handling of the package removal
with the UpgradeRemovePackages parameter (no change to the interface)

The yaml-validate also now checks for duplicate 'when:' statements

Q upgrade spec @ Ibde21e6efae3a7d311bee526d63c5692c4e27b28
Related Blueprint: major-upgrade-workflow
[0]: 394a92f761/tripleo_common/utils/config.py (L141)
Change-Id: I6adc5619a28099f4e241351b63377f1e96933810
2018-01-08 13:57:47 +02:00
Zuul 107b610923 Merge "Change template names to queens" 2017-11-24 16:21:27 +00:00
Rajesh Tailor 1ccb030120 docker/internal TLS: spawn extra container for ec2api TLS proxy
This spawns extra containers that runs httpd to run the TLS proxy
that will go in front of ec2-api and ec2-api-metadata.

bp tls-via-certmonger-containers

Depends-On: I214fe20e12487395e1c6e247e92b2f53ba158ff9
Depends-On: Iae8e61cb5be4faeea8861296629dd6a5f3ed4f01
Change-Id: I847e13c22354aab7759364e04e009f40e6a75b79
2017-11-23 10:51:33 +00:00
Carlos Camacho 927495fe3d Change template names to queens
The new master branch should point now to queens instead of pike.

So, HOT templates should specify that they might contain features
for queens release [1]

[1]: https://docs.openstack.org/heat/latest/template_guide/hot_spec.html#queens

Change-Id: I7654d1c59db0c4508a9d7045f452612d22493004
2017-11-23 10:15:32 +01:00
Zuul 0e75343d01 Merge "Add validation task in docker services [Mixed-1]" 2017-11-22 18:30:02 +00:00
Zuul 6a72a9f1b4 Merge "Drop step_config as top level docker requirement" 2017-11-17 08:33:29 +00:00
Dan Prince a307fe7ffc Drop step_config as top level docker requirement
Step config is only required within the puppet_configs section
of docker/services/*. This patch drops the top level 'step_config'
and updates the unit tests accordingly.

Change-Id: I7dc7cfae3ef1965ec95b1d9ef23e7f162418c034
2017-11-15 16:01:16 -05:00
Jiri Stransky 85ec193403 Write readme.txt into old log directories
This should help operators find the new log files. We do have them
documented, but not everybody reads every word in the docs :)

The readme creation has ignore_errors: true so that if the directory
isn't present at all (e.g. on deployed server environments, which
don't have openstack packages installed), we don't fail the deployment
when we're not able to create the readme.

Change-Id: I6b36db7b7ce8b3e4da566eb7828d0c3b8646a14f
Partial-Bug: #1730957
2017-11-14 10:35:11 +01:00
Jose Luis Franco Arza 07c154f3ed Add validation task in docker services [Mixed-1]
Docker services are missing the pre-upgrade validation task
in the upgrade_tasks section which verifies if the service
is running before going on with the upgrade.

Change-Id: Idf93d01521af4ae08702168d68941025cffeca44
Partial-Bug: #1704389
2017-11-03 10:53:05 +01:00
Juan Badia Payno 5dbe1121e9 docker: add logging(source & groups)
The services that docker depends on, have logging_sources and logging_groups;
but those are not set on the docker outputs so they are not used when dockers
are deployed.

Added logging_source & logging_groups as docker optional parameters in
tools/yaml-validate.py

Closes-Bug: #1718110
Change-Id: I8795eaf4bd06051e9b94aa50450dee0d8761e526
2017-09-27 07:37:14 +00:00
Damien Ciabrini 0cb45d65c6 Generate MySQL client config if service requires database
Services that access database have to read an extra MySQL configuration file
/etc/my.cnf.d/tripleo.cnf which holds client-only settings, like client bind
address and SSL configuration. The configuration file is thus used by
containerized services, but also by non-containerized services that still
run on the host.

In order to generate that client configuration file appropriately both on the
host and for containers, 1) the MySQLClient service must be included by the
role; 2) every containerized service which uses the database must include the
mysql::client profile in the docker-puppet config generation step.

By including the mysql::client profile in each containerized service, we ensure
that any change in configuration file will be reflected in the service's
/var/lib/config-data/{service}, and that paunch will restart the service's
container automatically.

We now only rely on MySQLClient from puppet/services, to make it possible to
generate /etc/my.cnf.d/tripleo.cnf on the host, and to set the hiera keys that
drive the generation of that config file in containers via docker-puppet.

We include a new YAML validation step to ensure that any service which depends
on MySQL will initialize the mysql::client profile during the docker-puppet
step.

Change-Id: I0dab1dc9caef1e749f1c42cfefeba179caebc8d7
2017-07-27 13:41:13 -04:00
Jenkins 2185b83560 Merge "Use a single configuration file for specifying docker containers." 2017-07-15 06:19:13 +00:00
Ian Main e76d84f784 Use a single configuration file for specifying docker containers.
This removes the default container names from all the templates
and uses a single environment file to specify the full container
name and registry from which to pull.  Also does away with most
of DockerNamespace.

Change-Id: Ieaedac33f0a25a352ab432cdb00b5c888be4ba27
Depends-On: Ibc108871ebc2beb1baae437105b2da1d0123ba60
Co-Authored-By: Dan Prince <dprince@redhat.com>
Co-Authored-By: Steve Baker <sbaker@redhat.com>
2017-07-14 22:23:02 +00:00
Giulio Fidente baf6eee501 Adds network/cidr mapping into a new service property
Makes it possible to resolve network subnets within a service
template; the data is transported into a new property ServiceData
wired into every service which hopefully is generic enough to
be extended in the future and transport more data.

Data can be consumed in service templates to set config values
which need to know what is the subnet where a deamon operates (for
example the Ceph Public vs Cluster network).

Change-Id: I28e21c46f1ef609517175f7e7ee19e28d1c0cba2
2017-07-14 13:44:04 +02:00
Martin André cf18e865d1 Copy only generated puppet files into the container
This solves a problem with bind-mounts when the containers are holding
files descriptors open.

At the same time this makes the template more robust to puppet changes
since new config files will be available in the containers without
needing to update the templates.

Partial-Bug: #1698323
Change-Id: Ia4ad6d77387e3dc354cd131c2f9756939fb8f736
2017-07-10 11:13:25 +02:00
Sven Anderson 630030a431 Fix typo in config_volume
Change-Id: I6a53a56c534f24cb514d8aeb8cec3d7865b93448
2017-06-30 08:30:36 +02:00
Martin André a474ae82d5 Add heat parameter for all of config_volume images
This commit consistently defines a heat template parameter in the form
of DockerXXXConfigImage where XXX represents the name of the
config_volume that is used by docker-puppet.

The goal is to mitigate hard to debug errors where the templates would
set different defaults for the image docker-puppet.py uses to run, for
the same config_volume name.

This fixes a couple of inconsistencies on the way.

Change-Id: I212020a76622a03521385a6cae4ce73e51ce5b6b
Closes-Bug: #1699791
2017-06-28 10:48:53 +02:00
Sven Anderson c4a247d580 Add Ec2Api to container based deployment
Depends-On: I5dc10ef5cccf6d378c20c68fc4a32d2d3c38233f
Change-Id: Ib96040c2e27ad76b1fa6ecb9468bb9d97b3c4518
2017-06-15 17:40:10 +02:00