Re-Add facter cache for container configurations

Rather than running all the facts for every invocation of
container-puppet.py, let's cache the facts for the entire container
config step.  This is similar to what used to happen when we ran puppet
in a single instance for a given step. This improves the time it takes
puppet tp execute within the containers. When systems have large number
of interfaces, the fact generation for puppet can come to a crawl given
that puppet is not supposed to be invoked multiple times in parallel. By
pre-caching the facts, we eliminate the need for puppet to do the
network (and other fact) related lookups when running container
configuration generation.

This patch includes logic to exclude the puppet caching dirs that we
mount as read only in the config containers.

Conflicts:
	common/container-puppet.py

Change-Id: I94002572ec7ac1767fbfbfdf4c6fa6fbf0cd58d5
Closes-Bug: #1837082
Closes-Bug: #1835959
(cherry picked from commit 08e34dbcae)
This commit is contained in:
Alex Schultz 2019-07-18 10:36:05 -06:00
parent 4305cf9075
commit 10e8506e16
2 changed files with 67 additions and 1 deletions

View File

@ -336,8 +336,17 @@ if not os.path.exists(sh_script):
exclude_files+=" --exclude=$p"
fi
done
rsync -a -R --delay-updates --delete-after $exclude_files $rsync_srcs /var/lib/config-data/${NAME}
# Exclude read-only mounted directories/files which we do not want
# to copy or delete.
ro_files="/etc/puppetlabs/ /opt/puppetlabs/"
for ro in $ro_files; do
if [ -e "$ro" ]; then
exclude_files+=" --exclude=$ro"
fi
done
rsync -a -R --delay-updates --delete-after $exclude_files $rsync_srcs /var/lib/config-data/${NAME}
# Also make a copy of files modified during puppet run
# This is useful for debugging
@ -418,6 +427,9 @@ def mp_puppet_config(*args):
'--volume', '/etc/pki/tls/certs/ca-bundle.trust.crt:/etc/pki/tls/certs/ca-bundle.trust.crt:ro',
'--volume', '/etc/pki/tls/cert.pem:/etc/pki/tls/cert.pem:ro',
'--volume', '%s:/var/lib/config-data/:rw' % config_volume_prefix,
# facter caching
'--volume', '/var/lib/container-puppet/puppetlabs/facter.conf:/etc/puppetlabs/facter/facter.conf:ro',
'--volume', '/var/lib/container-puppet/puppetlabs/:/opt/puppetlabs/:ro',
# Syslog socket for puppet logs
'--volume', '/dev/log:/dev/log:rw']
if privileged:

View File

@ -421,6 +421,60 @@
- host_config
ignore_errors: "{{ ansible_check_mode|bool }}"
#########################################
# Pre-cache facts for container-puppet.py
#########################################
- name: Create puppet caching structures
file:
path: /var/lib/container-puppet/puppetlabs
state: directory
setype: svirt_sandbox_file_t
selevel: s0
recurse: True
tags:
- container_config
- container_config_tasks
- name: Write facter cache config
copy:
dest: /var/lib/container-puppet/puppetlabs/facter.conf
content: |
facts : {
ttls: [
{ "kernel" : 8 hour },
{ "memory" : 8 hour },
{ "networking" : 8 hour },
{ "operating system" : 8 hour },
{ "processor" : 8 hour },
]
}
tags:
- container_config
- container_config_tasks
- name: Cleanup facter cache if exists
file:
path: /opt/puppetlabs/facter
state: absent
ignore_errors: True
tags:
- container_config
- container_config_tasks
- name: Pre-cache facts
command: facter --config /var/lib/container-puppet/puppetlabs/facter.conf
no_log: True
ignore_errors: True
tags:
- container_config
- container_config_tasks
- name: Sync cached facts
synchronize:
src: /opt/puppetlabs/
dest: /var/lib/container-puppet/puppetlabs/
delegate_to: "{{ inventory_hostname }}"
tags:
- container_config
- container_config_tasks
######################################
# Generate config via container-puppet.py
######################################