This is necessary to ensure it is possible for apt to write to /tmp
when accessing the container image as a chroot during cache prep.
In addition fix up cross-repo zuul jobs to be in line with the rest of OSA
by dropping the Suse job and switching debian stable(stretch) for buster.
Temporarily make the cross-repo jobs non-voting to break a circular
dependancy between fixes in nspawn_hosts and nspawn_container_create.
Depends-On: https://review.opendev.org/728995
Change-Id: I8c6034f49600b9f1f431eb2a7c02ae567ea6d32c
This patch adds the Debian jobs for this role to make sure
it's always passing as well as updates the meta and creates
debian.yml var file to reflect it's support of Debian accordingly.
Change-Id: I2653aea2aebc95bfc67fe49bc66fb68a4f996170
The nspawn cache prep is timing out regularly so use eatmydata to minimise
pressure on the filesystem during package installation.
dpkg calls fsync for each file installed from each package which can
result in very poor performance on some filesystems. This patch gives
around a 4x improvement in "slow" gate runs, and 2x on esxi hdd storage.
Change-Id: I80efd3f8044f377a497492f09689c459c6094742
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
Co-authored-by: Jonathan Rosser <jonathan.rosser@rd.bbc.co.uk>
This change updates the unit file for systemd-nspawn to allow it to
better confine containers and have them reliabily start/stop on host
restart.
Change-Id: I3c7a07a94c94a81ac8380a4e336cf744615a6b5b
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
The ssh service on ubuntu based systems is "ssh" which is established by
the service unit path `/lib/systemd/system/ssh.service`. When running
the service will respond to the name "sshd" however this is just an
alias.
The image prep task used to mask ssh after it was disabled. While
in practice thats OK, its really a little overkill and provides little
benifit. In prep the containers will have ssh disabled, but in the event
its needed it can be easily enabled wihout having to remember to unmask.
Change-Id: I2edb3ca7cae6e05f15ab29b1d50ff1281762905e
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
Bionic requires a functioning gpg-agent to run apt-key add. This means
that gpg-agent must be working properly in the chroot when the nspawn
image preparation script runs.
Previous changes [1] have enabled apt-key to communicate with gpg-agent
during the nspawn_hosts role checks, however the cache prep fails almost
every time when nspawn_hosts is used within other role checks.
This is not a new issue, debian-installer is affected too [2].
This change adopts the same route as d-i, and simply copies the host
/etc/apt/trusted.gpg.d directory to the nspawn image, removing the
need for apt-key and in turn gpg-agent.
This is a re-implementation of https://review.openstack.org/588962
for nspawn.
[1] https://review.openstack.org/590431
[2] https://bugs.debian.org/cgi-bin/bugreport.cgi?bug=851774
Change-Id: I3c56da445377d7ba27a623fb7ebe95c20d28a327
When running without bridges the local routing needs to be scoped to the
host. This change ensures that is the care.
When re-preping an environment the GPG setup will require aceess to fd's
which may not exist at the time the base cache is re-prep'd.
nspawn mtu settings will now follow the primary interface.
Change-Id: I74e9301a98cf92161feb31e6808e9e02a07f662c
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
Ensure that the devices required for apt are present. Ensure
that the required directories for gpg-agent are present. Ensure
that the gpg-agent is running.
Change-Id: I057ac43a7f88654affd14410c06e68bffe34104d
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
In modern kernels sysfs is largely shared with the host in a read only
state when an nspawn container is created. This sharing results in the
container seeing network interfaces that don't belong to them. This
change forces network isolation by instructing the container to begin
with a fully isolated network stack.
Change-Id: I9ef22c5e321dce8025cafc622bdfb6ad252ef0cf
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
The tempfiles template was being setup on the host, this causes host
machines to have an issue with journald on reboot due to the directory
being part of systemd-tmpfiles.
Nspawn container journals were not being written to the host. This was
fixed by changing try-guest to try-host, which will fall back to the
guest in the event that host journalling is not possible.
Systemd-nspawn containers were not starting on boot due to them needing
to wait for networking to be online. The base template has been updated
to ensure networking is online before starting containers.
Change-Id: I6af3923bd10091172e75cfb16c9146cd47f827a4
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
The host copy process was taking too much, this makes that process more
selective.
Change-Id: Ic25559d9b68b6208ffdbed020cc79bd6b3d52fdd
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
Update container pkg list and use qgroups
Other roles within OSA require additional packages that were removed
from the base cache. This change puts those packages back.
Additionally this change implements qgroups which will resolve an issue
where the containers are spamming a reoccuring error message indicating
an issue with the loopback file system.
Change-Id: If658e5f332e4580cc51182ac92331a2ef91fbcb8
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
Use the common roles throught this role to remove all of the boilerplate
code we had. The common modules do most of the heavy lifing.
Update to fix the resolve.conf issue with the image cache prep.
Add legacy image support and use smaller upstream images by default.
Now that suse supports systemd-networkd we can enable suse support in
nspawn.
Change-Id: I5f6ceb928f5c0902adf2e34f96a5998840400777
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
While the legacy resolveconf service is fairly stable this change
updates us to use more of the built-in services and removes additional
dependencies.
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
When the cache is created the lock file my be left over, this can cause
some conflicts later in the container provisioning process. This change
removes the file if found.
Signed-off-by: Kevin Carter <kevin.carter@rackspace.com>
Dmitriy Rabotyagov